=== RUN TestStartStop/group/old-k8s-version/serial/FirstStart
start_stop_delete_test.go:186: (dbg) Run: out/minikube-darwin-amd64 start -p old-k8s-version-378000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0
E0717 13:37:02.454675 38325 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/auto-859000/client.crt: no such file or directory
start_stop_delete_test.go:186: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p old-k8s-version-378000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0: exit status 109 (4m16.73742814s)
-- stdout --
* [old-k8s-version-378000] minikube v1.30.1 on Darwin 13.4.1
- MINIKUBE_LOCATION=16890
- KUBECONFIG=/Users/jenkins/minikube-integration/16890-37879/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/16890-37879/.minikube
- MINIKUBE_FORCE_SYSTEMD=
* Using the docker driver based on user configuration
* Using Docker Desktop driver with root privileges
* Starting control plane node old-k8s-version-378000 in cluster old-k8s-version-378000
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.16.0 on Docker 24.0.4 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
-- /stdout --
** stderr **
I0717 13:37:00.895219 53894 out.go:296] Setting OutFile to fd 1 ...
I0717 13:37:00.895417 53894 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0717 13:37:00.895422 53894 out.go:309] Setting ErrFile to fd 2...
I0717 13:37:00.895426 53894 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0717 13:37:00.895618 53894 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/16890-37879/.minikube/bin
I0717 13:37:00.897932 53894 out.go:303] Setting JSON to false
I0717 13:37:00.919119 53894 start.go:128] hostinfo: {"hostname":"MacOS-Agent-2.local","uptime":16591,"bootTime":1689609629,"procs":399,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.4.1","kernelVersion":"22.5.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"2965c349-98a5-5970-aaa9-9eedd3ae5959"}
W0717 13:37:00.919238 53894 start.go:136] gopshost.Virtualization returned error: not implemented yet
I0717 13:37:00.942267 53894 out.go:177] * [old-k8s-version-378000] minikube v1.30.1 on Darwin 13.4.1
I0717 13:37:00.985455 53894 notify.go:220] Checking for updates...
I0717 13:37:01.012217 53894 out.go:177] - MINIKUBE_LOCATION=16890
I0717 13:37:01.054148 53894 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/16890-37879/kubeconfig
I0717 13:37:01.096342 53894 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0717 13:37:01.139157 53894 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0717 13:37:01.182294 53894 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/16890-37879/.minikube
I0717 13:37:01.225203 53894 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0717 13:37:01.246704 53894 config.go:182] Loaded profile config "kubenet-859000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.27.3
I0717 13:37:01.246799 53894 driver.go:373] Setting default libvirt URI to qemu:///system
I0717 13:37:01.303200 53894 docker.go:121] docker version: linux-24.0.2:Docker Desktop 4.21.1 (114176)
I0717 13:37:01.303337 53894 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0717 13:37:01.407803 53894 info.go:266] docker info: {ID:9dd12a49-41d2-44e8-aa64-4ab7fa99394e Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:59 OomKillDisable:false NGoroutines:68 SystemTime:2023-07-17 20:37:01.395972065 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:5.15.49-linuxkit-pr OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexS
erverAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:3dce8eb055cbb6872793272b4f20ed16117344f8 Expected:3dce8eb055cbb6872793272b4f20ed16117344f8} RuncCommit:{ID:v1.1.7-0-g860f061 Expected:v1.1.7-0-g860f061} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=built
in name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.11.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.19.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.20] map[Name:init Pat
h:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.6] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Command line tool for Docker Scout Vendor:Docker Inc. Version:0.16.1]] Warnings:<n
il>}}
I0717 13:37:01.453309 53894 out.go:177] * Using the docker driver based on user configuration
I0717 13:37:01.474422 53894 start.go:298] selected driver: docker
I0717 13:37:01.474440 53894 start.go:880] validating driver "docker" against <nil>
I0717 13:37:01.474455 53894 start.go:891] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0717 13:37:01.478476 53894 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0717 13:37:01.585145 53894 info.go:266] docker info: {ID:9dd12a49-41d2-44e8-aa64-4ab7fa99394e Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:59 OomKillDisable:false NGoroutines:68 SystemTime:2023-07-17 20:37:01.574671534 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:5.15.49-linuxkit-pr OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexS
erverAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:6 MemTotal:6231715840 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:3dce8eb055cbb6872793272b4f20ed16117344f8 Expected:3dce8eb055cbb6872793272b4f20ed16117344f8} RuncCommit:{ID:v1.1.7-0-g860f061 Expected:v1.1.7-0-g860f061} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=built
in name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.11.0] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.19.1] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Manages Docker extensions Vendor:Docker Inc. Version:v0.2.20] map[Name:init Pat
h:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.6] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Command line tool for Docker Scout Vendor:Docker Inc. Version:0.16.1]] Warnings:<n
il>}}
I0717 13:37:01.585328 53894 start_flags.go:305] no existing cluster config was found, will generate one from the flags
I0717 13:37:01.585517 53894 start_flags.go:919] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0717 13:37:01.606965 53894 out.go:177] * Using Docker Desktop driver with root privileges
I0717 13:37:01.627735 53894 cni.go:84] Creating CNI manager for ""
I0717 13:37:01.627788 53894 cni.go:168] CNI unnecessary in this configuration, recommending no CNI
I0717 13:37:01.627806 53894 start_flags.go:319] config:
{Name:old-k8s-version-378000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.40@sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-378000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocke
t: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0}
I0717 13:37:01.664933 53894 out.go:177] * Starting control plane node old-k8s-version-378000 in cluster old-k8s-version-378000
I0717 13:37:01.686871 53894 cache.go:122] Beginning downloading kic base image for docker with docker
I0717 13:37:01.708794 53894 out.go:177] * Pulling base image ...
I0717 13:37:01.753762 53894 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0717 13:37:01.753855 53894 image.go:79] Checking for gcr.io/k8s-minikube/kicbase:v0.0.40@sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631 in local docker daemon
I0717 13:37:01.753881 53894 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4
I0717 13:37:01.753905 53894 cache.go:57] Caching tarball of preloaded images
I0717 13:37:01.754167 53894 preload.go:174] Found /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0717 13:37:01.754193 53894 cache.go:60] Finished verifying existence of preloaded tar for v1.16.0 on docker
I0717 13:37:01.755212 53894 profile.go:148] Saving config to /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/config.json ...
I0717 13:37:01.755334 53894 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/config.json: {Name:mk4706edef70dff77fd69cdb25a9366fc972e7a3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0717 13:37:01.806730 53894 image.go:83] Found gcr.io/k8s-minikube/kicbase:v0.0.40@sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631 in local docker daemon, skipping pull
I0717 13:37:01.806766 53894 cache.go:145] gcr.io/k8s-minikube/kicbase:v0.0.40@sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631 exists in daemon, skipping load
I0717 13:37:01.806788 53894 cache.go:195] Successfully downloaded all kic artifacts
I0717 13:37:01.806835 53894 start.go:365] acquiring machines lock for old-k8s-version-378000: {Name:mk1fa5bdcb933442ff3b09d713656e27b57c768b Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0717 13:37:01.806997 53894 start.go:369] acquired machines lock for "old-k8s-version-378000" in 150.88µs
I0717 13:37:01.807028 53894 start.go:93] Provisioning new machine with config: &{Name:old-k8s-version-378000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.40@sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-378000 Namespace:default APIServerName:minikubeCA APIServe
rNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:fals
e CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0} &{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0717 13:37:01.807116 53894 start.go:125] createHost starting for "" (driver="docker")
I0717 13:37:01.817175 53894 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0717 13:37:01.817375 53894 start.go:159] libmachine.API.Create for "old-k8s-version-378000" (driver="docker")
I0717 13:37:01.817417 53894 client.go:168] LocalClient.Create starting
I0717 13:37:01.817527 53894 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/ca.pem
I0717 13:37:01.817563 53894 main.go:141] libmachine: Decoding PEM data...
I0717 13:37:01.817580 53894 main.go:141] libmachine: Parsing certificate...
I0717 13:37:01.817658 53894 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/cert.pem
I0717 13:37:01.817683 53894 main.go:141] libmachine: Decoding PEM data...
I0717 13:37:01.817692 53894 main.go:141] libmachine: Parsing certificate...
I0717 13:37:01.825449 53894 cli_runner.go:164] Run: docker network inspect old-k8s-version-378000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0717 13:37:01.878636 53894 cli_runner.go:211] docker network inspect old-k8s-version-378000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0717 13:37:01.878753 53894 network_create.go:281] running [docker network inspect old-k8s-version-378000] to gather additional debugging logs...
I0717 13:37:01.878771 53894 cli_runner.go:164] Run: docker network inspect old-k8s-version-378000
W0717 13:37:01.928426 53894 cli_runner.go:211] docker network inspect old-k8s-version-378000 returned with exit code 1
I0717 13:37:01.928454 53894 network_create.go:284] error running [docker network inspect old-k8s-version-378000]: docker network inspect old-k8s-version-378000: exit status 1
stdout:
[]
stderr:
Error response from daemon: network old-k8s-version-378000 not found
I0717 13:37:01.928474 53894 network_create.go:286] output of [docker network inspect old-k8s-version-378000]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network old-k8s-version-378000 not found
** /stderr **
I0717 13:37:01.928571 53894 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0717 13:37:02.071794 53894 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0717 13:37:02.072138 53894 network.go:209] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000e4a4a0}
I0717 13:37:02.072152 53894 network_create.go:123] attempt to create docker network old-k8s-version-378000 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 65535 ...
I0717 13:37:02.072230 53894 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-378000 old-k8s-version-378000
W0717 13:37:02.121040 53894 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-378000 old-k8s-version-378000 returned with exit code 1
W0717 13:37:02.121075 53894 network_create.go:148] failed to create docker network old-k8s-version-378000 192.168.58.0/24 with gateway 192.168.58.1 and mtu of 65535: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-378000 old-k8s-version-378000: exit status 1
stdout:
stderr:
Error response from daemon: Pool overlaps with other one on this address space
W0717 13:37:02.121093 53894 network_create.go:115] failed to create docker network old-k8s-version-378000 192.168.58.0/24, will retry: subnet is taken
I0717 13:37:02.122419 53894 network.go:212] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}
I0717 13:37:02.122726 53894 network.go:209] using free private subnet 192.168.67.0/24: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000e4b630}
I0717 13:37:02.122740 53894 network_create.go:123] attempt to create docker network old-k8s-version-378000 192.168.67.0/24 with gateway 192.168.67.1 and MTU of 65535 ...
I0717 13:37:02.122814 53894 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=old-k8s-version-378000 old-k8s-version-378000
I0717 13:37:02.204283 53894 network_create.go:107] docker network old-k8s-version-378000 192.168.67.0/24 created
I0717 13:37:02.204325 53894 kic.go:117] calculated static IP "192.168.67.2" for the "old-k8s-version-378000" container
I0717 13:37:02.204461 53894 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0717 13:37:02.252869 53894 cli_runner.go:164] Run: docker volume create old-k8s-version-378000 --label name.minikube.sigs.k8s.io=old-k8s-version-378000 --label created_by.minikube.sigs.k8s.io=true
I0717 13:37:02.302729 53894 oci.go:103] Successfully created a docker volume old-k8s-version-378000
I0717 13:37:02.302859 53894 cli_runner.go:164] Run: docker run --rm --name old-k8s-version-378000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-378000 --entrypoint /usr/bin/test -v old-k8s-version-378000:/var gcr.io/k8s-minikube/kicbase:v0.0.40@sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631 -d /var/lib
I0717 13:37:02.686565 53894 oci.go:107] Successfully prepared a docker volume old-k8s-version-378000
I0717 13:37:02.686601 53894 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0717 13:37:02.686615 53894 kic.go:190] Starting extracting preloaded images to volume ...
I0717 13:37:02.686733 53894 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-378000:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.40@sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631 -I lz4 -xf /preloaded.tar -C /extractDir
I0717 13:37:05.428494 53894 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v old-k8s-version-378000:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.40@sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631 -I lz4 -xf /preloaded.tar -C /extractDir: (2.741664668s)
I0717 13:37:05.428524 53894 kic.go:199] duration metric: took 2.741899 seconds to extract preloaded images to volume
I0717 13:37:05.428648 53894 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0717 13:37:05.526232 53894 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname old-k8s-version-378000 --name old-k8s-version-378000 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=old-k8s-version-378000 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=old-k8s-version-378000 --network old-k8s-version-378000 --ip 192.168.67.2 --volume old-k8s-version-378000:/var --security-opt apparmor=unconfined --memory=2200mb --memory-swap=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.40@sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631
I0717 13:37:05.801871 53894 cli_runner.go:164] Run: docker container inspect old-k8s-version-378000 --format={{.State.Running}}
I0717 13:37:05.852936 53894 cli_runner.go:164] Run: docker container inspect old-k8s-version-378000 --format={{.State.Status}}
I0717 13:37:05.932596 53894 cli_runner.go:164] Run: docker exec old-k8s-version-378000 stat /var/lib/dpkg/alternatives/iptables
I0717 13:37:06.031184 53894 oci.go:144] the created container "old-k8s-version-378000" has a running status.
I0717 13:37:06.031226 53894 kic.go:221] Creating ssh key for kic: /Users/jenkins/minikube-integration/16890-37879/.minikube/machines/old-k8s-version-378000/id_rsa...
I0717 13:37:06.118072 53894 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/16890-37879/.minikube/machines/old-k8s-version-378000/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0717 13:37:06.177222 53894 cli_runner.go:164] Run: docker container inspect old-k8s-version-378000 --format={{.State.Status}}
I0717 13:37:06.228392 53894 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0717 13:37:06.228417 53894 kic_runner.go:114] Args: [docker exec --privileged old-k8s-version-378000 chown docker:docker /home/docker/.ssh/authorized_keys]
I0717 13:37:06.320826 53894 cli_runner.go:164] Run: docker container inspect old-k8s-version-378000 --format={{.State.Status}}
I0717 13:37:06.371998 53894 machine.go:88] provisioning docker machine ...
I0717 13:37:06.372042 53894 ubuntu.go:169] provisioning hostname "old-k8s-version-378000"
I0717 13:37:06.372138 53894 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-378000
I0717 13:37:06.421320 53894 main.go:141] libmachine: Using SSH client type: native
I0717 13:37:06.421698 53894 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140d2c0] 0x1410360 <nil> [] 0s} 127.0.0.1 59146 <nil> <nil>}
I0717 13:37:06.421714 53894 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-378000 && echo "old-k8s-version-378000" | sudo tee /etc/hostname
I0717 13:37:06.559214 53894 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-378000
I0717 13:37:06.559313 53894 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-378000
I0717 13:37:06.608682 53894 main.go:141] libmachine: Using SSH client type: native
I0717 13:37:06.609026 53894 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140d2c0] 0x1410360 <nil> [] 0s} 127.0.0.1 59146 <nil> <nil>}
I0717 13:37:06.609040 53894 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-378000' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-378000/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-378000' | sudo tee -a /etc/hosts;
fi
fi
I0717 13:37:06.737833 53894 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0717 13:37:06.737861 53894 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/16890-37879/.minikube CaCertPath:/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/16890-37879/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/16890-37879/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/16890-37879/.minikube}
I0717 13:37:06.737880 53894 ubuntu.go:177] setting up certificates
I0717 13:37:06.737885 53894 provision.go:83] configureAuth start
I0717 13:37:06.737958 53894 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-378000
I0717 13:37:06.787713 53894 provision.go:138] copyHostCerts
I0717 13:37:06.787819 53894 exec_runner.go:144] found /Users/jenkins/minikube-integration/16890-37879/.minikube/ca.pem, removing ...
I0717 13:37:06.787829 53894 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/16890-37879/.minikube/ca.pem
I0717 13:37:06.787957 53894 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/16890-37879/.minikube/ca.pem (1078 bytes)
I0717 13:37:06.788167 53894 exec_runner.go:144] found /Users/jenkins/minikube-integration/16890-37879/.minikube/cert.pem, removing ...
I0717 13:37:06.788173 53894 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/16890-37879/.minikube/cert.pem
I0717 13:37:06.788237 53894 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/16890-37879/.minikube/cert.pem (1123 bytes)
I0717 13:37:06.788391 53894 exec_runner.go:144] found /Users/jenkins/minikube-integration/16890-37879/.minikube/key.pem, removing ...
I0717 13:37:06.788397 53894 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/16890-37879/.minikube/key.pem
I0717 13:37:06.788457 53894 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/16890-37879/.minikube/key.pem (1679 bytes)
I0717 13:37:06.788584 53894 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/16890-37879/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-378000 san=[192.168.67.2 127.0.0.1 localhost 127.0.0.1 minikube old-k8s-version-378000]
I0717 13:37:06.842014 53894 provision.go:172] copyRemoteCerts
I0717 13:37:06.842077 53894 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0717 13:37:06.842131 53894 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-378000
I0717 13:37:06.892053 53894 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59146 SSHKeyPath:/Users/jenkins/minikube-integration/16890-37879/.minikube/machines/old-k8s-version-378000/id_rsa Username:docker}
I0717 13:37:06.984663 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I0717 13:37:07.006194 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0717 13:37:07.028018 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0717 13:37:07.049455 53894 provision.go:86] duration metric: configureAuth took 311.555132ms
I0717 13:37:07.049469 53894 ubuntu.go:193] setting minikube options for container-runtime
I0717 13:37:07.049629 53894 config.go:182] Loaded profile config "old-k8s-version-378000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.16.0
I0717 13:37:07.049723 53894 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-378000
I0717 13:37:07.099594 53894 main.go:141] libmachine: Using SSH client type: native
I0717 13:37:07.099958 53894 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140d2c0] 0x1410360 <nil> [] 0s} 127.0.0.1 59146 <nil> <nil>}
I0717 13:37:07.099970 53894 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0717 13:37:07.228163 53894 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0717 13:37:07.228177 53894 ubuntu.go:71] root file system type: overlay
I0717 13:37:07.228260 53894 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0717 13:37:07.228346 53894 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-378000
I0717 13:37:07.278564 53894 main.go:141] libmachine: Using SSH client type: native
I0717 13:37:07.278923 53894 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140d2c0] 0x1410360 <nil> [] 0s} 127.0.0.1 59146 <nil> <nil>}
I0717 13:37:07.278977 53894 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0717 13:37:07.418860 53894 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0717 13:37:07.418984 53894 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-378000
I0717 13:37:07.469033 53894 main.go:141] libmachine: Using SSH client type: native
I0717 13:37:07.469375 53894 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140d2c0] 0x1410360 <nil> [] 0s} 127.0.0.1 59146 <nil> <nil>}
I0717 13:37:07.469388 53894 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0717 13:37:08.119988 53894 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2023-07-07 14:50:55.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2023-07-17 20:37:07.415979828 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service time-set.target
-Wants=network-online.target containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
+Wants=network-online.target
Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutStartSec=0
-RestartSec=2
-Restart=always
+Restart=on-failure
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0717 13:37:08.120013 53894 machine.go:91] provisioned docker machine in 1.747989081s
I0717 13:37:08.120021 53894 client.go:171] LocalClient.Create took 6.302582077s
I0717 13:37:08.120040 53894 start.go:167] duration metric: libmachine.API.Create for "old-k8s-version-378000" took 6.302648795s
I0717 13:37:08.120049 53894 start.go:300] post-start starting for "old-k8s-version-378000" (driver="docker")
I0717 13:37:08.120059 53894 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0717 13:37:08.120131 53894 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0717 13:37:08.120200 53894 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-378000
I0717 13:37:08.170074 53894 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59146 SSHKeyPath:/Users/jenkins/minikube-integration/16890-37879/.minikube/machines/old-k8s-version-378000/id_rsa Username:docker}
I0717 13:37:08.263657 53894 ssh_runner.go:195] Run: cat /etc/os-release
I0717 13:37:08.267731 53894 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0717 13:37:08.267759 53894 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0717 13:37:08.267768 53894 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0717 13:37:08.267773 53894 info.go:137] Remote host: Ubuntu 22.04.2 LTS
I0717 13:37:08.267783 53894 filesync.go:126] Scanning /Users/jenkins/minikube-integration/16890-37879/.minikube/addons for local assets ...
I0717 13:37:08.267870 53894 filesync.go:126] Scanning /Users/jenkins/minikube-integration/16890-37879/.minikube/files for local assets ...
I0717 13:37:08.268055 53894 filesync.go:149] local asset: /Users/jenkins/minikube-integration/16890-37879/.minikube/files/etc/ssl/certs/383252.pem -> 383252.pem in /etc/ssl/certs
I0717 13:37:08.268242 53894 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0717 13:37:08.276839 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/files/etc/ssl/certs/383252.pem --> /etc/ssl/certs/383252.pem (1708 bytes)
I0717 13:37:08.299141 53894 start.go:303] post-start completed in 179.072701ms
I0717 13:37:08.299641 53894 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-378000
I0717 13:37:08.350540 53894 profile.go:148] Saving config to /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/config.json ...
I0717 13:37:08.350997 53894 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0717 13:37:08.351069 53894 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-378000
I0717 13:37:08.400179 53894 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59146 SSHKeyPath:/Users/jenkins/minikube-integration/16890-37879/.minikube/machines/old-k8s-version-378000/id_rsa Username:docker}
I0717 13:37:08.490053 53894 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0717 13:37:08.495719 53894 start.go:128] duration metric: createHost completed in 6.688577904s
I0717 13:37:08.495737 53894 start.go:83] releasing machines lock for "old-k8s-version-378000", held for 6.688715292s
I0717 13:37:08.495829 53894 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-378000
I0717 13:37:08.545148 53894 ssh_runner.go:195] Run: cat /version.json
I0717 13:37:08.545188 53894 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0717 13:37:08.545243 53894 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-378000
I0717 13:37:08.545263 53894 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-378000
I0717 13:37:08.596453 53894 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59146 SSHKeyPath:/Users/jenkins/minikube-integration/16890-37879/.minikube/machines/old-k8s-version-378000/id_rsa Username:docker}
I0717 13:37:08.596451 53894 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:59146 SSHKeyPath:/Users/jenkins/minikube-integration/16890-37879/.minikube/machines/old-k8s-version-378000/id_rsa Username:docker}
W0717 13:37:08.686292 53894 out.go:239] ! Image was not built for the current minikube version. To resolve this you can delete and recreate your minikube cluster using the latest images. Expected minikube version: v1.31.0 -> Actual minikube version: v1.30.1
! Image was not built for the current minikube version. To resolve this you can delete and recreate your minikube cluster using the latest images. Expected minikube version: v1.31.0 -> Actual minikube version: v1.30.1
I0717 13:37:08.686372 53894 ssh_runner.go:195] Run: systemctl --version
I0717 13:37:08.809870 53894 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0717 13:37:08.815766 53894 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0717 13:37:08.838776 53894 cni.go:236] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0717 13:37:08.838857 53894 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I0717 13:37:08.854865 53894 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I0717 13:37:08.870269 53894 cni.go:314] configured [/etc/cni/net.d/100-crio-bridge.conf, /etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0717 13:37:08.870289 53894 start.go:469] detecting cgroup driver to use...
I0717 13:37:08.870300 53894 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0717 13:37:08.870406 53894 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0717 13:37:08.885786 53894 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.1"|' /etc/containerd/config.toml"
I0717 13:37:08.896605 53894 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0717 13:37:08.906419 53894 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...
I0717 13:37:08.906489 53894 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0717 13:37:08.916787 53894 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0717 13:37:08.926460 53894 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0717 13:37:08.936442 53894 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0717 13:37:08.946125 53894 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0717 13:37:08.955704 53894 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0717 13:37:08.965523 53894 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0717 13:37:08.974123 53894 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0717 13:37:08.982612 53894 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0717 13:37:09.053791 53894 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0717 13:37:09.130460 53894 start.go:469] detecting cgroup driver to use...
I0717 13:37:09.130481 53894 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0717 13:37:09.130551 53894 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0717 13:37:09.142454 53894 cruntime.go:276] skipping containerd shutdown because we are bound to it
I0717 13:37:09.142526 53894 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0717 13:37:09.153724 53894 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0717 13:37:09.170019 53894 ssh_runner.go:195] Run: which cri-dockerd
I0717 13:37:09.175109 53894 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I0717 13:37:09.206139 53894 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)
I0717 13:37:09.223437 53894 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0717 13:37:09.315581 53894 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0717 13:37:09.403756 53894 docker.go:535] configuring docker to use "cgroupfs" as cgroup driver...
I0717 13:37:09.403771 53894 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)
I0717 13:37:09.421976 53894 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0717 13:37:09.505739 53894 ssh_runner.go:195] Run: sudo systemctl restart docker
I0717 13:37:09.774175 53894 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0717 13:37:09.801117 53894 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0717 13:37:09.904096 53894 out.go:204] * Preparing Kubernetes v1.16.0 on Docker 24.0.4 ...
I0717 13:37:09.904234 53894 cli_runner.go:164] Run: docker exec -t old-k8s-version-378000 dig +short host.docker.internal
I0717 13:37:10.018171 53894 network.go:96] got host ip for mount in container by digging dns: 192.168.65.254
I0717 13:37:10.018303 53894 ssh_runner.go:195] Run: grep 192.168.65.254 host.minikube.internal$ /etc/hosts
I0717 13:37:10.023316 53894 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.254 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0717 13:37:10.034371 53894 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" old-k8s-version-378000
I0717 13:37:10.083527 53894 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker
I0717 13:37:10.083600 53894 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0717 13:37:10.104509 53894 docker.go:636] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0717 13:37:10.104522 53894 docker.go:642] registry.k8s.io/kube-apiserver:v1.16.0 wasn't preloaded
I0717 13:37:10.104588 53894 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json
I0717 13:37:10.113749 53894 ssh_runner.go:195] Run: which lz4
I0717 13:37:10.118408 53894 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0717 13:37:10.122666 53894 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/preloaded.tar.lz4': No such file or directory
I0717 13:37:10.122698 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 --> /preloaded.tar.lz4 (369789069 bytes)
I0717 13:37:14.965329 53894 docker.go:600] Took 4.846984 seconds to copy over tarball
I0717 13:37:14.965380 53894 ssh_runner.go:195] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4
I0717 13:37:17.186995 53894 ssh_runner.go:235] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (2.221594303s)
I0717 13:37:17.187012 53894 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0717 13:37:17.235506 53894 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json
I0717 13:37:17.246085 53894 ssh_runner.go:362] scp memory --> /var/lib/docker/image/overlay2/repositories.json (2499 bytes)
I0717 13:37:17.264293 53894 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0717 13:37:17.332419 53894 ssh_runner.go:195] Run: sudo systemctl restart docker
I0717 13:37:18.092735 53894 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0717 13:37:18.113913 53894 docker.go:636] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
k8s.gcr.io/pause:3.1
-- /stdout --
I0717 13:37:18.113931 53894 docker.go:642] registry.k8s.io/kube-apiserver:v1.16.0 wasn't preloaded
I0717 13:37:18.113940 53894 cache_images.go:88] LoadImages start: [registry.k8s.io/kube-apiserver:v1.16.0 registry.k8s.io/kube-controller-manager:v1.16.0 registry.k8s.io/kube-scheduler:v1.16.0 registry.k8s.io/kube-proxy:v1.16.0 registry.k8s.io/pause:3.1 registry.k8s.io/etcd:3.3.15-0 registry.k8s.io/coredns:1.6.2 gcr.io/k8s-minikube/storage-provisioner:v5]
I0717 13:37:18.121580 53894 image.go:134] retrieving image: registry.k8s.io/etcd:3.3.15-0
I0717 13:37:18.121979 53894 image.go:134] retrieving image: registry.k8s.io/coredns:1.6.2
I0717 13:37:18.122027 53894 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.16.0
I0717 13:37:18.122128 53894 image.go:134] retrieving image: registry.k8s.io/pause:3.1
I0717 13:37:18.122161 53894 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.16.0
I0717 13:37:18.122168 53894 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0717 13:37:18.122185 53894 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.16.0
I0717 13:37:18.122188 53894 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.16.0
I0717 13:37:18.128240 53894 image.go:177] daemon lookup for registry.k8s.io/coredns:1.6.2: Error response from daemon: No such image: registry.k8s.io/coredns:1.6.2
I0717 13:37:18.128262 53894 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.16.0
I0717 13:37:18.128328 53894 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0717 13:37:18.128396 53894 image.go:177] daemon lookup for registry.k8s.io/pause:3.1: Error response from daemon: No such image: registry.k8s.io/pause:3.1
I0717 13:37:18.128480 53894 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.16.0
I0717 13:37:18.128530 53894 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.16.0
I0717 13:37:18.128557 53894 image.go:177] daemon lookup for registry.k8s.io/etcd:3.3.15-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.3.15-0
I0717 13:37:18.128580 53894 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.16.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.16.0
I0717 13:37:19.288843 53894 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.16.0
I0717 13:37:19.313207 53894 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.16.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.16.0" does not exist at hash "301ddc62b80b16315d3c2653cf3888370394277afb3187614cfa20edc352ca0a" in container runtime
I0717 13:37:19.313270 53894 docker.go:316] Removing image: registry.k8s.io/kube-scheduler:v1.16.0
I0717 13:37:19.313361 53894 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-scheduler:v1.16.0
I0717 13:37:19.336507 53894 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.16.0
I0717 13:37:19.474564 53894 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/coredns:1.6.2
I0717 13:37:19.496596 53894 cache_images.go:116] "registry.k8s.io/coredns:1.6.2" needs transfer: "registry.k8s.io/coredns:1.6.2" does not exist at hash "bf261d157914477ee1a5969d28ec687f3fbfc9fb5a664b22df78e57023b0e03b" in container runtime
I0717 13:37:19.496623 53894 docker.go:316] Removing image: registry.k8s.io/coredns:1.6.2
I0717 13:37:19.496672 53894 ssh_runner.go:195] Run: docker rmi registry.k8s.io/coredns:1.6.2
I0717 13:37:19.520546 53894 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.6.2
I0717 13:37:19.817749 53894 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/pause:3.1
I0717 13:37:19.836889 53894 cache_images.go:116] "registry.k8s.io/pause:3.1" needs transfer: "registry.k8s.io/pause:3.1" does not exist at hash "da86e6ba6ca197bf6bc5e9d900febd906b133eaa4750e6bed647b0fbe50ed43e" in container runtime
I0717 13:37:19.836918 53894 docker.go:316] Removing image: registry.k8s.io/pause:3.1
I0717 13:37:19.836967 53894 ssh_runner.go:195] Run: docker rmi registry.k8s.io/pause:3.1
I0717 13:37:19.858448 53894 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/images/amd64/registry.k8s.io/pause_3.1
I0717 13:37:19.958772 53894 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5
I0717 13:37:20.066267 53894 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.16.0
I0717 13:37:20.091794 53894 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.16.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.16.0" does not exist at hash "b305571ca60a5a7818bda47da122683d75e8a1907475681ee8b1efbd06bff12e" in container runtime
I0717 13:37:20.091832 53894 docker.go:316] Removing image: registry.k8s.io/kube-apiserver:v1.16.0
I0717 13:37:20.091920 53894 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-apiserver:v1.16.0
I0717 13:37:20.116002 53894 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.16.0
I0717 13:37:20.315323 53894 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.16.0
I0717 13:37:20.338199 53894 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.16.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.16.0" does not exist at hash "06a629a7e51cdcc81a5ed6a3e6650348312f20c954ac52ee489a023628ec9c7d" in container runtime
I0717 13:37:20.338231 53894 docker.go:316] Removing image: registry.k8s.io/kube-controller-manager:v1.16.0
I0717 13:37:20.338290 53894 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-controller-manager:v1.16.0
I0717 13:37:20.361802 53894 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.16.0
I0717 13:37:20.640880 53894 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/etcd:3.3.15-0
I0717 13:37:20.662137 53894 cache_images.go:116] "registry.k8s.io/etcd:3.3.15-0" needs transfer: "registry.k8s.io/etcd:3.3.15-0" does not exist at hash "b2756210eeabf84f3221da9959e9483f3919dc2aaab4cd45e7cd072fcbde27ed" in container runtime
I0717 13:37:20.662163 53894 docker.go:316] Removing image: registry.k8s.io/etcd:3.3.15-0
I0717 13:37:20.662204 53894 ssh_runner.go:195] Run: docker rmi registry.k8s.io/etcd:3.3.15-0
I0717 13:37:20.682731 53894 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.3.15-0
I0717 13:37:20.974511 53894 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.16.0
I0717 13:37:20.997185 53894 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.16.0" needs transfer: "registry.k8s.io/kube-proxy:v1.16.0" does not exist at hash "c21b0c7400f988db4777858edd13b6d3930d62d7ccf026d2415485a52037f384" in container runtime
I0717 13:37:20.997213 53894 docker.go:316] Removing image: registry.k8s.io/kube-proxy:v1.16.0
I0717 13:37:20.997271 53894 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-proxy:v1.16.0
I0717 13:37:21.016263 53894 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.16.0
I0717 13:37:21.016314 53894 cache_images.go:92] LoadImages completed in 2.902355024s
W0717 13:37:21.016364 53894 out.go:239] X Unable to load cached images: loading cached images: stat /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.16.0: no such file or directory
X Unable to load cached images: loading cached images: stat /Users/jenkins/minikube-integration/16890-37879/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.16.0: no such file or directory
I0717 13:37:21.016447 53894 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0717 13:37:21.071743 53894 cni.go:84] Creating CNI manager for ""
I0717 13:37:21.071778 53894 cni.go:168] CNI unnecessary in this configuration, recommending no CNI
I0717 13:37:21.071802 53894 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0717 13:37:21.071819 53894 kubeadm.go:176] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.67.2 APIServerPort:8443 KubernetesVersion:v1.16.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-378000 NodeName:old-k8s-version-378000 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.67.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.67.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticP
odPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0717 13:37:21.071916 53894 kubeadm.go:181] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.67.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-378000"
kubeletExtraArgs:
node-ip: 192.168.67.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.67.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: old-k8s-version-378000
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
listen-metrics-urls: http://127.0.0.1:2381,http://192.168.67.2:2381
kubernetesVersion: v1.16.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0717 13:37:21.071980 53894 kubeadm.go:976] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.16.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-378000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.67.2
[Install]
config:
{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-378000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0717 13:37:21.072064 53894 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.16.0
I0717 13:37:21.081728 53894 binaries.go:44] Found k8s binaries, skipping transfer
I0717 13:37:21.081802 53894 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0717 13:37:21.091603 53894 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I0717 13:37:21.109940 53894 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0717 13:37:21.126615 53894 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2174 bytes)
I0717 13:37:21.143298 53894 ssh_runner.go:195] Run: grep 192.168.67.2 control-plane.minikube.internal$ /etc/hosts
I0717 13:37:21.147879 53894 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.67.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0717 13:37:21.158872 53894 certs.go:56] Setting up /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000 for IP: 192.168.67.2
I0717 13:37:21.158891 53894 certs.go:190] acquiring lock for shared ca certs: {Name:mkcb761e9710dc67a00cbdee9d78e096db7e9bb9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0717 13:37:21.159076 53894 certs.go:199] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/16890-37879/.minikube/ca.key
I0717 13:37:21.159136 53894 certs.go:199] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/16890-37879/.minikube/proxy-client-ca.key
I0717 13:37:21.159185 53894 certs.go:319] generating minikube-user signed cert: /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/client.key
I0717 13:37:21.159197 53894 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/client.crt with IP's: []
I0717 13:37:21.475547 53894 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/client.crt ...
I0717 13:37:21.475568 53894 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/client.crt: {Name:mk127283dbd0c2f1f3370862de35894d692e7a46 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0717 13:37:21.475938 53894 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/client.key ...
I0717 13:37:21.475948 53894 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/client.key: {Name:mk25f59a3d3f2e34cdf61f8653f0ae64e59deceb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0717 13:37:21.476187 53894 certs.go:319] generating minikube signed cert: /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.key.c7fa3a9e
I0717 13:37:21.476208 53894 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.crt.c7fa3a9e with IP's: [192.168.67.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0717 13:37:21.551900 53894 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.crt.c7fa3a9e ...
I0717 13:37:21.551913 53894 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.crt.c7fa3a9e: {Name:mk698d28ea167d8dd079ab5c0e3a196882512ea5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0717 13:37:21.552214 53894 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.key.c7fa3a9e ...
I0717 13:37:21.552222 53894 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.key.c7fa3a9e: {Name:mk8f93d0efc5e23ced6a8dbe10a5080173627e8d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0717 13:37:21.552410 53894 certs.go:337] copying /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.crt.c7fa3a9e -> /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.crt
I0717 13:37:21.552573 53894 certs.go:341] copying /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.key.c7fa3a9e -> /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.key
I0717 13:37:21.552733 53894 certs.go:319] generating aggregator signed cert: /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/proxy-client.key
I0717 13:37:21.552746 53894 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/proxy-client.crt with IP's: []
I0717 13:37:21.669087 53894 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/proxy-client.crt ...
I0717 13:37:21.669100 53894 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/proxy-client.crt: {Name:mk5767ef62185dd14948b457a6b5c56b8a1b97aa Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0717 13:37:21.669408 53894 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/proxy-client.key ...
I0717 13:37:21.669416 53894 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/proxy-client.key: {Name:mkbdb4b6e12da121e0c2d626d9fdb3a160d44d25 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0717 13:37:21.669844 53894 certs.go:437] found cert: /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/38325.pem (1338 bytes)
W0717 13:37:21.669893 53894 certs.go:433] ignoring /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/38325_empty.pem, impossibly tiny 0 bytes
I0717 13:37:21.669906 53894 certs.go:437] found cert: /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/ca-key.pem (1679 bytes)
I0717 13:37:21.669944 53894 certs.go:437] found cert: /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/ca.pem (1078 bytes)
I0717 13:37:21.669974 53894 certs.go:437] found cert: /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/cert.pem (1123 bytes)
I0717 13:37:21.670004 53894 certs.go:437] found cert: /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/Users/jenkins/minikube-integration/16890-37879/.minikube/certs/key.pem (1679 bytes)
I0717 13:37:21.670074 53894 certs.go:437] found cert: /Users/jenkins/minikube-integration/16890-37879/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/16890-37879/.minikube/files/etc/ssl/certs/383252.pem (1708 bytes)
I0717 13:37:21.670590 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0717 13:37:21.694298 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0717 13:37:21.717389 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0717 13:37:21.741683 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/profiles/old-k8s-version-378000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0717 13:37:21.764672 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0717 13:37:21.788650 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0717 13:37:21.812014 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0717 13:37:21.836102 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0717 13:37:21.859535 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0717 13:37:21.885256 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/certs/38325.pem --> /usr/share/ca-certificates/38325.pem (1338 bytes)
I0717 13:37:21.909483 53894 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16890-37879/.minikube/files/etc/ssl/certs/383252.pem --> /usr/share/ca-certificates/383252.pem (1708 bytes)
I0717 13:37:21.931765 53894 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0717 13:37:21.947973 53894 ssh_runner.go:195] Run: openssl version
I0717 13:37:21.953903 53894 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/383252.pem && ln -fs /usr/share/ca-certificates/383252.pem /etc/ssl/certs/383252.pem"
I0717 13:37:21.963849 53894 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/383252.pem
I0717 13:37:21.968793 53894 certs.go:480] hashing: -rw-r--r-- 1 root root 1708 Jul 17 19:49 /usr/share/ca-certificates/383252.pem
I0717 13:37:21.968853 53894 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/383252.pem
I0717 13:37:21.976271 53894 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/383252.pem /etc/ssl/certs/3ec20f2e.0"
I0717 13:37:21.986707 53894 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0717 13:37:21.996705 53894 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0717 13:37:22.001426 53894 certs.go:480] hashing: -rw-r--r-- 1 root root 1111 Jul 17 19:44 /usr/share/ca-certificates/minikubeCA.pem
I0717 13:37:22.001472 53894 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0717 13:37:22.008679 53894 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0717 13:37:22.019498 53894 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/38325.pem && ln -fs /usr/share/ca-certificates/38325.pem /etc/ssl/certs/38325.pem"
I0717 13:37:22.029942 53894 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/38325.pem
I0717 13:37:22.034976 53894 certs.go:480] hashing: -rw-r--r-- 1 root root 1338 Jul 17 19:49 /usr/share/ca-certificates/38325.pem
I0717 13:37:22.035043 53894 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/38325.pem
I0717 13:37:22.043196 53894 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/38325.pem /etc/ssl/certs/51391683.0"
I0717 13:37:22.053406 53894 ssh_runner.go:195] Run: ls /var/lib/minikube/certs/etcd
I0717 13:37:22.058047 53894 certs.go:353] certs directory doesn't exist, likely first start: ls /var/lib/minikube/certs/etcd: Process exited with status 2
stdout:
stderr:
ls: cannot access '/var/lib/minikube/certs/etcd': No such file or directory
I0717 13:37:22.058091 53894 kubeadm.go:404] StartCluster: {Name:old-k8s-version-378000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.40@sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:old-k8s-version-378000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs
:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.67.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQe
muFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0}
I0717 13:37:22.058180 53894 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0717 13:37:22.078164 53894 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0717 13:37:22.087819 53894 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0717 13:37:22.097237 53894 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0717 13:37:22.097299 53894 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0717 13:37:22.107680 53894 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0717 13:37:22.107713 53894 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0717 13:37:22.163055 53894 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0717 13:37:22.163118 53894 kubeadm.go:322] [preflight] Running pre-flight checks
I0717 13:37:22.427585 53894 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0717 13:37:22.427683 53894 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0717 13:37:22.427777 53894 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0717 13:37:22.620190 53894 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0717 13:37:22.620978 53894 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0717 13:37:22.628338 53894 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0717 13:37:22.695569 53894 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0717 13:37:22.753462 53894 out.go:204] - Generating certificates and keys ...
I0717 13:37:22.753581 53894 kubeadm.go:322] [certs] Using existing ca certificate authority
I0717 13:37:22.753710 53894 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0717 13:37:23.004716 53894 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key
I0717 13:37:23.051824 53894 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key
I0717 13:37:23.309527 53894 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key
I0717 13:37:23.497744 53894 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key
I0717 13:37:23.616543 53894 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key
I0717 13:37:23.616666 53894 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [old-k8s-version-378000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
I0717 13:37:23.722323 53894 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key
I0717 13:37:23.722427 53894 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-378000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
I0717 13:37:23.800425 53894 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key
I0717 13:37:24.225050 53894 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key
I0717 13:37:24.460381 53894 kubeadm.go:322] [certs] Generating "sa" key and public key
I0717 13:37:24.460462 53894 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0717 13:37:24.504182 53894 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0717 13:37:24.557199 53894 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0717 13:37:24.668349 53894 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0717 13:37:24.809826 53894 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0717 13:37:24.811070 53894 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0717 13:37:24.832499 53894 out.go:204] - Booting up control plane ...
I0717 13:37:24.832635 53894 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0717 13:37:24.832768 53894 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0717 13:37:24.832845 53894 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0717 13:37:24.832949 53894 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0717 13:37:24.833141 53894 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0717 13:38:04.821613 53894 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0717 13:38:04.822469 53894 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0717 13:38:04.822718 53894 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0717 13:38:09.823407 53894 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0717 13:38:09.823615 53894 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0717 13:38:19.825201 53894 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0717 13:38:19.825468 53894 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0717 13:38:39.826023 53894 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0717 13:38:39.826236 53894 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0717 13:39:19.826861 53894 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0717 13:39:19.827034 53894 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0717 13:39:19.827048 53894 kubeadm.go:322]
I0717 13:39:19.827110 53894 kubeadm.go:322] Unfortunately, an error has occurred:
I0717 13:39:19.827176 53894 kubeadm.go:322] timed out waiting for the condition
I0717 13:39:19.827187 53894 kubeadm.go:322]
I0717 13:39:19.827226 53894 kubeadm.go:322] This error is likely caused by:
I0717 13:39:19.827256 53894 kubeadm.go:322] - The kubelet is not running
I0717 13:39:19.827337 53894 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0717 13:39:19.827346 53894 kubeadm.go:322]
I0717 13:39:19.827423 53894 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0717 13:39:19.827456 53894 kubeadm.go:322] - 'systemctl status kubelet'
I0717 13:39:19.827490 53894 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0717 13:39:19.827496 53894 kubeadm.go:322]
I0717 13:39:19.827605 53894 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0717 13:39:19.827691 53894 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0717 13:39:19.827756 53894 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0717 13:39:19.827790 53894 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0717 13:39:19.827846 53894 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0717 13:39:19.827877 53894 kubeadm.go:322] - 'docker logs CONTAINERID'
I0717 13:39:19.830169 53894 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0717 13:39:19.830279 53894 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0717 13:39:19.830392 53894 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.4. Latest validated version: 18.09
I0717 13:39:19.830478 53894 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0717 13:39:19.830554 53894 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0717 13:39:19.830620 53894 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
W0717 13:39:19.830703 53894 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-378000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-378000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.4. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [old-k8s-version-378000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [old-k8s-version-378000 localhost] and IPs [192.168.67.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.4. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0717 13:39:19.830736 53894 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0717 13:39:20.251446 53894 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0717 13:39:20.263905 53894 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver
I0717 13:39:20.263956 53894 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0717 13:39:20.273111 53894 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0717 13:39:20.273140 53894 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0717 13:39:20.324040 53894 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0
I0717 13:39:20.324089 53894 kubeadm.go:322] [preflight] Running pre-flight checks
I0717 13:39:20.581703 53894 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster
I0717 13:39:20.581815 53894 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0717 13:39:20.581897 53894 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0717 13:39:20.768854 53894 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0717 13:39:20.769867 53894 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0717 13:39:20.776554 53894 kubeadm.go:322] [kubelet-start] Activating the kubelet service
I0717 13:39:20.841580 53894 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0717 13:39:20.863177 53894 out.go:204] - Generating certificates and keys ...
I0717 13:39:20.863314 53894 kubeadm.go:322] [certs] Using existing ca certificate authority
I0717 13:39:20.863517 53894 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk
I0717 13:39:20.863666 53894 kubeadm.go:322] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0717 13:39:20.863755 53894 kubeadm.go:322] [certs] Using existing front-proxy-ca certificate authority
I0717 13:39:20.863864 53894 kubeadm.go:322] [certs] Using existing front-proxy-client certificate and key on disk
I0717 13:39:20.863920 53894 kubeadm.go:322] [certs] Using existing etcd/ca certificate authority
I0717 13:39:20.863995 53894 kubeadm.go:322] [certs] Using existing etcd/server certificate and key on disk
I0717 13:39:20.864102 53894 kubeadm.go:322] [certs] Using existing etcd/peer certificate and key on disk
I0717 13:39:20.864179 53894 kubeadm.go:322] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0717 13:39:20.864308 53894 kubeadm.go:322] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0717 13:39:20.864359 53894 kubeadm.go:322] [certs] Using the existing "sa" key
I0717 13:39:20.864445 53894 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0717 13:39:21.204051 53894 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file
I0717 13:39:21.620596 53894 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0717 13:39:21.816709 53894 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0717 13:39:22.030990 53894 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0717 13:39:22.031558 53894 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0717 13:39:22.052118 53894 out.go:204] - Booting up control plane ...
I0717 13:39:22.052231 53894 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0717 13:39:22.052341 53894 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0717 13:39:22.052431 53894 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0717 13:39:22.052522 53894 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0717 13:39:22.052704 53894 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0717 13:40:02.048704 53894 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.
I0717 13:40:02.049692 53894 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0717 13:40:02.049886 53894 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0717 13:40:07.052773 53894 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0717 13:40:07.053006 53894 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0717 13:40:17.056560 53894 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0717 13:40:17.056779 53894 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0717 13:40:37.058830 53894 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0717 13:40:37.059043 53894 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0717 13:41:17.061780 53894 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0717 13:41:17.062000 53894 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
I0717 13:41:17.062019 53894 kubeadm.go:322]
I0717 13:41:17.062071 53894 kubeadm.go:322] Unfortunately, an error has occurred:
I0717 13:41:17.062128 53894 kubeadm.go:322] timed out waiting for the condition
I0717 13:41:17.062142 53894 kubeadm.go:322]
I0717 13:41:17.062190 53894 kubeadm.go:322] This error is likely caused by:
I0717 13:41:17.062229 53894 kubeadm.go:322] - The kubelet is not running
I0717 13:41:17.062368 53894 kubeadm.go:322] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0717 13:41:17.062378 53894 kubeadm.go:322]
I0717 13:41:17.062525 53894 kubeadm.go:322] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0717 13:41:17.062564 53894 kubeadm.go:322] - 'systemctl status kubelet'
I0717 13:41:17.062599 53894 kubeadm.go:322] - 'journalctl -xeu kubelet'
I0717 13:41:17.062605 53894 kubeadm.go:322]
I0717 13:41:17.062735 53894 kubeadm.go:322] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0717 13:41:17.062882 53894 kubeadm.go:322] To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
I0717 13:41:17.062970 53894 kubeadm.go:322] Here is one example how you may list all Kubernetes containers running in docker:
I0717 13:41:17.063014 53894 kubeadm.go:322] - 'docker ps -a | grep kube | grep -v pause'
I0717 13:41:17.063080 53894 kubeadm.go:322] Once you have found the failing container, you can inspect its logs with:
I0717 13:41:17.063108 53894 kubeadm.go:322] - 'docker logs CONTAINERID'
I0717 13:41:17.064739 53894 kubeadm.go:322] [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0717 13:41:17.064809 53894 kubeadm.go:322] [WARNING Swap]: running with swap on is not supported. Please disable swap
I0717 13:41:17.064934 53894 kubeadm.go:322] [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.4. Latest validated version: 18.09
I0717 13:41:17.065052 53894 kubeadm.go:322] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0717 13:41:17.065129 53894 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0717 13:41:17.065190 53894 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher
I0717 13:41:17.065225 53894 kubeadm.go:406] StartCluster complete in 3m54.991440132s
I0717 13:41:17.065323 53894 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0717 13:41:17.085598 53894 logs.go:284] 0 containers: []
W0717 13:41:17.085611 53894 logs.go:286] No container was found matching "kube-apiserver"
I0717 13:41:17.085684 53894 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0717 13:41:17.104591 53894 logs.go:284] 0 containers: []
W0717 13:41:17.104605 53894 logs.go:286] No container was found matching "etcd"
I0717 13:41:17.104676 53894 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0717 13:41:17.123359 53894 logs.go:284] 0 containers: []
W0717 13:41:17.123373 53894 logs.go:286] No container was found matching "coredns"
I0717 13:41:17.123447 53894 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0717 13:41:17.143673 53894 logs.go:284] 0 containers: []
W0717 13:41:17.143685 53894 logs.go:286] No container was found matching "kube-scheduler"
I0717 13:41:17.143767 53894 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0717 13:41:17.164317 53894 logs.go:284] 0 containers: []
W0717 13:41:17.164331 53894 logs.go:286] No container was found matching "kube-proxy"
I0717 13:41:17.164406 53894 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0717 13:41:17.183348 53894 logs.go:284] 0 containers: []
W0717 13:41:17.183361 53894 logs.go:286] No container was found matching "kube-controller-manager"
I0717 13:41:17.183436 53894 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0717 13:41:17.202224 53894 logs.go:284] 0 containers: []
W0717 13:41:17.202239 53894 logs.go:286] No container was found matching "kindnet"
I0717 13:41:17.202246 53894 logs.go:123] Gathering logs for kubelet ...
I0717 13:41:17.202254 53894 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0717 13:41:17.240713 53894 logs.go:123] Gathering logs for dmesg ...
I0717 13:41:17.240731 53894 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0717 13:41:17.255113 53894 logs.go:123] Gathering logs for describe nodes ...
I0717 13:41:17.255128 53894 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0717 13:41:17.310359 53894 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0717 13:41:17.310373 53894 logs.go:123] Gathering logs for Docker ...
I0717 13:41:17.310379 53894 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"
I0717 13:41:17.326951 53894 logs.go:123] Gathering logs for container status ...
I0717 13:41:17.326967 53894 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
W0717 13:41:17.379061 53894 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.4. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0717 13:41:17.379079 53894 out.go:239] *
*
W0717 13:41:17.379153 53894 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.4. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.4. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0717 13:41:17.379182 53894 out.go:239] *
*
W0717 13:41:17.379867 53894 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0717 13:41:17.457919 53894 out.go:177]
W0717 13:41:17.499942 53894 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.4. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.16.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
stderr:
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[WARNING Swap]: running with swap on is not supported. Please disable swap
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.4. Latest validated version: 18.09
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0717 13:41:17.500005 53894 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0717 13:41:17.500027 53894 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0717 13:41:17.541918 53894 out.go:177]
** /stderr **
start_stop_delete_test.go:188: failed starting minikube -first start-. args "out/minikube-darwin-amd64 start -p old-k8s-version-378000 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --kubernetes-version=v1.16.0": exit status 109
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/FirstStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-378000
helpers_test.go:235: (dbg) docker inspect old-k8s-version-378000:
-- stdout --
[
{
"Id": "b5cf72528f711ac4193d0a9a8f59a539b1c9a82e56b492c4504ac5a55d9b9666",
"Created": "2023-07-17T20:37:05.574347632Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 718537,
"ExitCode": 0,
"Error": "",
"StartedAt": "2023-07-17T20:37:05.794684114Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:c6cc01e6091959400f260dc442708e7c71630b58dab1f7c344cb00926bd84950",
"ResolvConfPath": "/var/lib/docker/containers/b5cf72528f711ac4193d0a9a8f59a539b1c9a82e56b492c4504ac5a55d9b9666/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/b5cf72528f711ac4193d0a9a8f59a539b1c9a82e56b492c4504ac5a55d9b9666/hostname",
"HostsPath": "/var/lib/docker/containers/b5cf72528f711ac4193d0a9a8f59a539b1c9a82e56b492c4504ac5a55d9b9666/hosts",
"LogPath": "/var/lib/docker/containers/b5cf72528f711ac4193d0a9a8f59a539b1c9a82e56b492c4504ac5a55d9b9666/b5cf72528f711ac4193d0a9a8f59a539b1c9a82e56b492c4504ac5a55d9b9666-json.log",
"Name": "/old-k8s-version-378000",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-378000:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-378000",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "0"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 2306867200,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/92e903a37111c1be0b41a42b0b482279b759da84e66b3f0a99d79bad046a816b-init/diff:/var/lib/docker/overlay2/e56ac82b253363a3e2a8ef1d32b035837a0160e70c091e0204df14a88b273cb0/diff",
"MergedDir": "/var/lib/docker/overlay2/92e903a37111c1be0b41a42b0b482279b759da84e66b3f0a99d79bad046a816b/merged",
"UpperDir": "/var/lib/docker/overlay2/92e903a37111c1be0b41a42b0b482279b759da84e66b3f0a99d79bad046a816b/diff",
"WorkDir": "/var/lib/docker/overlay2/92e903a37111c1be0b41a42b0b482279b759da84e66b3f0a99d79bad046a816b/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-378000",
"Source": "/var/lib/docker/volumes/old-k8s-version-378000/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-378000",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase:v0.0.40@sha256:8cadf23777709e43eca447c47a45f5a4635615129267ce025193040ec92a1631",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-378000",
"name.minikube.sigs.k8s.io": "old-k8s-version-378000",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "c95dcad38d994f0eefc35e3377946f33ace9c263664067236394916b531fb3c1",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59146"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59147"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59148"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59149"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "59150"
}
]
},
"SandboxKey": "/var/run/docker/netns/c95dcad38d99",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-378000": {
"IPAMConfig": {
"IPv4Address": "192.168.67.2"
},
"Links": null,
"Aliases": [
"b5cf72528f71",
"old-k8s-version-378000"
],
"NetworkID": "c3d985d4d6f8171a299a582295ee1a9b4b599d36307c61b13f7920634885fa85",
"EndpointID": "c253e59524ffcac002d9239041f417095609bffd96e4ac16dab03e933a4af6a9",
"Gateway": "192.168.67.1",
"IPAddress": "192.168.67.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:43:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-378000 -n old-k8s-version-378000
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-378000 -n old-k8s-version-378000: exit status 6 (355.918478ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E0717 13:41:18.042187 54988 status.go:415] kubeconfig endpoint: extract IP: "old-k8s-version-378000" does not appear in /Users/jenkins/minikube-integration/16890-37879/kubeconfig
** /stderr **
helpers_test.go:239: status error: exit status 6 (may be ok)
helpers_test.go:241: "old-k8s-version-378000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")
--- FAIL: TestStartStop/group/old-k8s-version/serial/FirstStart (257.19s)