=== RUN   TestOffline

=== PAUSE TestOffline

=== CONT  TestOffline

aab_offline_test.go:55: (dbg) Run:  out/minikube-darwin-amd64 start -p offline-docker-301000 --alsologtostderr -v=1 --memory=2048 --wait=true --driver=docker 

aab_offline_test.go:55: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p offline-docker-301000 --alsologtostderr -v=1 --memory=2048 --wait=true --driver=docker : exit status 52 (12m35.94512641s)

-- stdout --

	* [offline-docker-301000] minikube v1.32.0 on Darwin 14.2.1

	  - MINIKUBE_LOCATION=17953

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the docker driver based on user configuration

	* Using Docker Desktop driver with root privileges

	* Starting control plane node offline-docker-301000 in cluster offline-docker-301000

	* Pulling base image v0.0.42-1704759386-17866 ...

	* Creating docker container (CPUs=2, Memory=2048MB) ...

	* docker "offline-docker-301000" container is missing, will recreate.

	* Creating docker container (CPUs=2, Memory=2048MB) ...

-- /stdout --

** stderr ** 

	I0115 06:12:22.784247   72225 out.go:296] Setting OutFile to fd 1 ...

	I0115 06:12:22.784596   72225 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 06:12:22.784604   72225 out.go:309] Setting ErrFile to fd 2...

	I0115 06:12:22.784608   72225 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 06:12:22.784790   72225 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 06:12:22.786398   72225 out.go:303] Setting JSON to false

	I0115 06:12:22.810025   72225 start.go:128] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":34685,"bootTime":1705293257,"procs":506,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.2.1","kernelVersion":"23.2.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}

	W0115 06:12:22.810129   72225 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0115 06:12:22.831944   72225 out.go:177] * [offline-docker-301000] minikube v1.32.0 on Darwin 14.2.1

	I0115 06:12:22.873384   72225 out.go:177]   - MINIKUBE_LOCATION=17953

	I0115 06:12:22.873466   72225 notify.go:220] Checking for updates...

	I0115 06:12:22.915470   72225 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	I0115 06:12:22.936305   72225 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-amd64

	I0115 06:12:22.957534   72225 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0115 06:12:22.978507   72225 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	I0115 06:12:22.999299   72225 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0115 06:12:23.020743   72225 driver.go:392] Setting default libvirt URI to qemu:///system

	I0115 06:12:23.077379   72225 docker.go:122] docker version: linux-24.0.7:Docker Desktop 4.26.0 (130397)

	I0115 06:12:23.077578   72225 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0115 06:12:23.241235   72225 info.go:266] docker info: {ID:37b3081a-8cd6-457e-b2a4-79dc82345b06 Containers:8 ContainersRunning:0 ContainersPaused:0 ContainersStopped:8 Images:2 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:65 OomKillDisable:false NGoroutines:143 SystemTime:2024-01-15 14:12:23.062077828 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:6.5.11-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ

erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6221279232 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f Expected:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f} RuncCommit:{ID:v1.1.10-0-g18a0cb0 Expected:v1.1.10-0-g18a0cb0} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=uncon

fined name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.0-desktop.2] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.23.3-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Man

ages Docker extensions Vendor:Docker Inc. Version:v0.2.21] map[Name:feedback Path:/Users/jenkins/.docker/cli-plugins/docker-feedback SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-feedback] ShortDescription:Provide feedback, right in your terminal! Vendor:Docker Inc. Version:0.1] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.10] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/

docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Scout Vendor:Docker Inc. Version:v1.2.0]] Warnings:<nil>}}

	I0115 06:12:23.262871   72225 out.go:177] * Using the docker driver based on user configuration

	I0115 06:12:23.284034   72225 start.go:298] selected driver: docker

	I0115 06:12:23.284053   72225 start.go:902] validating driver "docker" against <nil>

	I0115 06:12:23.284066   72225 start.go:913] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0115 06:12:23.287000   72225 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0115 06:12:23.387762   72225 info.go:266] docker info: {ID:37b3081a-8cd6-457e-b2a4-79dc82345b06 Containers:8 ContainersRunning:0 ContainersPaused:0 ContainersStopped:8 Images:2 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:65 OomKillDisable:false NGoroutines:143 SystemTime:2024-01-15 14:12:23.240732488 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:6.5.11-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ

erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6221279232 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f Expected:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f} RuncCommit:{ID:v1.1.10-0-g18a0cb0 Expected:v1.1.10-0-g18a0cb0} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=uncon

fined name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.0-desktop.2] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.23.3-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Man

ages Docker extensions Vendor:Docker Inc. Version:v0.2.21] map[Name:feedback Path:/Users/jenkins/.docker/cli-plugins/docker-feedback SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-feedback] ShortDescription:Provide feedback, right in your terminal! Vendor:Docker Inc. Version:0.1] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.10] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/

docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Scout Vendor:Docker Inc. Version:v1.2.0]] Warnings:<nil>}}

	I0115 06:12:23.387914   72225 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0115 06:12:23.388105   72225 start_flags.go:927] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0115 06:12:23.409062   72225 out.go:177] * Using Docker Desktop driver with root privileges

	I0115 06:12:23.430087   72225 cni.go:84] Creating CNI manager for ""

	I0115 06:12:23.430131   72225 cni.go:158] "docker" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0115 06:12:23.430150   72225 start_flags.go:316] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0115 06:12:23.430180   72225 start_flags.go:321] config:

	{Name:offline-docker-301000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:offline-docker-301000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Container

Runtime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs:}

	I0115 06:12:23.473157   72225 out.go:177] * Starting control plane node offline-docker-301000 in cluster offline-docker-301000

	I0115 06:12:23.515113   72225 cache.go:121] Beginning downloading kic base image for docker with docker

	I0115 06:12:23.579093   72225 out.go:177] * Pulling base image v0.0.42-1704759386-17866 ...

	I0115 06:12:23.620716   72225 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0115 06:12:23.620761   72225 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4

	I0115 06:12:23.620771   72225 cache.go:56] Caching tarball of preloaded images

	I0115 06:12:23.620788   72225 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 in local docker daemon

	I0115 06:12:23.620903   72225 preload.go:174] Found /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4 in cache, skipping download

	I0115 06:12:23.620913   72225 cache.go:59] Finished verifying existence of preloaded tar for  v1.28.4 on docker

	I0115 06:12:23.621787   72225 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/offline-docker-301000/config.json ...

	I0115 06:12:23.621866   72225 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/offline-docker-301000/config.json: {Name:mkc6bf2c25fc53bb045b56218ed4b6f05326b9c2 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0115 06:12:23.672931   72225 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 in local docker daemon, skipping pull

	I0115 06:12:23.673152   72225 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 exists in daemon, skipping load

	I0115 06:12:23.673170   72225 cache.go:194] Successfully downloaded all kic artifacts

	I0115 06:12:23.673212   72225 start.go:365] acquiring machines lock for offline-docker-301000: {Name:mk777770c680d16be68699b00e5578abf6398a0c Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0115 06:12:23.673383   72225 start.go:369] acquired machines lock for "offline-docker-301000" in 159.978µs

	I0115 06:12:23.673409   72225 start.go:93] Provisioning new machine with config: &{Name:offline-docker-301000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:offline-docker-301000 Namespace:default APIServerN

ame:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations

:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs:} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0115 06:12:23.673484   72225 start.go:125] createHost starting for "" (driver="docker")

	I0115 06:12:23.694888   72225 out.go:204] * Creating docker container (CPUs=2, Memory=2048MB) ...

	I0115 06:12:23.695066   72225 start.go:159] libmachine.API.Create for "offline-docker-301000" (driver="docker")

	I0115 06:12:23.695096   72225 client.go:168] LocalClient.Create starting

	I0115 06:12:23.695228   72225 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem

	I0115 06:12:23.695273   72225 main.go:141] libmachine: Decoding PEM data...

	I0115 06:12:23.695296   72225 main.go:141] libmachine: Parsing certificate...

	I0115 06:12:23.695372   72225 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem

	I0115 06:12:23.695407   72225 main.go:141] libmachine: Decoding PEM data...

	I0115 06:12:23.695416   72225 main.go:141] libmachine: Parsing certificate...

	I0115 06:12:23.695927   72225 cli_runner.go:164] Run: docker network inspect offline-docker-301000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	W0115 06:12:23.767283   72225 cli_runner.go:211] docker network inspect offline-docker-301000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1

	I0115 06:12:23.767380   72225 network_create.go:281] running [docker network inspect offline-docker-301000] to gather additional debugging logs...

	I0115 06:12:23.767397   72225 cli_runner.go:164] Run: docker network inspect offline-docker-301000

	W0115 06:12:23.818944   72225 cli_runner.go:211] docker network inspect offline-docker-301000 returned with exit code 1

	I0115 06:12:23.818971   72225 network_create.go:284] error running [docker network inspect offline-docker-301000]: docker network inspect offline-docker-301000: exit status 1

	stdout:

	[]

	stderr:

	Error response from daemon: network offline-docker-301000 not found

	I0115 06:12:23.818984   72225 network_create.go:286] output of [docker network inspect offline-docker-301000]: -- stdout --

	[]

	-- /stdout --

	** stderr ** 

	Error response from daemon: network offline-docker-301000 not found

	** /stderr **

	I0115 06:12:23.819103   72225 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 06:12:23.893799   72225 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}

	I0115 06:12:23.894171   72225 network.go:209] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0022bc3b0}

	I0115 06:12:23.894188   72225 network_create.go:124] attempt to create docker network offline-docker-301000 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 65535 ...

	I0115 06:12:23.894262   72225 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=offline-docker-301000 offline-docker-301000

	I0115 06:12:23.981689   72225 network_create.go:108] docker network offline-docker-301000 192.168.58.0/24 created

	I0115 06:12:23.981729   72225 kic.go:121] calculated static IP "192.168.58.2" for the "offline-docker-301000" container

	I0115 06:12:23.981865   72225 cli_runner.go:164] Run: docker ps -a --format {{.Names}}

	I0115 06:12:24.034949   72225 cli_runner.go:164] Run: docker volume create offline-docker-301000 --label name.minikube.sigs.k8s.io=offline-docker-301000 --label created_by.minikube.sigs.k8s.io=true

	I0115 06:12:24.088268   72225 oci.go:103] Successfully created a docker volume offline-docker-301000

	I0115 06:12:24.088387   72225 cli_runner.go:164] Run: docker run --rm --name offline-docker-301000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=offline-docker-301000 --entrypoint /usr/bin/test -v offline-docker-301000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -d /var/lib

	I0115 06:12:24.907296   72225 oci.go:107] Successfully prepared a docker volume offline-docker-301000

	I0115 06:12:24.907331   72225 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0115 06:12:24.907345   72225 kic.go:194] Starting extracting preloaded images to volume ...

	I0115 06:12:24.907447   72225 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v offline-docker-301000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -I lz4 -xf /preloaded.tar -C /extractDir

	I0115 06:18:23.683590   72225 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0115 06:18:23.683690   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:18:23.736166   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:18:23.736272   72225 retry.go:31] will retry after 220.33958ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:23.958321   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:18:24.010942   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:18:24.011053   72225 retry.go:31] will retry after 189.636369ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:24.202969   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:18:24.257843   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:18:24.257939   72225 retry.go:31] will retry after 652.635853ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:24.912444   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:18:24.967449   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	W0115 06:18:24.967571   72225 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	W0115 06:18:24.967595   72225 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:24.967652   72225 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0115 06:18:24.967712   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:18:25.018529   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:18:25.018647   72225 retry.go:31] will retry after 194.799042ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:25.215006   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:18:25.269306   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:18:25.269423   72225 retry.go:31] will retry after 561.481183ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:25.833321   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:18:25.887844   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:18:25.887943   72225 retry.go:31] will retry after 301.391128ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:26.191700   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:18:26.245811   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:18:26.245911   72225 retry.go:31] will retry after 594.092863ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:26.841081   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:18:26.895339   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	W0115 06:18:26.895439   72225 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	W0115 06:18:26.895456   72225 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:26.895472   72225 start.go:128] duration metric: createHost completed in 6m3.23440049s

	I0115 06:18:26.895480   72225 start.go:83] releasing machines lock for "offline-docker-301000", held for 6m3.234511506s

	W0115 06:18:26.895493   72225 start.go:694] error starting host: creating host: create host timed out in 360.000000 seconds

	I0115 06:18:26.895920   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:26.946721   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	I0115 06:18:26.946780   72225 delete.go:82] Unable to get host status for offline-docker-301000, assuming it has already been deleted: state: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	W0115 06:18:26.946847   72225 out.go:239] ! StartHost failed, but will try again: creating host: create host timed out in 360.000000 seconds

	! StartHost failed, but will try again: creating host: create host timed out in 360.000000 seconds

	I0115 06:18:26.946858   72225 start.go:709] Will try again in 5 seconds ...

	I0115 06:18:31.949248   72225 start.go:365] acquiring machines lock for offline-docker-301000: {Name:mk777770c680d16be68699b00e5578abf6398a0c Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0115 06:18:31.949399   72225 start.go:369] acquired machines lock for "offline-docker-301000" in 115.274µs

	I0115 06:18:31.949430   72225 start.go:96] Skipping create...Using existing machine configuration

	I0115 06:18:31.949441   72225 fix.go:54] fixHost starting: 

	I0115 06:18:31.949791   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:32.002302   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	I0115 06:18:32.002348   72225 fix.go:102] recreateIfNeeded on offline-docker-301000: state= err=unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:32.002367   72225 fix.go:107] machineExists: false. err=machine does not exist

	I0115 06:18:32.025951   72225 out.go:177] * docker "offline-docker-301000" container is missing, will recreate.

	I0115 06:18:32.067671   72225 delete.go:124] DEMOLISHING offline-docker-301000 ...

	I0115 06:18:32.067819   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:32.119503   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	W0115 06:18:32.119575   72225 stop.go:75] unable to get state: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:32.119597   72225 delete.go:128] stophost failed (probably ok): ssh power off: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:32.119986   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:32.170303   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	I0115 06:18:32.170352   72225 delete.go:82] Unable to get host status for offline-docker-301000, assuming it has already been deleted: state: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:32.170426   72225 cli_runner.go:164] Run: docker container inspect -f {{.Id}} offline-docker-301000

	W0115 06:18:32.220907   72225 cli_runner.go:211] docker container inspect -f {{.Id}} offline-docker-301000 returned with exit code 1

	I0115 06:18:32.220953   72225 kic.go:371] could not find the container offline-docker-301000 to remove it. will try anyways

	I0115 06:18:32.221035   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:32.271367   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	W0115 06:18:32.271414   72225 oci.go:84] error getting container status, will try to delete anyways: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:32.271492   72225 cli_runner.go:164] Run: docker exec --privileged -t offline-docker-301000 /bin/bash -c "sudo init 0"

	W0115 06:18:32.322239   72225 cli_runner.go:211] docker exec --privileged -t offline-docker-301000 /bin/bash -c "sudo init 0" returned with exit code 1

	I0115 06:18:32.322273   72225 oci.go:650] error shutdown offline-docker-301000: docker exec --privileged -t offline-docker-301000 /bin/bash -c "sudo init 0": exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:33.324440   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:33.380185   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	I0115 06:18:33.380236   72225 oci.go:662] temporary error verifying shutdown: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:33.380246   72225 oci.go:664] temporary error: container offline-docker-301000 status is  but expect it to be exited

	I0115 06:18:33.380272   72225 retry.go:31] will retry after 683.914082ms: couldn't verify container is exited. %v: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:34.065500   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:34.118665   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	I0115 06:18:34.118715   72225 oci.go:662] temporary error verifying shutdown: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:34.118724   72225 oci.go:664] temporary error: container offline-docker-301000 status is  but expect it to be exited

	I0115 06:18:34.118746   72225 retry.go:31] will retry after 476.752011ms: couldn't verify container is exited. %v: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:34.595805   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:34.650483   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	I0115 06:18:34.650536   72225 oci.go:662] temporary error verifying shutdown: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:34.650550   72225 oci.go:664] temporary error: container offline-docker-301000 status is  but expect it to be exited

	I0115 06:18:34.650575   72225 retry.go:31] will retry after 744.418658ms: couldn't verify container is exited. %v: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:35.396146   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:35.451233   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	I0115 06:18:35.451279   72225 oci.go:662] temporary error verifying shutdown: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:35.451292   72225 oci.go:664] temporary error: container offline-docker-301000 status is  but expect it to be exited

	I0115 06:18:35.451316   72225 retry.go:31] will retry after 1.827715145s: couldn't verify container is exited. %v: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:37.280227   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:37.336346   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	I0115 06:18:37.336401   72225 oci.go:662] temporary error verifying shutdown: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:37.336412   72225 oci.go:664] temporary error: container offline-docker-301000 status is  but expect it to be exited

	I0115 06:18:37.336441   72225 retry.go:31] will retry after 2.69210934s: couldn't verify container is exited. %v: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:40.030776   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:40.086541   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	I0115 06:18:40.086589   72225 oci.go:662] temporary error verifying shutdown: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:40.086598   72225 oci.go:664] temporary error: container offline-docker-301000 status is  but expect it to be exited

	I0115 06:18:40.086633   72225 retry.go:31] will retry after 4.431348079s: couldn't verify container is exited. %v: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:44.518427   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:44.572409   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	I0115 06:18:44.572459   72225 oci.go:662] temporary error verifying shutdown: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:44.572469   72225 oci.go:664] temporary error: container offline-docker-301000 status is  but expect it to be exited

	I0115 06:18:44.572495   72225 retry.go:31] will retry after 6.72502865s: couldn't verify container is exited. %v: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:51.298293   72225 cli_runner.go:164] Run: docker container inspect offline-docker-301000 --format={{.State.Status}}

	W0115 06:18:51.349592   72225 cli_runner.go:211] docker container inspect offline-docker-301000 --format={{.State.Status}} returned with exit code 1

	I0115 06:18:51.349641   72225 oci.go:662] temporary error verifying shutdown: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:51.349654   72225 oci.go:664] temporary error: container offline-docker-301000 status is  but expect it to be exited

	I0115 06:18:51.349685   72225 oci.go:88] couldn't shut down offline-docker-301000 (might be okay): verify shutdown: couldn't verify container is exited. %v: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:18:51.349754   72225 cli_runner.go:164] Run: docker rm -f -v offline-docker-301000

	I0115 06:18:51.401599   72225 cli_runner.go:164] Run: docker container inspect -f {{.Id}} offline-docker-301000

	W0115 06:18:51.453213   72225 cli_runner.go:211] docker container inspect -f {{.Id}} offline-docker-301000 returned with exit code 1

	I0115 06:18:51.453334   72225 cli_runner.go:164] Run: docker network inspect offline-docker-301000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 06:18:51.504038   72225 cli_runner.go:164] Run: docker network rm offline-docker-301000

	I0115 06:18:51.605838   72225 fix.go:114] Sleeping 1 second for extra luck!

	I0115 06:18:52.605932   72225 start.go:125] createHost starting for "" (driver="docker")

	I0115 06:18:52.629272   72225 out.go:204] * Creating docker container (CPUs=2, Memory=2048MB) ...

	I0115 06:18:52.629421   72225 start.go:159] libmachine.API.Create for "offline-docker-301000" (driver="docker")

	I0115 06:18:52.629443   72225 client.go:168] LocalClient.Create starting

	I0115 06:18:52.629610   72225 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem

	I0115 06:18:52.629687   72225 main.go:141] libmachine: Decoding PEM data...

	I0115 06:18:52.629707   72225 main.go:141] libmachine: Parsing certificate...

	I0115 06:18:52.629770   72225 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem

	I0115 06:18:52.629823   72225 main.go:141] libmachine: Decoding PEM data...

	I0115 06:18:52.629835   72225 main.go:141] libmachine: Parsing certificate...

	I0115 06:18:52.630367   72225 cli_runner.go:164] Run: docker network inspect offline-docker-301000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	W0115 06:18:52.684422   72225 cli_runner.go:211] docker network inspect offline-docker-301000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1

	I0115 06:18:52.684517   72225 network_create.go:281] running [docker network inspect offline-docker-301000] to gather additional debugging logs...

	I0115 06:18:52.684532   72225 cli_runner.go:164] Run: docker network inspect offline-docker-301000

	W0115 06:18:52.735668   72225 cli_runner.go:211] docker network inspect offline-docker-301000 returned with exit code 1

	I0115 06:18:52.735696   72225 network_create.go:284] error running [docker network inspect offline-docker-301000]: docker network inspect offline-docker-301000: exit status 1

	stdout:

	[]

	stderr:

	Error response from daemon: network offline-docker-301000 not found

	I0115 06:18:52.735711   72225 network_create.go:286] output of [docker network inspect offline-docker-301000]: -- stdout --

	[]

	-- /stdout --

	** stderr ** 

	Error response from daemon: network offline-docker-301000 not found

	** /stderr **

	I0115 06:18:52.735863   72225 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 06:18:52.803122   72225 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}

	I0115 06:18:52.804582   72225 network.go:212] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}

	I0115 06:18:52.804936   72225 network.go:209] using free private subnet 192.168.67.0/24: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0022bda30}

	I0115 06:18:52.804948   72225 network_create.go:124] attempt to create docker network offline-docker-301000 192.168.67.0/24 with gateway 192.168.67.1 and MTU of 65535 ...

	I0115 06:18:52.805019   72225 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=offline-docker-301000 offline-docker-301000

	W0115 06:18:52.856327   72225 cli_runner.go:211] docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=offline-docker-301000 offline-docker-301000 returned with exit code 1

	W0115 06:18:52.856369   72225 network_create.go:149] failed to create docker network offline-docker-301000 192.168.67.0/24 with gateway 192.168.67.1 and mtu of 65535: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=offline-docker-301000 offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: Pool overlaps with other one on this address space

	W0115 06:18:52.856385   72225 network_create.go:116] failed to create docker network offline-docker-301000 192.168.67.0/24, will retry: subnet is taken

	I0115 06:18:52.857767   72225 network.go:212] skipping subnet 192.168.67.0/24 that is reserved: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}

	I0115 06:18:52.858125   72225 network.go:209] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0020ee510}

	I0115 06:18:52.858137   72225 network_create.go:124] attempt to create docker network offline-docker-301000 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 65535 ...

	I0115 06:18:52.858205   72225 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=offline-docker-301000 offline-docker-301000

	I0115 06:18:52.944706   72225 network_create.go:108] docker network offline-docker-301000 192.168.76.0/24 created

	I0115 06:18:52.944746   72225 kic.go:121] calculated static IP "192.168.76.2" for the "offline-docker-301000" container

	I0115 06:18:52.944894   72225 cli_runner.go:164] Run: docker ps -a --format {{.Names}}

	I0115 06:18:52.998249   72225 cli_runner.go:164] Run: docker volume create offline-docker-301000 --label name.minikube.sigs.k8s.io=offline-docker-301000 --label created_by.minikube.sigs.k8s.io=true

	I0115 06:18:53.048757   72225 oci.go:103] Successfully created a docker volume offline-docker-301000

	I0115 06:18:53.048897   72225 cli_runner.go:164] Run: docker run --rm --name offline-docker-301000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=offline-docker-301000 --entrypoint /usr/bin/test -v offline-docker-301000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -d /var/lib

	I0115 06:18:53.356418   72225 oci.go:107] Successfully prepared a docker volume offline-docker-301000

	I0115 06:18:53.356472   72225 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0115 06:18:53.356484   72225 kic.go:194] Starting extracting preloaded images to volume ...

	I0115 06:18:53.356599   72225 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v offline-docker-301000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -I lz4 -xf /preloaded.tar -C /extractDir

	I0115 06:24:52.617693   72225 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0115 06:24:52.617821   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:52.673190   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:52.673307   72225 retry.go:31] will retry after 184.770323ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:52.860438   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:52.911824   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:52.911923   72225 retry.go:31] will retry after 511.501536ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:53.423688   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:53.474872   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:53.474972   72225 retry.go:31] will retry after 353.601996ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:53.830074   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:53.884363   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	W0115 06:24:53.884480   72225 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	W0115 06:24:53.884506   72225 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:53.884567   72225 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0115 06:24:53.884652   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:53.935133   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:53.935236   72225 retry.go:31] will retry after 265.436168ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:54.202962   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:54.256848   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:54.256949   72225 retry.go:31] will retry after 466.68895ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:54.725996   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:54.780647   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:54.780760   72225 retry.go:31] will retry after 732.23224ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:55.513434   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:55.567828   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	W0115 06:24:55.567926   72225 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	W0115 06:24:55.567961   72225 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:55.567977   72225 start.go:128] duration metric: createHost completed in 6m2.974375379s

	I0115 06:24:55.568040   72225 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0115 06:24:55.568112   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:55.619332   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:55.619433   72225 retry.go:31] will retry after 232.506422ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:55.853189   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:55.907372   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:55.907482   72225 retry.go:31] will retry after 454.792155ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:56.363073   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:56.415819   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:56.415909   72225 retry.go:31] will retry after 580.083423ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:56.996611   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:57.051810   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	W0115 06:24:57.051910   72225 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	W0115 06:24:57.051928   72225 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:57.051994   72225 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0115 06:24:57.052048   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:57.103923   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:57.104018   72225 retry.go:31] will retry after 344.361076ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:57.450700   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:57.503413   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:57.503506   72225 retry.go:31] will retry after 295.366774ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:57.799336   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:57.854774   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	I0115 06:24:57.854864   72225 retry.go:31] will retry after 585.936228ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:58.443110   72225 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000

	W0115 06:24:58.496886   72225 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000 returned with exit code 1

	W0115 06:24:58.496994   72225 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	W0115 06:24:58.497020   72225 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "offline-docker-301000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" offline-docker-301000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

	I0115 06:24:58.497033   72225 fix.go:56] fixHost completed within 6m26.560789767s

	I0115 06:24:58.497039   72225 start.go:83] releasing machines lock for "offline-docker-301000", held for 6m26.560826808s

	W0115 06:24:58.497113   72225 out.go:239] * Failed to start docker container. Running "minikube delete -p offline-docker-301000" may fix it: recreate: creating host: create host timed out in 360.000000 seconds

	* Failed to start docker container. Running "minikube delete -p offline-docker-301000" may fix it: recreate: creating host: create host timed out in 360.000000 seconds

	I0115 06:24:58.540610   72225 out.go:177] 

	W0115 06:24:58.562303   72225 out.go:239] X Exiting due to DRV_CREATE_TIMEOUT: Failed to start host: recreate: creating host: create host timed out in 360.000000 seconds

	X Exiting due to DRV_CREATE_TIMEOUT: Failed to start host: recreate: creating host: create host timed out in 360.000000 seconds

	W0115 06:24:58.562341   72225 out.go:239] * Suggestion: Try 'minikube delete', and disable any conflicting VPN or firewall software

	* Suggestion: Try 'minikube delete', and disable any conflicting VPN or firewall software

	W0115 06:24:58.562357   72225 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/7072

	* Related issue: https://github.com/kubernetes/minikube/issues/7072

	I0115 06:24:58.583492   72225 out.go:177] 

** /stderr **

aab_offline_test.go:58: out/minikube-darwin-amd64 start -p offline-docker-301000 --alsologtostderr -v=1 --memory=2048 --wait=true --driver=docker  failed: exit status 52

panic.go:523: *** TestOffline FAILED at 2024-01-15 06:24:58.659929 -0800 PST m=+4994.761572276

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestOffline]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect offline-docker-301000

helpers_test.go:235: (dbg) docker inspect offline-docker-301000:

-- stdout --

	[

	    {

	        "Name": "offline-docker-301000",

	        "Id": "8ca869fe5d44da08c8e16299a0cd035694d96c15d834808e6216a158dc4e25e1",

	        "Created": "2024-01-15T14:18:52.906991768Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.76.0/24",

	                    "Gateway": "192.168.76.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "offline-docker-301000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p offline-docker-301000 -n offline-docker-301000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p offline-docker-301000 -n offline-docker-301000: exit status 7 (109.778251ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 06:24:58.822726   73086 status.go:249] status error: host: state: unknown state "offline-docker-301000": docker container inspect offline-docker-301000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: offline-docker-301000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "offline-docker-301000" host is not running, skipping log retrieval (state="Nonexistent")

helpers_test.go:175: Cleaning up "offline-docker-301000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-amd64 delete -p offline-docker-301000

--- FAIL: TestOffline (756.86s)

=== RUN   TestIngressAddonLegacy/StartLegacyK8sCluster

ingress_addon_legacy_test.go:39: (dbg) Run:  out/minikube-darwin-amd64 start -p ingress-addon-legacy-482000 --kubernetes-version=v1.18.20 --memory=4096 --wait=true --alsologtostderr -v=5 --driver=docker 

E0115 05:14:54.017946   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 05:15:21.726027   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 05:15:56.754375   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:15:56.760658   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:15:56.771965   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:15:56.794195   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:15:56.835561   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:15:56.917803   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:15:57.080000   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:15:57.401068   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:15:58.041414   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:15:59.322232   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:16:01.883936   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:16:07.003890   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:16:17.243837   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:16:37.723699   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:17:18.682221   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

ingress_addon_legacy_test.go:39: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p ingress-addon-legacy-482000 --kubernetes-version=v1.18.20 --memory=4096 --wait=true --alsologtostderr -v=5 --driver=docker : exit status 109 (4m17.666617229s)

-- stdout --

	* [ingress-addon-legacy-482000] minikube v1.32.0 on Darwin 14.2.1

	  - MINIKUBE_LOCATION=17953

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the docker driver based on user configuration

	* Using Docker Desktop driver with root privileges

	* Starting control plane node ingress-addon-legacy-482000 in cluster ingress-addon-legacy-482000

	* Pulling base image v0.0.42-1704759386-17866 ...

	* Downloading Kubernetes v1.18.20 preload ...

	* Creating docker container (CPUs=2, Memory=4096MB) ...

	* Preparing Kubernetes v1.18.20 on Docker 24.0.7 ...

	  - Generating certificates and keys ...

	  - Booting up control plane ...

	  - Generating certificates and keys ...

	  - Booting up control plane ...

-- /stdout --

** stderr ** 

	I0115 05:13:17.523696   68490 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:13:17.523896   68490 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:13:17.523901   68490 out.go:309] Setting ErrFile to fd 2...

	I0115 05:13:17.523905   68490 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:13:17.524085   68490 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:13:17.525569   68490 out.go:303] Setting JSON to false

	I0115 05:13:17.547961   68490 start.go:128] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":31140,"bootTime":1705293257,"procs":490,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.2.1","kernelVersion":"23.2.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}

	W0115 05:13:17.548117   68490 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0115 05:13:17.570030   68490 out.go:177] * [ingress-addon-legacy-482000] minikube v1.32.0 on Darwin 14.2.1

	I0115 05:13:17.613776   68490 out.go:177]   - MINIKUBE_LOCATION=17953

	I0115 05:13:17.613831   68490 notify.go:220] Checking for updates...

	I0115 05:13:17.636003   68490 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	I0115 05:13:17.657817   68490 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-amd64

	I0115 05:13:17.679522   68490 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0115 05:13:17.700758   68490 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	I0115 05:13:17.722801   68490 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0115 05:13:17.745024   68490 driver.go:392] Setting default libvirt URI to qemu:///system

	I0115 05:13:17.801556   68490 docker.go:122] docker version: linux-24.0.7:Docker Desktop 4.26.0 (130397)

	I0115 05:13:17.801719   68490 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0115 05:13:17.906237   68490 info.go:266] docker info: {ID:37b3081a-8cd6-457e-b2a4-79dc82345b06 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:2 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:46 OomKillDisable:false NGoroutines:63 SystemTime:2024-01-15 13:13:17.896795038 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:6.5.11-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServe

rAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6221279232 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f Expected:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f} RuncCommit:{ID:v1.1.10-0-g18a0cb0 Expected:v1.1.10-0-g18a0cb0} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=unconf

ined name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.0-desktop.2] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.23.3-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Mana

ges Docker extensions Vendor:Docker Inc. Version:v0.2.21] map[Name:feedback Path:/Users/jenkins/.docker/cli-plugins/docker-feedback SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-feedback] ShortDescription:Provide feedback, right in your terminal! Vendor:Docker Inc. Version:0.1] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.10] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/d

ocker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Scout Vendor:Docker Inc. Version:v1.2.0]] Warnings:<nil>}}

	I0115 05:13:17.927993   68490 out.go:177] * Using the docker driver based on user configuration

	I0115 05:13:17.970601   68490 start.go:298] selected driver: docker

	I0115 05:13:17.970624   68490 start.go:902] validating driver "docker" against <nil>

	I0115 05:13:17.970637   68490 start.go:913] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0115 05:13:17.974535   68490 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0115 05:13:18.078475   68490 info.go:266] docker info: {ID:37b3081a-8cd6-457e-b2a4-79dc82345b06 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:2 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:46 OomKillDisable:false NGoroutines:63 SystemTime:2024-01-15 13:13:18.070483754 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:6.5.11-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServe

rAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6221279232 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f Expected:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f} RuncCommit:{ID:v1.1.10-0-g18a0cb0 Expected:v1.1.10-0-g18a0cb0} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=unconf

ined name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.0-desktop.2] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.23.3-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Mana

ges Docker extensions Vendor:Docker Inc. Version:v0.2.21] map[Name:feedback Path:/Users/jenkins/.docker/cli-plugins/docker-feedback SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-feedback] ShortDescription:Provide feedback, right in your terminal! Vendor:Docker Inc. Version:0.1] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.10] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/d

ocker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Scout Vendor:Docker Inc. Version:v1.2.0]] Warnings:<nil>}}

	I0115 05:13:18.078657   68490 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0115 05:13:18.078844   68490 start_flags.go:927] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0115 05:13:18.100712   68490 out.go:177] * Using Docker Desktop driver with root privileges

	I0115 05:13:18.121883   68490 cni.go:84] Creating CNI manager for ""

	I0115 05:13:18.121922   68490 cni.go:162] CNI unnecessary in this configuration, recommending no CNI

	I0115 05:13:18.121942   68490 start_flags.go:321] config:

	{Name:ingress-addon-legacy-482000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 Memory:4096 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.18.20 ClusterName:ingress-addon-legacy-482000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.lo

cal ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs:}

	I0115 05:13:18.165694   68490 out.go:177] * Starting control plane node ingress-addon-legacy-482000 in cluster ingress-addon-legacy-482000

	I0115 05:13:18.187576   68490 cache.go:121] Beginning downloading kic base image for docker with docker

	I0115 05:13:18.230702   68490 out.go:177] * Pulling base image v0.0.42-1704759386-17866 ...

	I0115 05:13:18.252655   68490 preload.go:132] Checking if preload exists for k8s version v1.18.20 and runtime docker

	I0115 05:13:18.252715   68490 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 in local docker daemon

	I0115 05:13:18.306994   68490 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 in local docker daemon, skipping pull

	I0115 05:13:18.307017   68490 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 exists in daemon, skipping load

	I0115 05:13:18.316110   68490 preload.go:119] Found remote preload: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.18.20/preloaded-images-k8s-v18-v1.18.20-docker-overlay2-amd64.tar.lz4

	I0115 05:13:18.316130   68490 cache.go:56] Caching tarball of preloaded images

	I0115 05:13:18.316383   68490 preload.go:132] Checking if preload exists for k8s version v1.18.20 and runtime docker

	I0115 05:13:18.337593   68490 out.go:177] * Downloading Kubernetes v1.18.20 preload ...

	I0115 05:13:18.379675   68490 preload.go:238] getting checksum for preloaded-images-k8s-v18-v1.18.20-docker-overlay2-amd64.tar.lz4 ...

	I0115 05:13:18.461516   68490 download.go:107] Downloading: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.18.20/preloaded-images-k8s-v18-v1.18.20-docker-overlay2-amd64.tar.lz4?checksum=md5:ff35f06d4f6c0bac9297b8f85d8ebf70 -> /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.18.20-docker-overlay2-amd64.tar.lz4

	I0115 05:13:21.376100   68490 preload.go:249] saving checksum for preloaded-images-k8s-v18-v1.18.20-docker-overlay2-amd64.tar.lz4 ...

	I0115 05:13:21.376286   68490 preload.go:256] verifying checksum of /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.18.20-docker-overlay2-amd64.tar.lz4 ...

	I0115 05:13:22.007348   68490 cache.go:59] Finished verifying existence of preloaded tar for  v1.18.20 on docker

	I0115 05:13:22.007602   68490 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/config.json ...

	I0115 05:13:22.007628   68490 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/config.json: {Name:mk25d800f452f25dcdbf3f13f03dc1f97733d386 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0115 05:13:22.007921   68490 cache.go:194] Successfully downloaded all kic artifacts

	I0115 05:13:22.007951   68490 start.go:365] acquiring machines lock for ingress-addon-legacy-482000: {Name:mk0930bd0b28507c16b010a48d1c10ae87dd3dc4 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0115 05:13:22.008057   68490 start.go:369] acquired machines lock for "ingress-addon-legacy-482000" in 98.221µs

	I0115 05:13:22.008078   68490 start.go:93] Provisioning new machine with config: &{Name:ingress-addon-legacy-482000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 Memory:4096 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.18.20 ClusterName:ingress-addon-legacy-482000 Namespace:defau

lt APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.18.20 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOp

timizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs:} &{Name: IP: Port:8443 KubernetesVersion:v1.18.20 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0115 05:13:22.008118   68490 start.go:125] createHost starting for "" (driver="docker")

	I0115 05:13:22.059365   68490 out.go:204] * Creating docker container (CPUs=2, Memory=4096MB) ...

	I0115 05:13:22.059682   68490 start.go:159] libmachine.API.Create for "ingress-addon-legacy-482000" (driver="docker")

	I0115 05:13:22.059733   68490 client.go:168] LocalClient.Create starting

	I0115 05:13:22.059926   68490 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem

	I0115 05:13:22.059997   68490 main.go:141] libmachine: Decoding PEM data...

	I0115 05:13:22.060020   68490 main.go:141] libmachine: Parsing certificate...

	I0115 05:13:22.060081   68490 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem

	I0115 05:13:22.060132   68490 main.go:141] libmachine: Decoding PEM data...

	I0115 05:13:22.060151   68490 main.go:141] libmachine: Parsing certificate...

	I0115 05:13:22.060804   68490 cli_runner.go:164] Run: docker network inspect ingress-addon-legacy-482000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	W0115 05:13:22.112543   68490 cli_runner.go:211] docker network inspect ingress-addon-legacy-482000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1

	I0115 05:13:22.112661   68490 network_create.go:281] running [docker network inspect ingress-addon-legacy-482000] to gather additional debugging logs...

	I0115 05:13:22.112679   68490 cli_runner.go:164] Run: docker network inspect ingress-addon-legacy-482000

	W0115 05:13:22.163159   68490 cli_runner.go:211] docker network inspect ingress-addon-legacy-482000 returned with exit code 1

	I0115 05:13:22.163195   68490 network_create.go:284] error running [docker network inspect ingress-addon-legacy-482000]: docker network inspect ingress-addon-legacy-482000: exit status 1

	stdout:

	[]

	stderr:

	Error response from daemon: network ingress-addon-legacy-482000 not found

	I0115 05:13:22.163212   68490 network_create.go:286] output of [docker network inspect ingress-addon-legacy-482000]: -- stdout --

	[]

	-- /stdout --

	** stderr ** 

	Error response from daemon: network ingress-addon-legacy-482000 not found

	** /stderr **

	I0115 05:13:22.163351   68490 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 05:13:22.214750   68490 network.go:209] using free private subnet 192.168.49.0/24: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000013230}

	I0115 05:13:22.214797   68490 network_create.go:124] attempt to create docker network ingress-addon-legacy-482000 192.168.49.0/24 with gateway 192.168.49.1 and MTU of 65535 ...

	I0115 05:13:22.214870   68490 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=ingress-addon-legacy-482000 ingress-addon-legacy-482000

	I0115 05:13:22.302368   68490 network_create.go:108] docker network ingress-addon-legacy-482000 192.168.49.0/24 created

	I0115 05:13:22.302417   68490 kic.go:121] calculated static IP "192.168.49.2" for the "ingress-addon-legacy-482000" container

	I0115 05:13:22.302558   68490 cli_runner.go:164] Run: docker ps -a --format {{.Names}}

	I0115 05:13:22.356821   68490 cli_runner.go:164] Run: docker volume create ingress-addon-legacy-482000 --label name.minikube.sigs.k8s.io=ingress-addon-legacy-482000 --label created_by.minikube.sigs.k8s.io=true

	I0115 05:13:22.409370   68490 oci.go:103] Successfully created a docker volume ingress-addon-legacy-482000

	I0115 05:13:22.409526   68490 cli_runner.go:164] Run: docker run --rm --name ingress-addon-legacy-482000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=ingress-addon-legacy-482000 --entrypoint /usr/bin/test -v ingress-addon-legacy-482000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -d /var/lib

	I0115 05:13:22.809037   68490 oci.go:107] Successfully prepared a docker volume ingress-addon-legacy-482000

	I0115 05:13:22.809074   68490 preload.go:132] Checking if preload exists for k8s version v1.18.20 and runtime docker

	I0115 05:13:22.809089   68490 kic.go:194] Starting extracting preloaded images to volume ...

	I0115 05:13:22.809204   68490 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.18.20-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v ingress-addon-legacy-482000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -I lz4 -xf /preloaded.tar -C /extractDir

	I0115 05:13:25.373614   68490 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.18.20-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v ingress-addon-legacy-482000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -I lz4 -xf /preloaded.tar -C /extractDir: (2.56448383s)

	I0115 05:13:25.373648   68490 kic.go:203] duration metric: took 2.564699 seconds to extract preloaded images to volume

	I0115 05:13:25.373761   68490 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"

	I0115 05:13:25.476310   68490 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname ingress-addon-legacy-482000 --name ingress-addon-legacy-482000 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=ingress-addon-legacy-482000 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=ingress-addon-legacy-482000 --network ingress-addon-legacy-482000 --ip 192.168.49.2 --volume ingress-addon-legacy-482000:/var --security-opt apparmor=unconfined --memory=4096mb --memory-swap=4096mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0

	I0115 05:13:25.746892   68490 cli_runner.go:164] Run: docker container inspect ingress-addon-legacy-482000 --format={{.State.Running}}

	I0115 05:13:25.804901   68490 cli_runner.go:164] Run: docker container inspect ingress-addon-legacy-482000 --format={{.State.Status}}

	I0115 05:13:25.865308   68490 cli_runner.go:164] Run: docker exec ingress-addon-legacy-482000 stat /var/lib/dpkg/alternatives/iptables

	I0115 05:13:25.999636   68490 oci.go:144] the created container "ingress-addon-legacy-482000" has a running status.

	I0115 05:13:25.999685   68490 kic.go:225] Creating ssh key for kic: /Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa...

	I0115 05:13:26.298580   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa.pub -> /home/docker/.ssh/authorized_keys

	I0115 05:13:26.298644   68490 kic_runner.go:191] docker (temp): /Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)

	I0115 05:13:26.364308   68490 cli_runner.go:164] Run: docker container inspect ingress-addon-legacy-482000 --format={{.State.Status}}

	I0115 05:13:26.418678   68490 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys

	I0115 05:13:26.418702   68490 kic_runner.go:114] Args: [docker exec --privileged ingress-addon-legacy-482000 chown docker:docker /home/docker/.ssh/authorized_keys]

	I0115 05:13:26.515333   68490 cli_runner.go:164] Run: docker container inspect ingress-addon-legacy-482000 --format={{.State.Status}}

	I0115 05:13:26.567606   68490 machine.go:88] provisioning docker machine ...

	I0115 05:13:26.567670   68490 ubuntu.go:169] provisioning hostname "ingress-addon-legacy-482000"

	I0115 05:13:26.567812   68490 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:13:26.622123   68490 main.go:141] libmachine: Using SSH client type: native

	I0115 05:13:26.622486   68490 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x14075e0] 0x140a2c0 <nil>  [] 0s} 127.0.0.1 54699 <nil> <nil>}

	I0115 05:13:26.622503   68490 main.go:141] libmachine: About to run SSH command:

	sudo hostname ingress-addon-legacy-482000 && echo "ingress-addon-legacy-482000" | sudo tee /etc/hostname

	I0115 05:13:26.768147   68490 main.go:141] libmachine: SSH cmd err, output: <nil>: ingress-addon-legacy-482000

	I0115 05:13:26.768244   68490 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:13:26.820241   68490 main.go:141] libmachine: Using SSH client type: native

	I0115 05:13:26.820555   68490 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x14075e0] 0x140a2c0 <nil>  [] 0s} 127.0.0.1 54699 <nil> <nil>}

	I0115 05:13:26.820576   68490 main.go:141] libmachine: About to run SSH command:

			if ! grep -xq '.*\singress-addon-legacy-482000' /etc/hosts; then

				if grep -xq '127.0.1.1\s.*' /etc/hosts; then

					sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 ingress-addon-legacy-482000/g' /etc/hosts;

				else 

					echo '127.0.1.1 ingress-addon-legacy-482000' | sudo tee -a /etc/hosts; 

				fi

			fi

	I0115 05:13:26.957781   68490 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	I0115 05:13:26.957803   68490 ubuntu.go:175] set auth options {CertDir:/Users/jenkins/minikube-integration/17953-64881/.minikube CaCertPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/17953-64881/.minikube}

	I0115 05:13:26.957826   68490 ubuntu.go:177] setting up certificates

	I0115 05:13:26.957845   68490 provision.go:83] configureAuth start

	I0115 05:13:26.957926   68490 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" ingress-addon-legacy-482000

	I0115 05:13:27.008737   68490 provision.go:138] copyHostCerts

	I0115 05:13:27.008804   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem -> /Users/jenkins/minikube-integration/17953-64881/.minikube/ca.pem

	I0115 05:13:27.008855   68490 exec_runner.go:144] found /Users/jenkins/minikube-integration/17953-64881/.minikube/ca.pem, removing ...

	I0115 05:13:27.008863   68490 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17953-64881/.minikube/ca.pem

	I0115 05:13:27.008971   68490 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/17953-64881/.minikube/ca.pem (1082 bytes)

	I0115 05:13:27.009151   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem -> /Users/jenkins/minikube-integration/17953-64881/.minikube/cert.pem

	I0115 05:13:27.009188   68490 exec_runner.go:144] found /Users/jenkins/minikube-integration/17953-64881/.minikube/cert.pem, removing ...

	I0115 05:13:27.009193   68490 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17953-64881/.minikube/cert.pem

	I0115 05:13:27.009277   68490 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/17953-64881/.minikube/cert.pem (1123 bytes)

	I0115 05:13:27.009424   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/key.pem -> /Users/jenkins/minikube-integration/17953-64881/.minikube/key.pem

	I0115 05:13:27.009461   68490 exec_runner.go:144] found /Users/jenkins/minikube-integration/17953-64881/.minikube/key.pem, removing ...

	I0115 05:13:27.009472   68490 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17953-64881/.minikube/key.pem

	I0115 05:13:27.009547   68490 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/17953-64881/.minikube/key.pem (1675 bytes)

	I0115 05:13:27.009689   68490 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/17953-64881/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca-key.pem org=jenkins.ingress-addon-legacy-482000 san=[192.168.49.2 127.0.0.1 localhost 127.0.0.1 minikube ingress-addon-legacy-482000]

	I0115 05:13:27.129693   68490 provision.go:172] copyRemoteCerts

	I0115 05:13:27.129750   68490 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker

	I0115 05:13:27.129804   68490 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:13:27.181589   68490 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:54699 SSHKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa Username:docker}

	I0115 05:13:27.275665   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem -> /etc/docker/ca.pem

	I0115 05:13:27.275743   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)

	I0115 05:13:27.295400   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/machines/server.pem -> /etc/docker/server.pem

	I0115 05:13:27.295467   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/machines/server.pem --> /etc/docker/server.pem (1257 bytes)

	I0115 05:13:27.315293   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/machines/server-key.pem -> /etc/docker/server-key.pem

	I0115 05:13:27.315374   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)

	I0115 05:13:27.335495   68490 provision.go:86] duration metric: configureAuth took 377.654996ms

	I0115 05:13:27.335513   68490 ubuntu.go:193] setting minikube options for container-runtime

	I0115 05:13:27.335655   68490 config.go:182] Loaded profile config "ingress-addon-legacy-482000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.18.20

	I0115 05:13:27.335724   68490 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:13:27.387241   68490 main.go:141] libmachine: Using SSH client type: native

	I0115 05:13:27.387597   68490 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x14075e0] 0x140a2c0 <nil>  [] 0s} 127.0.0.1 54699 <nil> <nil>}

	I0115 05:13:27.387613   68490 main.go:141] libmachine: About to run SSH command:

	df --output=fstype / | tail -n 1

	I0115 05:13:27.521855   68490 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay

	I0115 05:13:27.521873   68490 ubuntu.go:71] root file system type: overlay

	I0115 05:13:27.521960   68490 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...

	I0115 05:13:27.522043   68490 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:13:27.574157   68490 main.go:141] libmachine: Using SSH client type: native

	I0115 05:13:27.574495   68490 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x14075e0] 0x140a2c0 <nil>  [] 0s} 127.0.0.1 54699 <nil> <nil>}

	I0115 05:13:27.574552   68490 main.go:141] libmachine: About to run SSH command:

	sudo mkdir -p /lib/systemd/system && printf %s "[Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	BindsTo=containerd.service

	After=network-online.target firewalld.service containerd.service

	Wants=network-online.target

	Requires=docker.socket

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP \$MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	" | sudo tee /lib/systemd/system/docker.service.new

	I0115 05:13:27.717276   68490 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	BindsTo=containerd.service

	After=network-online.target firewalld.service containerd.service

	Wants=network-online.target

	Requires=docker.socket

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP $MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	I0115 05:13:27.717388   68490 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:13:27.769354   68490 main.go:141] libmachine: Using SSH client type: native

	I0115 05:13:27.769666   68490 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x14075e0] 0x140a2c0 <nil>  [] 0s} 127.0.0.1 54699 <nil> <nil>}

	I0115 05:13:27.769679   68490 main.go:141] libmachine: About to run SSH command:

	sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }

	I0115 05:13:28.349239   68490 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service	2023-10-26 09:06:22.000000000 +0000

	+++ /lib/systemd/system/docker.service.new	2024-01-15 13:13:27.714916959 +0000

	@@ -1,30 +1,32 @@

	 [Unit]

	 Description=Docker Application Container Engine

	 Documentation=https://docs.docker.com

	-After=network-online.target docker.socket firewalld.service containerd.service time-set.target

	-Wants=network-online.target containerd.service

	+BindsTo=containerd.service

	+After=network-online.target firewalld.service containerd.service

	+Wants=network-online.target

	 Requires=docker.socket

	+StartLimitBurst=3

	+StartLimitIntervalSec=60

	 [Service]

	 Type=notify

	-# the default is not to use systemd for cgroups because the delegate issues still

	-# exists and systemd currently does not support the cgroup feature set required

	-# for containers run by docker

	-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

	-ExecReload=/bin/kill -s HUP $MAINPID

	-TimeoutStartSec=0

	-RestartSec=2

	-Restart=always

	+Restart=on-failure

	-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.

	-# Both the old, and new location are accepted by systemd 229 and up, so using the old location

	-# to make them work for either version of systemd.

	-StartLimitBurst=3

	-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.

	-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make

	-# this option work for either version of systemd.

	-StartLimitInterval=60s

	+

	+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	+# The base configuration already specifies an 'ExecStart=...' command. The first directive

	+# here is to clear out that command inherited from the base configuration. Without this,

	+# the command from the base configuration and the command specified here are treated as

	+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	+# will catch this invalid input and refuse to start the service with an error like:

	+#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	+

	+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	+ExecStart=

	+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12 

	+ExecReload=/bin/kill -s HUP $MAINPID

	 # Having non-zero Limit*s causes performance problems due to accounting overhead

	 # in the kernel. We recommend using cgroups to do container-local accounting.

	@@ -32,16 +34,16 @@

	 LimitNPROC=infinity

	 LimitCORE=infinity

	-# Comment TasksMax if your systemd version does not support it.

	-# Only systemd 226 and above support this option.

	+# Uncomment TasksMax if your systemd version supports it.

	+# Only systemd 226 and above support this version.

	 TasksMax=infinity

	+TimeoutStartSec=0

	 # set delegate yes so that systemd does not reset the cgroups of docker containers

	 Delegate=yes

	 # kill only the docker process, not all processes in the cgroup

	 KillMode=process

	-OOMScoreAdjust=-500

	 [Install]

	 WantedBy=multi-user.target

	Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.

	Executing: /lib/systemd/systemd-sysv-install enable docker

	I0115 05:13:28.349263   68490 machine.go:91] provisioned docker machine in 1.781711736s

	I0115 05:13:28.349270   68490 client.go:171] LocalClient.Create took 6.289873831s

	I0115 05:13:28.349318   68490 start.go:167] duration metric: libmachine.API.Create for "ingress-addon-legacy-482000" took 6.289982776s

	I0115 05:13:28.349335   68490 start.go:300] post-start starting for "ingress-addon-legacy-482000" (driver="docker")

	I0115 05:13:28.349359   68490 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]

	I0115 05:13:28.349441   68490 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs

	I0115 05:13:28.349565   68490 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:13:28.404401   68490 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:54699 SSHKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa Username:docker}

	I0115 05:13:28.499611   68490 ssh_runner.go:195] Run: cat /etc/os-release

	I0115 05:13:28.503660   68490 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found

	I0115 05:13:28.503684   68490 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found

	I0115 05:13:28.503691   68490 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found

	I0115 05:13:28.503699   68490 info.go:137] Remote host: Ubuntu 22.04.3 LTS

	I0115 05:13:28.503709   68490 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17953-64881/.minikube/addons for local assets ...

	I0115 05:13:28.503818   68490 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17953-64881/.minikube/files for local assets ...

	I0115 05:13:28.504003   68490 filesync.go:149] local asset: /Users/jenkins/minikube-integration/17953-64881/.minikube/files/etc/ssl/certs/656302.pem -> 656302.pem in /etc/ssl/certs

	I0115 05:13:28.504009   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/files/etc/ssl/certs/656302.pem -> /etc/ssl/certs/656302.pem

	I0115 05:13:28.504217   68490 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs

	I0115 05:13:28.512139   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/files/etc/ssl/certs/656302.pem --> /etc/ssl/certs/656302.pem (1708 bytes)

	I0115 05:13:28.532653   68490 start.go:303] post-start completed in 183.315151ms

	I0115 05:13:28.533237   68490 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" ingress-addon-legacy-482000

	I0115 05:13:28.586073   68490 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/config.json ...

	I0115 05:13:28.586563   68490 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0115 05:13:28.586629   68490 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:13:28.638438   68490 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:54699 SSHKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa Username:docker}

	I0115 05:13:28.731024   68490 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0115 05:13:28.735739   68490 start.go:128] duration metric: createHost completed in 6.727974982s

	I0115 05:13:28.735758   68490 start.go:83] releasing machines lock for "ingress-addon-legacy-482000", held for 6.728063648s

	I0115 05:13:28.735851   68490 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" ingress-addon-legacy-482000

	I0115 05:13:28.786941   68490 ssh_runner.go:195] Run: cat /version.json

	I0115 05:13:28.786955   68490 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/

	I0115 05:13:28.787019   68490 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:13:28.787033   68490 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:13:28.848291   68490 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:54699 SSHKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa Username:docker}

	I0115 05:13:28.848265   68490 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:54699 SSHKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa Username:docker}

	I0115 05:13:29.049864   68490 ssh_runner.go:195] Run: systemctl --version

	I0115 05:13:29.054708   68490 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"

	I0115 05:13:29.059652   68490 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;

	I0115 05:13:29.081190   68490 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found

	I0115 05:13:29.081282   68490 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;

	I0115 05:13:29.096252   68490 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;

	I0115 05:13:29.110975   68490 cni.go:308] configured [/etc/cni/net.d/100-crio-bridge.conf, /etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)

	I0115 05:13:29.110994   68490 start.go:475] detecting cgroup driver to use...

	I0115 05:13:29.111009   68490 detect.go:196] detected "cgroupfs" cgroup driver on host os

	I0115 05:13:29.111127   68490 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock

	" | sudo tee /etc/crictl.yaml"

	I0115 05:13:29.125845   68490 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"

	I0115 05:13:29.135228   68490 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"

	I0115 05:13:29.144625   68490 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...

	I0115 05:13:29.144686   68490 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"

	I0115 05:13:29.153815   68490 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0115 05:13:29.163223   68490 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"

	I0115 05:13:29.172581   68490 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0115 05:13:29.181909   68490 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"

	I0115 05:13:29.190860   68490 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"

	I0115 05:13:29.199976   68490 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables

	I0115 05:13:29.208080   68490 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

	I0115 05:13:29.216167   68490 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0115 05:13:29.265335   68490 ssh_runner.go:195] Run: sudo systemctl restart containerd

	I0115 05:13:29.341241   68490 start.go:475] detecting cgroup driver to use...

	I0115 05:13:29.341278   68490 detect.go:196] detected "cgroupfs" cgroup driver on host os

	I0115 05:13:29.341355   68490 ssh_runner.go:195] Run: sudo systemctl cat docker.service

	I0115 05:13:29.353715   68490 cruntime.go:279] skipping containerd shutdown because we are bound to it

	I0115 05:13:29.353790   68490 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I0115 05:13:29.365211   68490 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock

	" | sudo tee /etc/crictl.yaml"

	I0115 05:13:29.382257   68490 ssh_runner.go:195] Run: which cri-dockerd

	I0115 05:13:29.386950   68490 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d

	I0115 05:13:29.396739   68490 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)

	I0115 05:13:29.415157   68490 ssh_runner.go:195] Run: sudo systemctl unmask docker.service

	I0115 05:13:29.508344   68490 ssh_runner.go:195] Run: sudo systemctl enable docker.socket

	I0115 05:13:29.595964   68490 docker.go:574] configuring docker to use "cgroupfs" as cgroup driver...

	I0115 05:13:29.596114   68490 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)

	I0115 05:13:29.613130   68490 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0115 05:13:29.668060   68490 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0115 05:13:29.987672   68490 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0115 05:13:30.011499   68490 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0115 05:13:30.077738   68490 out.go:204] * Preparing Kubernetes v1.18.20 on Docker 24.0.7 ...

	I0115 05:13:30.077894   68490 cli_runner.go:164] Run: docker exec -t ingress-addon-legacy-482000 dig +short host.docker.internal

	I0115 05:13:30.191378   68490 network.go:96] got host ip for mount in container by digging dns: 192.168.65.254

	I0115 05:13:30.191494   68490 ssh_runner.go:195] Run: grep 192.168.65.254	host.minikube.internal$ /etc/hosts

	I0115 05:13:30.196036   68490 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.65.254	host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0115 05:13:30.206585   68490 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "8443/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:13:30.259522   68490 preload.go:132] Checking if preload exists for k8s version v1.18.20 and runtime docker

	I0115 05:13:30.259601   68490 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0115 05:13:30.277922   68490 docker.go:685] Got preloaded images: -- stdout --

	k8s.gcr.io/kube-proxy:v1.18.20

	k8s.gcr.io/kube-apiserver:v1.18.20

	k8s.gcr.io/kube-scheduler:v1.18.20

	k8s.gcr.io/kube-controller-manager:v1.18.20

	gcr.io/k8s-minikube/storage-provisioner:v5

	k8s.gcr.io/pause:3.2

	k8s.gcr.io/coredns:1.6.7

	k8s.gcr.io/etcd:3.4.3-0

	-- /stdout --

	I0115 05:13:30.277938   68490 docker.go:691] registry.k8s.io/kube-apiserver:v1.18.20 wasn't preloaded

	I0115 05:13:30.278012   68490 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0115 05:13:30.286809   68490 ssh_runner.go:195] Run: which lz4

	I0115 05:13:30.291092   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.18.20-docker-overlay2-amd64.tar.lz4 -> /preloaded.tar.lz4

	I0115 05:13:30.291209   68490 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4

	I0115 05:13:30.295173   68490 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1

	stdout:

	stderr:

	stat: cannot statx '/preloaded.tar.lz4': No such file or directory

	I0115 05:13:30.295196   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.18.20-docker-overlay2-amd64.tar.lz4 --> /preloaded.tar.lz4 (424164442 bytes)

	I0115 05:13:35.860514   68490 docker.go:649] Took 5.569658 seconds to copy over tarball

	I0115 05:13:35.860596   68490 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4

	I0115 05:13:37.512914   68490 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (1.652345447s)

	I0115 05:13:37.512948   68490 ssh_runner.go:146] rm: /preloaded.tar.lz4

	I0115 05:13:37.558894   68490 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0115 05:13:37.567647   68490 ssh_runner.go:362] scp memory --> /var/lib/docker/image/overlay2/repositories.json (2502 bytes)

	I0115 05:13:37.582829   68490 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0115 05:13:37.633169   68490 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0115 05:13:38.699392   68490 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.06626346s)

	I0115 05:13:38.699496   68490 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0115 05:13:38.717697   68490 docker.go:685] Got preloaded images: -- stdout --

	k8s.gcr.io/kube-proxy:v1.18.20

	k8s.gcr.io/kube-apiserver:v1.18.20

	k8s.gcr.io/kube-scheduler:v1.18.20

	k8s.gcr.io/kube-controller-manager:v1.18.20

	gcr.io/k8s-minikube/storage-provisioner:v5

	k8s.gcr.io/pause:3.2

	k8s.gcr.io/coredns:1.6.7

	k8s.gcr.io/etcd:3.4.3-0

	-- /stdout --

	I0115 05:13:38.717714   68490 docker.go:691] registry.k8s.io/kube-apiserver:v1.18.20 wasn't preloaded

	I0115 05:13:38.717725   68490 cache_images.go:88] LoadImages start: [registry.k8s.io/kube-apiserver:v1.18.20 registry.k8s.io/kube-controller-manager:v1.18.20 registry.k8s.io/kube-scheduler:v1.18.20 registry.k8s.io/kube-proxy:v1.18.20 registry.k8s.io/pause:3.2 registry.k8s.io/etcd:3.4.3-0 registry.k8s.io/coredns:1.6.7 gcr.io/k8s-minikube/storage-provisioner:v5]

	I0115 05:13:38.723231   68490 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.18.20

	I0115 05:13:38.724730   68490 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5

	I0115 05:13:38.724821   68490 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.18.20

	I0115 05:13:38.725231   68490 image.go:134] retrieving image: registry.k8s.io/etcd:3.4.3-0

	I0115 05:13:38.725334   68490 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.18.20

	I0115 05:13:38.725383   68490 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.18.20

	I0115 05:13:38.725476   68490 image.go:134] retrieving image: registry.k8s.io/coredns:1.6.7

	I0115 05:13:38.725520   68490 image.go:134] retrieving image: registry.k8s.io/pause:3.2

	I0115 05:13:38.733050   68490 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.18.20: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.18.20

	I0115 05:13:38.733499   68490 image.go:177] daemon lookup for registry.k8s.io/pause:3.2: Error response from daemon: No such image: registry.k8s.io/pause:3.2

	I0115 05:13:38.735375   68490 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5

	I0115 05:13:38.735377   68490 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.18.20: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.18.20

	I0115 05:13:38.735401   68490 image.go:177] daemon lookup for registry.k8s.io/etcd:3.4.3-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.4.3-0

	I0115 05:13:38.735504   68490 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.18.20: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.18.20

	I0115 05:13:38.735586   68490 image.go:177] daemon lookup for registry.k8s.io/coredns:1.6.7: Error response from daemon: No such image: registry.k8s.io/coredns:1.6.7

	I0115 05:13:38.735636   68490 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.18.20: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.18.20

	I0115 05:13:39.175786   68490 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/pause:3.2

	I0115 05:13:39.183284   68490 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-apiserver:v1.18.20

	I0115 05:13:39.199822   68490 cache_images.go:116] "registry.k8s.io/pause:3.2" needs transfer: "registry.k8s.io/pause:3.2" does not exist at hash "80d28bedfe5dec59da9ebf8e6260224ac9008ab5c11dbbe16ee3ba3e4439ac2c" in container runtime

	I0115 05:13:39.199933   68490 docker.go:337] Removing image: registry.k8s.io/pause:3.2

	I0115 05:13:39.200041   68490 ssh_runner.go:195] Run: docker rmi registry.k8s.io/pause:3.2

	I0115 05:13:39.206903   68490 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.18.20" needs transfer: "registry.k8s.io/kube-apiserver:v1.18.20" does not exist at hash "7d8d2960de69688eab5698081441539a1662f47e092488973e455a8334955cb1" in container runtime

	I0115 05:13:39.206963   68490 docker.go:337] Removing image: registry.k8s.io/kube-apiserver:v1.18.20

	I0115 05:13:39.207056   68490 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-apiserver:v1.18.20

	I0115 05:13:39.223298   68490 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2

	I0115 05:13:39.228994   68490 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.18.20

	I0115 05:13:39.237719   68490 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-controller-manager:v1.18.20

	I0115 05:13:39.256010   68490 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.18.20" needs transfer: "registry.k8s.io/kube-controller-manager:v1.18.20" does not exist at hash "e7c545a60706cf009a893afdc7dba900cc2e342b8042b9c421d607ca41e8b290" in container runtime

	I0115 05:13:39.256047   68490 docker.go:337] Removing image: registry.k8s.io/kube-controller-manager:v1.18.20

	I0115 05:13:39.256107   68490 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-controller-manager:v1.18.20

	I0115 05:13:39.275869   68490 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.18.20

	I0115 05:13:39.312429   68490 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/etcd:3.4.3-0

	I0115 05:13:39.331833   68490 cache_images.go:116] "registry.k8s.io/etcd:3.4.3-0" needs transfer: "registry.k8s.io/etcd:3.4.3-0" does not exist at hash "303ce5db0e90dab1c5728ec70d21091201a23cdf8aeca70ab54943bbaaf0833f" in container runtime

	I0115 05:13:39.331863   68490 docker.go:337] Removing image: registry.k8s.io/etcd:3.4.3-0

	I0115 05:13:39.331937   68490 ssh_runner.go:195] Run: docker rmi registry.k8s.io/etcd:3.4.3-0

	I0115 05:13:39.350209   68490 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.3-0

	I0115 05:13:39.366916   68490 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/coredns:1.6.7

	I0115 05:13:39.386076   68490 cache_images.go:116] "registry.k8s.io/coredns:1.6.7" needs transfer: "registry.k8s.io/coredns:1.6.7" does not exist at hash "67da37a9a360e600e74464da48437257b00a754c77c40f60c65e4cb327c34bd5" in container runtime

	I0115 05:13:39.386100   68490 docker.go:337] Removing image: registry.k8s.io/coredns:1.6.7

	I0115 05:13:39.386164   68490 ssh_runner.go:195] Run: docker rmi registry.k8s.io/coredns:1.6.7

	I0115 05:13:39.403557   68490 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.6.7

	I0115 05:13:39.436460   68490 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-proxy:v1.18.20

	I0115 05:13:39.455940   68490 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.18.20" needs transfer: "registry.k8s.io/kube-proxy:v1.18.20" does not exist at hash "27f8b8d51985f755cfb3ffea424fa34865cc0da63e99378d8202f923c3c5a8ba" in container runtime

	I0115 05:13:39.455966   68490 docker.go:337] Removing image: registry.k8s.io/kube-proxy:v1.18.20

	I0115 05:13:39.456035   68490 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-proxy:v1.18.20

	I0115 05:13:39.476163   68490 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.18.20

	I0115 05:13:39.510542   68490 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} gcr.io/k8s-minikube/storage-provisioner:v5

	I0115 05:13:39.525063   68490 ssh_runner.go:195] Run: docker image inspect --format {{.Id}} registry.k8s.io/kube-scheduler:v1.18.20

	I0115 05:13:39.546194   68490 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.18.20" needs transfer: "registry.k8s.io/kube-scheduler:v1.18.20" does not exist at hash "a05a1a79adaad058478b7638d2e73cf408b283305081516fbe02706b0e205346" in container runtime

	I0115 05:13:39.546222   68490 docker.go:337] Removing image: registry.k8s.io/kube-scheduler:v1.18.20

	I0115 05:13:39.546291   68490 ssh_runner.go:195] Run: docker rmi registry.k8s.io/kube-scheduler:v1.18.20

	I0115 05:13:39.564001   68490 cache_images.go:286] Loading image from: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.18.20

	I0115 05:13:39.564046   68490 cache_images.go:92] LoadImages completed in 846.355343ms

	W0115 05:13:39.564097   68490 out.go:239] X Unable to load cached images: loading cached images: stat /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2: no such file or directory

	X Unable to load cached images: loading cached images: stat /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/images/amd64/registry.k8s.io/pause_3.2: no such file or directory

	I0115 05:13:39.564170   68490 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}

	I0115 05:13:39.611506   68490 cni.go:84] Creating CNI manager for ""

	I0115 05:13:39.611524   68490 cni.go:162] CNI unnecessary in this configuration, recommending no CNI

	I0115 05:13:39.611539   68490 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16

	I0115 05:13:39.611555   68490 kubeadm.go:176] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.49.2 APIServerPort:8443 KubernetesVersion:v1.18.20 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:ingress-addon-legacy-482000 NodeName:ingress-addon-legacy-482000 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.49.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.49.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.

crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}

	I0115 05:13:39.611656   68490 kubeadm.go:181] kubeadm config:

	apiVersion: kubeadm.k8s.io/v1beta2

	kind: InitConfiguration

	localAPIEndpoint:

	  advertiseAddress: 192.168.49.2

	  bindPort: 8443

	bootstrapTokens:

	  - groups:

	      - system:bootstrappers:kubeadm:default-node-token

	    ttl: 24h0m0s

	    usages:

	      - signing

	      - authentication

	nodeRegistration:

	  criSocket: /var/run/dockershim.sock

	  name: "ingress-addon-legacy-482000"

	  kubeletExtraArgs:

	    node-ip: 192.168.49.2

	  taints: []

	---

	apiVersion: kubeadm.k8s.io/v1beta2

	kind: ClusterConfiguration

	apiServer:

	  certSANs: ["127.0.0.1", "localhost", "192.168.49.2"]

	  extraArgs:

	    enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"

	controllerManager:

	  extraArgs:

	    allocate-node-cidrs: "true"

	    leader-elect: "false"

	scheduler:

	  extraArgs:

	    leader-elect: "false"

	certificatesDir: /var/lib/minikube/certs

	clusterName: mk

	controlPlaneEndpoint: control-plane.minikube.internal:8443

	dns:

	  type: CoreDNS

	etcd:

	  local:

	    dataDir: /var/lib/minikube/etcd

	    extraArgs:

	      proxy-refresh-interval: "70000"

	kubernetesVersion: v1.18.20

	networking:

	  dnsDomain: cluster.local

	  podSubnet: "10.244.0.0/16"

	  serviceSubnet: 10.96.0.0/12

	---

	apiVersion: kubelet.config.k8s.io/v1beta1

	kind: KubeletConfiguration

	authentication:

	  x509:

	    clientCAFile: /var/lib/minikube/certs/ca.crt

	cgroupDriver: cgroupfs

	hairpinMode: hairpin-veth

	runtimeRequestTimeout: 15m

	clusterDomain: "cluster.local"

	# disable disk resource management by default

	imageGCHighThresholdPercent: 100

	evictionHard:

	  nodefs.available: "0%"

	  nodefs.inodesFree: "0%"

	  imagefs.available: "0%"

	failSwapOn: false

	staticPodPath: /etc/kubernetes/manifests

	---

	apiVersion: kubeproxy.config.k8s.io/v1alpha1

	kind: KubeProxyConfiguration

	clusterCIDR: "10.244.0.0/16"

	metricsBindAddress: 0.0.0.0:10249

	conntrack:

	  maxPerCore: 0

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"

	  tcpEstablishedTimeout: 0s

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"

	  tcpCloseWaitTimeout: 0s

	I0115 05:13:39.611743   68490 kubeadm.go:976] kubelet [Unit]

	Wants=docker.socket

	[Service]

	ExecStart=

	ExecStart=/var/lib/minikube/binaries/v1.18.20/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=ingress-addon-legacy-482000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.49.2

	[Install]

	 config:

	{KubernetesVersion:v1.18.20 ClusterName:ingress-addon-legacy-482000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}

	I0115 05:13:39.611822   68490 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.18.20

	I0115 05:13:39.620476   68490 binaries.go:44] Found k8s binaries, skipping transfer

	I0115 05:13:39.620547   68490 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube

	I0115 05:13:39.628698   68490 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (354 bytes)

	I0115 05:13:39.644176   68490 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (353 bytes)

	I0115 05:13:39.660198   68490 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2124 bytes)

	I0115 05:13:39.676072   68490 ssh_runner.go:195] Run: grep 192.168.49.2	control-plane.minikube.internal$ /etc/hosts

	I0115 05:13:39.680437   68490 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.49.2	control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0115 05:13:39.690632   68490 certs.go:56] Setting up /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000 for IP: 192.168.49.2

	I0115 05:13:39.690651   68490 certs.go:190] acquiring lock for shared ca certs: {Name:mk6c13ac30e7b90f60feff8c8c9de7894c05f68c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0115 05:13:39.690869   68490 certs.go:199] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/17953-64881/.minikube/ca.key

	I0115 05:13:39.690952   68490 certs.go:199] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/17953-64881/.minikube/proxy-client-ca.key

	I0115 05:13:39.691004   68490 certs.go:319] generating minikube-user signed cert: /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/client.key

	I0115 05:13:39.691021   68490 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/client.crt with IP's: []

	I0115 05:13:39.791494   68490 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/client.crt ...

	I0115 05:13:39.791507   68490 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/client.crt: {Name:mkec7f250db9d306b935715f1b6906d417032968 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0115 05:13:39.791867   68490 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/client.key ...

	I0115 05:13:39.791877   68490 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/client.key: {Name:mk10e8cf240ecb2db2c2a535cdc22006e4c68b5b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0115 05:13:39.792147   68490 certs.go:319] generating minikube signed cert: /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.key.dd3b5fb2

	I0115 05:13:39.792163   68490 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.crt.dd3b5fb2 with IP's: [192.168.49.2 10.96.0.1 127.0.0.1 10.0.0.1]

	I0115 05:13:39.886100   68490 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.crt.dd3b5fb2 ...

	I0115 05:13:39.886109   68490 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.crt.dd3b5fb2: {Name:mk503b842c6fe81bcaa07da5a286cac0cd30d6c9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0115 05:13:39.886351   68490 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.key.dd3b5fb2 ...

	I0115 05:13:39.886359   68490 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.key.dd3b5fb2: {Name:mkcdd6a7dab40037ce525f2f116e7b974f5425b0 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0115 05:13:39.886562   68490 certs.go:337] copying /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.crt.dd3b5fb2 -> /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.crt

	I0115 05:13:39.886739   68490 certs.go:341] copying /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.key.dd3b5fb2 -> /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.key

	I0115 05:13:39.886922   68490 certs.go:319] generating aggregator signed cert: /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/proxy-client.key

	I0115 05:13:39.886935   68490 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/proxy-client.crt with IP's: []

	I0115 05:13:39.955402   68490 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/proxy-client.crt ...

	I0115 05:13:39.955411   68490 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/proxy-client.crt: {Name:mk15271420b452085c597a55df0aee8291d34b1b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0115 05:13:39.955639   68490 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/proxy-client.key ...

	I0115 05:13:39.955648   68490 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/proxy-client.key: {Name:mkdb948c00b256404f2164e7a349d9b705989283 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0115 05:13:39.955845   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.crt -> /var/lib/minikube/certs/apiserver.crt

	I0115 05:13:39.955870   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.key -> /var/lib/minikube/certs/apiserver.key

	I0115 05:13:39.955891   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/proxy-client.crt -> /var/lib/minikube/certs/proxy-client.crt

	I0115 05:13:39.955933   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/proxy-client.key -> /var/lib/minikube/certs/proxy-client.key

	I0115 05:13:39.955951   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/ca.crt -> /var/lib/minikube/certs/ca.crt

	I0115 05:13:39.955969   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/ca.key -> /var/lib/minikube/certs/ca.key

	I0115 05:13:39.955987   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/proxy-client-ca.crt -> /var/lib/minikube/certs/proxy-client-ca.crt

	I0115 05:13:39.956003   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/proxy-client-ca.key -> /var/lib/minikube/certs/proxy-client-ca.key

	I0115 05:13:39.956111   68490 certs.go:437] found cert: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/65630.pem (1338 bytes)

	W0115 05:13:39.956157   68490 certs.go:433] ignoring /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/65630_empty.pem, impossibly tiny 0 bytes

	I0115 05:13:39.956166   68490 certs.go:437] found cert: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca-key.pem (1675 bytes)

	I0115 05:13:39.956197   68490 certs.go:437] found cert: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem (1082 bytes)

	I0115 05:13:39.956231   68490 certs.go:437] found cert: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem (1123 bytes)

	I0115 05:13:39.956263   68490 certs.go:437] found cert: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/Users/jenkins/minikube-integration/17953-64881/.minikube/certs/key.pem (1675 bytes)

	I0115 05:13:39.956331   68490 certs.go:437] found cert: /Users/jenkins/minikube-integration/17953-64881/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/17953-64881/.minikube/files/etc/ssl/certs/656302.pem (1708 bytes)

	I0115 05:13:39.956364   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/ca.crt -> /usr/share/ca-certificates/minikubeCA.pem

	I0115 05:13:39.956384   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/65630.pem -> /usr/share/ca-certificates/65630.pem

	I0115 05:13:39.956401   68490 vm_assets.go:163] NewFileAsset: /Users/jenkins/minikube-integration/17953-64881/.minikube/files/etc/ssl/certs/656302.pem -> /usr/share/ca-certificates/656302.pem

	I0115 05:13:39.956848   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)

	I0115 05:13:39.978775   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)

	I0115 05:13:39.999345   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)

	I0115 05:13:40.020320   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/ingress-addon-legacy-482000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)

	I0115 05:13:40.041114   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)

	I0115 05:13:40.062075   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)

	I0115 05:13:40.082643   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)

	I0115 05:13:40.103303   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)

	I0115 05:13:40.124503   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)

	I0115 05:13:40.145143   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/65630.pem --> /usr/share/ca-certificates/65630.pem (1338 bytes)

	I0115 05:13:40.166357   68490 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17953-64881/.minikube/files/etc/ssl/certs/656302.pem --> /usr/share/ca-certificates/656302.pem (1708 bytes)

	I0115 05:13:40.187022   68490 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)

	I0115 05:13:40.202603   68490 ssh_runner.go:195] Run: openssl version

	I0115 05:13:40.208489   68490 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"

	I0115 05:13:40.217600   68490 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem

	I0115 05:13:40.221987   68490 certs.go:480] hashing: -rw-r--r-- 1 root root 1111 Jan 15 13:02 /usr/share/ca-certificates/minikubeCA.pem

	I0115 05:13:40.222048   68490 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem

	I0115 05:13:40.228432   68490 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"

	I0115 05:13:40.237532   68490 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/65630.pem && ln -fs /usr/share/ca-certificates/65630.pem /etc/ssl/certs/65630.pem"

	I0115 05:13:40.246512   68490 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/65630.pem

	I0115 05:13:40.250910   68490 certs.go:480] hashing: -rw-r--r-- 1 root root 1338 Jan 15 13:07 /usr/share/ca-certificates/65630.pem

	I0115 05:13:40.250959   68490 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/65630.pem

	I0115 05:13:40.257873   68490 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/65630.pem /etc/ssl/certs/51391683.0"

	I0115 05:13:40.267015   68490 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/656302.pem && ln -fs /usr/share/ca-certificates/656302.pem /etc/ssl/certs/656302.pem"

	I0115 05:13:40.276377   68490 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/656302.pem

	I0115 05:13:40.280364   68490 certs.go:480] hashing: -rw-r--r-- 1 root root 1708 Jan 15 13:07 /usr/share/ca-certificates/656302.pem

	I0115 05:13:40.280408   68490 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/656302.pem

	I0115 05:13:40.286843   68490 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/656302.pem /etc/ssl/certs/3ec20f2e.0"

	I0115 05:13:40.296126   68490 ssh_runner.go:195] Run: ls /var/lib/minikube/certs/etcd

	I0115 05:13:40.300340   68490 certs.go:353] certs directory doesn't exist, likely first start: ls /var/lib/minikube/certs/etcd: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/var/lib/minikube/certs/etcd': No such file or directory

	I0115 05:13:40.300386   68490 kubeadm.go:404] StartCluster: {Name:ingress-addon-legacy-482000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 Memory:4096 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.18.20 ClusterName:ingress-addon-legacy-482000 Namespace:default APIServerName:minik

ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.18.20 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimization

s:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs:}

	I0115 05:13:40.300482   68490 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}

	I0115 05:13:40.319453   68490 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd

	I0115 05:13:40.328220   68490 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml

	I0115 05:13:40.336415   68490 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver

	I0115 05:13:40.336477   68490 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf

	I0115 05:13:40.344784   68490 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory

	I0115 05:13:40.344856   68490 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.18.20:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"

	I0115 05:13:40.398559   68490 kubeadm.go:322] [init] Using Kubernetes version: v1.18.20

	I0115 05:13:40.398649   68490 kubeadm.go:322] [preflight] Running pre-flight checks

	I0115 05:13:40.643731   68490 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster

	I0115 05:13:40.643810   68490 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection

	I0115 05:13:40.643882   68490 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	I0115 05:13:40.809008   68490 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	I0115 05:13:40.809772   68490 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	I0115 05:13:40.809806   68490 kubeadm.go:322] [kubelet-start] Starting the kubelet

	I0115 05:13:40.878341   68490 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"

	I0115 05:13:40.923678   68490 out.go:204]   - Generating certificates and keys ...

	I0115 05:13:40.923749   68490 kubeadm.go:322] [certs] Using existing ca certificate authority

	I0115 05:13:40.923844   68490 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk

	I0115 05:13:41.227800   68490 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key

	I0115 05:13:41.350653   68490 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key

	I0115 05:13:41.397161   68490 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key

	I0115 05:13:41.554434   68490 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key

	I0115 05:13:41.661300   68490 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key

	I0115 05:13:41.661505   68490 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [ingress-addon-legacy-482000 localhost] and IPs [192.168.49.2 127.0.0.1 ::1]

	I0115 05:13:41.742690   68490 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key

	I0115 05:13:41.742869   68490 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [ingress-addon-legacy-482000 localhost] and IPs [192.168.49.2 127.0.0.1 ::1]

	I0115 05:13:41.941844   68490 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key

	I0115 05:13:42.098476   68490 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key

	I0115 05:13:42.240757   68490 kubeadm.go:322] [certs] Generating "sa" key and public key

	I0115 05:13:42.240814   68490 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	I0115 05:13:42.538308   68490 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file

	I0115 05:13:42.686341   68490 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file

	I0115 05:13:42.785000   68490 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file

	I0115 05:13:42.902787   68490 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file

	I0115 05:13:42.903345   68490 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"

	I0115 05:13:42.924950   68490 out.go:204]   - Booting up control plane ...

	I0115 05:13:42.925057   68490 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"

	I0115 05:13:42.925162   68490 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"

	I0115 05:13:42.925266   68490 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"

	I0115 05:13:42.925363   68490 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	I0115 05:13:42.925533   68490 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	I0115 05:14:22.911004   68490 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.

	I0115 05:14:22.912142   68490 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.

	I0115 05:14:22.912367   68490 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	I0115 05:14:27.913195   68490 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.

	I0115 05:14:27.913422   68490 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	I0115 05:14:37.914806   68490 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.

	I0115 05:14:37.915016   68490 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	I0115 05:14:57.915231   68490 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.

	I0115 05:14:57.915546   68490 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	I0115 05:15:37.915365   68490 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.

	I0115 05:15:37.915595   68490 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	I0115 05:15:37.915620   68490 kubeadm.go:322] 

	I0115 05:15:37.915666   68490 kubeadm.go:322] 	Unfortunately, an error has occurred:

	I0115 05:15:37.915709   68490 kubeadm.go:322] 		timed out waiting for the condition

	I0115 05:15:37.915717   68490 kubeadm.go:322] 

	I0115 05:15:37.915766   68490 kubeadm.go:322] 	This error is likely caused by:

	I0115 05:15:37.915815   68490 kubeadm.go:322] 		- The kubelet is not running

	I0115 05:15:37.915937   68490 kubeadm.go:322] 		- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

	I0115 05:15:37.915947   68490 kubeadm.go:322] 

	I0115 05:15:37.916079   68490 kubeadm.go:322] 	If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:

	I0115 05:15:37.916135   68490 kubeadm.go:322] 		- 'systemctl status kubelet'

	I0115 05:15:37.916187   68490 kubeadm.go:322] 		- 'journalctl -xeu kubelet'

	I0115 05:15:37.916198   68490 kubeadm.go:322] 

	I0115 05:15:37.916373   68490 kubeadm.go:322] 	Additionally, a control plane component may have crashed or exited when started by the container runtime.

	I0115 05:15:37.916485   68490 kubeadm.go:322] 	To troubleshoot, list all containers using your preferred container runtimes CLI.

	I0115 05:15:37.916498   68490 kubeadm.go:322] 

	I0115 05:15:37.916608   68490 kubeadm.go:322] 	Here is one example how you may list all Kubernetes containers running in docker:

	I0115 05:15:37.916681   68490 kubeadm.go:322] 		- 'docker ps -a | grep kube | grep -v pause'

	I0115 05:15:37.916763   68490 kubeadm.go:322] 		Once you have found the failing container, you can inspect its logs with:

	I0115 05:15:37.916794   68490 kubeadm.go:322] 		- 'docker logs CONTAINERID'

	I0115 05:15:37.916805   68490 kubeadm.go:322] 

	I0115 05:15:37.917977   68490 kubeadm.go:322] W0115 13:13:40.398594    1693 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]

	I0115 05:15:37.918133   68490 kubeadm.go:322] 	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

	I0115 05:15:37.918224   68490 kubeadm.go:322] 	[WARNING Swap]: running with swap on is not supported. Please disable swap

	I0115 05:15:37.918370   68490 kubeadm.go:322] 	[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.7. Latest validated version: 19.03

	I0115 05:15:37.918474   68490 kubeadm.go:322] 	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	I0115 05:15:37.918582   68490 kubeadm.go:322] W0115 13:13:42.907987    1693 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	I0115 05:15:37.918686   68490 kubeadm.go:322] W0115 13:13:42.908829    1693 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	I0115 05:15:37.918752   68490 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster

	I0115 05:15:37.918823   68490 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher

	W0115 05:15:37.918923   68490 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.18.20:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1

	stdout:

	[init] Using Kubernetes version: v1.18.20

	[preflight] Running pre-flight checks

	[preflight] Pulling images required for setting up a Kubernetes cluster

	[preflight] This might take a minute or two, depending on the speed of your internet connection

	[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	[kubelet-start] Starting the kubelet

	[certs] Using certificateDir folder "/var/lib/minikube/certs"

	[certs] Using existing ca certificate authority

	[certs] Using existing apiserver certificate and key on disk

	[certs] Generating "apiserver-kubelet-client" certificate and key

	[certs] Generating "front-proxy-ca" certificate and key

	[certs] Generating "front-proxy-client" certificate and key

	[certs] Generating "etcd/ca" certificate and key

	[certs] Generating "etcd/server" certificate and key

	[certs] etcd/server serving cert is signed for DNS names [ingress-addon-legacy-482000 localhost] and IPs [192.168.49.2 127.0.0.1 ::1]

	[certs] Generating "etcd/peer" certificate and key

	[certs] etcd/peer serving cert is signed for DNS names [ingress-addon-legacy-482000 localhost] and IPs [192.168.49.2 127.0.0.1 ::1]

	[certs] Generating "etcd/healthcheck-client" certificate and key

	[certs] Generating "apiserver-etcd-client" certificate and key

	[certs] Generating "sa" key and public key

	[kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	[kubeconfig] Writing "admin.conf" kubeconfig file

	[kubeconfig] Writing "kubelet.conf" kubeconfig file

	[kubeconfig] Writing "controller-manager.conf" kubeconfig file

	[kubeconfig] Writing "scheduler.conf" kubeconfig file

	[control-plane] Using manifest folder "/etc/kubernetes/manifests"

	[control-plane] Creating static Pod manifest for "kube-apiserver"

	[control-plane] Creating static Pod manifest for "kube-controller-manager"

	[control-plane] Creating static Pod manifest for "kube-scheduler"

	[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	[kubelet-check] Initial timeout of 40s passed.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

		Unfortunately, an error has occurred:

			timed out waiting for the condition

		This error is likely caused by:

			- The kubelet is not running

			- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

		If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:

			- 'systemctl status kubelet'

			- 'journalctl -xeu kubelet'

		Additionally, a control plane component may have crashed or exited when started by the container runtime.

		To troubleshoot, list all containers using your preferred container runtimes CLI.

		Here is one example how you may list all Kubernetes containers running in docker:

			- 'docker ps -a | grep kube | grep -v pause'

			Once you have found the failing container, you can inspect its logs with:

			- 'docker logs CONTAINERID'

	stderr:

	W0115 13:13:40.398594    1693 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]

		[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

		[WARNING Swap]: running with swap on is not supported. Please disable swap

		[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.7. Latest validated version: 19.03

		[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	W0115 13:13:42.907987    1693 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	W0115 13:13:42.908829    1693 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster

	To see the stack trace of this error execute with --v=5 or higher

	! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.18.20:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1

	stdout:

	[init] Using Kubernetes version: v1.18.20

	[preflight] Running pre-flight checks

	[preflight] Pulling images required for setting up a Kubernetes cluster

	[preflight] This might take a minute or two, depending on the speed of your internet connection

	[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	[kubelet-start] Starting the kubelet

	[certs] Using certificateDir folder "/var/lib/minikube/certs"

	[certs] Using existing ca certificate authority

	[certs] Using existing apiserver certificate and key on disk

	[certs] Generating "apiserver-kubelet-client" certificate and key

	[certs] Generating "front-proxy-ca" certificate and key

	[certs] Generating "front-proxy-client" certificate and key

	[certs] Generating "etcd/ca" certificate and key

	[certs] Generating "etcd/server" certificate and key

	[certs] etcd/server serving cert is signed for DNS names [ingress-addon-legacy-482000 localhost] and IPs [192.168.49.2 127.0.0.1 ::1]

	[certs] Generating "etcd/peer" certificate and key

	[certs] etcd/peer serving cert is signed for DNS names [ingress-addon-legacy-482000 localhost] and IPs [192.168.49.2 127.0.0.1 ::1]

	[certs] Generating "etcd/healthcheck-client" certificate and key

	[certs] Generating "apiserver-etcd-client" certificate and key

	[certs] Generating "sa" key and public key

	[kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	[kubeconfig] Writing "admin.conf" kubeconfig file

	[kubeconfig] Writing "kubelet.conf" kubeconfig file

	[kubeconfig] Writing "controller-manager.conf" kubeconfig file

	[kubeconfig] Writing "scheduler.conf" kubeconfig file

	[control-plane] Using manifest folder "/etc/kubernetes/manifests"

	[control-plane] Creating static Pod manifest for "kube-apiserver"

	[control-plane] Creating static Pod manifest for "kube-controller-manager"

	[control-plane] Creating static Pod manifest for "kube-scheduler"

	[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	[kubelet-check] Initial timeout of 40s passed.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

		Unfortunately, an error has occurred:

			timed out waiting for the condition

		This error is likely caused by:

			- The kubelet is not running

			- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

		If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:

			- 'systemctl status kubelet'

			- 'journalctl -xeu kubelet'

		Additionally, a control plane component may have crashed or exited when started by the container runtime.

		To troubleshoot, list all containers using your preferred container runtimes CLI.

		Here is one example how you may list all Kubernetes containers running in docker:

			- 'docker ps -a | grep kube | grep -v pause'

			Once you have found the failing container, you can inspect its logs with:

			- 'docker logs CONTAINERID'

	stderr:

	W0115 13:13:40.398594    1693 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]

		[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

		[WARNING Swap]: running with swap on is not supported. Please disable swap

		[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.7. Latest validated version: 19.03

		[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	W0115 13:13:42.907987    1693 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	W0115 13:13:42.908829    1693 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster

	To see the stack trace of this error execute with --v=5 or higher

	I0115 05:15:37.918958   68490 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.18.20:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"

	I0115 05:15:38.324392   68490 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I0115 05:15:38.334882   68490 kubeadm.go:226] ignoring SystemVerification for kubeadm because of docker driver

	I0115 05:15:38.334946   68490 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf

	I0115 05:15:38.343146   68490 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory

	I0115 05:15:38.343180   68490 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.18.20:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"

	I0115 05:15:38.394995   68490 kubeadm.go:322] [init] Using Kubernetes version: v1.18.20

	I0115 05:15:38.395049   68490 kubeadm.go:322] [preflight] Running pre-flight checks

	I0115 05:15:38.623298   68490 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster

	I0115 05:15:38.623396   68490 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection

	I0115 05:15:38.623488   68490 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	I0115 05:15:38.794051   68490 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	I0115 05:15:38.794621   68490 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	I0115 05:15:38.794662   68490 kubeadm.go:322] [kubelet-start] Starting the kubelet

	I0115 05:15:38.866070   68490 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"

	I0115 05:15:38.888089   68490 out.go:204]   - Generating certificates and keys ...

	I0115 05:15:38.888163   68490 kubeadm.go:322] [certs] Using existing ca certificate authority

	I0115 05:15:38.888244   68490 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk

	I0115 05:15:38.888327   68490 kubeadm.go:322] [certs] Using existing apiserver-kubelet-client certificate and key on disk

	I0115 05:15:38.888385   68490 kubeadm.go:322] [certs] Using existing front-proxy-ca certificate authority

	I0115 05:15:38.888441   68490 kubeadm.go:322] [certs] Using existing front-proxy-client certificate and key on disk

	I0115 05:15:38.888482   68490 kubeadm.go:322] [certs] Using existing etcd/ca certificate authority

	I0115 05:15:38.888543   68490 kubeadm.go:322] [certs] Using existing etcd/server certificate and key on disk

	I0115 05:15:38.888610   68490 kubeadm.go:322] [certs] Using existing etcd/peer certificate and key on disk

	I0115 05:15:38.888715   68490 kubeadm.go:322] [certs] Using existing etcd/healthcheck-client certificate and key on disk

	I0115 05:15:38.888782   68490 kubeadm.go:322] [certs] Using existing apiserver-etcd-client certificate and key on disk

	I0115 05:15:38.888811   68490 kubeadm.go:322] [certs] Using the existing "sa" key

	I0115 05:15:38.888868   68490 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	I0115 05:15:39.070503   68490 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file

	I0115 05:15:39.221395   68490 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file

	I0115 05:15:39.431710   68490 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file

	I0115 05:15:39.554212   68490 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file

	I0115 05:15:39.554837   68490 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"

	I0115 05:15:39.576890   68490 out.go:204]   - Booting up control plane ...

	I0115 05:15:39.577041   68490 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"

	I0115 05:15:39.577175   68490 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"

	I0115 05:15:39.577301   68490 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"

	I0115 05:15:39.577426   68490 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	I0115 05:15:39.577687   68490 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	I0115 05:16:19.561445   68490 kubeadm.go:322] [kubelet-check] Initial timeout of 40s passed.

	I0115 05:16:19.562612   68490 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.

	I0115 05:16:19.562788   68490 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	I0115 05:16:24.563577   68490 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.

	I0115 05:16:24.563785   68490 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	I0115 05:16:34.564459   68490 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.

	I0115 05:16:34.564681   68490 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	I0115 05:16:54.564402   68490 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.

	I0115 05:16:54.564623   68490 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	I0115 05:17:34.563880   68490 kubeadm.go:322] [kubelet-check] It seems like the kubelet isn't running or healthy.

	I0115 05:17:34.564112   68490 kubeadm.go:322] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	I0115 05:17:34.564128   68490 kubeadm.go:322] 

	I0115 05:17:34.564193   68490 kubeadm.go:322] 	Unfortunately, an error has occurred:

	I0115 05:17:34.564260   68490 kubeadm.go:322] 		timed out waiting for the condition

	I0115 05:17:34.564274   68490 kubeadm.go:322] 

	I0115 05:17:34.564310   68490 kubeadm.go:322] 	This error is likely caused by:

	I0115 05:17:34.564347   68490 kubeadm.go:322] 		- The kubelet is not running

	I0115 05:17:34.564456   68490 kubeadm.go:322] 		- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

	I0115 05:17:34.564465   68490 kubeadm.go:322] 

	I0115 05:17:34.564588   68490 kubeadm.go:322] 	If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:

	I0115 05:17:34.564644   68490 kubeadm.go:322] 		- 'systemctl status kubelet'

	I0115 05:17:34.564686   68490 kubeadm.go:322] 		- 'journalctl -xeu kubelet'

	I0115 05:17:34.564695   68490 kubeadm.go:322] 

	I0115 05:17:34.564854   68490 kubeadm.go:322] 	Additionally, a control plane component may have crashed or exited when started by the container runtime.

	I0115 05:17:34.564992   68490 kubeadm.go:322] 	To troubleshoot, list all containers using your preferred container runtimes CLI.

	I0115 05:17:34.565009   68490 kubeadm.go:322] 

	I0115 05:17:34.565124   68490 kubeadm.go:322] 	Here is one example how you may list all Kubernetes containers running in docker:

	I0115 05:17:34.565186   68490 kubeadm.go:322] 		- 'docker ps -a | grep kube | grep -v pause'

	I0115 05:17:34.565293   68490 kubeadm.go:322] 		Once you have found the failing container, you can inspect its logs with:

	I0115 05:17:34.565335   68490 kubeadm.go:322] 		- 'docker logs CONTAINERID'

	I0115 05:17:34.565352   68490 kubeadm.go:322] 

	I0115 05:17:34.566660   68490 kubeadm.go:322] W0115 13:15:38.395198    4738 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]

	I0115 05:17:34.566850   68490 kubeadm.go:322] 	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

	I0115 05:17:34.566914   68490 kubeadm.go:322] 	[WARNING Swap]: running with swap on is not supported. Please disable swap

	I0115 05:17:34.567022   68490 kubeadm.go:322] 	[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.7. Latest validated version: 19.03

	I0115 05:17:34.567111   68490 kubeadm.go:322] 	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	I0115 05:17:34.567211   68490 kubeadm.go:322] W0115 13:15:39.560179    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	I0115 05:17:34.567318   68490 kubeadm.go:322] W0115 13:15:39.560851    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	I0115 05:17:34.567385   68490 kubeadm.go:322] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster

	I0115 05:17:34.567447   68490 kubeadm.go:322] To see the stack trace of this error execute with --v=5 or higher

	I0115 05:17:34.567485   68490 kubeadm.go:406] StartCluster complete in 3m54.279969776s

	I0115 05:17:34.567569   68490 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}

	I0115 05:17:34.585198   68490 logs.go:284] 0 containers: []

	W0115 05:17:34.585215   68490 logs.go:286] No container was found matching "kube-apiserver"

	I0115 05:17:34.585284   68490 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}

	I0115 05:17:34.604638   68490 logs.go:284] 0 containers: []

	W0115 05:17:34.604653   68490 logs.go:286] No container was found matching "etcd"

	I0115 05:17:34.604722   68490 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}

	I0115 05:17:34.621896   68490 logs.go:284] 0 containers: []

	W0115 05:17:34.621911   68490 logs.go:286] No container was found matching "coredns"

	I0115 05:17:34.621979   68490 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}

	I0115 05:17:34.639827   68490 logs.go:284] 0 containers: []

	W0115 05:17:34.639840   68490 logs.go:286] No container was found matching "kube-scheduler"

	I0115 05:17:34.639905   68490 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}

	I0115 05:17:34.658517   68490 logs.go:284] 0 containers: []

	W0115 05:17:34.658531   68490 logs.go:286] No container was found matching "kube-proxy"

	I0115 05:17:34.658598   68490 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}

	I0115 05:17:34.677607   68490 logs.go:284] 0 containers: []

	W0115 05:17:34.677621   68490 logs.go:286] No container was found matching "kube-controller-manager"

	I0115 05:17:34.677689   68490 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}

	I0115 05:17:34.696195   68490 logs.go:284] 0 containers: []

	W0115 05:17:34.696211   68490 logs.go:286] No container was found matching "kindnet"

	I0115 05:17:34.696219   68490 logs.go:123] Gathering logs for kubelet ...

	I0115 05:17:34.696227   68490 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"

	I0115 05:17:34.731785   68490 logs.go:123] Gathering logs for dmesg ...

	I0115 05:17:34.731805   68490 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"

	I0115 05:17:34.744197   68490 logs.go:123] Gathering logs for describe nodes ...

	I0115 05:17:34.744211   68490 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.18.20/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"

	W0115 05:17:34.817098   68490 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.18.20/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.18.20/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	 output: 

	** stderr ** 

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	** /stderr **

	I0115 05:17:34.817117   68490 logs.go:123] Gathering logs for Docker ...

	I0115 05:17:34.817124   68490 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"

	I0115 05:17:34.832330   68490 logs.go:123] Gathering logs for container status ...

	I0115 05:17:34.832346   68490 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"

	W0115 05:17:34.908173   68490 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.18.20:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1

	stdout:

	[init] Using Kubernetes version: v1.18.20

	[preflight] Running pre-flight checks

	[preflight] Pulling images required for setting up a Kubernetes cluster

	[preflight] This might take a minute or two, depending on the speed of your internet connection

	[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	[kubelet-start] Starting the kubelet

	[certs] Using certificateDir folder "/var/lib/minikube/certs"

	[certs] Using existing ca certificate authority

	[certs] Using existing apiserver certificate and key on disk

	[certs] Using existing apiserver-kubelet-client certificate and key on disk

	[certs] Using existing front-proxy-ca certificate authority

	[certs] Using existing front-proxy-client certificate and key on disk

	[certs] Using existing etcd/ca certificate authority

	[certs] Using existing etcd/server certificate and key on disk

	[certs] Using existing etcd/peer certificate and key on disk

	[certs] Using existing etcd/healthcheck-client certificate and key on disk

	[certs] Using existing apiserver-etcd-client certificate and key on disk

	[certs] Using the existing "sa" key

	[kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	[kubeconfig] Writing "admin.conf" kubeconfig file

	[kubeconfig] Writing "kubelet.conf" kubeconfig file

	[kubeconfig] Writing "controller-manager.conf" kubeconfig file

	[kubeconfig] Writing "scheduler.conf" kubeconfig file

	[control-plane] Using manifest folder "/etc/kubernetes/manifests"

	[control-plane] Creating static Pod manifest for "kube-apiserver"

	[control-plane] Creating static Pod manifest for "kube-controller-manager"

	[control-plane] Creating static Pod manifest for "kube-scheduler"

	[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	[kubelet-check] Initial timeout of 40s passed.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

		Unfortunately, an error has occurred:

			timed out waiting for the condition

		This error is likely caused by:

			- The kubelet is not running

			- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

		If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:

			- 'systemctl status kubelet'

			- 'journalctl -xeu kubelet'

		Additionally, a control plane component may have crashed or exited when started by the container runtime.

		To troubleshoot, list all containers using your preferred container runtimes CLI.

		Here is one example how you may list all Kubernetes containers running in docker:

			- 'docker ps -a | grep kube | grep -v pause'

			Once you have found the failing container, you can inspect its logs with:

			- 'docker logs CONTAINERID'

	stderr:

	W0115 13:15:38.395198    4738 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]

		[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

		[WARNING Swap]: running with swap on is not supported. Please disable swap

		[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.7. Latest validated version: 19.03

		[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	W0115 13:15:39.560179    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	W0115 13:15:39.560851    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster

	To see the stack trace of this error execute with --v=5 or higher

	W0115 05:17:34.908196   68490 out.go:239] * 

	* 

	W0115 05:17:34.908241   68490 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.18.20:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1

	stdout:

	[init] Using Kubernetes version: v1.18.20

	[preflight] Running pre-flight checks

	[preflight] Pulling images required for setting up a Kubernetes cluster

	[preflight] This might take a minute or two, depending on the speed of your internet connection

	[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	[kubelet-start] Starting the kubelet

	[certs] Using certificateDir folder "/var/lib/minikube/certs"

	[certs] Using existing ca certificate authority

	[certs] Using existing apiserver certificate and key on disk

	[certs] Using existing apiserver-kubelet-client certificate and key on disk

	[certs] Using existing front-proxy-ca certificate authority

	[certs] Using existing front-proxy-client certificate and key on disk

	[certs] Using existing etcd/ca certificate authority

	[certs] Using existing etcd/server certificate and key on disk

	[certs] Using existing etcd/peer certificate and key on disk

	[certs] Using existing etcd/healthcheck-client certificate and key on disk

	[certs] Using existing apiserver-etcd-client certificate and key on disk

	[certs] Using the existing "sa" key

	[kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	[kubeconfig] Writing "admin.conf" kubeconfig file

	[kubeconfig] Writing "kubelet.conf" kubeconfig file

	[kubeconfig] Writing "controller-manager.conf" kubeconfig file

	[kubeconfig] Writing "scheduler.conf" kubeconfig file

	[control-plane] Using manifest folder "/etc/kubernetes/manifests"

	[control-plane] Creating static Pod manifest for "kube-apiserver"

	[control-plane] Creating static Pod manifest for "kube-controller-manager"

	[control-plane] Creating static Pod manifest for "kube-scheduler"

	[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	[kubelet-check] Initial timeout of 40s passed.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

		Unfortunately, an error has occurred:

			timed out waiting for the condition

		This error is likely caused by:

			- The kubelet is not running

			- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

		If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:

			- 'systemctl status kubelet'

			- 'journalctl -xeu kubelet'

		Additionally, a control plane component may have crashed or exited when started by the container runtime.

		To troubleshoot, list all containers using your preferred container runtimes CLI.

		Here is one example how you may list all Kubernetes containers running in docker:

			- 'docker ps -a | grep kube | grep -v pause'

			Once you have found the failing container, you can inspect its logs with:

			- 'docker logs CONTAINERID'

	stderr:

	W0115 13:15:38.395198    4738 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]

		[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

		[WARNING Swap]: running with swap on is not supported. Please disable swap

		[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.7. Latest validated version: 19.03

		[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	W0115 13:15:39.560179    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	W0115 13:15:39.560851    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster

	To see the stack trace of this error execute with --v=5 or higher

	X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.18.20:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1

	stdout:

	[init] Using Kubernetes version: v1.18.20

	[preflight] Running pre-flight checks

	[preflight] Pulling images required for setting up a Kubernetes cluster

	[preflight] This might take a minute or two, depending on the speed of your internet connection

	[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	[kubelet-start] Starting the kubelet

	[certs] Using certificateDir folder "/var/lib/minikube/certs"

	[certs] Using existing ca certificate authority

	[certs] Using existing apiserver certificate and key on disk

	[certs] Using existing apiserver-kubelet-client certificate and key on disk

	[certs] Using existing front-proxy-ca certificate authority

	[certs] Using existing front-proxy-client certificate and key on disk

	[certs] Using existing etcd/ca certificate authority

	[certs] Using existing etcd/server certificate and key on disk

	[certs] Using existing etcd/peer certificate and key on disk

	[certs] Using existing etcd/healthcheck-client certificate and key on disk

	[certs] Using existing apiserver-etcd-client certificate and key on disk

	[certs] Using the existing "sa" key

	[kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	[kubeconfig] Writing "admin.conf" kubeconfig file

	[kubeconfig] Writing "kubelet.conf" kubeconfig file

	[kubeconfig] Writing "controller-manager.conf" kubeconfig file

	[kubeconfig] Writing "scheduler.conf" kubeconfig file

	[control-plane] Using manifest folder "/etc/kubernetes/manifests"

	[control-plane] Creating static Pod manifest for "kube-apiserver"

	[control-plane] Creating static Pod manifest for "kube-controller-manager"

	[control-plane] Creating static Pod manifest for "kube-scheduler"

	[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	[kubelet-check] Initial timeout of 40s passed.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

		Unfortunately, an error has occurred:

			timed out waiting for the condition

		This error is likely caused by:

			- The kubelet is not running

			- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

		If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:

			- 'systemctl status kubelet'

			- 'journalctl -xeu kubelet'

		Additionally, a control plane component may have crashed or exited when started by the container runtime.

		To troubleshoot, list all containers using your preferred container runtimes CLI.

		Here is one example how you may list all Kubernetes containers running in docker:

			- 'docker ps -a | grep kube | grep -v pause'

			Once you have found the failing container, you can inspect its logs with:

			- 'docker logs CONTAINERID'

	stderr:

	W0115 13:15:38.395198    4738 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]

		[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

		[WARNING Swap]: running with swap on is not supported. Please disable swap

		[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.7. Latest validated version: 19.03

		[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	W0115 13:15:39.560179    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	W0115 13:15:39.560851    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster

	To see the stack trace of this error execute with --v=5 or higher

	W0115 05:17:34.908256   68490 out.go:239] * 

	* 

	W0115 05:17:34.908912   68490 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0115 05:17:34.993714   68490 out.go:177] 

	W0115 05:17:35.036202   68490 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.18.20:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1

	stdout:

	[init] Using Kubernetes version: v1.18.20

	[preflight] Running pre-flight checks

	[preflight] Pulling images required for setting up a Kubernetes cluster

	[preflight] This might take a minute or two, depending on the speed of your internet connection

	[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	[kubelet-start] Starting the kubelet

	[certs] Using certificateDir folder "/var/lib/minikube/certs"

	[certs] Using existing ca certificate authority

	[certs] Using existing apiserver certificate and key on disk

	[certs] Using existing apiserver-kubelet-client certificate and key on disk

	[certs] Using existing front-proxy-ca certificate authority

	[certs] Using existing front-proxy-client certificate and key on disk

	[certs] Using existing etcd/ca certificate authority

	[certs] Using existing etcd/server certificate and key on disk

	[certs] Using existing etcd/peer certificate and key on disk

	[certs] Using existing etcd/healthcheck-client certificate and key on disk

	[certs] Using existing apiserver-etcd-client certificate and key on disk

	[certs] Using the existing "sa" key

	[kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	[kubeconfig] Writing "admin.conf" kubeconfig file

	[kubeconfig] Writing "kubelet.conf" kubeconfig file

	[kubeconfig] Writing "controller-manager.conf" kubeconfig file

	[kubeconfig] Writing "scheduler.conf" kubeconfig file

	[control-plane] Using manifest folder "/etc/kubernetes/manifests"

	[control-plane] Creating static Pod manifest for "kube-apiserver"

	[control-plane] Creating static Pod manifest for "kube-controller-manager"

	[control-plane] Creating static Pod manifest for "kube-scheduler"

	[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	[kubelet-check] Initial timeout of 40s passed.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

		Unfortunately, an error has occurred:

			timed out waiting for the condition

		This error is likely caused by:

			- The kubelet is not running

			- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

		If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:

			- 'systemctl status kubelet'

			- 'journalctl -xeu kubelet'

		Additionally, a control plane component may have crashed or exited when started by the container runtime.

		To troubleshoot, list all containers using your preferred container runtimes CLI.

		Here is one example how you may list all Kubernetes containers running in docker:

			- 'docker ps -a | grep kube | grep -v pause'

			Once you have found the failing container, you can inspect its logs with:

			- 'docker logs CONTAINERID'

	stderr:

	W0115 13:15:38.395198    4738 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]

		[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

		[WARNING Swap]: running with swap on is not supported. Please disable swap

		[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.7. Latest validated version: 19.03

		[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	W0115 13:15:39.560179    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	W0115 13:15:39.560851    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster

	To see the stack trace of this error execute with --v=5 or higher

	X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.18.20:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1

	stdout:

	[init] Using Kubernetes version: v1.18.20

	[preflight] Running pre-flight checks

	[preflight] Pulling images required for setting up a Kubernetes cluster

	[preflight] This might take a minute or two, depending on the speed of your internet connection

	[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	[kubelet-start] Starting the kubelet

	[certs] Using certificateDir folder "/var/lib/minikube/certs"

	[certs] Using existing ca certificate authority

	[certs] Using existing apiserver certificate and key on disk

	[certs] Using existing apiserver-kubelet-client certificate and key on disk

	[certs] Using existing front-proxy-ca certificate authority

	[certs] Using existing front-proxy-client certificate and key on disk

	[certs] Using existing etcd/ca certificate authority

	[certs] Using existing etcd/server certificate and key on disk

	[certs] Using existing etcd/peer certificate and key on disk

	[certs] Using existing etcd/healthcheck-client certificate and key on disk

	[certs] Using existing apiserver-etcd-client certificate and key on disk

	[certs] Using the existing "sa" key

	[kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	[kubeconfig] Writing "admin.conf" kubeconfig file

	[kubeconfig] Writing "kubelet.conf" kubeconfig file

	[kubeconfig] Writing "controller-manager.conf" kubeconfig file

	[kubeconfig] Writing "scheduler.conf" kubeconfig file

	[control-plane] Using manifest folder "/etc/kubernetes/manifests"

	[control-plane] Creating static Pod manifest for "kube-apiserver"

	[control-plane] Creating static Pod manifest for "kube-controller-manager"

	[control-plane] Creating static Pod manifest for "kube-scheduler"

	[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	[kubelet-check] Initial timeout of 40s passed.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

	[kubelet-check] It seems like the kubelet isn't running or healthy.

	[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused.

		Unfortunately, an error has occurred:

			timed out waiting for the condition

		This error is likely caused by:

			- The kubelet is not running

			- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

		If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:

			- 'systemctl status kubelet'

			- 'journalctl -xeu kubelet'

		Additionally, a control plane component may have crashed or exited when started by the container runtime.

		To troubleshoot, list all containers using your preferred container runtimes CLI.

		Here is one example how you may list all Kubernetes containers running in docker:

			- 'docker ps -a | grep kube | grep -v pause'

			Once you have found the failing container, you can inspect its logs with:

			- 'docker logs CONTAINERID'

	stderr:

	W0115 13:15:38.395198    4738 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]

		[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

		[WARNING Swap]: running with swap on is not supported. Please disable swap

		[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.7. Latest validated version: 19.03

		[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	W0115 13:15:39.560179    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	W0115 13:15:39.560851    4738 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"

	error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster

	To see the stack trace of this error execute with --v=5 or higher

	W0115 05:17:35.036273   68490 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start

	* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start

	W0115 05:17:35.036308   68490 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172

	* Related issue: https://github.com/kubernetes/minikube/issues/4172

	I0115 05:17:35.057739   68490 out.go:177] 

** /stderr **

ingress_addon_legacy_test.go:41: failed to start minikube with args: "out/minikube-darwin-amd64 start -p ingress-addon-legacy-482000 --kubernetes-version=v1.18.20 --memory=4096 --wait=true --alsologtostderr -v=5 --driver=docker " : exit status 109

--- FAIL: TestIngressAddonLegacy/StartLegacyK8sCluster (257.71s)

=== RUN   TestIngressAddonLegacy/serial/ValidateIngressAddonActivation

ingress_addon_legacy_test.go:70: (dbg) Run:  out/minikube-darwin-amd64 -p ingress-addon-legacy-482000 addons enable ingress --alsologtostderr -v=5

E0115 05:18:40.598253   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

ingress_addon_legacy_test.go:70: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p ingress-addon-legacy-482000 addons enable ingress --alsologtostderr -v=5: exit status 10 (1m47.325531837s)

-- stdout --

	* ingress is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.

	You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS

	* After the addon is enabled, please run "minikube tunnel" and your ingress resources would be available at "127.0.0.1"

	  - Using image docker.io/jettech/kube-webhook-certgen:v1.5.1

	  - Using image docker.io/jettech/kube-webhook-certgen:v1.5.1

	  - Using image registry.k8s.io/ingress-nginx/controller:v0.49.3

	* Verifying ingress addon...

-- /stdout --

** stderr ** 

	I0115 05:17:35.216953   68643 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:17:35.218286   68643 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:17:35.218294   68643 out.go:309] Setting ErrFile to fd 2...

	I0115 05:17:35.218300   68643 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:17:35.218502   68643 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:17:35.218880   68643 mustload.go:65] Loading cluster: ingress-addon-legacy-482000

	I0115 05:17:35.219269   68643 config.go:182] Loaded profile config "ingress-addon-legacy-482000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.18.20

	I0115 05:17:35.219287   68643 addons.go:597] checking whether the cluster is paused

	I0115 05:17:35.219445   68643 config.go:182] Loaded profile config "ingress-addon-legacy-482000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.18.20

	I0115 05:17:35.219483   68643 host.go:66] Checking if "ingress-addon-legacy-482000" exists ...

	I0115 05:17:35.220098   68643 cli_runner.go:164] Run: docker container inspect ingress-addon-legacy-482000 --format={{.State.Status}}

	I0115 05:17:35.276243   68643 ssh_runner.go:195] Run: systemctl --version

	I0115 05:17:35.276339   68643 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:17:35.327798   68643 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:54699 SSHKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa Username:docker}

	I0115 05:17:35.418264   68643 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}

	I0115 05:17:35.457989   68643 out.go:177] * ingress is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.

	You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS

	I0115 05:17:35.479651   68643 config.go:182] Loaded profile config "ingress-addon-legacy-482000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.18.20

	I0115 05:17:35.479663   68643 addons.go:69] Setting ingress=true in profile "ingress-addon-legacy-482000"

	I0115 05:17:35.479669   68643 addons.go:234] Setting addon ingress=true in "ingress-addon-legacy-482000"

	I0115 05:17:35.479697   68643 host.go:66] Checking if "ingress-addon-legacy-482000" exists ...

	I0115 05:17:35.479996   68643 cli_runner.go:164] Run: docker container inspect ingress-addon-legacy-482000 --format={{.State.Status}}

	I0115 05:17:35.550525   68643 out.go:177] * After the addon is enabled, please run "minikube tunnel" and your ingress resources would be available at "127.0.0.1"

	I0115 05:17:35.571381   68643 out.go:177]   - Using image docker.io/jettech/kube-webhook-certgen:v1.5.1

	I0115 05:17:35.592457   68643 out.go:177]   - Using image docker.io/jettech/kube-webhook-certgen:v1.5.1

	I0115 05:17:35.615622   68643 out.go:177]   - Using image registry.k8s.io/ingress-nginx/controller:v0.49.3

	I0115 05:17:35.636684   68643 addons.go:426] installing /etc/kubernetes/addons/ingress-deploy.yaml

	I0115 05:17:35.636707   68643 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ingress-deploy.yaml (15618 bytes)

	I0115 05:17:35.636833   68643 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:17:35.687616   68643 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:54699 SSHKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa Username:docker}

	I0115 05:17:35.786627   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:17:35.833782   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:35.833815   68643 retry.go:31] will retry after 371.564893ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:36.205983   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:17:36.254710   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:36.254739   68643 retry.go:31] will retry after 494.271308ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:36.749475   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:17:36.804486   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:36.804506   68643 retry.go:31] will retry after 340.184796ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:37.145073   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:17:37.196283   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:37.196304   68643 retry.go:31] will retry after 707.835729ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:37.904981   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:17:37.954333   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:37.954358   68643 retry.go:31] will retry after 642.095497ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:38.596784   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:17:38.660136   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:38.660155   68643 retry.go:31] will retry after 1.599241496s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:40.261486   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:17:40.313181   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:40.313205   68643 retry.go:31] will retry after 2.3846559s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:42.697901   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:17:42.768454   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:42.768471   68643 retry.go:31] will retry after 2.195307932s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:44.963784   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:17:45.013701   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:45.013719   68643 retry.go:31] will retry after 9.242354262s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:54.256702   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:17:54.317957   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:54.317974   68643 retry.go:31] will retry after 5.291386123s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:59.609387   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:17:59.661077   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:17:59.661094   68643 retry.go:31] will retry after 15.578437605s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:18:15.239312   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:18:15.291316   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:18:15.291333   68643 retry.go:31] will retry after 18.710892223s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:18:34.002674   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:18:34.053895   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:18:34.053919   68643 retry.go:31] will retry after 48.182365264s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:22.233915   68643 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml

	W0115 05:19:22.363608   68643 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:22.363633   68643 addons.go:470] Verifying addon ingress=true in "ingress-addon-legacy-482000"

	I0115 05:19:22.385215   68643 out.go:177] * Verifying ingress addon...

	I0115 05:19:22.407281   68643 out.go:177] 

	W0115 05:19:22.428762   68643 out.go:239] X Exiting due to MK_ADDON_ENABLE: enable failed: run callbacks: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	 get kube-client to validate ingress addon: client config: context "ingress-addon-legacy-482000" does not exist: client config: context "ingress-addon-legacy-482000" does not exist]

	X Exiting due to MK_ADDON_ENABLE: enable failed: run callbacks: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-deploy.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	 get kube-client to validate ingress addon: client config: context "ingress-addon-legacy-482000" does not exist: client config: context "ingress-addon-legacy-482000" does not exist]

	W0115 05:19:22.428779   68643 out.go:239] * 

	* 

	W0115 05:19:22.435495   68643 out.go:239] ╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                           │

	│    * If the above advice does not help, please let us know:                                                               │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                             │

	│                                                                                                                           │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                  │

	│    * Please also attach the following file to the GitHub issue:                                                           │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_addons_ecab7b1157b569c129811d3c2b680fbca2a6f3d2_0.log    │

	│                                                                                                                           │

	╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                           │

	│    * If the above advice does not help, please let us know:                                                               │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                             │

	│                                                                                                                           │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                  │

	│    * Please also attach the following file to the GitHub issue:                                                           │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_addons_ecab7b1157b569c129811d3c2b680fbca2a6f3d2_0.log    │

	│                                                                                                                           │

	╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	I0115 05:19:22.456940   68643 out.go:177] 

** /stderr **

ingress_addon_legacy_test.go:71: failed to enable ingress addon: exit status 10

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestIngressAddonLegacy/serial/ValidateIngressAddonActivation]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect ingress-addon-legacy-482000

helpers_test.go:235: (dbg) docker inspect ingress-addon-legacy-482000:

-- stdout --

	[

	    {

	        "Id": "01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5",

	        "Created": "2024-01-15T13:13:25.527112214Z",

	        "Path": "/usr/local/bin/entrypoint",

	        "Args": [

	            "/sbin/init"

	        ],

	        "State": {

	            "Status": "running",

	            "Running": true,

	            "Paused": false,

	            "Restarting": false,

	            "OOMKilled": false,

	            "Dead": false,

	            "Pid": 51552,

	            "ExitCode": 0,

	            "Error": "",

	            "StartedAt": "2024-01-15T13:13:25.739041481Z",

	            "FinishedAt": "0001-01-01T00:00:00Z"

	        },

	        "Image": "sha256:9941de2e064a4a6a7155bfc66cedd2854b8c725b77bb8d4eaf81bef39f951dd7",

	        "ResolvConfPath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/resolv.conf",

	        "HostnamePath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/hostname",

	        "HostsPath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/hosts",

	        "LogPath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5-json.log",

	        "Name": "/ingress-addon-legacy-482000",

	        "RestartCount": 0,

	        "Driver": "overlay2",

	        "Platform": "linux",

	        "MountLabel": "",

	        "ProcessLabel": "",

	        "AppArmorProfile": "",

	        "ExecIDs": null,

	        "HostConfig": {

	            "Binds": [

	                "ingress-addon-legacy-482000:/var",

	                "/lib/modules:/lib/modules:ro"

	            ],

	            "ContainerIDFile": "",

	            "LogConfig": {

	                "Type": "json-file",

	                "Config": {}

	            },

	            "NetworkMode": "ingress-addon-legacy-482000",

	            "PortBindings": {

	                "22/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "2376/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "32443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "5000/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "8443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ]

	            },

	            "RestartPolicy": {

	                "Name": "no",

	                "MaximumRetryCount": 0

	            },

	            "AutoRemove": false,

	            "VolumeDriver": "",

	            "VolumesFrom": null,

	            "ConsoleSize": [

	                0,

	                0

	            ],

	            "CapAdd": null,

	            "CapDrop": null,

	            "CgroupnsMode": "private",

	            "Dns": [],

	            "DnsOptions": [],

	            "DnsSearch": [],

	            "ExtraHosts": null,

	            "GroupAdd": null,

	            "IpcMode": "private",

	            "Cgroup": "",

	            "Links": null,

	            "OomScoreAdj": 0,

	            "PidMode": "",

	            "Privileged": true,

	            "PublishAllPorts": false,

	            "ReadonlyRootfs": false,

	            "SecurityOpt": [

	                "seccomp=unconfined",

	                "apparmor=unconfined",

	                "label=disable"

	            ],

	            "Tmpfs": {

	                "/run": "",

	                "/tmp": ""

	            },

	            "UTSMode": "",

	            "UsernsMode": "",

	            "ShmSize": 67108864,

	            "Runtime": "runc",

	            "Isolation": "",

	            "CpuShares": 0,

	            "Memory": 4294967296,

	            "NanoCpus": 2000000000,

	            "CgroupParent": "",

	            "BlkioWeight": 0,

	            "BlkioWeightDevice": [],

	            "BlkioDeviceReadBps": [],

	            "BlkioDeviceWriteBps": [],

	            "BlkioDeviceReadIOps": [],

	            "BlkioDeviceWriteIOps": [],

	            "CpuPeriod": 0,

	            "CpuQuota": 0,

	            "CpuRealtimePeriod": 0,

	            "CpuRealtimeRuntime": 0,

	            "CpusetCpus": "",

	            "CpusetMems": "",

	            "Devices": [],

	            "DeviceCgroupRules": null,

	            "DeviceRequests": null,

	            "MemoryReservation": 0,

	            "MemorySwap": 4294967296,

	            "MemorySwappiness": null,

	            "OomKillDisable": null,

	            "PidsLimit": null,

	            "Ulimits": null,

	            "CpuCount": 0,

	            "CpuPercent": 0,

	            "IOMaximumIOps": 0,

	            "IOMaximumBandwidth": 0,

	            "MaskedPaths": null,

	            "ReadonlyPaths": null

	        },

	        "GraphDriver": {

	            "Data": {

	                "LowerDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63-init/diff:/var/lib/docker/overlay2/b3cb78fe399645181979e767fd2b27916778197e6245b2db21b3eb1fe7dda1f5/diff",

	                "MergedDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63/merged",

	                "UpperDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63/diff",

	                "WorkDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63/work"

	            },

	            "Name": "overlay2"

	        },

	        "Mounts": [

	            {

	                "Type": "volume",

	                "Name": "ingress-addon-legacy-482000",

	                "Source": "/var/lib/docker/volumes/ingress-addon-legacy-482000/_data",

	                "Destination": "/var",

	                "Driver": "local",

	                "Mode": "z",

	                "RW": true,

	                "Propagation": ""

	            },

	            {

	                "Type": "bind",

	                "Source": "/lib/modules",

	                "Destination": "/lib/modules",

	                "Mode": "ro",

	                "RW": false,

	                "Propagation": "rprivate"

	            }

	        ],

	        "Config": {

	            "Hostname": "ingress-addon-legacy-482000",

	            "Domainname": "",

	            "User": "",

	            "AttachStdin": false,

	            "AttachStdout": false,

	            "AttachStderr": false,

	            "ExposedPorts": {

	                "22/tcp": {},

	                "2376/tcp": {},

	                "32443/tcp": {},

	                "5000/tcp": {},

	                "8443/tcp": {}

	            },

	            "Tty": true,

	            "OpenStdin": false,

	            "StdinOnce": false,

	            "Env": [

	                "container=docker",

	                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

	            ],

	            "Cmd": null,

	            "Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0",

	            "Volumes": null,

	            "WorkingDir": "/",

	            "Entrypoint": [

	                "/usr/local/bin/entrypoint",

	                "/sbin/init"

	            ],

	            "OnBuild": null,

	            "Labels": {

	                "created_by.minikube.sigs.k8s.io": "true",

	                "mode.minikube.sigs.k8s.io": "ingress-addon-legacy-482000",

	                "name.minikube.sigs.k8s.io": "ingress-addon-legacy-482000",

	                "role.minikube.sigs.k8s.io": ""

	            },

	            "StopSignal": "SIGRTMIN+3"

	        },

	        "NetworkSettings": {

	            "Bridge": "",

	            "SandboxID": "c52d563e1da90ce414962fa1df8d9f1f7b3e7e151030bd1bd475447b08ef956f",

	            "HairpinMode": false,

	            "LinkLocalIPv6Address": "",

	            "LinkLocalIPv6PrefixLen": 0,

	            "Ports": {

	                "22/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54699"

	                    }

	                ],

	                "2376/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54695"

	                    }

	                ],

	                "32443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54696"

	                    }

	                ],

	                "5000/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54697"

	                    }

	                ],

	                "8443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54698"

	                    }

	                ]

	            },

	            "SandboxKey": "/var/run/docker/netns/c52d563e1da9",

	            "SecondaryIPAddresses": null,

	            "SecondaryIPv6Addresses": null,

	            "EndpointID": "",

	            "Gateway": "",

	            "GlobalIPv6Address": "",

	            "GlobalIPv6PrefixLen": 0,

	            "IPAddress": "",

	            "IPPrefixLen": 0,

	            "IPv6Gateway": "",

	            "MacAddress": "",

	            "Networks": {

	                "ingress-addon-legacy-482000": {

	                    "IPAMConfig": {

	                        "IPv4Address": "192.168.49.2"

	                    },

	                    "Links": null,

	                    "Aliases": [

	                        "01e6adecdd3d",

	                        "ingress-addon-legacy-482000"

	                    ],

	                    "MacAddress": "02:42:c0:a8:31:02",

	                    "NetworkID": "1736f1945b5c282c51c1427aefc8d7c5b95fb9883f9d2b3660576aa7e5f3df5e",

	                    "EndpointID": "d7fd6caa78b65da813d9019e0f38370b5eed09e2397f731aa620e12eae53c3e3",

	                    "Gateway": "192.168.49.1",

	                    "IPAddress": "192.168.49.2",

	                    "IPPrefixLen": 24,

	                    "IPv6Gateway": "",

	                    "GlobalIPv6Address": "",

	                    "GlobalIPv6PrefixLen": 0,

	                    "DriverOpts": null

	                }

	            }

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p ingress-addon-legacy-482000 -n ingress-addon-legacy-482000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p ingress-addon-legacy-482000 -n ingress-addon-legacy-482000: exit status 6 (377.659416ms)

-- stdout --

	Running

	WARNING: Your kubectl is pointing to stale minikube-vm.

	To fix the kubectl context, run `minikube update-context`

-- /stdout --

** stderr ** 

	E0115 05:19:22.901673   68667 status.go:415] kubeconfig endpoint: extract IP: "ingress-addon-legacy-482000" does not appear in /Users/jenkins/minikube-integration/17953-64881/kubeconfig

** /stderr **

helpers_test.go:239: status error: exit status 6 (may be ok)

helpers_test.go:241: "ingress-addon-legacy-482000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")

--- FAIL: TestIngressAddonLegacy/serial/ValidateIngressAddonActivation (107.76s)

=== RUN   TestIngressAddonLegacy/serial/ValidateIngressDNSAddonActivation

ingress_addon_legacy_test.go:79: (dbg) Run:  out/minikube-darwin-amd64 -p ingress-addon-legacy-482000 addons enable ingress-dns --alsologtostderr -v=5

E0115 05:19:54.001612   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

ingress_addon_legacy_test.go:79: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p ingress-addon-legacy-482000 addons enable ingress-dns --alsologtostderr -v=5: exit status 10 (1m23.932628634s)

-- stdout --

	* ingress-dns is an addon maintained by minikube. For any concerns contact minikube on GitHub.

	You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS

	* After the addon is enabled, please run "minikube tunnel" and your ingress resources would be available at "127.0.0.1"

	  - Using image cryptexlabs/minikube-ingress-dns:0.3.0

-- /stdout --

** stderr ** 

	I0115 05:19:22.968272   68677 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:19:22.969401   68677 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:19:22.969409   68677 out.go:309] Setting ErrFile to fd 2...

	I0115 05:19:22.969413   68677 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:19:22.969610   68677 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:19:22.969981   68677 mustload.go:65] Loading cluster: ingress-addon-legacy-482000

	I0115 05:19:22.970263   68677 config.go:182] Loaded profile config "ingress-addon-legacy-482000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.18.20

	I0115 05:19:22.970279   68677 addons.go:597] checking whether the cluster is paused

	I0115 05:19:22.970362   68677 config.go:182] Loaded profile config "ingress-addon-legacy-482000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.18.20

	I0115 05:19:22.970380   68677 host.go:66] Checking if "ingress-addon-legacy-482000" exists ...

	I0115 05:19:22.970778   68677 cli_runner.go:164] Run: docker container inspect ingress-addon-legacy-482000 --format={{.State.Status}}

	I0115 05:19:23.021494   68677 ssh_runner.go:195] Run: systemctl --version

	I0115 05:19:23.021585   68677 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:19:23.071637   68677 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:54699 SSHKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa Username:docker}

	I0115 05:19:23.162452   68677 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}

	I0115 05:19:23.202642   68677 out.go:177] * ingress-dns is an addon maintained by minikube. For any concerns contact minikube on GitHub.

	You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS

	I0115 05:19:23.223615   68677 config.go:182] Loaded profile config "ingress-addon-legacy-482000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.18.20

	I0115 05:19:23.223633   68677 addons.go:69] Setting ingress-dns=true in profile "ingress-addon-legacy-482000"

	I0115 05:19:23.223647   68677 addons.go:234] Setting addon ingress-dns=true in "ingress-addon-legacy-482000"

	I0115 05:19:23.223705   68677 host.go:66] Checking if "ingress-addon-legacy-482000" exists ...

	I0115 05:19:23.224094   68677 cli_runner.go:164] Run: docker container inspect ingress-addon-legacy-482000 --format={{.State.Status}}

	I0115 05:19:23.296529   68677 out.go:177] * After the addon is enabled, please run "minikube tunnel" and your ingress resources would be available at "127.0.0.1"

	I0115 05:19:23.317411   68677 out.go:177]   - Using image cryptexlabs/minikube-ingress-dns:0.3.0

	I0115 05:19:23.338612   68677 addons.go:426] installing /etc/kubernetes/addons/ingress-dns-pod.yaml

	I0115 05:19:23.338633   68677 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ingress-dns-pod.yaml (2434 bytes)

	I0115 05:19:23.338739   68677 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" ingress-addon-legacy-482000

	I0115 05:19:23.389584   68677 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:54699 SSHKeyPath:/Users/jenkins/minikube-integration/17953-64881/.minikube/machines/ingress-addon-legacy-482000/id_rsa Username:docker}

	I0115 05:19:23.491205   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:19:23.540344   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:23.540372   68677 retry.go:31] will retry after 331.490486ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:23.872896   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:19:23.920318   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:23.920344   68677 retry.go:31] will retry after 524.620002ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:24.447263   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:19:24.508227   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:24.508245   68677 retry.go:31] will retry after 659.097097ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:25.167483   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:19:25.217306   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:25.217322   68677 retry.go:31] will retry after 639.652196ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:25.857319   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:19:25.914300   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:25.914320   68677 retry.go:31] will retry after 936.529292ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:26.851254   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:19:26.916051   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:26.916069   68677 retry.go:31] will retry after 2.28411149s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:29.200174   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:19:29.251736   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:29.251756   68677 retry.go:31] will retry after 4.129021641s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:33.381335   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:19:33.439314   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:33.439337   68677 retry.go:31] will retry after 3.033003624s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:36.474484   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:19:36.530507   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:36.530526   68677 retry.go:31] will retry after 4.640597695s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:41.173060   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:19:41.222824   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:41.222859   68677 retry.go:31] will retry after 8.920623802s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:50.143175   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:19:50.202136   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:19:50.202153   68677 retry.go:31] will retry after 15.064109223s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:20:05.266052   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:20:05.322155   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:20:05.322178   68677 retry.go:31] will retry after 23.993574502s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:20:29.315544   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:20:29.367880   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:20:29.367897   68677 retry.go:31] will retry after 17.328363265s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:20:46.696044   68677 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	W0115 05:20:46.744042   68677 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	I0115 05:20:46.765857   68677 out.go:177] 

	W0115 05:20:46.787935   68677 out.go:239] X Exiting due to MK_ADDON_ENABLE: enable failed: run callbacks: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	]

	X Exiting due to MK_ADDON_ENABLE: enable failed: run callbacks: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.20/kubectl apply --force -f /etc/kubernetes/addons/ingress-dns-pod.yaml: Process exited with status 1

	stdout:

	stderr:

	The connection to the server localhost:8443 was refused - did you specify the right host or port?

	]

	W0115 05:20:46.787967   68677 out.go:239] * 

	* 

	W0115 05:20:46.795939   68677 out.go:239] ╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                           │

	│    * If the above advice does not help, please let us know:                                                               │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                             │

	│                                                                                                                           │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                  │

	│    * Please also attach the following file to the GitHub issue:                                                           │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_addons_26091442b04c5e26589fdfa18b5031c2ff11dd6b_0.log    │

	│                                                                                                                           │

	╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                           │

	│    * If the above advice does not help, please let us know:                                                               │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                             │

	│                                                                                                                           │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                  │

	│    * Please also attach the following file to the GitHub issue:                                                           │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_addons_26091442b04c5e26589fdfa18b5031c2ff11dd6b_0.log    │

	│                                                                                                                           │

	╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	I0115 05:20:46.816885   68677 out.go:177] 

** /stderr **

ingress_addon_legacy_test.go:80: failed to enable ingress-dns addon: exit status 10

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestIngressAddonLegacy/serial/ValidateIngressDNSAddonActivation]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect ingress-addon-legacy-482000

helpers_test.go:235: (dbg) docker inspect ingress-addon-legacy-482000:

-- stdout --

	[

	    {

	        "Id": "01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5",

	        "Created": "2024-01-15T13:13:25.527112214Z",

	        "Path": "/usr/local/bin/entrypoint",

	        "Args": [

	            "/sbin/init"

	        ],

	        "State": {

	            "Status": "running",

	            "Running": true,

	            "Paused": false,

	            "Restarting": false,

	            "OOMKilled": false,

	            "Dead": false,

	            "Pid": 51552,

	            "ExitCode": 0,

	            "Error": "",

	            "StartedAt": "2024-01-15T13:13:25.739041481Z",

	            "FinishedAt": "0001-01-01T00:00:00Z"

	        },

	        "Image": "sha256:9941de2e064a4a6a7155bfc66cedd2854b8c725b77bb8d4eaf81bef39f951dd7",

	        "ResolvConfPath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/resolv.conf",

	        "HostnamePath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/hostname",

	        "HostsPath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/hosts",

	        "LogPath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5-json.log",

	        "Name": "/ingress-addon-legacy-482000",

	        "RestartCount": 0,

	        "Driver": "overlay2",

	        "Platform": "linux",

	        "MountLabel": "",

	        "ProcessLabel": "",

	        "AppArmorProfile": "",

	        "ExecIDs": null,

	        "HostConfig": {

	            "Binds": [

	                "ingress-addon-legacy-482000:/var",

	                "/lib/modules:/lib/modules:ro"

	            ],

	            "ContainerIDFile": "",

	            "LogConfig": {

	                "Type": "json-file",

	                "Config": {}

	            },

	            "NetworkMode": "ingress-addon-legacy-482000",

	            "PortBindings": {

	                "22/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "2376/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "32443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "5000/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "8443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ]

	            },

	            "RestartPolicy": {

	                "Name": "no",

	                "MaximumRetryCount": 0

	            },

	            "AutoRemove": false,

	            "VolumeDriver": "",

	            "VolumesFrom": null,

	            "ConsoleSize": [

	                0,

	                0

	            ],

	            "CapAdd": null,

	            "CapDrop": null,

	            "CgroupnsMode": "private",

	            "Dns": [],

	            "DnsOptions": [],

	            "DnsSearch": [],

	            "ExtraHosts": null,

	            "GroupAdd": null,

	            "IpcMode": "private",

	            "Cgroup": "",

	            "Links": null,

	            "OomScoreAdj": 0,

	            "PidMode": "",

	            "Privileged": true,

	            "PublishAllPorts": false,

	            "ReadonlyRootfs": false,

	            "SecurityOpt": [

	                "seccomp=unconfined",

	                "apparmor=unconfined",

	                "label=disable"

	            ],

	            "Tmpfs": {

	                "/run": "",

	                "/tmp": ""

	            },

	            "UTSMode": "",

	            "UsernsMode": "",

	            "ShmSize": 67108864,

	            "Runtime": "runc",

	            "Isolation": "",

	            "CpuShares": 0,

	            "Memory": 4294967296,

	            "NanoCpus": 2000000000,

	            "CgroupParent": "",

	            "BlkioWeight": 0,

	            "BlkioWeightDevice": [],

	            "BlkioDeviceReadBps": [],

	            "BlkioDeviceWriteBps": [],

	            "BlkioDeviceReadIOps": [],

	            "BlkioDeviceWriteIOps": [],

	            "CpuPeriod": 0,

	            "CpuQuota": 0,

	            "CpuRealtimePeriod": 0,

	            "CpuRealtimeRuntime": 0,

	            "CpusetCpus": "",

	            "CpusetMems": "",

	            "Devices": [],

	            "DeviceCgroupRules": null,

	            "DeviceRequests": null,

	            "MemoryReservation": 0,

	            "MemorySwap": 4294967296,

	            "MemorySwappiness": null,

	            "OomKillDisable": null,

	            "PidsLimit": null,

	            "Ulimits": null,

	            "CpuCount": 0,

	            "CpuPercent": 0,

	            "IOMaximumIOps": 0,

	            "IOMaximumBandwidth": 0,

	            "MaskedPaths": null,

	            "ReadonlyPaths": null

	        },

	        "GraphDriver": {

	            "Data": {

	                "LowerDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63-init/diff:/var/lib/docker/overlay2/b3cb78fe399645181979e767fd2b27916778197e6245b2db21b3eb1fe7dda1f5/diff",

	                "MergedDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63/merged",

	                "UpperDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63/diff",

	                "WorkDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63/work"

	            },

	            "Name": "overlay2"

	        },

	        "Mounts": [

	            {

	                "Type": "volume",

	                "Name": "ingress-addon-legacy-482000",

	                "Source": "/var/lib/docker/volumes/ingress-addon-legacy-482000/_data",

	                "Destination": "/var",

	                "Driver": "local",

	                "Mode": "z",

	                "RW": true,

	                "Propagation": ""

	            },

	            {

	                "Type": "bind",

	                "Source": "/lib/modules",

	                "Destination": "/lib/modules",

	                "Mode": "ro",

	                "RW": false,

	                "Propagation": "rprivate"

	            }

	        ],

	        "Config": {

	            "Hostname": "ingress-addon-legacy-482000",

	            "Domainname": "",

	            "User": "",

	            "AttachStdin": false,

	            "AttachStdout": false,

	            "AttachStderr": false,

	            "ExposedPorts": {

	                "22/tcp": {},

	                "2376/tcp": {},

	                "32443/tcp": {},

	                "5000/tcp": {},

	                "8443/tcp": {}

	            },

	            "Tty": true,

	            "OpenStdin": false,

	            "StdinOnce": false,

	            "Env": [

	                "container=docker",

	                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

	            ],

	            "Cmd": null,

	            "Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0",

	            "Volumes": null,

	            "WorkingDir": "/",

	            "Entrypoint": [

	                "/usr/local/bin/entrypoint",

	                "/sbin/init"

	            ],

	            "OnBuild": null,

	            "Labels": {

	                "created_by.minikube.sigs.k8s.io": "true",

	                "mode.minikube.sigs.k8s.io": "ingress-addon-legacy-482000",

	                "name.minikube.sigs.k8s.io": "ingress-addon-legacy-482000",

	                "role.minikube.sigs.k8s.io": ""

	            },

	            "StopSignal": "SIGRTMIN+3"

	        },

	        "NetworkSettings": {

	            "Bridge": "",

	            "SandboxID": "c52d563e1da90ce414962fa1df8d9f1f7b3e7e151030bd1bd475447b08ef956f",

	            "HairpinMode": false,

	            "LinkLocalIPv6Address": "",

	            "LinkLocalIPv6PrefixLen": 0,

	            "Ports": {

	                "22/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54699"

	                    }

	                ],

	                "2376/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54695"

	                    }

	                ],

	                "32443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54696"

	                    }

	                ],

	                "5000/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54697"

	                    }

	                ],

	                "8443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54698"

	                    }

	                ]

	            },

	            "SandboxKey": "/var/run/docker/netns/c52d563e1da9",

	            "SecondaryIPAddresses": null,

	            "SecondaryIPv6Addresses": null,

	            "EndpointID": "",

	            "Gateway": "",

	            "GlobalIPv6Address": "",

	            "GlobalIPv6PrefixLen": 0,

	            "IPAddress": "",

	            "IPPrefixLen": 0,

	            "IPv6Gateway": "",

	            "MacAddress": "",

	            "Networks": {

	                "ingress-addon-legacy-482000": {

	                    "IPAMConfig": {

	                        "IPv4Address": "192.168.49.2"

	                    },

	                    "Links": null,

	                    "Aliases": [

	                        "01e6adecdd3d",

	                        "ingress-addon-legacy-482000"

	                    ],

	                    "MacAddress": "02:42:c0:a8:31:02",

	                    "NetworkID": "1736f1945b5c282c51c1427aefc8d7c5b95fb9883f9d2b3660576aa7e5f3df5e",

	                    "EndpointID": "d7fd6caa78b65da813d9019e0f38370b5eed09e2397f731aa620e12eae53c3e3",

	                    "Gateway": "192.168.49.1",

	                    "IPAddress": "192.168.49.2",

	                    "IPPrefixLen": 24,

	                    "IPv6Gateway": "",

	                    "GlobalIPv6Address": "",

	                    "GlobalIPv6PrefixLen": 0,

	                    "DriverOpts": null

	                }

	            }

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p ingress-addon-legacy-482000 -n ingress-addon-legacy-482000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p ingress-addon-legacy-482000 -n ingress-addon-legacy-482000: exit status 6 (384.906033ms)

-- stdout --

	Running

	WARNING: Your kubectl is pointing to stale minikube-vm.

	To fix the kubectl context, run `minikube update-context`

-- /stdout --

** stderr ** 

	E0115 05:20:47.268429   68691 status.go:415] kubeconfig endpoint: extract IP: "ingress-addon-legacy-482000" does not appear in /Users/jenkins/minikube-integration/17953-64881/kubeconfig

** /stderr **

helpers_test.go:239: status error: exit status 6 (may be ok)

helpers_test.go:241: "ingress-addon-legacy-482000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")

--- FAIL: TestIngressAddonLegacy/serial/ValidateIngressDNSAddonActivation (84.37s)

=== RUN   TestIngressAddonLegacy/serial/ValidateIngressAddons

addons_test.go:201: failed to get Kubernetes client: <nil>

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestIngressAddonLegacy/serial/ValidateIngressAddons]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect ingress-addon-legacy-482000

helpers_test.go:235: (dbg) docker inspect ingress-addon-legacy-482000:

-- stdout --

	[

	    {

	        "Id": "01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5",

	        "Created": "2024-01-15T13:13:25.527112214Z",

	        "Path": "/usr/local/bin/entrypoint",

	        "Args": [

	            "/sbin/init"

	        ],

	        "State": {

	            "Status": "running",

	            "Running": true,

	            "Paused": false,

	            "Restarting": false,

	            "OOMKilled": false,

	            "Dead": false,

	            "Pid": 51552,

	            "ExitCode": 0,

	            "Error": "",

	            "StartedAt": "2024-01-15T13:13:25.739041481Z",

	            "FinishedAt": "0001-01-01T00:00:00Z"

	        },

	        "Image": "sha256:9941de2e064a4a6a7155bfc66cedd2854b8c725b77bb8d4eaf81bef39f951dd7",

	        "ResolvConfPath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/resolv.conf",

	        "HostnamePath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/hostname",

	        "HostsPath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/hosts",

	        "LogPath": "/var/lib/docker/containers/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5/01e6adecdd3d3aa99a947e08750856ed7335a481446f5627e0beeb852c1fdbd5-json.log",

	        "Name": "/ingress-addon-legacy-482000",

	        "RestartCount": 0,

	        "Driver": "overlay2",

	        "Platform": "linux",

	        "MountLabel": "",

	        "ProcessLabel": "",

	        "AppArmorProfile": "",

	        "ExecIDs": null,

	        "HostConfig": {

	            "Binds": [

	                "ingress-addon-legacy-482000:/var",

	                "/lib/modules:/lib/modules:ro"

	            ],

	            "ContainerIDFile": "",

	            "LogConfig": {

	                "Type": "json-file",

	                "Config": {}

	            },

	            "NetworkMode": "ingress-addon-legacy-482000",

	            "PortBindings": {

	                "22/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "2376/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "32443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "5000/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ],

	                "8443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "0"

	                    }

	                ]

	            },

	            "RestartPolicy": {

	                "Name": "no",

	                "MaximumRetryCount": 0

	            },

	            "AutoRemove": false,

	            "VolumeDriver": "",

	            "VolumesFrom": null,

	            "ConsoleSize": [

	                0,

	                0

	            ],

	            "CapAdd": null,

	            "CapDrop": null,

	            "CgroupnsMode": "private",

	            "Dns": [],

	            "DnsOptions": [],

	            "DnsSearch": [],

	            "ExtraHosts": null,

	            "GroupAdd": null,

	            "IpcMode": "private",

	            "Cgroup": "",

	            "Links": null,

	            "OomScoreAdj": 0,

	            "PidMode": "",

	            "Privileged": true,

	            "PublishAllPorts": false,

	            "ReadonlyRootfs": false,

	            "SecurityOpt": [

	                "seccomp=unconfined",

	                "apparmor=unconfined",

	                "label=disable"

	            ],

	            "Tmpfs": {

	                "/run": "",

	                "/tmp": ""

	            },

	            "UTSMode": "",

	            "UsernsMode": "",

	            "ShmSize": 67108864,

	            "Runtime": "runc",

	            "Isolation": "",

	            "CpuShares": 0,

	            "Memory": 4294967296,

	            "NanoCpus": 2000000000,

	            "CgroupParent": "",

	            "BlkioWeight": 0,

	            "BlkioWeightDevice": [],

	            "BlkioDeviceReadBps": [],

	            "BlkioDeviceWriteBps": [],

	            "BlkioDeviceReadIOps": [],

	            "BlkioDeviceWriteIOps": [],

	            "CpuPeriod": 0,

	            "CpuQuota": 0,

	            "CpuRealtimePeriod": 0,

	            "CpuRealtimeRuntime": 0,

	            "CpusetCpus": "",

	            "CpusetMems": "",

	            "Devices": [],

	            "DeviceCgroupRules": null,

	            "DeviceRequests": null,

	            "MemoryReservation": 0,

	            "MemorySwap": 4294967296,

	            "MemorySwappiness": null,

	            "OomKillDisable": null,

	            "PidsLimit": null,

	            "Ulimits": null,

	            "CpuCount": 0,

	            "CpuPercent": 0,

	            "IOMaximumIOps": 0,

	            "IOMaximumBandwidth": 0,

	            "MaskedPaths": null,

	            "ReadonlyPaths": null

	        },

	        "GraphDriver": {

	            "Data": {

	                "LowerDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63-init/diff:/var/lib/docker/overlay2/b3cb78fe399645181979e767fd2b27916778197e6245b2db21b3eb1fe7dda1f5/diff",

	                "MergedDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63/merged",

	                "UpperDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63/diff",

	                "WorkDir": "/var/lib/docker/overlay2/2cb0888d187b97af0912977b10ebd3b19657ae1014924785c29b2fd6cdbb2e63/work"

	            },

	            "Name": "overlay2"

	        },

	        "Mounts": [

	            {

	                "Type": "volume",

	                "Name": "ingress-addon-legacy-482000",

	                "Source": "/var/lib/docker/volumes/ingress-addon-legacy-482000/_data",

	                "Destination": "/var",

	                "Driver": "local",

	                "Mode": "z",

	                "RW": true,

	                "Propagation": ""

	            },

	            {

	                "Type": "bind",

	                "Source": "/lib/modules",

	                "Destination": "/lib/modules",

	                "Mode": "ro",

	                "RW": false,

	                "Propagation": "rprivate"

	            }

	        ],

	        "Config": {

	            "Hostname": "ingress-addon-legacy-482000",

	            "Domainname": "",

	            "User": "",

	            "AttachStdin": false,

	            "AttachStdout": false,

	            "AttachStderr": false,

	            "ExposedPorts": {

	                "22/tcp": {},

	                "2376/tcp": {},

	                "32443/tcp": {},

	                "5000/tcp": {},

	                "8443/tcp": {}

	            },

	            "Tty": true,

	            "OpenStdin": false,

	            "StdinOnce": false,

	            "Env": [

	                "container=docker",

	                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

	            ],

	            "Cmd": null,

	            "Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0",

	            "Volumes": null,

	            "WorkingDir": "/",

	            "Entrypoint": [

	                "/usr/local/bin/entrypoint",

	                "/sbin/init"

	            ],

	            "OnBuild": null,

	            "Labels": {

	                "created_by.minikube.sigs.k8s.io": "true",

	                "mode.minikube.sigs.k8s.io": "ingress-addon-legacy-482000",

	                "name.minikube.sigs.k8s.io": "ingress-addon-legacy-482000",

	                "role.minikube.sigs.k8s.io": ""

	            },

	            "StopSignal": "SIGRTMIN+3"

	        },

	        "NetworkSettings": {

	            "Bridge": "",

	            "SandboxID": "c52d563e1da90ce414962fa1df8d9f1f7b3e7e151030bd1bd475447b08ef956f",

	            "HairpinMode": false,

	            "LinkLocalIPv6Address": "",

	            "LinkLocalIPv6PrefixLen": 0,

	            "Ports": {

	                "22/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54699"

	                    }

	                ],

	                "2376/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54695"

	                    }

	                ],

	                "32443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54696"

	                    }

	                ],

	                "5000/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54697"

	                    }

	                ],

	                "8443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "54698"

	                    }

	                ]

	            },

	            "SandboxKey": "/var/run/docker/netns/c52d563e1da9",

	            "SecondaryIPAddresses": null,

	            "SecondaryIPv6Addresses": null,

	            "EndpointID": "",

	            "Gateway": "",

	            "GlobalIPv6Address": "",

	            "GlobalIPv6PrefixLen": 0,

	            "IPAddress": "",

	            "IPPrefixLen": 0,

	            "IPv6Gateway": "",

	            "MacAddress": "",

	            "Networks": {

	                "ingress-addon-legacy-482000": {

	                    "IPAMConfig": {

	                        "IPv4Address": "192.168.49.2"

	                    },

	                    "Links": null,

	                    "Aliases": [

	                        "01e6adecdd3d",

	                        "ingress-addon-legacy-482000"

	                    ],

	                    "MacAddress": "02:42:c0:a8:31:02",

	                    "NetworkID": "1736f1945b5c282c51c1427aefc8d7c5b95fb9883f9d2b3660576aa7e5f3df5e",

	                    "EndpointID": "d7fd6caa78b65da813d9019e0f38370b5eed09e2397f731aa620e12eae53c3e3",

	                    "Gateway": "192.168.49.1",

	                    "IPAddress": "192.168.49.2",

	                    "IPPrefixLen": 24,

	                    "IPv6Gateway": "",

	                    "GlobalIPv6Address": "",

	                    "GlobalIPv6PrefixLen": 0,

	                    "DriverOpts": null

	                }

	            }

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p ingress-addon-legacy-482000 -n ingress-addon-legacy-482000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p ingress-addon-legacy-482000 -n ingress-addon-legacy-482000: exit status 6 (404.028118ms)

-- stdout --

	Running

	WARNING: Your kubectl is pointing to stale minikube-vm.

	To fix the kubectl context, run `minikube update-context`

-- /stdout --

** stderr ** 

	E0115 05:20:47.724016   68703 status.go:415] kubeconfig endpoint: extract IP: "ingress-addon-legacy-482000" does not appear in /Users/jenkins/minikube-integration/17953-64881/kubeconfig

** /stderr **

helpers_test.go:239: status error: exit status 6 (may be ok)

helpers_test.go:241: "ingress-addon-legacy-482000" host is not running, skipping log retrieval (state="Running\nWARNING: Your kubectl is pointing to stale minikube-vm.\nTo fix the kubectl context, run `minikube update-context`")

--- FAIL: TestIngressAddonLegacy/serial/ValidateIngressAddons (0.46s)

=== RUN   TestMultiNode/serial/FreshStart2Nodes

multinode_test.go:86: (dbg) Run:  out/minikube-darwin-amd64 start -p multinode-456000 --wait=true --memory=2200 --nodes=2 -v=8 --alsologtostderr --driver=docker 

E0115 05:25:56.721434   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:26:17.050438   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 05:29:53.968477   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 05:30:56.704464   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:32:19.756960   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:34:53.952601   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 05:35:56.689167   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

multinode_test.go:86: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p multinode-456000 --wait=true --memory=2200 --nodes=2 -v=8 --alsologtostderr --driver=docker : exit status 52 (12m35.650452605s)

-- stdout --

	* [multinode-456000] minikube v1.32.0 on Darwin 14.2.1

	  - MINIKUBE_LOCATION=17953

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the docker driver based on user configuration

	* Using Docker Desktop driver with root privileges

	* Starting control plane node multinode-456000 in cluster multinode-456000

	* Pulling base image v0.0.42-1704759386-17866 ...

	* Creating docker container (CPUs=2, Memory=2200MB) ...

	* docker "multinode-456000" container is missing, will recreate.

	* Creating docker container (CPUs=2, Memory=2200MB) ...

-- /stdout --

** stderr ** 

	I0115 05:25:04.862422   70438 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:25:04.862722   70438 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:25:04.862729   70438 out.go:309] Setting ErrFile to fd 2...

	I0115 05:25:04.862733   70438 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:25:04.862925   70438 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:25:04.864399   70438 out.go:303] Setting JSON to false

	I0115 05:25:04.887015   70438 start.go:128] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":31847,"bootTime":1705293257,"procs":495,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.2.1","kernelVersion":"23.2.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}

	W0115 05:25:04.887125   70438 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0115 05:25:04.908965   70438 out.go:177] * [multinode-456000] minikube v1.32.0 on Darwin 14.2.1

	I0115 05:25:04.952818   70438 out.go:177]   - MINIKUBE_LOCATION=17953

	I0115 05:25:04.952890   70438 notify.go:220] Checking for updates...

	I0115 05:25:04.995448   70438 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	I0115 05:25:05.017656   70438 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-amd64

	I0115 05:25:05.039700   70438 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0115 05:25:05.061337   70438 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	I0115 05:25:05.082534   70438 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0115 05:25:05.104000   70438 driver.go:392] Setting default libvirt URI to qemu:///system

	I0115 05:25:05.160233   70438 docker.go:122] docker version: linux-24.0.7:Docker Desktop 4.26.0 (130397)

	I0115 05:25:05.160408   70438 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0115 05:25:05.264285   70438 info.go:266] docker info: {ID:37b3081a-8cd6-457e-b2a4-79dc82345b06 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:2 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:42 OomKillDisable:false NGoroutines:63 SystemTime:2024-01-15 13:25:05.255192323 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:6.5.11-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServe

rAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6221279232 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f Expected:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f} RuncCommit:{ID:v1.1.10-0-g18a0cb0 Expected:v1.1.10-0-g18a0cb0} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=unconf

ined name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.0-desktop.2] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.23.3-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Mana

ges Docker extensions Vendor:Docker Inc. Version:v0.2.21] map[Name:feedback Path:/Users/jenkins/.docker/cli-plugins/docker-feedback SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-feedback] ShortDescription:Provide feedback, right in your terminal! Vendor:Docker Inc. Version:0.1] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.10] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/d

ocker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Scout Vendor:Docker Inc. Version:v1.2.0]] Warnings:<nil>}}

	I0115 05:25:05.306759   70438 out.go:177] * Using the docker driver based on user configuration

	I0115 05:25:05.327943   70438 start.go:298] selected driver: docker

	I0115 05:25:05.327977   70438 start.go:902] validating driver "docker" against <nil>

	I0115 05:25:05.327994   70438 start.go:913] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0115 05:25:05.332460   70438 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0115 05:25:05.435888   70438 info.go:266] docker info: {ID:37b3081a-8cd6-457e-b2a4-79dc82345b06 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:2 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:42 OomKillDisable:false NGoroutines:63 SystemTime:2024-01-15 13:25:05.426104182 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:6.5.11-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServe

rAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6221279232 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f Expected:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f} RuncCommit:{ID:v1.1.10-0-g18a0cb0 Expected:v1.1.10-0-g18a0cb0} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=unconf

ined name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.0-desktop.2] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.23.3-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Mana

ges Docker extensions Vendor:Docker Inc. Version:v0.2.21] map[Name:feedback Path:/Users/jenkins/.docker/cli-plugins/docker-feedback SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-feedback] ShortDescription:Provide feedback, right in your terminal! Vendor:Docker Inc. Version:0.1] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.10] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/d

ocker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Scout Vendor:Docker Inc. Version:v1.2.0]] Warnings:<nil>}}

	I0115 05:25:05.436074   70438 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0115 05:25:05.436280   70438 start_flags.go:927] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0115 05:25:05.457167   70438 out.go:177] * Using Docker Desktop driver with root privileges

	I0115 05:25:05.478172   70438 cni.go:84] Creating CNI manager for ""

	I0115 05:25:05.478196   70438 cni.go:136] 0 nodes found, recommending kindnet

	I0115 05:25:05.478207   70438 start_flags.go:316] Found "CNI" CNI - setting NetworkPlugin=cni

	I0115 05:25:05.478223   70438 start_flags.go:321] config:

	{Name:multinode-456000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:multinode-456000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:do

cker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs:}

	I0115 05:25:05.500181   70438 out.go:177] * Starting control plane node multinode-456000 in cluster multinode-456000

	I0115 05:25:05.523082   70438 cache.go:121] Beginning downloading kic base image for docker with docker

	I0115 05:25:05.545280   70438 out.go:177] * Pulling base image v0.0.42-1704759386-17866 ...

	I0115 05:25:05.589097   70438 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0115 05:25:05.589181   70438 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4

	I0115 05:25:05.589198   70438 cache.go:56] Caching tarball of preloaded images

	I0115 05:25:05.589195   70438 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 in local docker daemon

	I0115 05:25:05.589411   70438 preload.go:174] Found /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4 in cache, skipping download

	I0115 05:25:05.589432   70438 cache.go:59] Finished verifying existence of preloaded tar for  v1.28.4 on docker

	I0115 05:25:05.591029   70438 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/multinode-456000/config.json ...

	I0115 05:25:05.591130   70438 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/multinode-456000/config.json: {Name:mk57cdc7a31a0f7c77791552212d8f40b0ba927e Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0115 05:25:05.641170   70438 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 in local docker daemon, skipping pull

	I0115 05:25:05.641205   70438 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 exists in daemon, skipping load

	I0115 05:25:05.641233   70438 cache.go:194] Successfully downloaded all kic artifacts

	I0115 05:25:05.641281   70438 start.go:365] acquiring machines lock for multinode-456000: {Name:mk3c781fec38dc7197a8eca34d9ca558cb93e4e6 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0115 05:25:05.641435   70438 start.go:369] acquired machines lock for "multinode-456000" in 142.071µs

	I0115 05:25:05.641460   70438 start.go:93] Provisioning new machine with config: &{Name:multinode-456000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:multinode-456000 Namespace:default APIServerName:miniku

beCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Disa

bleMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs:} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0115 05:25:05.641550   70438 start.go:125] createHost starting for "" (driver="docker")

	I0115 05:25:05.684168   70438 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...

	I0115 05:25:05.684536   70438 start.go:159] libmachine.API.Create for "multinode-456000" (driver="docker")

	I0115 05:25:05.684614   70438 client.go:168] LocalClient.Create starting

	I0115 05:25:05.684830   70438 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem

	I0115 05:25:05.684934   70438 main.go:141] libmachine: Decoding PEM data...

	I0115 05:25:05.684967   70438 main.go:141] libmachine: Parsing certificate...

	I0115 05:25:05.685083   70438 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem

	I0115 05:25:05.685156   70438 main.go:141] libmachine: Decoding PEM data...

	I0115 05:25:05.685173   70438 main.go:141] libmachine: Parsing certificate...

	I0115 05:25:05.686084   70438 cli_runner.go:164] Run: docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	W0115 05:25:05.737243   70438 cli_runner.go:211] docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1

	I0115 05:25:05.737340   70438 network_create.go:281] running [docker network inspect multinode-456000] to gather additional debugging logs...

	I0115 05:25:05.737360   70438 cli_runner.go:164] Run: docker network inspect multinode-456000

	W0115 05:25:05.787312   70438 cli_runner.go:211] docker network inspect multinode-456000 returned with exit code 1

	I0115 05:25:05.787345   70438 network_create.go:284] error running [docker network inspect multinode-456000]: docker network inspect multinode-456000: exit status 1

	stdout:

	[]

	stderr:

	Error response from daemon: network multinode-456000 not found

	I0115 05:25:05.787354   70438 network_create.go:286] output of [docker network inspect multinode-456000]: -- stdout --

	[]

	-- /stdout --

	** stderr ** 

	Error response from daemon: network multinode-456000 not found

	** /stderr **

	I0115 05:25:05.787477   70438 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 05:25:05.840668   70438 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}

	I0115 05:25:05.841037   70438 network.go:209] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc0021f52c0}

	I0115 05:25:05.841059   70438 network_create.go:124] attempt to create docker network multinode-456000 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 65535 ...

	I0115 05:25:05.841128   70438 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=multinode-456000 multinode-456000

	I0115 05:25:05.926740   70438 network_create.go:108] docker network multinode-456000 192.168.58.0/24 created

	I0115 05:25:05.926776   70438 kic.go:121] calculated static IP "192.168.58.2" for the "multinode-456000" container

	I0115 05:25:05.926890   70438 cli_runner.go:164] Run: docker ps -a --format {{.Names}}

	I0115 05:25:05.979385   70438 cli_runner.go:164] Run: docker volume create multinode-456000 --label name.minikube.sigs.k8s.io=multinode-456000 --label created_by.minikube.sigs.k8s.io=true

	I0115 05:25:06.031321   70438 oci.go:103] Successfully created a docker volume multinode-456000

	I0115 05:25:06.031460   70438 cli_runner.go:164] Run: docker run --rm --name multinode-456000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=multinode-456000 --entrypoint /usr/bin/test -v multinode-456000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -d /var/lib

	I0115 05:25:06.425164   70438 oci.go:107] Successfully prepared a docker volume multinode-456000

	I0115 05:25:06.425207   70438 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0115 05:25:06.425220   70438 kic.go:194] Starting extracting preloaded images to volume ...

	I0115 05:25:06.425324   70438 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v multinode-456000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -I lz4 -xf /preloaded.tar -C /extractDir

	I0115 05:31:05.665770   70438 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0115 05:31:05.665960   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:31:05.720709   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:31:05.720816   70438 retry.go:31] will retry after 264.399242ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:05.985645   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:31:06.039384   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:31:06.039492   70438 retry.go:31] will retry after 370.350747ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:06.410198   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:31:06.462858   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:31:06.462967   70438 retry.go:31] will retry after 395.551363ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:06.859036   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:31:06.914439   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:31:06.914539   70438 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:31:06.914561   70438 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:06.914616   70438 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0115 05:31:06.914687   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:31:06.965755   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:31:06.965851   70438 retry.go:31] will retry after 223.288616ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:07.190000   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:31:07.242041   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:31:07.242145   70438 retry.go:31] will retry after 478.925606ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:07.721644   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:31:07.774627   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:31:07.774724   70438 retry.go:31] will retry after 618.794964ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:08.394000   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:31:08.447953   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:31:08.448049   70438 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:31:08.448069   70438 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:08.448100   70438 start.go:128] duration metric: createHost completed in 6m2.826465747s

	I0115 05:31:08.448107   70438 start.go:83] releasing machines lock for "multinode-456000", held for 6m2.826591801s

	W0115 05:31:08.448119   70438 start.go:694] error starting host: creating host: create host timed out in 360.000000 seconds

	I0115 05:31:08.448530   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:08.498172   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:31:08.498221   70438 delete.go:82] Unable to get host status for multinode-456000, assuming it has already been deleted: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:31:08.498310   70438 out.go:239] ! StartHost failed, but will try again: creating host: create host timed out in 360.000000 seconds

	! StartHost failed, but will try again: creating host: create host timed out in 360.000000 seconds

	I0115 05:31:08.498322   70438 start.go:709] Will try again in 5 seconds ...

	I0115 05:31:13.498704   70438 start.go:365] acquiring machines lock for multinode-456000: {Name:mk3c781fec38dc7197a8eca34d9ca558cb93e4e6 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0115 05:31:13.498858   70438 start.go:369] acquired machines lock for "multinode-456000" in 114.592µs

	I0115 05:31:13.498882   70438 start.go:96] Skipping create...Using existing machine configuration

	I0115 05:31:13.498893   70438 fix.go:54] fixHost starting: 

	I0115 05:31:13.499201   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:13.553472   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:31:13.553521   70438 fix.go:102] recreateIfNeeded on multinode-456000: state= err=unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:13.553540   70438 fix.go:107] machineExists: false. err=machine does not exist

	I0115 05:31:13.575183   70438 out.go:177] * docker "multinode-456000" container is missing, will recreate.

	I0115 05:31:13.617921   70438 delete.go:124] DEMOLISHING multinode-456000 ...

	I0115 05:31:13.618159   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:13.669195   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	W0115 05:31:13.669237   70438 stop.go:75] unable to get state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:13.669259   70438 delete.go:128] stophost failed (probably ok): ssh power off: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:13.669629   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:13.720322   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:31:13.720375   70438 delete.go:82] Unable to get host status for multinode-456000, assuming it has already been deleted: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:13.720457   70438 cli_runner.go:164] Run: docker container inspect -f {{.Id}} multinode-456000

	W0115 05:31:13.770600   70438 cli_runner.go:211] docker container inspect -f {{.Id}} multinode-456000 returned with exit code 1

	I0115 05:31:13.770634   70438 kic.go:371] could not find the container multinode-456000 to remove it. will try anyways

	I0115 05:31:13.770709   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:13.820032   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	W0115 05:31:13.820074   70438 oci.go:84] error getting container status, will try to delete anyways: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:13.820154   70438 cli_runner.go:164] Run: docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0"

	W0115 05:31:13.869966   70438 cli_runner.go:211] docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0" returned with exit code 1

	I0115 05:31:13.869997   70438 oci.go:650] error shutdown multinode-456000: docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0": exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:14.870733   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:14.924523   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:31:14.924565   70438 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:14.924601   70438 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:31:14.924625   70438 retry.go:31] will retry after 446.845126ms: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:15.373667   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:15.428268   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:31:15.428313   70438 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:15.428325   70438 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:31:15.428350   70438 retry.go:31] will retry after 668.076439ms: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:16.096655   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:16.149113   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:31:16.149163   70438 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:16.149172   70438 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:31:16.149206   70438 retry.go:31] will retry after 821.319547ms: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:16.972822   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:17.026821   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:31:17.026867   70438 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:17.026877   70438 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:31:17.026900   70438 retry.go:31] will retry after 1.332225664s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:18.359505   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:18.413551   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:31:18.413599   70438 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:18.413608   70438 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:31:18.413632   70438 retry.go:31] will retry after 2.94320096s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:21.356969   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:21.409372   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:31:21.409417   70438 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:21.409425   70438 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:31:21.409451   70438 retry.go:31] will retry after 3.86325067s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:25.274751   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:25.329020   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:31:25.329062   70438 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:25.329071   70438 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:31:25.329096   70438 retry.go:31] will retry after 6.995576204s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:32.324862   70438 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:31:32.376918   70438 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:31:32.376967   70438 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:32.376977   70438 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:31:32.377003   70438 oci.go:88] couldn't shut down multinode-456000 (might be okay): verify shutdown: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:31:32.377089   70438 cli_runner.go:164] Run: docker rm -f -v multinode-456000

	I0115 05:31:32.427821   70438 cli_runner.go:164] Run: docker container inspect -f {{.Id}} multinode-456000

	W0115 05:31:32.478374   70438 cli_runner.go:211] docker container inspect -f {{.Id}} multinode-456000 returned with exit code 1

	I0115 05:31:32.478486   70438 cli_runner.go:164] Run: docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 05:31:32.528951   70438 cli_runner.go:164] Run: docker network rm multinode-456000

	I0115 05:31:32.630980   70438 fix.go:114] Sleeping 1 second for extra luck!

	I0115 05:31:33.633089   70438 start.go:125] createHost starting for "" (driver="docker")

	I0115 05:31:33.655180   70438 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...

	I0115 05:31:33.655346   70438 start.go:159] libmachine.API.Create for "multinode-456000" (driver="docker")

	I0115 05:31:33.655388   70438 client.go:168] LocalClient.Create starting

	I0115 05:31:33.655586   70438 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem

	I0115 05:31:33.655673   70438 main.go:141] libmachine: Decoding PEM data...

	I0115 05:31:33.655705   70438 main.go:141] libmachine: Parsing certificate...

	I0115 05:31:33.655786   70438 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem

	I0115 05:31:33.655857   70438 main.go:141] libmachine: Decoding PEM data...

	I0115 05:31:33.655880   70438 main.go:141] libmachine: Parsing certificate...

	I0115 05:31:33.656566   70438 cli_runner.go:164] Run: docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	W0115 05:31:33.710275   70438 cli_runner.go:211] docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1

	I0115 05:31:33.710364   70438 network_create.go:281] running [docker network inspect multinode-456000] to gather additional debugging logs...

	I0115 05:31:33.710384   70438 cli_runner.go:164] Run: docker network inspect multinode-456000

	W0115 05:31:33.761829   70438 cli_runner.go:211] docker network inspect multinode-456000 returned with exit code 1

	I0115 05:31:33.761859   70438 network_create.go:284] error running [docker network inspect multinode-456000]: docker network inspect multinode-456000: exit status 1

	stdout:

	[]

	stderr:

	Error response from daemon: network multinode-456000 not found

	I0115 05:31:33.761870   70438 network_create.go:286] output of [docker network inspect multinode-456000]: -- stdout --

	[]

	-- /stdout --

	** stderr ** 

	Error response from daemon: network multinode-456000 not found

	** /stderr **

	I0115 05:31:33.761994   70438 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 05:31:33.814183   70438 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}

	I0115 05:31:33.815779   70438 network.go:212] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}

	I0115 05:31:33.816138   70438 network.go:209] using free private subnet 192.168.67.0/24: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc00052b3e0}

	I0115 05:31:33.816156   70438 network_create.go:124] attempt to create docker network multinode-456000 192.168.67.0/24 with gateway 192.168.67.1 and MTU of 65535 ...

	I0115 05:31:33.816229   70438 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=multinode-456000 multinode-456000

	I0115 05:31:33.901372   70438 network_create.go:108] docker network multinode-456000 192.168.67.0/24 created

	I0115 05:31:33.901420   70438 kic.go:121] calculated static IP "192.168.67.2" for the "multinode-456000" container

	I0115 05:31:33.901550   70438 cli_runner.go:164] Run: docker ps -a --format {{.Names}}

	I0115 05:31:33.952806   70438 cli_runner.go:164] Run: docker volume create multinode-456000 --label name.minikube.sigs.k8s.io=multinode-456000 --label created_by.minikube.sigs.k8s.io=true

	I0115 05:31:34.002483   70438 oci.go:103] Successfully created a docker volume multinode-456000

	I0115 05:31:34.002606   70438 cli_runner.go:164] Run: docker run --rm --name multinode-456000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=multinode-456000 --entrypoint /usr/bin/test -v multinode-456000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -d /var/lib

	I0115 05:31:34.319725   70438 oci.go:107] Successfully prepared a docker volume multinode-456000

	I0115 05:31:34.319755   70438 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0115 05:31:34.319768   70438 kic.go:194] Starting extracting preloaded images to volume ...

	I0115 05:31:34.319875   70438 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v multinode-456000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -I lz4 -xf /preloaded.tar -C /extractDir

	I0115 05:37:33.636104   70438 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0115 05:37:33.636238   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:33.689809   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:33.689925   70438 retry.go:31] will retry after 276.577026ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:33.967069   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:34.020049   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:34.020165   70438 retry.go:31] will retry after 311.080493ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:34.331563   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:34.385902   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:34.386021   70438 retry.go:31] will retry after 342.845753ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:34.729751   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:34.784207   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:34.784308   70438 retry.go:31] will retry after 841.433136ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:35.626978   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:35.681582   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:37:35.681685   70438 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:37:35.681703   70438 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:35.681766   70438 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0115 05:37:35.681825   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:35.733991   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:35.734089   70438 retry.go:31] will retry after 161.514157ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:35.896075   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:35.949065   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:35.949161   70438 retry.go:31] will retry after 383.533344ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:36.335016   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:36.388751   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:36.388875   70438 retry.go:31] will retry after 808.205182ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:37.197737   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:37.250808   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:37:37.250910   70438 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:37:37.250924   70438 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:37.250938   70438 start.go:128] duration metric: createHost completed in 6m3.637759302s

	I0115 05:37:37.251003   70438 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0115 05:37:37.251058   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:37.301242   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:37.301336   70438 retry.go:31] will retry after 247.527655ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:37.551131   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:37.604195   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:37.604307   70438 retry.go:31] will retry after 397.116814ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:38.003419   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:38.056179   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:38.056271   70438 retry.go:31] will retry after 523.876776ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:38.581785   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:38.635802   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:37:38.635902   70438 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:37:38.635931   70438 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:38.635991   70438 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0115 05:37:38.636042   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:38.686221   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:38.686311   70438 retry.go:31] will retry after 373.34137ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:39.061958   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:39.116561   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:39.116654   70438 retry.go:31] will retry after 364.736541ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:39.482897   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:39.536646   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:37:39.536746   70438 retry.go:31] will retry after 671.145659ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:40.209229   70438 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:37:40.260401   70438 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:37:40.260507   70438 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:37:40.260523   70438 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:37:40.260533   70438 fix.go:56] fixHost completed within 6m26.782884023s

	I0115 05:37:40.260541   70438 start.go:83] releasing machines lock for "multinode-456000", held for 6m26.782915755s

	W0115 05:37:40.260631   70438 out.go:239] * Failed to start docker container. Running "minikube delete -p multinode-456000" may fix it: recreate: creating host: create host timed out in 360.000000 seconds

	* Failed to start docker container. Running "minikube delete -p multinode-456000" may fix it: recreate: creating host: create host timed out in 360.000000 seconds

	I0115 05:37:40.304134   70438 out.go:177] 

	W0115 05:37:40.326020   70438 out.go:239] X Exiting due to DRV_CREATE_TIMEOUT: Failed to start host: recreate: creating host: create host timed out in 360.000000 seconds

	X Exiting due to DRV_CREATE_TIMEOUT: Failed to start host: recreate: creating host: create host timed out in 360.000000 seconds

	W0115 05:37:40.326091   70438 out.go:239] * Suggestion: Try 'minikube delete', and disable any conflicting VPN or firewall software

	* Suggestion: Try 'minikube delete', and disable any conflicting VPN or firewall software

	W0115 05:37:40.326188   70438 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/7072

	* Related issue: https://github.com/kubernetes/minikube/issues/7072

	I0115 05:37:40.370285   70438 out.go:177] 

** /stderr **

multinode_test.go:88: failed to start cluster. args "out/minikube-darwin-amd64 start -p multinode-456000 --wait=true --memory=2200 --nodes=2 -v=8 --alsologtostderr --driver=docker " : exit status 52

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/FreshStart2Nodes]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "e5bb0cc2be31ba8ad0bf13b88e9218d75aadde41d113cc24fb4fdaded1fe001b",

	        "Created": "2024-01-15T13:31:33.863554312Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (109.634655ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:37:40.588695   70653 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/FreshStart2Nodes (755.83s)

=== RUN   TestMultiNode/serial/DeployApp2Nodes

multinode_test.go:509: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- apply -f ./testdata/multinodes/multinode-pod-dns-test.yaml

multinode_test.go:509: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- apply -f ./testdata/multinodes/multinode-pod-dns-test.yaml: exit status 1 (95.563053ms)

** stderr ** 

	error: cluster "multinode-456000" does not exist

** /stderr **

multinode_test.go:511: failed to create busybox deployment to multinode cluster

multinode_test.go:514: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- rollout status deployment/busybox

multinode_test.go:514: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- rollout status deployment/busybox: exit status 1 (96.713019ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:516: failed to deploy busybox to multinode cluster

multinode_test.go:521: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:521: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (94.851974ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:524: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:521: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:521: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (100.882393ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:524: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:521: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:521: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (98.053963ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:524: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:521: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:521: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (93.600171ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:524: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:521: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:521: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (99.870076ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:524: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:521: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:521: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (99.571265ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:524: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:521: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:521: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (97.809055ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:524: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:521: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:521: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (94.954869ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:524: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:521: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:521: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (99.036828ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:524: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:521: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:521: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (96.845209ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:524: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:540: failed to resolve pod IPs: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:544: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].metadata.name}'

multinode_test.go:544: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].metadata.name}': exit status 1 (95.723675ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:546: failed get Pod names

multinode_test.go:552: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- exec  -- nslookup kubernetes.io

multinode_test.go:552: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- exec  -- nslookup kubernetes.io: exit status 1 (96.35862ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:554: Pod  could not resolve 'kubernetes.io': exit status 1

multinode_test.go:562: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- exec  -- nslookup kubernetes.default

multinode_test.go:562: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- exec  -- nslookup kubernetes.default: exit status 1 (96.205738ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:564: Pod  could not resolve 'kubernetes.default': exit status 1

multinode_test.go:570: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- exec  -- nslookup kubernetes.default.svc.cluster.local

multinode_test.go:570: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- exec  -- nslookup kubernetes.default.svc.cluster.local: exit status 1 (95.29604ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:572: Pod  could not resolve local service (kubernetes.default.svc.cluster.local): exit status 1

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/DeployApp2Nodes]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "e5bb0cc2be31ba8ad0bf13b88e9218d75aadde41d113cc24fb4fdaded1fe001b",

	        "Created": "2024-01-15T13:31:33.863554312Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (110.224138ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:39:04.726880   70715 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/DeployApp2Nodes (84.14s)

=== RUN   TestMultiNode/serial/PingHostFrom2Pods

multinode_test.go:580: (dbg) Run:  out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].metadata.name}'

multinode_test.go:580: (dbg) Non-zero exit: out/minikube-darwin-amd64 kubectl -p multinode-456000 -- get pods -o jsonpath='{.items[*].metadata.name}': exit status 1 (93.522737ms)

** stderr ** 

	error: no server found for cluster "multinode-456000"

** /stderr **

multinode_test.go:582: failed to get Pod names: exit status 1

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/PingHostFrom2Pods]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "e5bb0cc2be31ba8ad0bf13b88e9218d75aadde41d113cc24fb4fdaded1fe001b",

	        "Created": "2024-01-15T13:31:33.863554312Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (109.497265ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:39:04.984884   70724 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/PingHostFrom2Pods (0.26s)

=== RUN   TestMultiNode/serial/AddNode

multinode_test.go:111: (dbg) Run:  out/minikube-darwin-amd64 node add -p multinode-456000 -v 3 --alsologtostderr

multinode_test.go:111: (dbg) Non-zero exit: out/minikube-darwin-amd64 node add -p multinode-456000 -v 3 --alsologtostderr: exit status 80 (207.010698ms)

-- stdout --

-- /stdout --

** stderr ** 

	I0115 05:39:05.041682   70728 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:39:05.042910   70728 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:39:05.042917   70728 out.go:309] Setting ErrFile to fd 2...

	I0115 05:39:05.042921   70728 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:39:05.043105   70728 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:39:05.043436   70728 mustload.go:65] Loading cluster: multinode-456000

	I0115 05:39:05.043722   70728 config.go:182] Loaded profile config "multinode-456000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0115 05:39:05.044109   70728 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:05.093656   70728 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:05.117920   70728 out.go:177] 

	W0115 05:39:05.139844   70728 out.go:239] X Exiting due to GUEST_STATUS: Unable to get machine status: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format=<no value>: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	X Exiting due to GUEST_STATUS: Unable to get machine status: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format=<no value>: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:39:05.139873   70728 out.go:239] * 

	* 

	W0115 05:39:05.148081   70728 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_node_040ea7097fd6ed71e65be9a474587f81f0ccd21d_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_node_040ea7097fd6ed71e65be9a474587f81f0ccd21d_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	I0115 05:39:05.169781   70728 out.go:177] 

** /stderr **

multinode_test.go:113: failed to add node to current cluster. args "out/minikube-darwin-amd64 node add -p multinode-456000 -v 3 --alsologtostderr" : exit status 80

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/AddNode]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "e5bb0cc2be31ba8ad0bf13b88e9218d75aadde41d113cc24fb4fdaded1fe001b",

	        "Created": "2024-01-15T13:31:33.863554312Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (109.59049ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:39:05.356220   70734 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/AddNode (0.37s)

=== RUN   TestMultiNode/serial/MultiNodeLabels

multinode_test.go:211: (dbg) Run:  kubectl --context multinode-456000 get nodes -o "jsonpath=[{range .items[*]}{.metadata.labels},{end}]"

multinode_test.go:211: (dbg) Non-zero exit: kubectl --context multinode-456000 get nodes -o "jsonpath=[{range .items[*]}{.metadata.labels},{end}]": exit status 1 (35.473625ms)

** stderr ** 

	Error in configuration: context was not found for specified context: multinode-456000

** /stderr **

multinode_test.go:213: failed to 'kubectl get nodes' with args "kubectl --context multinode-456000 get nodes -o \"jsonpath=[{range .items[*]}{.metadata.labels},{end}]\"": exit status 1

multinode_test.go:220: failed to decode json from label list: args "kubectl --context multinode-456000 get nodes -o \"jsonpath=[{range .items[*]}{.metadata.labels},{end}]\"": unexpected end of JSON input

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/MultiNodeLabels]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "e5bb0cc2be31ba8ad0bf13b88e9218d75aadde41d113cc24fb4fdaded1fe001b",

	        "Created": "2024-01-15T13:31:33.863554312Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (109.291316ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:39:05.555661   70741 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/MultiNodeLabels (0.20s)

=== RUN   TestMultiNode/serial/ProfileList

multinode_test.go:133: (dbg) Run:  out/minikube-darwin-amd64 profile list --output json

multinode_test.go:156: expected profile "multinode-456000" in json of 'profile list' include 3 nodes but have 1 nodes. got *"{\"invalid\":[],\"valid\":[{\"Name\":\"multinode-456000\",\"Status\":\"Unknown\",\"Config\":{\"Name\":\"multinode-456000\",\"KeepContext\":false,\"EmbedCerts\":false,\"MinikubeISO\":\"\",\"KicBaseImage\":\"gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0\",\"Memory\":2200,\"CPUs\":2,\"DiskSize\":20000,\"VMDriver\":\"\",\"Driver\":\"docker\",\"HyperkitVpnKitSock\":\"\",\"HyperkitVSockPorts\":[],\"DockerEnv\":null,\"ContainerVolumeMounts\":null,\"InsecureRegistry\":null,\"RegistryMirror\":[],\"HostOnlyCIDR\":\"192.168.59.1/24\",\"HypervVirtualSwitch\":\"\",\"HypervUseExternalSwitch\":false,\"HypervExternalAdapter\":\"\",\"KVMNetwork\":\"default\",\"KVMQemuURI\":\"qemu:///system\",\"KVMGPU\":false,\"KVMHidden\":false,\"KVMNUMACount\":1,\"APIServerPort\":0,\"DockerOpt\":null,\"DisableDriverMounts\":false,

\"NFSShare\":[],\"NFSSharesRoot\":\"/nfsshares\",\"UUID\":\"\",\"NoVTXCheck\":false,\"DNSProxy\":false,\"HostDNSResolver\":true,\"HostOnlyNicType\":\"virtio\",\"NatNicType\":\"virtio\",\"SSHIPAddress\":\"\",\"SSHUser\":\"root\",\"SSHKey\":\"\",\"SSHPort\":22,\"KubernetesConfig\":{\"KubernetesVersion\":\"v1.28.4\",\"ClusterName\":\"multinode-456000\",\"Namespace\":\"default\",\"APIServerName\":\"minikubeCA\",\"APIServerNames\":null,\"APIServerIPs\":null,\"DNSDomain\":\"cluster.local\",\"ContainerRuntime\":\"docker\",\"CRISocket\":\"\",\"NetworkPlugin\":\"cni\",\"FeatureGates\":\"\",\"ServiceCIDR\":\"10.96.0.0/12\",\"ImageRepository\":\"\",\"LoadBalancerStartIP\":\"\",\"LoadBalancerEndIP\":\"\",\"CustomIngressCert\":\"\",\"RegistryAliases\":\"\",\"ExtraOptions\":null,\"ShouldLoadCachedImages\":true,\"EnableDefaultCNI\":false,\"CNI\":\"\",\"NodeIP\":\"\",\"NodePort\":8443,\"NodeName\":\"\"},\"Nodes\":[{\"Name\":\"\",\"IP\":\"\",\"Port\":8443,\"KubernetesVersion\":\"v1.28.4\",\"ContainerRuntime\":\"docker\",\"Con

trolPlane\":true,\"Worker\":true}],\"Addons\":null,\"CustomAddonImages\":null,\"CustomAddonRegistries\":null,\"VerifyComponents\":{\"apiserver\":true,\"apps_running\":true,\"default_sa\":true,\"extra\":true,\"kubelet\":true,\"node_ready\":true,\"system_pods\":true},\"StartHostTimeout\":360000000000,\"ScheduledStop\":null,\"ExposedPorts\":[],\"ListenAddress\":\"\",\"Network\":\"\",\"Subnet\":\"\",\"MultiNodeRequested\":true,\"ExtraDisks\":0,\"CertExpiration\":94608000000000000,\"Mount\":false,\"MountString\":\"/Users:/minikube-host\",\"Mount9PVersion\":\"9p2000.L\",\"MountGID\":\"docker\",\"MountIP\":\"\",\"MountMSize\":262144,\"MountOptions\":[],\"MountPort\":0,\"MountType\":\"9p\",\"MountUID\":\"docker\",\"BinaryMirror\":\"\",\"DisableOptimizations\":false,\"DisableMetrics\":false,\"CustomQemuFirmwarePath\":\"\",\"SocketVMnetClientPath\":\"\",\"SocketVMnetPath\":\"\",\"StaticIP\":\"\",\"SSHAuthSock\":\"\",\"SSHAgentPID\":0,\"GPUs\":\"\"},\"Active\":false}]}"*. args: "out/minikube-darwin-amd64 profile list --

output json"

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/ProfileList]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "e5bb0cc2be31ba8ad0bf13b88e9218d75aadde41d113cc24fb4fdaded1fe001b",

	        "Created": "2024-01-15T13:31:33.863554312Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (109.857403ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:39:05.901928   70753 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/ProfileList (0.35s)

=== RUN   TestMultiNode/serial/CopyFile

multinode_test.go:174: (dbg) Run:  out/minikube-darwin-amd64 -p multinode-456000 status --output json --alsologtostderr

multinode_test.go:174: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p multinode-456000 status --output json --alsologtostderr: exit status 7 (109.256216ms)

-- stdout --

	{"Name":"multinode-456000","Host":"Nonexistent","Kubelet":"Nonexistent","APIServer":"Nonexistent","Kubeconfig":"Nonexistent","Worker":false}

-- /stdout --

** stderr ** 

	I0115 05:39:05.959169   70757 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:39:05.959482   70757 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:39:05.959488   70757 out.go:309] Setting ErrFile to fd 2...

	I0115 05:39:05.959492   70757 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:39:05.959673   70757 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:39:05.959856   70757 out.go:303] Setting JSON to true

	I0115 05:39:05.959895   70757 mustload.go:65] Loading cluster: multinode-456000

	I0115 05:39:05.959923   70757 notify.go:220] Checking for updates...

	I0115 05:39:05.960201   70757 config.go:182] Loaded profile config "multinode-456000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0115 05:39:05.960212   70757 status.go:255] checking status of multinode-456000 ...

	I0115 05:39:05.960650   70757 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:06.011255   70757 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:06.011309   70757 status.go:330] multinode-456000 host status = "" (err=state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	)

	I0115 05:39:06.011332   70757 status.go:257] multinode-456000 status: &{Name:multinode-456000 Host:Nonexistent Kubelet:Nonexistent APIServer:Nonexistent Kubeconfig:Nonexistent Worker:false TimeToStop: DockerEnv: PodManEnv:}

	E0115 05:39:06.011349   70757 status.go:260] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	E0115 05:39:06.011356   70757 status.go:263] The "multinode-456000" host does not exist!

** /stderr **

multinode_test.go:181: failed to decode json from status: args "out/minikube-darwin-amd64 -p multinode-456000 status --output json --alsologtostderr": json: cannot unmarshal object into Go value of type []cmd.Status

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/CopyFile]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "e5bb0cc2be31ba8ad0bf13b88e9218d75aadde41d113cc24fb4fdaded1fe001b",

	        "Created": "2024-01-15T13:31:33.863554312Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (110.759911ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:39:06.176492   70763 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/CopyFile (0.27s)

=== RUN   TestMultiNode/serial/StopNode

multinode_test.go:238: (dbg) Run:  out/minikube-darwin-amd64 -p multinode-456000 node stop m03

multinode_test.go:238: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p multinode-456000 node stop m03: exit status 85 (155.08459ms)

-- stdout --

-- /stdout --

** stderr ** 

	X Exiting due to GUEST_NODE_RETRIEVE: retrieving node: Could not find node m03

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_node_295f67d8757edd996fe5c1e7ccde72c355ccf4dc_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

multinode_test.go:240: node stop returned an error. args "out/minikube-darwin-amd64 -p multinode-456000 node stop m03": exit status 85

multinode_test.go:244: (dbg) Run:  out/minikube-darwin-amd64 -p multinode-456000 status

multinode_test.go:244: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p multinode-456000 status: exit status 7 (110.571664ms)

-- stdout --

	multinode-456000

	type: Control Plane

	host: Nonexistent

	kubelet: Nonexistent

	apiserver: Nonexistent

	kubeconfig: Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:39:06.442811   70769 status.go:260] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	E0115 05:39:06.442821   70769 status.go:263] The "multinode-456000" host does not exist!

** /stderr **

multinode_test.go:251: (dbg) Run:  out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr

multinode_test.go:251: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr: exit status 7 (108.781532ms)

-- stdout --

	multinode-456000

	type: Control Plane

	host: Nonexistent

	kubelet: Nonexistent

	apiserver: Nonexistent

	kubeconfig: Nonexistent

-- /stdout --

** stderr ** 

	I0115 05:39:06.500296   70773 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:39:06.500523   70773 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:39:06.500528   70773 out.go:309] Setting ErrFile to fd 2...

	I0115 05:39:06.500533   70773 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:39:06.500722   70773 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:39:06.500903   70773 out.go:303] Setting JSON to false

	I0115 05:39:06.500925   70773 mustload.go:65] Loading cluster: multinode-456000

	I0115 05:39:06.500964   70773 notify.go:220] Checking for updates...

	I0115 05:39:06.501222   70773 config.go:182] Loaded profile config "multinode-456000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0115 05:39:06.501232   70773 status.go:255] checking status of multinode-456000 ...

	I0115 05:39:06.501623   70773 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:06.551648   70773 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:06.551695   70773 status.go:330] multinode-456000 host status = "" (err=state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	)

	I0115 05:39:06.551715   70773 status.go:257] multinode-456000 status: &{Name:multinode-456000 Host:Nonexistent Kubelet:Nonexistent APIServer:Nonexistent Kubeconfig:Nonexistent Worker:false TimeToStop: DockerEnv: PodManEnv:}

	E0115 05:39:06.551733   70773 status.go:260] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	E0115 05:39:06.551741   70773 status.go:263] The "multinode-456000" host does not exist!

** /stderr **

multinode_test.go:257: incorrect number of running kubelets: args "out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr": multinode-456000

type: Control Plane

host: Nonexistent

kubelet: Nonexistent

apiserver: Nonexistent

kubeconfig: Nonexistent

multinode_test.go:261: incorrect number of stopped hosts: args "out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr": multinode-456000

type: Control Plane

host: Nonexistent

kubelet: Nonexistent

apiserver: Nonexistent

kubeconfig: Nonexistent

multinode_test.go:265: incorrect number of stopped kubelets: args "out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr": multinode-456000

type: Control Plane

host: Nonexistent

kubelet: Nonexistent

apiserver: Nonexistent

kubeconfig: Nonexistent

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/StopNode]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "e5bb0cc2be31ba8ad0bf13b88e9218d75aadde41d113cc24fb4fdaded1fe001b",

	        "Created": "2024-01-15T13:31:33.863554312Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (107.883406ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:39:06.714293   70779 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/StopNode (0.54s)

=== RUN   TestMultiNode/serial/StartAfterStop

multinode_test.go:272: (dbg) Run:  docker version -f {{.Server.Version}}

multinode_test.go:282: (dbg) Run:  out/minikube-darwin-amd64 -p multinode-456000 node start m03 --alsologtostderr

multinode_test.go:282: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p multinode-456000 node start m03 --alsologtostderr: exit status 85 (155.159819ms)

-- stdout --

-- /stdout --

** stderr ** 

	I0115 05:39:06.827831   70785 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:39:06.828748   70785 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:39:06.828753   70785 out.go:309] Setting ErrFile to fd 2...

	I0115 05:39:06.828757   70785 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:39:06.828960   70785 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:39:06.829294   70785 mustload.go:65] Loading cluster: multinode-456000

	I0115 05:39:06.829577   70785 config.go:182] Loaded profile config "multinode-456000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0115 05:39:06.851462   70785 out.go:177] 

	W0115 05:39:06.872476   70785 out.go:239] X Exiting due to GUEST_NODE_RETRIEVE: retrieving node: Could not find node m03

	X Exiting due to GUEST_NODE_RETRIEVE: retrieving node: Could not find node m03

	W0115 05:39:06.872505   70785 out.go:239] * 

	* 

	W0115 05:39:06.880705   70785 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_node_1c3a1297795327375b61f3ff5a4ef34c9b2fc69b_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_node_1c3a1297795327375b61f3ff5a4ef34c9b2fc69b_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	I0115 05:39:06.902569   70785 out.go:177] 

** /stderr **

multinode_test.go:284: I0115 05:39:06.827831   70785 out.go:296] Setting OutFile to fd 1 ...

I0115 05:39:06.828748   70785 out.go:343] TERM=,COLORTERM=, which probably does not support color

I0115 05:39:06.828753   70785 out.go:309] Setting ErrFile to fd 2...

I0115 05:39:06.828757   70785 out.go:343] TERM=,COLORTERM=, which probably does not support color

I0115 05:39:06.828960   70785 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

I0115 05:39:06.829294   70785 mustload.go:65] Loading cluster: multinode-456000

I0115 05:39:06.829577   70785 config.go:182] Loaded profile config "multinode-456000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.4

I0115 05:39:06.851462   70785 out.go:177] 

W0115 05:39:06.872476   70785 out.go:239] X Exiting due to GUEST_NODE_RETRIEVE: retrieving node: Could not find node m03

X Exiting due to GUEST_NODE_RETRIEVE: retrieving node: Could not find node m03

W0115 05:39:06.872505   70785 out.go:239] * 

* 

W0115 05:39:06.880705   70785 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

│                                                                                                                         │

│    * If the above advice does not help, please let us know:                                                             │

│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

│                                                                                                                         │

│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

│    * Please also attach the following file to the GitHub issue:                                                         │

│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_node_1c3a1297795327375b61f3ff5a4ef34c9b2fc69b_0.log    │

│                                                                                                                         │

╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

│                                                                                                                         │

│    * If the above advice does not help, please let us know:                                                             │

│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

│                                                                                                                         │

│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

│    * Please also attach the following file to the GitHub issue:                                                         │

│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_node_1c3a1297795327375b61f3ff5a4ef34c9b2fc69b_0.log    │

│                                                                                                                         │

╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

I0115 05:39:06.902569   70785 out.go:177] 

multinode_test.go:285: node start returned an error. args "out/minikube-darwin-amd64 -p multinode-456000 node start m03 --alsologtostderr": exit status 85

multinode_test.go:289: (dbg) Run:  out/minikube-darwin-amd64 -p multinode-456000 status

multinode_test.go:289: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p multinode-456000 status: exit status 7 (109.21175ms)

-- stdout --

	multinode-456000

	type: Control Plane

	host: Nonexistent

	kubelet: Nonexistent

	apiserver: Nonexistent

	kubeconfig: Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:39:07.034552   70787 status.go:260] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	E0115 05:39:07.034563   70787 status.go:263] The "multinode-456000" host does not exist!

** /stderr **

multinode_test.go:291: failed to run minikube status. args "out/minikube-darwin-amd64 -p multinode-456000 status" : exit status 7

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/StartAfterStop]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "e5bb0cc2be31ba8ad0bf13b88e9218d75aadde41d113cc24fb4fdaded1fe001b",

	        "Created": "2024-01-15T13:31:33.863554312Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (109.453247ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:39:07.199098   70793 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/StartAfterStop (0.48s)

=== RUN   TestMultiNode/serial/RestartKeepsNodes

multinode_test.go:311: (dbg) Run:  out/minikube-darwin-amd64 node list -p multinode-456000

multinode_test.go:318: (dbg) Run:  out/minikube-darwin-amd64 stop -p multinode-456000

multinode_test.go:318: (dbg) Non-zero exit: out/minikube-darwin-amd64 stop -p multinode-456000: exit status 82 (12.858540882s)

-- stdout --

	* Stopping node "multinode-456000"  ...

	* Stopping node "multinode-456000"  ...

	* Stopping node "multinode-456000"  ...

	* Stopping node "multinode-456000"  ...

	* Stopping node "multinode-456000"  ...

	* Stopping node "multinode-456000"  ...

-- /stdout --

** stderr ** 

	X Exiting due to GUEST_STOP_TIMEOUT: Unable to stop VM: docker container inspect multinode-456000 --format=<no value>: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_stop_24ef9d461bcadf056806ccb9bba8d5f9f54754a6_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

multinode_test.go:320: failed to run minikube stop. args "out/minikube-darwin-amd64 node list -p multinode-456000" : exit status 82

multinode_test.go:323: (dbg) Run:  out/minikube-darwin-amd64 start -p multinode-456000 --wait=true -v=8 --alsologtostderr

E0115 05:39:53.935528   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 05:40:56.671785   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:42:57.087719   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 05:44:54.011354   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 05:45:56.747658   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:48:59.793885   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 05:49:53.995936   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 05:50:56.730530   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

multinode_test.go:323: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p multinode-456000 --wait=true -v=8 --alsologtostderr: exit status 52 (12m54.526982459s)

-- stdout --

	* [multinode-456000] minikube v1.32.0 on Darwin 14.2.1

	  - MINIKUBE_LOCATION=17953

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the docker driver based on existing profile

	* Starting control plane node multinode-456000 in cluster multinode-456000

	* Pulling base image v0.0.42-1704759386-17866 ...

	* docker "multinode-456000" container is missing, will recreate.

	* Creating docker container (CPUs=2, Memory=2200MB) ...

	* docker "multinode-456000" container is missing, will recreate.

	* Creating docker container (CPUs=2, Memory=2200MB) ...

-- /stdout --

** stderr ** 

	I0115 05:39:20.172407   70816 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:39:20.172698   70816 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:39:20.172705   70816 out.go:309] Setting ErrFile to fd 2...

	I0115 05:39:20.172709   70816 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:39:20.172891   70816 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:39:20.174378   70816 out.go:303] Setting JSON to false

	I0115 05:39:20.196500   70816 start.go:128] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":32703,"bootTime":1705293257,"procs":488,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.2.1","kernelVersion":"23.2.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}

	W0115 05:39:20.196597   70816 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0115 05:39:20.218977   70816 out.go:177] * [multinode-456000] minikube v1.32.0 on Darwin 14.2.1

	I0115 05:39:20.262840   70816 out.go:177]   - MINIKUBE_LOCATION=17953

	I0115 05:39:20.262946   70816 notify.go:220] Checking for updates...

	I0115 05:39:20.306443   70816 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	I0115 05:39:20.327596   70816 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-amd64

	I0115 05:39:20.370386   70816 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0115 05:39:20.391785   70816 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	I0115 05:39:20.413768   70816 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0115 05:39:20.436109   70816 config.go:182] Loaded profile config "multinode-456000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0115 05:39:20.436278   70816 driver.go:392] Setting default libvirt URI to qemu:///system

	I0115 05:39:20.492844   70816 docker.go:122] docker version: linux-24.0.7:Docker Desktop 4.26.0 (130397)

	I0115 05:39:20.493009   70816 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0115 05:39:20.594702   70816 info.go:266] docker info: {ID:37b3081a-8cd6-457e-b2a4-79dc82345b06 Containers:2 ContainersRunning:0 ContainersPaused:0 ContainersStopped:2 Images:2 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:47 OomKillDisable:false NGoroutines:83 SystemTime:2024-01-15 13:39:20.585442447 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:6.5.11-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServe

rAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6221279232 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f Expected:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f} RuncCommit:{ID:v1.1.10-0-g18a0cb0 Expected:v1.1.10-0-g18a0cb0} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=unconf

ined name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.0-desktop.2] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.23.3-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Mana

ges Docker extensions Vendor:Docker Inc. Version:v0.2.21] map[Name:feedback Path:/Users/jenkins/.docker/cli-plugins/docker-feedback SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-feedback] ShortDescription:Provide feedback, right in your terminal! Vendor:Docker Inc. Version:0.1] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.10] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/d

ocker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Scout Vendor:Docker Inc. Version:v1.2.0]] Warnings:<nil>}}

	I0115 05:39:20.637449   70816 out.go:177] * Using the docker driver based on existing profile

	I0115 05:39:20.658455   70816 start.go:298] selected driver: docker

	I0115 05:39:20.658526   70816 start.go:902] validating driver "docker" against &{Name:multinode-456000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:multinode-456000 Namespace:default APIServerName:minikube

CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Disabl

eMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs:}

	I0115 05:39:20.658646   70816 start.go:913] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0115 05:39:20.658851   70816 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0115 05:39:20.761323   70816 info.go:266] docker info: {ID:37b3081a-8cd6-457e-b2a4-79dc82345b06 Containers:2 ContainersRunning:0 ContainersPaused:0 ContainersStopped:2 Images:2 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:47 OomKillDisable:false NGoroutines:83 SystemTime:2024-01-15 13:39:20.752337162 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:6.5.11-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServe

rAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6221279232 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f Expected:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f} RuncCommit:{ID:v1.1.10-0-g18a0cb0 Expected:v1.1.10-0-g18a0cb0} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=unconf

ined name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.0-desktop.2] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.23.3-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Mana

ges Docker extensions Vendor:Docker Inc. Version:v0.2.21] map[Name:feedback Path:/Users/jenkins/.docker/cli-plugins/docker-feedback SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-feedback] ShortDescription:Provide feedback, right in your terminal! Vendor:Docker Inc. Version:0.1] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.10] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/d

ocker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Scout Vendor:Docker Inc. Version:v1.2.0]] Warnings:<nil>}}

	I0115 05:39:20.764441   70816 start_flags.go:927] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0115 05:39:20.764515   70816 cni.go:84] Creating CNI manager for ""

	I0115 05:39:20.764524   70816 cni.go:136] 1 nodes found, recommending kindnet

	I0115 05:39:20.764533   70816 start_flags.go:321] config:

	{Name:multinode-456000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:multinode-456000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:do

cker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: S

taticIP: SSHAuthSock: SSHAgentPID:0 GPUs:}

	I0115 05:39:20.785957   70816 out.go:177] * Starting control plane node multinode-456000 in cluster multinode-456000

	I0115 05:39:20.807036   70816 cache.go:121] Beginning downloading kic base image for docker with docker

	I0115 05:39:20.850936   70816 out.go:177] * Pulling base image v0.0.42-1704759386-17866 ...

	I0115 05:39:20.872850   70816 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0115 05:39:20.872902   70816 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 in local docker daemon

	I0115 05:39:20.872931   70816 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4

	I0115 05:39:20.872951   70816 cache.go:56] Caching tarball of preloaded images

	I0115 05:39:20.873154   70816 preload.go:174] Found /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4 in cache, skipping download

	I0115 05:39:20.873173   70816 cache.go:59] Finished verifying existence of preloaded tar for  v1.28.4 on docker

	I0115 05:39:20.873361   70816 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/multinode-456000/config.json ...

	I0115 05:39:20.925220   70816 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 in local docker daemon, skipping pull

	I0115 05:39:20.925290   70816 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 exists in daemon, skipping load

	I0115 05:39:20.925321   70816 cache.go:194] Successfully downloaded all kic artifacts

	I0115 05:39:20.925361   70816 start.go:365] acquiring machines lock for multinode-456000: {Name:mk3c781fec38dc7197a8eca34d9ca558cb93e4e6 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0115 05:39:20.925447   70816 start.go:369] acquired machines lock for "multinode-456000" in 68.604µs

	I0115 05:39:20.925467   70816 start.go:96] Skipping create...Using existing machine configuration

	I0115 05:39:20.925476   70816 fix.go:54] fixHost starting: 

	I0115 05:39:20.925712   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:20.975616   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:20.975663   70816 fix.go:102] recreateIfNeeded on multinode-456000: state= err=unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:20.975685   70816 fix.go:107] machineExists: false. err=machine does not exist

	I0115 05:39:20.997510   70816 out.go:177] * docker "multinode-456000" container is missing, will recreate.

	I0115 05:39:21.041260   70816 delete.go:124] DEMOLISHING multinode-456000 ...

	I0115 05:39:21.041424   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:21.092952   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	W0115 05:39:21.092999   70816 stop.go:75] unable to get state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:21.093018   70816 delete.go:128] stophost failed (probably ok): ssh power off: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:21.093401   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:21.144067   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:21.144124   70816 delete.go:82] Unable to get host status for multinode-456000, assuming it has already been deleted: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:21.144206   70816 cli_runner.go:164] Run: docker container inspect -f {{.Id}} multinode-456000

	W0115 05:39:21.194668   70816 cli_runner.go:211] docker container inspect -f {{.Id}} multinode-456000 returned with exit code 1

	I0115 05:39:21.194707   70816 kic.go:371] could not find the container multinode-456000 to remove it. will try anyways

	I0115 05:39:21.194781   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:21.244804   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	W0115 05:39:21.244849   70816 oci.go:84] error getting container status, will try to delete anyways: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:21.244930   70816 cli_runner.go:164] Run: docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0"

	W0115 05:39:21.295608   70816 cli_runner.go:211] docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0" returned with exit code 1

	I0115 05:39:21.295642   70816 oci.go:650] error shutdown multinode-456000: docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0": exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:22.296744   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:22.349449   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:22.349505   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:22.349519   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:39:22.349555   70816 retry.go:31] will retry after 566.252015ms: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:22.917441   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:22.971129   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:22.971181   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:22.971217   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:39:22.971238   70816 retry.go:31] will retry after 500.832938ms: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:23.473400   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:23.525253   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:23.525300   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:23.525312   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:39:23.525339   70816 retry.go:31] will retry after 873.949848ms: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:24.400089   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:24.452483   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:24.452543   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:24.452554   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:39:24.452576   70816 retry.go:31] will retry after 1.673631963s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:26.127109   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:26.178608   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:26.178655   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:26.178665   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:39:26.178689   70816 retry.go:31] will retry after 3.329562781s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:29.508790   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:29.606666   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:29.606722   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:29.606748   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:39:29.606772   70816 retry.go:31] will retry after 2.457316192s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:32.064298   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:32.118267   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:32.118316   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:32.118325   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:39:32.118348   70816 retry.go:31] will retry after 7.03438726s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:39.153320   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:39:39.207702   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:39:39.207758   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:39.207768   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:39:39.207793   70816 oci.go:88] couldn't shut down multinode-456000 (might be okay): verify shutdown: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:39:39.207862   70816 cli_runner.go:164] Run: docker rm -f -v multinode-456000

	I0115 05:39:39.258515   70816 cli_runner.go:164] Run: docker container inspect -f {{.Id}} multinode-456000

	W0115 05:39:39.308152   70816 cli_runner.go:211] docker container inspect -f {{.Id}} multinode-456000 returned with exit code 1

	I0115 05:39:39.308261   70816 cli_runner.go:164] Run: docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 05:39:39.359055   70816 cli_runner.go:164] Run: docker network rm multinode-456000

	I0115 05:39:39.456992   70816 fix.go:114] Sleeping 1 second for extra luck!

	I0115 05:39:40.457833   70816 start.go:125] createHost starting for "" (driver="docker")

	I0115 05:39:40.480205   70816 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...

	I0115 05:39:40.480378   70816 start.go:159] libmachine.API.Create for "multinode-456000" (driver="docker")

	I0115 05:39:40.480430   70816 client.go:168] LocalClient.Create starting

	I0115 05:39:40.480618   70816 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem

	I0115 05:39:40.480709   70816 main.go:141] libmachine: Decoding PEM data...

	I0115 05:39:40.480745   70816 main.go:141] libmachine: Parsing certificate...

	I0115 05:39:40.480846   70816 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem

	I0115 05:39:40.480923   70816 main.go:141] libmachine: Decoding PEM data...

	I0115 05:39:40.480939   70816 main.go:141] libmachine: Parsing certificate...

	I0115 05:39:40.481641   70816 cli_runner.go:164] Run: docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	W0115 05:39:40.533458   70816 cli_runner.go:211] docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1

	I0115 05:39:40.533542   70816 network_create.go:281] running [docker network inspect multinode-456000] to gather additional debugging logs...

	I0115 05:39:40.533562   70816 cli_runner.go:164] Run: docker network inspect multinode-456000

	W0115 05:39:40.583888   70816 cli_runner.go:211] docker network inspect multinode-456000 returned with exit code 1

	I0115 05:39:40.583917   70816 network_create.go:284] error running [docker network inspect multinode-456000]: docker network inspect multinode-456000: exit status 1

	stdout:

	[]

	stderr:

	Error response from daemon: network multinode-456000 not found

	I0115 05:39:40.583949   70816 network_create.go:286] output of [docker network inspect multinode-456000]: -- stdout --

	[]

	-- /stdout --

	** stderr ** 

	Error response from daemon: network multinode-456000 not found

	** /stderr **

	I0115 05:39:40.584071   70816 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 05:39:40.635858   70816 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}

	I0115 05:39:40.636242   70816 network.go:209] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc002586060}

	I0115 05:39:40.636258   70816 network_create.go:124] attempt to create docker network multinode-456000 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 65535 ...

	I0115 05:39:40.636337   70816 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=multinode-456000 multinode-456000

	I0115 05:39:40.721443   70816 network_create.go:108] docker network multinode-456000 192.168.58.0/24 created

	I0115 05:39:40.721481   70816 kic.go:121] calculated static IP "192.168.58.2" for the "multinode-456000" container

	I0115 05:39:40.721594   70816 cli_runner.go:164] Run: docker ps -a --format {{.Names}}

	I0115 05:39:40.773538   70816 cli_runner.go:164] Run: docker volume create multinode-456000 --label name.minikube.sigs.k8s.io=multinode-456000 --label created_by.minikube.sigs.k8s.io=true

	I0115 05:39:40.823455   70816 oci.go:103] Successfully created a docker volume multinode-456000

	I0115 05:39:40.823585   70816 cli_runner.go:164] Run: docker run --rm --name multinode-456000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=multinode-456000 --entrypoint /usr/bin/test -v multinode-456000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -d /var/lib

	I0115 05:39:41.123575   70816 oci.go:107] Successfully prepared a docker volume multinode-456000

	I0115 05:39:41.123612   70816 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0115 05:39:41.123627   70816 kic.go:194] Starting extracting preloaded images to volume ...

	I0115 05:39:41.123724   70816 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v multinode-456000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -I lz4 -xf /preloaded.tar -C /extractDir

	I0115 05:45:40.553941   70816 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0115 05:45:40.554075   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:40.607346   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:40.607466   70816 retry.go:31] will retry after 262.826552ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:40.870768   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:40.922759   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:40.922878   70816 retry.go:31] will retry after 399.10528ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:41.322766   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:41.377227   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:41.377327   70816 retry.go:31] will retry after 608.984491ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:41.987719   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:42.040649   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:45:42.040774   70816 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:45:42.040795   70816 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:42.040857   70816 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0115 05:45:42.040924   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:42.092509   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:42.092612   70816 retry.go:31] will retry after 165.331702ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:42.258844   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:42.312602   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:42.312702   70816 retry.go:31] will retry after 370.472443ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:42.683542   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:42.737522   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:42.737622   70816 retry.go:31] will retry after 445.525112ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:43.185511   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:43.240209   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:45:43.240310   70816 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:45:43.240332   70816 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:43.240347   70816 start.go:128] duration metric: createHost completed in 6m2.71084566s

	I0115 05:45:43.240409   70816 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0115 05:45:43.240461   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:43.292406   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:43.292491   70816 retry.go:31] will retry after 232.995601ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:43.526153   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:43.578202   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:43.578289   70816 retry.go:31] will retry after 479.499551ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:44.058333   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:44.111380   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:44.111471   70816 retry.go:31] will retry after 398.889322ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:44.512617   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:44.566908   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:45:44.567015   70816 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:45:44.567030   70816 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:44.567085   70816 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0115 05:45:44.567140   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:44.678528   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:44.678610   70816 retry.go:31] will retry after 165.756821ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:44.845202   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:44.899520   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:44.899611   70816 retry.go:31] will retry after 401.816217ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:45.302625   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:45.357355   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:45.357455   70816 retry.go:31] will retry after 319.26478ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:45.678396   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:45.730187   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:45:45.730280   70816 retry.go:31] will retry after 712.439723ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:46.443955   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:45:46.497614   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:45:46.497723   70816 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:45:46.497738   70816 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:46.497751   70816 fix.go:56] fixHost completed within 6m25.501882411s

	I0115 05:45:46.497764   70816 start.go:83] releasing machines lock for "multinode-456000", held for 6m25.501915167s

	W0115 05:45:46.497777   70816 start.go:694] error starting host: recreate: creating host: create host timed out in 360.000000 seconds

	W0115 05:45:46.497845   70816 out.go:239] ! StartHost failed, but will try again: recreate: creating host: create host timed out in 360.000000 seconds

	! StartHost failed, but will try again: recreate: creating host: create host timed out in 360.000000 seconds

	I0115 05:45:46.497851   70816 start.go:709] Will try again in 5 seconds ...

	I0115 05:45:51.498179   70816 start.go:365] acquiring machines lock for multinode-456000: {Name:mk3c781fec38dc7197a8eca34d9ca558cb93e4e6 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0115 05:45:51.498367   70816 start.go:369] acquired machines lock for "multinode-456000" in 150.72µs

	I0115 05:45:51.498409   70816 start.go:96] Skipping create...Using existing machine configuration

	I0115 05:45:51.498418   70816 fix.go:54] fixHost starting: 

	I0115 05:45:51.498867   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:45:51.550265   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:45:51.550314   70816 fix.go:102] recreateIfNeeded on multinode-456000: state= err=unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:51.550334   70816 fix.go:107] machineExists: false. err=machine does not exist

	I0115 05:45:51.572238   70816 out.go:177] * docker "multinode-456000" container is missing, will recreate.

	I0115 05:45:51.615784   70816 delete.go:124] DEMOLISHING multinode-456000 ...

	I0115 05:45:51.615985   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:45:51.667151   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	W0115 05:45:51.667198   70816 stop.go:75] unable to get state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:51.667217   70816 delete.go:128] stophost failed (probably ok): ssh power off: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:51.667585   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:45:51.717824   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:45:51.717888   70816 delete.go:82] Unable to get host status for multinode-456000, assuming it has already been deleted: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:51.717973   70816 cli_runner.go:164] Run: docker container inspect -f {{.Id}} multinode-456000

	W0115 05:45:51.767515   70816 cli_runner.go:211] docker container inspect -f {{.Id}} multinode-456000 returned with exit code 1

	I0115 05:45:51.767543   70816 kic.go:371] could not find the container multinode-456000 to remove it. will try anyways

	I0115 05:45:51.767618   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:45:51.818992   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	W0115 05:45:51.819036   70816 oci.go:84] error getting container status, will try to delete anyways: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:51.819119   70816 cli_runner.go:164] Run: docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0"

	W0115 05:45:51.869549   70816 cli_runner.go:211] docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0" returned with exit code 1

	I0115 05:45:51.869578   70816 oci.go:650] error shutdown multinode-456000: docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0": exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:52.870649   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:45:52.923781   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:45:52.923825   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:52.923838   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:45:52.923872   70816 retry.go:31] will retry after 397.715553ms: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:53.321838   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:45:53.375339   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:45:53.375382   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:53.375395   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:45:53.375422   70816 retry.go:31] will retry after 789.423035ms: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:54.165116   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:45:54.218453   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:45:54.218500   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:54.218512   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:45:54.218538   70816 retry.go:31] will retry after 1.369482303s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:55.590293   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:45:55.643636   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:45:55.643677   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:55.643699   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:45:55.643723   70816 retry.go:31] will retry after 1.191475319s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:56.837071   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:45:56.892508   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:45:56.892557   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:56.892568   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:45:56.892589   70816 retry.go:31] will retry after 2.874119905s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:59.768396   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:45:59.820611   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:45:59.820658   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:45:59.820667   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:45:59.820696   70816 retry.go:31] will retry after 2.046631515s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:46:01.867632   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:46:01.921160   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:46:01.921205   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:46:01.921214   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:46:01.921238   70816 retry.go:31] will retry after 4.902148527s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:46:06.824157   70816 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:46:06.877973   70816 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:46:06.878016   70816 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:46:06.878027   70816 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:46:06.878058   70816 oci.go:88] couldn't shut down multinode-456000 (might be okay): verify shutdown: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:46:06.878143   70816 cli_runner.go:164] Run: docker rm -f -v multinode-456000

	I0115 05:46:06.929140   70816 cli_runner.go:164] Run: docker container inspect -f {{.Id}} multinode-456000

	W0115 05:46:06.979527   70816 cli_runner.go:211] docker container inspect -f {{.Id}} multinode-456000 returned with exit code 1

	I0115 05:46:06.979636   70816 cli_runner.go:164] Run: docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 05:46:07.030205   70816 cli_runner.go:164] Run: docker network rm multinode-456000

	I0115 05:46:07.138826   70816 fix.go:114] Sleeping 1 second for extra luck!

	I0115 05:46:08.140328   70816 start.go:125] createHost starting for "" (driver="docker")

	I0115 05:46:08.163417   70816 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...

	I0115 05:46:08.163581   70816 start.go:159] libmachine.API.Create for "multinode-456000" (driver="docker")

	I0115 05:46:08.163625   70816 client.go:168] LocalClient.Create starting

	I0115 05:46:08.163822   70816 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem

	I0115 05:46:08.163929   70816 main.go:141] libmachine: Decoding PEM data...

	I0115 05:46:08.163958   70816 main.go:141] libmachine: Parsing certificate...

	I0115 05:46:08.164039   70816 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem

	I0115 05:46:08.164108   70816 main.go:141] libmachine: Decoding PEM data...

	I0115 05:46:08.164124   70816 main.go:141] libmachine: Parsing certificate...

	I0115 05:46:08.164807   70816 cli_runner.go:164] Run: docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	W0115 05:46:08.217421   70816 cli_runner.go:211] docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1

	I0115 05:46:08.217514   70816 network_create.go:281] running [docker network inspect multinode-456000] to gather additional debugging logs...

	I0115 05:46:08.217531   70816 cli_runner.go:164] Run: docker network inspect multinode-456000

	W0115 05:46:08.267984   70816 cli_runner.go:211] docker network inspect multinode-456000 returned with exit code 1

	I0115 05:46:08.268011   70816 network_create.go:284] error running [docker network inspect multinode-456000]: docker network inspect multinode-456000: exit status 1

	stdout:

	[]

	stderr:

	Error response from daemon: network multinode-456000 not found

	I0115 05:46:08.268026   70816 network_create.go:286] output of [docker network inspect multinode-456000]: -- stdout --

	[]

	-- /stdout --

	** stderr ** 

	Error response from daemon: network multinode-456000 not found

	** /stderr **

	I0115 05:46:08.268177   70816 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 05:46:08.320136   70816 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}

	I0115 05:46:08.321739   70816 network.go:212] skipping subnet 192.168.58.0/24 that is reserved: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}

	I0115 05:46:08.322073   70816 network.go:209] using free private subnet 192.168.67.0/24: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc000d5bac0}

	I0115 05:46:08.322091   70816 network_create.go:124] attempt to create docker network multinode-456000 192.168.67.0/24 with gateway 192.168.67.1 and MTU of 65535 ...

	I0115 05:46:08.322170   70816 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.67.0/24 --gateway=192.168.67.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=multinode-456000 multinode-456000

	I0115 05:46:08.407556   70816 network_create.go:108] docker network multinode-456000 192.168.67.0/24 created

	I0115 05:46:08.407589   70816 kic.go:121] calculated static IP "192.168.67.2" for the "multinode-456000" container

	I0115 05:46:08.407713   70816 cli_runner.go:164] Run: docker ps -a --format {{.Names}}

	I0115 05:46:08.458723   70816 cli_runner.go:164] Run: docker volume create multinode-456000 --label name.minikube.sigs.k8s.io=multinode-456000 --label created_by.minikube.sigs.k8s.io=true

	I0115 05:46:08.508647   70816 oci.go:103] Successfully created a docker volume multinode-456000

	I0115 05:46:08.508770   70816 cli_runner.go:164] Run: docker run --rm --name multinode-456000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=multinode-456000 --entrypoint /usr/bin/test -v multinode-456000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -d /var/lib

	I0115 05:46:08.803509   70816 oci.go:107] Successfully prepared a docker volume multinode-456000

	I0115 05:46:08.803540   70816 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0115 05:46:08.803559   70816 kic.go:194] Starting extracting preloaded images to volume ...

	I0115 05:46:08.803669   70816 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v multinode-456000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -I lz4 -xf /preloaded.tar -C /extractDir

	I0115 05:52:08.145399   70816 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0115 05:52:08.145528   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:08.198388   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:08.198531   70816 retry.go:31] will retry after 292.278081ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:08.491794   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:08.543400   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:08.543498   70816 retry.go:31] will retry after 456.023333ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:09.000633   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:09.054661   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:09.054773   70816 retry.go:31] will retry after 759.770267ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:09.816070   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:09.871312   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:52:09.871421   70816 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:52:09.871438   70816 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:09.871495   70816 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0115 05:52:09.871550   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:09.920944   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:09.921043   70816 retry.go:31] will retry after 273.554179ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:10.196074   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:10.249662   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:10.249765   70816 retry.go:31] will retry after 438.90671ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:10.690098   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:10.742027   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:10.742126   70816 retry.go:31] will retry after 735.585288ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:11.479355   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:11.533543   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:52:11.533648   70816 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:52:11.533670   70816 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:11.533682   70816 start.go:128] duration metric: createHost completed in 6m3.413266275s

	I0115 05:52:11.533750   70816 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0115 05:52:11.533820   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:11.585229   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:11.585344   70816 retry.go:31] will retry after 356.202013ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:11.942549   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:11.996348   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:11.996439   70816 retry.go:31] will retry after 341.693744ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:12.340471   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:12.392784   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:12.392891   70816 retry.go:31] will retry after 759.740909ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:13.154277   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:13.206371   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:52:13.206471   70816 start.go:275] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:52:13.206486   70816 start.go:242] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:13.206545   70816 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0115 05:52:13.206606   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:13.256868   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:13.256964   70816 retry.go:31] will retry after 245.166444ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:13.504255   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:13.555601   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:13.555692   70816 retry.go:31] will retry after 204.793892ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:13.762823   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:13.815385   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	I0115 05:52:13.815474   70816 retry.go:31] will retry after 647.962403ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:14.465746   70816 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000

	W0115 05:52:14.519413   70816 cli_runner.go:211] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000 returned with exit code 1

	W0115 05:52:14.519523   70816 start.go:290] error running df -BG /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	W0115 05:52:14.519539   70816 start.go:247] error getting GiB of /var that is available: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "multinode-456000": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" multinode-456000: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:14.519550   70816 fix.go:56] fixHost completed within 6m23.042169872s

	I0115 05:52:14.519557   70816 start.go:83] releasing machines lock for "multinode-456000", held for 6m23.042211021s

	W0115 05:52:14.519636   70816 out.go:239] * Failed to start docker container. Running "minikube delete -p multinode-456000" may fix it: recreate: creating host: create host timed out in 360.000000 seconds

	* Failed to start docker container. Running "minikube delete -p multinode-456000" may fix it: recreate: creating host: create host timed out in 360.000000 seconds

	I0115 05:52:14.563003   70816 out.go:177] 

	W0115 05:52:14.586298   70816 out.go:239] X Exiting due to DRV_CREATE_TIMEOUT: Failed to start host: recreate: creating host: create host timed out in 360.000000 seconds

	X Exiting due to DRV_CREATE_TIMEOUT: Failed to start host: recreate: creating host: create host timed out in 360.000000 seconds

	W0115 05:52:14.586366   70816 out.go:239] * Suggestion: Try 'minikube delete', and disable any conflicting VPN or firewall software

	* Suggestion: Try 'minikube delete', and disable any conflicting VPN or firewall software

	W0115 05:52:14.586449   70816 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/7072

	* Related issue: https://github.com/kubernetes/minikube/issues/7072

	I0115 05:52:14.629912   70816 out.go:177] 

** /stderr **

multinode_test.go:325: failed to run minikube start. args "out/minikube-darwin-amd64 node list -p multinode-456000" : exit status 52

multinode_test.go:328: (dbg) Run:  out/minikube-darwin-amd64 node list -p multinode-456000

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/RestartKeepsNodes]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "8e43f70d1baf1704ac1bc2e5ec291c548ae587ed5b0694e18934c07f566bf186",

	        "Created": "2024-01-15T13:46:08.370563998Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (110.473314ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:52:14.930532   71094 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/RestartKeepsNodes (787.68s)

=== RUN   TestMultiNode/serial/DeleteNode

multinode_test.go:422: (dbg) Run:  out/minikube-darwin-amd64 -p multinode-456000 node delete m03

multinode_test.go:422: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p multinode-456000 node delete m03: exit status 80 (205.253183ms)

-- stdout --

-- /stdout --

** stderr ** 

	X Exiting due to GUEST_STATUS: Unable to get machine status: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format=<no value>: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_node_494011a6b05fec7d81170870a2aee2ef446d16a4_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

multinode_test.go:424: node stop returned an error. args "out/minikube-darwin-amd64 -p multinode-456000 node delete m03": exit status 80

multinode_test.go:428: (dbg) Run:  out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr

multinode_test.go:428: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr: exit status 7 (109.948286ms)

-- stdout --

	multinode-456000

	type: Control Plane

	host: Nonexistent

	kubelet: Nonexistent

	apiserver: Nonexistent

	kubeconfig: Nonexistent

-- /stdout --

** stderr ** 

	I0115 05:52:15.194470   71102 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:52:15.194953   71102 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:52:15.194959   71102 out.go:309] Setting ErrFile to fd 2...

	I0115 05:52:15.194963   71102 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:52:15.195160   71102 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:52:15.195335   71102 out.go:303] Setting JSON to false

	I0115 05:52:15.195358   71102 mustload.go:65] Loading cluster: multinode-456000

	I0115 05:52:15.195402   71102 notify.go:220] Checking for updates...

	I0115 05:52:15.195623   71102 config.go:182] Loaded profile config "multinode-456000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0115 05:52:15.195634   71102 status.go:255] checking status of multinode-456000 ...

	I0115 05:52:15.196088   71102 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:15.246099   71102 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:15.246162   71102 status.go:330] multinode-456000 host status = "" (err=state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	)

	I0115 05:52:15.246179   71102 status.go:257] multinode-456000 status: &{Name:multinode-456000 Host:Nonexistent Kubelet:Nonexistent APIServer:Nonexistent Kubeconfig:Nonexistent Worker:false TimeToStop: DockerEnv: PodManEnv:}

	E0115 05:52:15.246195   71102 status.go:260] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	E0115 05:52:15.246203   71102 status.go:263] The "multinode-456000" host does not exist!

** /stderr **

multinode_test.go:430: failed to run minikube status. args "out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr" : exit status 7

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/DeleteNode]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "8e43f70d1baf1704ac1bc2e5ec291c548ae587ed5b0694e18934c07f566bf186",

	        "Created": "2024-01-15T13:46:08.370563998Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (108.062659ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:52:15.408561   71108 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/DeleteNode (0.48s)

=== RUN   TestMultiNode/serial/StopMultiNode

multinode_test.go:342: (dbg) Run:  out/minikube-darwin-amd64 -p multinode-456000 stop

multinode_test.go:342: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p multinode-456000 stop: exit status 82 (11.882372484s)

-- stdout --

	* Stopping node "multinode-456000"  ...

	* Stopping node "multinode-456000"  ...

	* Stopping node "multinode-456000"  ...

	* Stopping node "multinode-456000"  ...

	* Stopping node "multinode-456000"  ...

	* Stopping node "multinode-456000"  ...

-- /stdout --

** stderr ** 

	X Exiting due to GUEST_STOP_TIMEOUT: Unable to stop VM: docker container inspect multinode-456000 --format=<no value>: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_stop_24ef9d461bcadf056806ccb9bba8d5f9f54754a6_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

multinode_test.go:344: node stop returned an error. args "out/minikube-darwin-amd64 -p multinode-456000 stop": exit status 82

multinode_test.go:348: (dbg) Run:  out/minikube-darwin-amd64 -p multinode-456000 status

multinode_test.go:348: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p multinode-456000 status: exit status 7 (109.232232ms)

-- stdout --

	multinode-456000

	type: Control Plane

	host: Nonexistent

	kubelet: Nonexistent

	apiserver: Nonexistent

	kubeconfig: Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:52:27.399867   71127 status.go:260] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	E0115 05:52:27.399879   71127 status.go:263] The "multinode-456000" host does not exist!

** /stderr **

multinode_test.go:355: (dbg) Run:  out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr

multinode_test.go:355: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr: exit status 7 (109.512416ms)

-- stdout --

	multinode-456000

	type: Control Plane

	host: Nonexistent

	kubelet: Nonexistent

	apiserver: Nonexistent

	kubeconfig: Nonexistent

-- /stdout --

** stderr ** 

	I0115 05:52:27.457782   71131 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:52:27.458094   71131 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:52:27.458100   71131 out.go:309] Setting ErrFile to fd 2...

	I0115 05:52:27.458105   71131 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:52:27.458302   71131 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:52:27.458501   71131 out.go:303] Setting JSON to false

	I0115 05:52:27.458525   71131 mustload.go:65] Loading cluster: multinode-456000

	I0115 05:52:27.458572   71131 notify.go:220] Checking for updates...

	I0115 05:52:27.458805   71131 config.go:182] Loaded profile config "multinode-456000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0115 05:52:27.458817   71131 status.go:255] checking status of multinode-456000 ...

	I0115 05:52:27.459289   71131 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:27.509381   71131 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:27.509439   71131 status.go:330] multinode-456000 host status = "" (err=state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	)

	I0115 05:52:27.509456   71131 status.go:257] multinode-456000 status: &{Name:multinode-456000 Host:Nonexistent Kubelet:Nonexistent APIServer:Nonexistent Kubeconfig:Nonexistent Worker:false TimeToStop: DockerEnv: PodManEnv:}

	E0115 05:52:27.509475   71131 status.go:260] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	E0115 05:52:27.509483   71131 status.go:263] The "multinode-456000" host does not exist!

** /stderr **

multinode_test.go:361: incorrect number of stopped hosts: args "out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr": multinode-456000

type: Control Plane

host: Nonexistent

kubelet: Nonexistent

apiserver: Nonexistent

kubeconfig: Nonexistent

multinode_test.go:365: incorrect number of stopped kubelets: args "out/minikube-darwin-amd64 -p multinode-456000 status --alsologtostderr": multinode-456000

type: Control Plane

host: Nonexistent

kubelet: Nonexistent

apiserver: Nonexistent

kubeconfig: Nonexistent

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/StopMultiNode]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "8e43f70d1baf1704ac1bc2e5ec291c548ae587ed5b0694e18934c07f566bf186",

	        "Created": "2024-01-15T13:46:08.370563998Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.67.0/24",

	                    "Gateway": "192.168.67.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (109.058551ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:52:27.673132   71137 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/StopMultiNode (12.27s)

=== RUN   TestMultiNode/serial/RestartMultiNode

multinode_test.go:372: (dbg) Run:  docker version -f {{.Server.Version}}

multinode_test.go:382: (dbg) Run:  out/minikube-darwin-amd64 start -p multinode-456000 --wait=true -v=8 --alsologtostderr --driver=docker 

E0115 05:54:53.978324   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

multinode_test.go:382: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p multinode-456000 --wait=true -v=8 --alsologtostderr --driver=docker : signal: killed (2m37.083150016s)

-- stdout --

	* [multinode-456000] minikube v1.32.0 on Darwin 14.2.1

	  - MINIKUBE_LOCATION=17953

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the docker driver based on existing profile

	* Starting control plane node multinode-456000 in cluster multinode-456000

	* Pulling base image v0.0.42-1704759386-17866 ...

	* docker "multinode-456000" container is missing, will recreate.

	* Creating docker container (CPUs=2, Memory=2200MB) ...

-- /stdout --

** stderr ** 

	I0115 05:52:27.785961   71143 out.go:296] Setting OutFile to fd 1 ...

	I0115 05:52:27.786182   71143 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:52:27.786187   71143 out.go:309] Setting ErrFile to fd 2...

	I0115 05:52:27.786191   71143 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0115 05:52:27.786362   71143 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17953-64881/.minikube/bin

	I0115 05:52:27.787755   71143 out.go:303] Setting JSON to false

	I0115 05:52:27.809970   71143 start.go:128] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":33490,"bootTime":1705293257,"procs":492,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.2.1","kernelVersion":"23.2.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}

	W0115 05:52:27.810087   71143 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0115 05:52:27.834139   71143 out.go:177] * [multinode-456000] minikube v1.32.0 on Darwin 14.2.1

	I0115 05:52:27.854968   71143 out.go:177]   - MINIKUBE_LOCATION=17953

	I0115 05:52:27.855084   71143 notify.go:220] Checking for updates...

	I0115 05:52:27.876820   71143 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	I0115 05:52:27.899096   71143 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-amd64

	I0115 05:52:27.921092   71143 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0115 05:52:27.941890   71143 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	I0115 05:52:27.962996   71143 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0115 05:52:27.984808   71143 config.go:182] Loaded profile config "multinode-456000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0115 05:52:27.985574   71143 driver.go:392] Setting default libvirt URI to qemu:///system

	I0115 05:52:28.042573   71143 docker.go:122] docker version: linux-24.0.7:Docker Desktop 4.26.0 (130397)

	I0115 05:52:28.042726   71143 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0115 05:52:28.141736   71143 info.go:266] docker info: {ID:37b3081a-8cd6-457e-b2a4-79dc82345b06 Containers:4 ContainersRunning:0 ContainersPaused:0 ContainersStopped:4 Images:2 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:53 OomKillDisable:false NGoroutines:103 SystemTime:2024-01-15 13:52:28.132333235 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:6.5.11-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServ

erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6221279232 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f Expected:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f} RuncCommit:{ID:v1.1.10-0-g18a0cb0 Expected:v1.1.10-0-g18a0cb0} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=uncon

fined name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.0-desktop.2] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.23.3-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Man

ages Docker extensions Vendor:Docker Inc. Version:v0.2.21] map[Name:feedback Path:/Users/jenkins/.docker/cli-plugins/docker-feedback SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-feedback] ShortDescription:Provide feedback, right in your terminal! Vendor:Docker Inc. Version:0.1] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.10] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/

docker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Scout Vendor:Docker Inc. Version:v1.2.0]] Warnings:<nil>}}

	I0115 05:52:28.163215   71143 out.go:177] * Using the docker driver based on existing profile

	I0115 05:52:28.184423   71143 start.go:298] selected driver: docker

	I0115 05:52:28.184456   71143 start.go:902] validating driver "docker" against &{Name:multinode-456000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:multinode-456000 Namespace:default APIServerName:minikube

CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Disabl

eMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs:}

	I0115 05:52:28.184571   71143 start.go:913] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0115 05:52:28.184769   71143 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0115 05:52:28.285540   71143 info.go:266] docker info: {ID:37b3081a-8cd6-457e-b2a4-79dc82345b06 Containers:4 ContainersRunning:0 ContainersPaused:0 ContainersStopped:4 Images:2 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:53 OomKillDisable:false NGoroutines:103 SystemTime:2024-01-15 13:52:28.27649053 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:8 KernelVersion:6.5.11-linuxkit OperatingSystem:Docker Desktop OSType:linux Architecture:x86_64 IndexServe

rAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:12 MemTotal:6221279232 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy:http.docker.internal:3128 HTTPSProxy:http.docker.internal:3128 NoProxy:hubproxy.docker.internal Name:docker-desktop Labels:[] ExperimentalBuild:false ServerVersion:24.0.7 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f Expected:d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f} RuncCommit:{ID:v1.1.10-0-g18a0cb0 Expected:v1.1.10-0-g18a0cb0} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=unconf

ined name=cgroupns] ProductLicense: Warnings:[WARNING: daemon is not using the default seccomp profile] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/Users/jenkins/.docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-buildx] ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.0-desktop.2] map[Name:compose Path:/Users/jenkins/.docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-compose] ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.23.3-desktop.2] map[Name:dev Path:/Users/jenkins/.docker/cli-plugins/docker-dev SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-dev] ShortDescription:Docker Dev Environments Vendor:Docker Inc. Version:v0.1.0] map[Name:extension Path:/Users/jenkins/.docker/cli-plugins/docker-extension SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-extension] ShortDescription:Mana

ges Docker extensions Vendor:Docker Inc. Version:v0.2.21] map[Name:feedback Path:/Users/jenkins/.docker/cli-plugins/docker-feedback SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-feedback] ShortDescription:Provide feedback, right in your terminal! Vendor:Docker Inc. Version:0.1] map[Name:init Path:/Users/jenkins/.docker/cli-plugins/docker-init SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-init] ShortDescription:Creates Docker-related starter files for your project Vendor:Docker Inc. Version:v0.1.0-beta.10] map[Name:sbom Path:/Users/jenkins/.docker/cli-plugins/docker-sbom SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-sbom] ShortDescription:View the packaged-based Software Bill Of Materials (SBOM) for an image URL:https://github.com/docker/sbom-cli-plugin Vendor:Anchore Inc. Version:0.6.0] map[Name:scan Path:/Users/jenkins/.docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/d

ocker-scan] ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.26.0] map[Name:scout Path:/Users/jenkins/.docker/cli-plugins/docker-scout SchemaVersion:0.1.0 ShadowedPaths:[/usr/local/lib/docker/cli-plugins/docker-scout] ShortDescription:Docker Scout Vendor:Docker Inc. Version:v1.2.0]] Warnings:<nil>}}

	I0115 05:52:28.288677   71143 start_flags.go:927] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0115 05:52:28.288714   71143 cni.go:84] Creating CNI manager for ""

	I0115 05:52:28.288722   71143 cni.go:136] 1 nodes found, recommending kindnet

	I0115 05:52:28.288730   71143 start_flags.go:321] config:

	{Name:multinode-456000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:multinode-456000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:do

cker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: S

taticIP: SSHAuthSock: SSHAgentPID:0 GPUs:}

	I0115 05:52:28.332270   71143 out.go:177] * Starting control plane node multinode-456000 in cluster multinode-456000

	I0115 05:52:28.353349   71143 cache.go:121] Beginning downloading kic base image for docker with docker

	I0115 05:52:28.397346   71143 out.go:177] * Pulling base image v0.0.42-1704759386-17866 ...

	I0115 05:52:28.418402   71143 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0115 05:52:28.418491   71143 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4

	I0115 05:52:28.418503   71143 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 in local docker daemon

	I0115 05:52:28.418521   71143 cache.go:56] Caching tarball of preloaded images

	I0115 05:52:28.418749   71143 preload.go:174] Found /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4 in cache, skipping download

	I0115 05:52:28.418768   71143 cache.go:59] Finished verifying existence of preloaded tar for  v1.28.4 on docker

	I0115 05:52:28.419820   71143 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/multinode-456000/config.json ...

	I0115 05:52:28.470803   71143 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 in local docker daemon, skipping pull

	I0115 05:52:28.470823   71143 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 exists in daemon, skipping load

	I0115 05:52:28.470843   71143 cache.go:194] Successfully downloaded all kic artifacts

	I0115 05:52:28.470893   71143 start.go:365] acquiring machines lock for multinode-456000: {Name:mk3c781fec38dc7197a8eca34d9ca558cb93e4e6 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0115 05:52:28.470982   71143 start.go:369] acquired machines lock for "multinode-456000" in 70.217µs

	I0115 05:52:28.471003   71143 start.go:96] Skipping create...Using existing machine configuration

	I0115 05:52:28.471012   71143 fix.go:54] fixHost starting: 

	I0115 05:52:28.471261   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:28.521350   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:28.521423   71143 fix.go:102] recreateIfNeeded on multinode-456000: state= err=unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:28.521443   71143 fix.go:107] machineExists: false. err=machine does not exist

	I0115 05:52:28.543203   71143 out.go:177] * docker "multinode-456000" container is missing, will recreate.

	I0115 05:52:28.585976   71143 delete.go:124] DEMOLISHING multinode-456000 ...

	I0115 05:52:28.586176   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:28.637014   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	W0115 05:52:28.637059   71143 stop.go:75] unable to get state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:28.637078   71143 delete.go:128] stophost failed (probably ok): ssh power off: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:28.637430   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:28.688281   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:28.688341   71143 delete.go:82] Unable to get host status for multinode-456000, assuming it has already been deleted: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:28.688433   71143 cli_runner.go:164] Run: docker container inspect -f {{.Id}} multinode-456000

	W0115 05:52:28.738503   71143 cli_runner.go:211] docker container inspect -f {{.Id}} multinode-456000 returned with exit code 1

	I0115 05:52:28.738537   71143 kic.go:371] could not find the container multinode-456000 to remove it. will try anyways

	I0115 05:52:28.738605   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:28.788220   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	W0115 05:52:28.788262   71143 oci.go:84] error getting container status, will try to delete anyways: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:28.788354   71143 cli_runner.go:164] Run: docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0"

	W0115 05:52:28.838413   71143 cli_runner.go:211] docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0" returned with exit code 1

	I0115 05:52:28.838447   71143 oci.go:650] error shutdown multinode-456000: docker exec --privileged -t multinode-456000 /bin/bash -c "sudo init 0": exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:29.840752   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:29.893789   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:29.893842   71143 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:29.893854   71143 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:52:29.893890   71143 retry.go:31] will retry after 265.799288ms: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:30.161954   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:30.214875   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:30.214919   71143 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:30.214933   71143 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:52:30.214958   71143 retry.go:31] will retry after 511.598206ms: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:30.727341   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:30.781555   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:30.781609   71143 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:30.781636   71143 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:52:30.781661   71143 retry.go:31] will retry after 816.701047ms: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:31.599272   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:31.653125   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:31.653167   71143 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:31.653176   71143 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:52:31.653201   71143 retry.go:31] will retry after 1.818521934s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:33.471871   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:33.522021   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:33.522064   71143 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:33.522082   71143 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:52:33.522107   71143 retry.go:31] will retry after 3.115894511s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:36.638437   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:36.690658   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:36.690708   71143 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:36.690726   71143 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:52:36.690753   71143 retry.go:31] will retry after 3.435831825s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:40.127387   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:40.181102   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:40.181154   71143 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:40.181164   71143 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:52:40.181189   71143 retry.go:31] will retry after 4.108056122s: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:44.290423   71143 cli_runner.go:164] Run: docker container inspect multinode-456000 --format={{.State.Status}}

	W0115 05:52:44.346051   71143 cli_runner.go:211] docker container inspect multinode-456000 --format={{.State.Status}} returned with exit code 1

	I0115 05:52:44.346094   71143 oci.go:662] temporary error verifying shutdown: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:44.346102   71143 oci.go:664] temporary error: container multinode-456000 status is  but expect it to be exited

	I0115 05:52:44.346132   71143 oci.go:88] couldn't shut down multinode-456000 (might be okay): verify shutdown: couldn't verify container is exited. %v: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

	I0115 05:52:44.346220   71143 cli_runner.go:164] Run: docker rm -f -v multinode-456000

	I0115 05:52:44.397284   71143 cli_runner.go:164] Run: docker container inspect -f {{.Id}} multinode-456000

	W0115 05:52:44.446949   71143 cli_runner.go:211] docker container inspect -f {{.Id}} multinode-456000 returned with exit code 1

	I0115 05:52:44.447055   71143 cli_runner.go:164] Run: docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 05:52:44.497435   71143 cli_runner.go:164] Run: docker network rm multinode-456000

	I0115 05:52:44.690114   71143 fix.go:114] Sleeping 1 second for extra luck!

	I0115 05:52:45.691407   71143 start.go:125] createHost starting for "" (driver="docker")

	I0115 05:52:45.713465   71143 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...

	I0115 05:52:45.713643   71143 start.go:159] libmachine.API.Create for "multinode-456000" (driver="docker")

	I0115 05:52:45.713691   71143 client.go:168] LocalClient.Create starting

	I0115 05:52:45.713886   71143 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/ca.pem

	I0115 05:52:45.713982   71143 main.go:141] libmachine: Decoding PEM data...

	I0115 05:52:45.714014   71143 main.go:141] libmachine: Parsing certificate...

	I0115 05:52:45.714134   71143 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17953-64881/.minikube/certs/cert.pem

	I0115 05:52:45.714222   71143 main.go:141] libmachine: Decoding PEM data...

	I0115 05:52:45.714239   71143 main.go:141] libmachine: Parsing certificate...

	I0115 05:52:45.735748   71143 cli_runner.go:164] Run: docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	W0115 05:52:45.788227   71143 cli_runner.go:211] docker network inspect multinode-456000 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1

	I0115 05:52:45.788325   71143 network_create.go:281] running [docker network inspect multinode-456000] to gather additional debugging logs...

	I0115 05:52:45.788351   71143 cli_runner.go:164] Run: docker network inspect multinode-456000

	W0115 05:52:45.838936   71143 cli_runner.go:211] docker network inspect multinode-456000 returned with exit code 1

	I0115 05:52:45.838970   71143 network_create.go:284] error running [docker network inspect multinode-456000]: docker network inspect multinode-456000: exit status 1

	stdout:

	[]

	stderr:

	Error response from daemon: network multinode-456000 not found

	I0115 05:52:45.838990   71143 network_create.go:286] output of [docker network inspect multinode-456000]: -- stdout --

	[]

	-- /stdout --

	** stderr ** 

	Error response from daemon: network multinode-456000 not found

	** /stderr **

	I0115 05:52:45.839134   71143 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0115 05:52:45.892672   71143 network.go:212] skipping subnet 192.168.49.0/24 that is reserved: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:<nil>}

	I0115 05:52:45.893048   71143 network.go:209] using free private subnet 192.168.58.0/24: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc002472180}

	I0115 05:52:45.893071   71143 network_create.go:124] attempt to create docker network multinode-456000 192.168.58.0/24 with gateway 192.168.58.1 and MTU of 65535 ...

	I0115 05:52:45.893140   71143 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.58.0/24 --gateway=192.168.58.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=65535 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=multinode-456000 multinode-456000

	I0115 05:52:45.978549   71143 network_create.go:108] docker network multinode-456000 192.168.58.0/24 created

	I0115 05:52:45.978594   71143 kic.go:121] calculated static IP "192.168.58.2" for the "multinode-456000" container

	I0115 05:52:45.978716   71143 cli_runner.go:164] Run: docker ps -a --format {{.Names}}

	I0115 05:52:46.030048   71143 cli_runner.go:164] Run: docker volume create multinode-456000 --label name.minikube.sigs.k8s.io=multinode-456000 --label created_by.minikube.sigs.k8s.io=true

	I0115 05:52:46.079836   71143 oci.go:103] Successfully created a docker volume multinode-456000

	I0115 05:52:46.079956   71143 cli_runner.go:164] Run: docker run --rm --name multinode-456000-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=multinode-456000 --entrypoint /usr/bin/test -v multinode-456000:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -d /var/lib

	I0115 05:52:46.373877   71143 oci.go:107] Successfully prepared a docker volume multinode-456000

	I0115 05:52:46.373918   71143 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0115 05:52:46.373932   71143 kic.go:194] Starting extracting preloaded images to volume ...

	I0115 05:52:46.374029   71143 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /Users/jenkins/minikube-integration/17953-64881/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v multinode-456000:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1704759386-17866@sha256:8c3c33047f9bc285e1f5f2a5aa14744a2fe04c58478f02f77b06169dea8dd3f0 -I lz4 -xf /preloaded.tar -C /extractDir

** /stderr **

multinode_test.go:384: failed to start cluster. args "out/minikube-darwin-amd64 start -p multinode-456000 --wait=true -v=8 --alsologtostderr --driver=docker " : signal: killed

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestMultiNode/serial/RestartMultiNode]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect multinode-456000

helpers_test.go:235: (dbg) docker inspect multinode-456000:

-- stdout --

	[

	    {

	        "Name": "multinode-456000",

	        "Id": "098b5e2c52aff468e4971dcd32317a6dc6b86c6bcc41e459f7b086e9a6783c92",

	        "Created": "2024-01-15T13:52:45.941910313Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.58.0/24",

	                    "Gateway": "192.168.58.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "multinode-456000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p multinode-456000 -n multinode-456000: exit status 7 (108.904626ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 05:55:04.970410   71266 status.go:249] status error: host: state: unknown state "multinode-456000": docker container inspect multinode-456000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: multinode-456000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-456000" host is not running, skipping log retrieval (state="Nonexistent")

--- FAIL: TestMultiNode/serial/RestartMultiNode (157.31s)

=== RUN   TestScheduledStopUnix

scheduled_stop_test.go:128: (dbg) Run:  out/minikube-darwin-amd64 start -p scheduled-stop-426000 --memory=2048 --driver=docker 

E0115 05:59:37.089558   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 05:59:54.018257   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 06:00:56.755244   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

scheduled_stop_test.go:128: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p scheduled-stop-426000 --memory=2048 --driver=docker : signal: killed (5m0.003532926s)

-- stdout --

	* [scheduled-stop-426000] minikube v1.32.0 on Darwin 14.2.1

	  - MINIKUBE_LOCATION=17953

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the docker driver based on user configuration

	* Using Docker Desktop driver with root privileges

	* Starting control plane node scheduled-stop-426000 in cluster scheduled-stop-426000

	* Pulling base image v0.0.42-1704759386-17866 ...

	* Creating docker container (CPUs=2, Memory=2048MB) ...

-- /stdout --

scheduled_stop_test.go:130: starting minikube: signal: killed

-- stdout --

	* [scheduled-stop-426000] minikube v1.32.0 on Darwin 14.2.1

	  - MINIKUBE_LOCATION=17953

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the docker driver based on user configuration

	* Using Docker Desktop driver with root privileges

	* Starting control plane node scheduled-stop-426000 in cluster scheduled-stop-426000

	* Pulling base image v0.0.42-1704759386-17866 ...

	* Creating docker container (CPUs=2, Memory=2048MB) ...

-- /stdout --

panic.go:523: *** TestScheduledStopUnix FAILED at 2024-01-15 06:02:20.089749 -0800 PST m=+3636.270785691

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestScheduledStopUnix]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect scheduled-stop-426000

helpers_test.go:235: (dbg) docker inspect scheduled-stop-426000:

-- stdout --

	[

	    {

	        "Name": "scheduled-stop-426000",

	        "Id": "1d5e4c905c6a6fe89114baa7f653e819ca1a0e8f319d1606afefe196d72e2213",

	        "Created": "2024-01-15T13:57:21.126789666Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.58.0/24",

	                    "Gateway": "192.168.58.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "scheduled-stop-426000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p scheduled-stop-426000 -n scheduled-stop-426000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p scheduled-stop-426000 -n scheduled-stop-426000: exit status 7 (110.420588ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 06:02:20.253233   71810 status.go:249] status error: host: state: unknown state "scheduled-stop-426000": docker container inspect scheduled-stop-426000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: scheduled-stop-426000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "scheduled-stop-426000" host is not running, skipping log retrieval (state="Nonexistent")

helpers_test.go:175: Cleaning up "scheduled-stop-426000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-amd64 delete -p scheduled-stop-426000

--- FAIL: TestScheduledStopUnix (300.90s)

=== RUN   TestSkaffold

skaffold_test.go:59: (dbg) Run:  /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/skaffold.exe943522914 version

skaffold_test.go:63: skaffold version: v2.10.0

skaffold_test.go:66: (dbg) Run:  out/minikube-darwin-amd64 start -p skaffold-709000 --memory=2600 --driver=docker 

E0115 06:04:54.002678   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/addons-744000/client.crt: no such file or directory

E0115 06:05:39.797368   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

E0115 06:05:56.736952   65630 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17953-64881/.minikube/profiles/functional-281000/client.crt: no such file or directory

skaffold_test.go:66: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p skaffold-709000 --memory=2600 --driver=docker : signal: killed (4m58.050228083s)

-- stdout --

	* [skaffold-709000] minikube v1.32.0 on Darwin 14.2.1

	  - MINIKUBE_LOCATION=17953

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the docker driver based on user configuration

	* Using Docker Desktop driver with root privileges

	* Starting control plane node skaffold-709000 in cluster skaffold-709000

	* Pulling base image v0.0.42-1704759386-17866 ...

	* Creating docker container (CPUs=2, Memory=2600MB) ...

-- /stdout --

skaffold_test.go:68: starting minikube: signal: killed

-- stdout --

	* [skaffold-709000] minikube v1.32.0 on Darwin 14.2.1

	  - MINIKUBE_LOCATION=17953

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17953-64881/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17953-64881/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the docker driver based on user configuration

	* Using Docker Desktop driver with root privileges

	* Starting control plane node skaffold-709000 in cluster skaffold-709000

	* Pulling base image v0.0.42-1704759386-17866 ...

	* Creating docker container (CPUs=2, Memory=2600MB) ...

-- /stdout --

panic.go:523: *** TestSkaffold FAILED at 2024-01-15 06:07:20.971668 -0800 PST m=+3937.169230456

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestSkaffold]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect skaffold-709000

helpers_test.go:235: (dbg) docker inspect skaffold-709000:

-- stdout --

	[

	    {

	        "Name": "skaffold-709000",

	        "Id": "7b542e7d056026e34a0659f54980b595dc59c38e90d0924bf25c8e43f40cc5b7",

	        "Created": "2024-01-15T14:02:24.016073991Z",

	        "Scope": "local",

	        "Driver": "bridge",

	        "EnableIPv6": false,

	        "IPAM": {

	            "Driver": "default",

	            "Options": {},

	            "Config": [

	                {

	                    "Subnet": "192.168.58.0/24",

	                    "Gateway": "192.168.58.1"

	                }

	            ]

	        },

	        "Internal": false,

	        "Attachable": false,

	        "Ingress": false,

	        "ConfigFrom": {

	            "Network": ""

	        },

	        "ConfigOnly": false,

	        "Containers": {},

	        "Options": {

	            "--icc": "",

	            "--ip-masq": "",

	            "com.docker.network.driver.mtu": "65535"

	        },

	        "Labels": {

	            "created_by.minikube.sigs.k8s.io": "true",

	            "name.minikube.sigs.k8s.io": "skaffold-709000"

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p skaffold-709000 -n skaffold-709000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-amd64 status --format={{.Host}} -p skaffold-709000 -n skaffold-709000: exit status 7 (109.984715ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0115 06:07:21.134490   71962 status.go:249] status error: host: state: unknown state "skaffold-709000": docker container inspect skaffold-709000 --format={{.State.Status}}: exit status 1

	stdout:

	stderr:

	Error response from daemon: No such container: skaffold-709000

** /stderr **

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "skaffold-709000" host is not running, skipping log retrieval (state="Nonexistent")

helpers_test.go:175: Cleaning up "skaffold-709000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-amd64 delete -p skaffold-709000

--- FAIL: TestSkaffold (300.91s)