=== RUN   TestDownloadOnly/v1.20.0/json-events

aaa_download_only_test.go:81: (dbg) Run:  out/minikube-darwin-arm64 start -o=json --download-only -p download-only-510000 --force --alsologtostderr --kubernetes-version=v1.20.0 --container-runtime=docker --driver=qemu2 

aaa_download_only_test.go:81: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -o=json --download-only -p download-only-510000 --force --alsologtostderr --kubernetes-version=v1.20.0 --container-runtime=docker --driver=qemu2 : exit status 40 (41.138072458s)

-- stdout --

	{"specversion":"1.0","id":"e9fada0e-662b-4c1a-bda5-2b7548d67dc2","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"0","message":"[download-only-510000] minikube v1.32.0 on Darwin 14.3.1 (arm64)","name":"Initial Minikube Setup","totalsteps":"19"}}

	{"specversion":"1.0","id":"df6cf61b-02ae-4b5d-9d5b-8724266a96d7","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_LOCATION=18320"}}

	{"specversion":"1.0","id":"2000e9db-72bb-421b-b4eb-ec5714c418ed","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig"}}

	{"specversion":"1.0","id":"8f9365b3-c00c-43f3-987a-2e48cc9f74db","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_BIN=out/minikube-darwin-arm64"}}

	{"specversion":"1.0","id":"b6998dce-d2a5-488c-9ef5-63cef020dda9","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true"}}

	{"specversion":"1.0","id":"8e0e0b5f-b02f-4d10-8907-606f9acbfbfe","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube"}}

	{"specversion":"1.0","id":"aa2cd1c0-4570-4768-b96d-56d7cab066b4","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.warning","datacontenttype":"application/json","data":{"message":"minikube skips various validations when --force is supplied; this may lead to unexpected behavior"}}

	{"specversion":"1.0","id":"5857f383-bf76-4181-a025-c86199977bc4","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"1","message":"Using the qemu2 driver based on user configuration","name":"Selecting Driver","totalsteps":"19"}}

	{"specversion":"1.0","id":"c215e9cb-dea0-4e9d-a22f-fe6d9130ed24","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"Automatically selected the socket_vmnet network"}}

	{"specversion":"1.0","id":"9580c3ed-b70c-4848-b546-69ec746bcbe5","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"1","message":"Downloading VM boot image ...","name":"Selecting Driver","totalsteps":"19"}}

	{"specversion":"1.0","id":"22c1c894-266b-4bf1-bba1-0141c95f086e","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"3","message":"Starting \"download-only-510000\" primary control-plane node in \"download-only-510000\" cluster","name":"Starting Node","totalsteps":"19"}}

	{"specversion":"1.0","id":"d7e42d66-cfb7-4722-abde-991c8213df4a","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"3","message":"Downloading Kubernetes v1.20.0 preload ...","name":"Starting Node","totalsteps":"19"}}

	{"specversion":"1.0","id":"93c65554-cf99-43a0-85e8-4e8cd67e64a4","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.error","datacontenttype":"application/json","data":{"advice":"","exitcode":"40","issues":"","message":"Failed to cache kubectl: download failed: https://dl.k8s.io/release/v1.20.0/bin/darwin/arm64/kubectl?checksum=file:https://dl.k8s.io/release/v1.20.0/bin/darwin/arm64/kubectl.sha256: getter: \u0026{Ctx:context.Background Src:https://dl.k8s.io/release/v1.20.0/bin/darwin/arm64/kubectl?checksum=file:https://dl.k8s.io/release/v1.20.0/bin/darwin/arm64/kubectl.sha256 Dst:/Users/jenkins/minikube-integration/18320-1153/.minikube/cache/darwin/arm64/v1.20.0/kubectl.download Pwd: Mode:2 Umask:---------- Detectors:[0x108e570a0 0x108e570a0 0x108e570a0 0x108e570a0 0x108e570a0 0x108e570a0 0x108e570a0] Decompressors:map[bz2:0x14000898df0 gz:0x14000898df8 tar:0x14000898da0 tar.bz2:0x14000898db0 tar.gz:0x14000898dc0 tar.xz:0x14000898dd0 tar.zst:0x14000898de0 tbz2:0x14000898db0 tgz:0x14

000898dc0 txz:0x14000898dd0 tzst:0x14000898de0 xz:0x14000898e00 zip:0x14000898e10 zst:0x14000898e08] Getters:map[file:0x140020d2570 http:0x140005fe230 https:0x140005fe280] Dir:false ProgressListener:\u003cnil\u003e Insecure:false DisableSymlinks:false Options:[]}: invalid checksum: Error downloading checksum file: bad response code: 404","name":"INET_CACHE_KUBECTL","url":""}}

	{"specversion":"1.0","id":"1ff246d6-b188-4a05-bfce-844b05680cf0","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.error","datacontenttype":"application/json","data":{"message":"╭───────────────────────────────────────────────────────────────────────────────────────────╮\n│                                                                                           │\n│    If the above advice does not help, please let us know:                                 │\n│    https://github.com/kubernetes/minikube/issues/new/choose                               │\n│                                                                                           │\n│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │\n│

│\n╰───────────────────────────────────────────────────────────────────────────────────────────╯"}}

-- /stdout --

** stderr ** 

	I0307 13:46:27.547614    1598 out.go:291] Setting OutFile to fd 1 ...

	I0307 13:46:27.547748    1598 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 13:46:27.547751    1598 out.go:304] Setting ErrFile to fd 2...

	I0307 13:46:27.547753    1598 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 13:46:27.547891    1598 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	W0307 13:46:27.547968    1598 root.go:314] Error reading config file at /Users/jenkins/minikube-integration/18320-1153/.minikube/config/config.json: open /Users/jenkins/minikube-integration/18320-1153/.minikube/config/config.json: no such file or directory

	I0307 13:46:27.549218    1598 out.go:298] Setting JSON to true

	I0307 13:46:27.566597    1598 start.go:129] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":944,"bootTime":1709847043,"procs":473,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.3.1","kernelVersion":"23.3.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0307 13:46:27.566659    1598 start.go:137] gopshost.Virtualization returned error: not implemented yet

	I0307 13:46:27.577159    1598 out.go:97] [download-only-510000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	I0307 13:46:27.581097    1598 out.go:169] MINIKUBE_LOCATION=18320

	I0307 13:46:27.577323    1598 notify.go:220] Checking for updates...

	W0307 13:46:27.577370    1598 preload.go:294] Failed to list preload files: open /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball: no such file or directory

	I0307 13:46:27.598234    1598 out.go:169] KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	I0307 13:46:27.601210    1598 out.go:169] MINIKUBE_BIN=out/minikube-darwin-arm64

	I0307 13:46:27.604153    1598 out.go:169] MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0307 13:46:27.608213    1598 out.go:169] MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	W0307 13:46:27.614162    1598 out.go:267] minikube skips various validations when --force is supplied; this may lead to unexpected behavior

	I0307 13:46:27.614394    1598 driver.go:392] Setting default libvirt URI to qemu:///system

	I0307 13:46:27.619207    1598 out.go:97] Using the qemu2 driver based on user configuration

	I0307 13:46:27.619228    1598 start.go:297] selected driver: qemu2

	I0307 13:46:27.619244    1598 start.go:901] validating driver "qemu2" against <nil>

	I0307 13:46:27.619322    1598 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0307 13:46:27.621294    1598 out.go:169] Automatically selected the socket_vmnet network

	I0307 13:46:27.626906    1598 start_flags.go:393] Using suggested 4000MB memory alloc based on sys=16384MB, container=0MB

	I0307 13:46:27.627022    1598 start_flags.go:929] Wait components to verify : map[apiserver:true system_pods:true]

	I0307 13:46:27.627117    1598 cni.go:84] Creating CNI manager for ""

	I0307 13:46:27.627135    1598 cni.go:162] CNI unnecessary in this configuration, recommending no CNI

	I0307 13:46:27.627187    1598 start.go:340] cluster config:

	{Name:download-only-510000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:4000 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:download-only-510000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Con

tainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSo

ck: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 13:46:27.633932    1598 iso.go:125] acquiring lock: {Name:mk9f11fe8095f5aa8805d47835226b5c64c27e19 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0307 13:46:27.637272    1598 out.go:97] Downloading VM boot image ...

	I0307 13:46:27.637294    1598 download.go:107] Downloading: https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso?checksum=file:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso.sha256 -> /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso

	I0307 13:46:46.368678    1598 out.go:97] Starting "download-only-510000" primary control-plane node in "download-only-510000" cluster

	I0307 13:46:46.368728    1598 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime docker

	I0307 13:46:46.656445    1598 preload.go:119] Found remote preload: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.20.0/preloaded-images-k8s-v18-v1.20.0-docker-overlay2-arm64.tar.lz4

	I0307 13:46:46.656524    1598 cache.go:56] Caching tarball of preloaded images

	I0307 13:46:46.657246    1598 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime docker

	I0307 13:46:46.662023    1598 out.go:97] Downloading Kubernetes v1.20.0 preload ...

	I0307 13:46:46.662051    1598 preload.go:237] getting checksum for preloaded-images-k8s-v18-v1.20.0-docker-overlay2-arm64.tar.lz4 ...

	I0307 13:46:47.271686    1598 download.go:107] Downloading: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.20.0/preloaded-images-k8s-v18-v1.20.0-docker-overlay2-arm64.tar.lz4?checksum=md5:1a3e8f9b29e6affec63d76d0d3000942 -> /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-docker-overlay2-arm64.tar.lz4

	I0307 13:47:07.206577    1598 preload.go:248] saving checksum for preloaded-images-k8s-v18-v1.20.0-docker-overlay2-arm64.tar.lz4 ...

	I0307 13:47:07.206760    1598 preload.go:255] verifying checksum of /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-docker-overlay2-arm64.tar.lz4 ...

	I0307 13:47:07.904369    1598 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on docker

	I0307 13:47:07.904551    1598 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/download-only-510000/config.json ...

	I0307 13:47:07.904567    1598 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/download-only-510000/config.json: {Name:mk7fd9b47ffcd4f27544635ead98cbb488fc03e3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:47:07.904805    1598 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime docker

	I0307 13:47:07.905002    1598 download.go:107] Downloading: https://dl.k8s.io/release/v1.20.0/bin/darwin/arm64/kubectl?checksum=file:https://dl.k8s.io/release/v1.20.0/bin/darwin/arm64/kubectl.sha256 -> /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/darwin/arm64/v1.20.0/kubectl

	I0307 13:47:08.608084    1598 out.go:169] 

	W0307 13:47:08.614137    1598 out_reason.go:110] Failed to cache kubectl: download failed: https://dl.k8s.io/release/v1.20.0/bin/darwin/arm64/kubectl?checksum=file:https://dl.k8s.io/release/v1.20.0/bin/darwin/arm64/kubectl.sha256: getter: &{Ctx:context.Background Src:https://dl.k8s.io/release/v1.20.0/bin/darwin/arm64/kubectl?checksum=file:https://dl.k8s.io/release/v1.20.0/bin/darwin/arm64/kubectl.sha256 Dst:/Users/jenkins/minikube-integration/18320-1153/.minikube/cache/darwin/arm64/v1.20.0/kubectl.download Pwd: Mode:2 Umask:---------- Detectors:[0x108e570a0 0x108e570a0 0x108e570a0 0x108e570a0 0x108e570a0 0x108e570a0 0x108e570a0] Decompressors:map[bz2:0x14000898df0 gz:0x14000898df8 tar:0x14000898da0 tar.bz2:0x14000898db0 tar.gz:0x14000898dc0 tar.xz:0x14000898dd0 tar.zst:0x14000898de0 tbz2:0x14000898db0 tgz:0x14000898dc0 txz:0x14000898dd0 tzst:0x14000898de0 xz:0x14000898e00 zip:0x14000898e10 zst:0x14000898e08] Getters:map[file:0x140020d2570 http:0x140005fe230 https:0x140005fe280] Dir:false ProgressList

ener:<nil> Insecure:false DisableSymlinks:false Options:[]}: invalid checksum: Error downloading checksum file: bad response code: 404

	W0307 13:47:08.614161    1598 out_reason.go:110] 

	W0307 13:47:08.622061    1598 out.go:229] ╭───────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                           │

	│    If the above advice does not help, please let us know:                                 │

	│    https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                           │

	│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                           │

	╰───────────────────────────────────────────────────────────────────────────────────────────╯

	I0307 13:47:08.626108    1598 out.go:169] 

** /stderr **

aaa_download_only_test.go:83: failed to download only. args: ["start" "-o=json" "--download-only" "-p" "download-only-510000" "--force" "--alsologtostderr" "--kubernetes-version=v1.20.0" "--container-runtime=docker" "--driver=qemu2" ""] exit status 40

--- FAIL: TestDownloadOnly/v1.20.0/json-events (41.14s)

=== RUN   TestDownloadOnly/v1.20.0/kubectl

aaa_download_only_test.go:175: expected the file for binary exist at "/Users/jenkins/minikube-integration/18320-1153/.minikube/cache/darwin/arm64/v1.20.0/kubectl" but got error stat /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/darwin/arm64/v1.20.0/kubectl: no such file or directory

--- FAIL: TestDownloadOnly/v1.20.0/kubectl (0.00s)

=== RUN   TestOffline

=== PAUSE TestOffline

=== CONT  TestOffline

aab_offline_test.go:55: (dbg) Run:  out/minikube-darwin-arm64 start -p offline-docker-139000 --alsologtostderr -v=1 --memory=2048 --wait=true --driver=qemu2 

aab_offline_test.go:55: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p offline-docker-139000 --alsologtostderr -v=1 --memory=2048 --wait=true --driver=qemu2 : exit status 80 (9.717821375s)

-- stdout --

	* [offline-docker-139000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting "offline-docker-139000" primary control-plane node in "offline-docker-139000" cluster

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "offline-docker-139000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0307 14:37:16.077207    3800 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:37:16.077368    3800 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:37:16.077371    3800 out.go:304] Setting ErrFile to fd 2...

	I0307 14:37:16.077373    3800 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:37:16.077512    3800 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:37:16.078752    3800 out.go:298] Setting JSON to false

	I0307 14:37:16.096315    3800 start.go:129] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":3993,"bootTime":1709847043,"procs":470,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.3.1","kernelVersion":"23.3.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0307 14:37:16.096391    3800 start.go:137] gopshost.Virtualization returned error: not implemented yet

	I0307 14:37:16.102070    3800 out.go:177] * [offline-docker-139000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	I0307 14:37:16.109044    3800 out.go:177]   - MINIKUBE_LOCATION=18320

	I0307 14:37:16.109068    3800 notify.go:220] Checking for updates...

	I0307 14:37:16.116956    3800 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	I0307 14:37:16.119939    3800 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0307 14:37:16.122964    3800 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0307 14:37:16.125969    3800 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	I0307 14:37:16.128995    3800 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0307 14:37:16.132328    3800 config.go:182] Loaded profile config "multinode-172000-m01": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:37:16.132388    3800 driver.go:392] Setting default libvirt URI to qemu:///system

	I0307 14:37:16.135968    3800 out.go:177] * Using the qemu2 driver based on user configuration

	I0307 14:37:16.143090    3800 start.go:297] selected driver: qemu2

	I0307 14:37:16.143112    3800 start.go:901] validating driver "qemu2" against <nil>

	I0307 14:37:16.143135    3800 start.go:912] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0307 14:37:16.145309    3800 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0307 14:37:16.148936    3800 out.go:177] * Automatically selected the socket_vmnet network

	I0307 14:37:16.153051    3800 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0307 14:37:16.153095    3800 cni.go:84] Creating CNI manager for ""

	I0307 14:37:16.153103    3800 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0307 14:37:16.153107    3800 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0307 14:37:16.153148    3800 start.go:340] cluster config:

	{Name:offline-docker-139000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:offline-docker-139000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C

ontainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bi

n/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 14:37:16.157803    3800 iso.go:125] acquiring lock: {Name:mk9f11fe8095f5aa8805d47835226b5c64c27e19 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0307 14:37:16.165985    3800 out.go:177] * Starting "offline-docker-139000" primary control-plane node in "offline-docker-139000" cluster

	I0307 14:37:16.169972    3800 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0307 14:37:16.170004    3800 preload.go:147] Found local preload: /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4

	I0307 14:37:16.170017    3800 cache.go:56] Caching tarball of preloaded images

	I0307 14:37:16.170096    3800 preload.go:173] Found /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0307 14:37:16.170102    3800 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on docker

	I0307 14:37:16.170166    3800 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/offline-docker-139000/config.json ...

	I0307 14:37:16.170176    3800 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/offline-docker-139000/config.json: {Name:mkfba3fc04f4892683d01369be1c78b637f71930 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 14:37:16.170457    3800 start.go:360] acquireMachinesLock for offline-docker-139000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:37:16.170488    3800 start.go:364] duration metric: took 21.875µs to acquireMachinesLock for "offline-docker-139000"

	I0307 14:37:16.170503    3800 start.go:93] Provisioning new machine with config: &{Name:offline-docker-139000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConf

ig:{KubernetesVersion:v1.28.4 ClusterName:offline-docker-139000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 Mou

ntOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 14:37:16.170535    3800 start.go:125] createHost starting for "" (driver="qemu2")

	I0307 14:37:16.174933    3800 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0307 14:37:16.190390    3800 start.go:159] libmachine.API.Create for "offline-docker-139000" (driver="qemu2")

	I0307 14:37:16.190423    3800 client.go:168] LocalClient.Create starting

	I0307 14:37:16.190488    3800 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem

	I0307 14:37:16.190521    3800 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:16.190530    3800 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:16.190574    3800 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem

	I0307 14:37:16.190594    3800 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:16.190598    3800 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:16.190993    3800 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso...

	I0307 14:37:16.337499    3800 main.go:141] libmachine: Creating SSH key...

	I0307 14:37:16.391272    3800 main.go:141] libmachine: Creating Disk image...

	I0307 14:37:16.391279    3800 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0307 14:37:16.391453    3800 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2.raw /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2

	I0307 14:37:16.404185    3800 main.go:141] libmachine: STDOUT: 

	I0307 14:37:16.404206    3800 main.go:141] libmachine: STDERR: 

	I0307 14:37:16.404266    3800 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2 +20000M

	I0307 14:37:16.416644    3800 main.go:141] libmachine: STDOUT: Image resized.

	I0307 14:37:16.416679    3800 main.go:141] libmachine: STDERR: 

	I0307 14:37:16.416694    3800 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2.raw and /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2

	I0307 14:37:16.416697    3800 main.go:141] libmachine: Starting QEMU VM...

	I0307 14:37:16.416736    3800 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/qemu.pid -device virtio-net-pci,netdev=net0,mac=8a:ad:d6:f5:ef:1e -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2

	I0307 14:37:16.418813    3800 main.go:141] libmachine: STDOUT: 

	I0307 14:37:16.418832    3800 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:37:16.418850    3800 client.go:171] duration metric: took 228.421667ms to LocalClient.Create

	I0307 14:37:18.420703    3800 start.go:128] duration metric: took 2.250155625s to createHost

	I0307 14:37:18.420719    3800 start.go:83] releasing machines lock for "offline-docker-139000", held for 2.250222959s

	W0307 14:37:18.420730    3800 start.go:713] error starting host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:18.429631    3800 out.go:177] * Deleting "offline-docker-139000" in qemu2 ...

	W0307 14:37:18.438875    3800 out.go:239] ! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:18.438883    3800 start.go:728] Will try again in 5 seconds ...

	I0307 14:37:23.441057    3800 start.go:360] acquireMachinesLock for offline-docker-139000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:37:23.441293    3800 start.go:364] duration metric: took 177.416µs to acquireMachinesLock for "offline-docker-139000"

	I0307 14:37:23.441351    3800 start.go:93] Provisioning new machine with config: &{Name:offline-docker-139000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConf

ig:{KubernetesVersion:v1.28.4 ClusterName:offline-docker-139000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 Mou

ntOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 14:37:23.441487    3800 start.go:125] createHost starting for "" (driver="qemu2")

	I0307 14:37:23.448891    3800 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0307 14:37:23.480910    3800 start.go:159] libmachine.API.Create for "offline-docker-139000" (driver="qemu2")

	I0307 14:37:23.480955    3800 client.go:168] LocalClient.Create starting

	I0307 14:37:23.481079    3800 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem

	I0307 14:37:23.481140    3800 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:23.481163    3800 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:23.481228    3800 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem

	I0307 14:37:23.481265    3800 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:23.481275    3800 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:23.481744    3800 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso...

	I0307 14:37:23.629136    3800 main.go:141] libmachine: Creating SSH key...

	I0307 14:37:23.689467    3800 main.go:141] libmachine: Creating Disk image...

	I0307 14:37:23.689473    3800 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0307 14:37:23.689658    3800 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2.raw /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2

	I0307 14:37:23.701736    3800 main.go:141] libmachine: STDOUT: 

	I0307 14:37:23.701756    3800 main.go:141] libmachine: STDERR: 

	I0307 14:37:23.701810    3800 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2 +20000M

	I0307 14:37:23.717435    3800 main.go:141] libmachine: STDOUT: Image resized.

	I0307 14:37:23.717455    3800 main.go:141] libmachine: STDERR: 

	I0307 14:37:23.717467    3800 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2.raw and /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2

	I0307 14:37:23.717470    3800 main.go:141] libmachine: Starting QEMU VM...

	I0307 14:37:23.717518    3800 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/qemu.pid -device virtio-net-pci,netdev=net0,mac=f2:c9:8a:bd:67:58 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/offline-docker-139000/disk.qcow2

	I0307 14:37:23.719265    3800 main.go:141] libmachine: STDOUT: 

	I0307 14:37:23.719282    3800 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:37:23.719294    3800 client.go:171] duration metric: took 238.333541ms to LocalClient.Create

	I0307 14:37:25.721475    3800 start.go:128] duration metric: took 2.279956291s to createHost

	I0307 14:37:25.721548    3800 start.go:83] releasing machines lock for "offline-docker-139000", held for 2.28023275s

	W0307 14:37:25.721966    3800 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p offline-docker-139000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p offline-docker-139000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:25.732561    3800 out.go:177] 

	W0307 14:37:25.736710    3800 out.go:239] X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0307 14:37:25.736749    3800 out.go:239] * 

	* 

	W0307 14:37:25.739812    3800 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0307 14:37:25.748523    3800 out.go:177] 

** /stderr **

aab_offline_test.go:58: out/minikube-darwin-arm64 start -p offline-docker-139000 --alsologtostderr -v=1 --memory=2048 --wait=true --driver=qemu2  failed: exit status 80

panic.go:626: *** TestOffline FAILED at 2024-03-07 14:37:25.765114 -0800 PST m=+3058.260611460

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p offline-docker-139000 -n offline-docker-139000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p offline-docker-139000 -n offline-docker-139000: exit status 7 (69.880792ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "offline-docker-139000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "offline-docker-139000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p offline-docker-139000

--- FAIL: TestOffline (9.90s)

=== RUN   TestAddons/parallel/Ingress

=== PAUSE TestAddons/parallel/Ingress

=== CONT  TestAddons/parallel/Ingress

addons_test.go:207: (dbg) Run:  kubectl --context addons-263000 wait --for=condition=ready --namespace=ingress-nginx pod --selector=app.kubernetes.io/component=controller --timeout=90s

addons_test.go:232: (dbg) Run:  kubectl --context addons-263000 replace --force -f testdata/nginx-ingress-v1.yaml

addons_test.go:245: (dbg) Run:  kubectl --context addons-263000 replace --force -f testdata/nginx-pod-svc.yaml

addons_test.go:250: (dbg) TestAddons/parallel/Ingress: waiting 8m0s for pods matching "run=nginx" in namespace "default" ...

helpers_test.go:344: "nginx" [6f929bbb-6156-470d-bcd8-346bf97fc817] Pending / Ready:ContainersNotReady (containers with unready status: [nginx]) / ContainersReady:ContainersNotReady (containers with unready status: [nginx])

helpers_test.go:344: "nginx" [6f929bbb-6156-470d-bcd8-346bf97fc817] Running

addons_test.go:250: (dbg) TestAddons/parallel/Ingress: run=nginx healthy within 10.003831292s

addons_test.go:262: (dbg) Run:  out/minikube-darwin-arm64 -p addons-263000 ssh "curl -s http://127.0.0.1/ -H 'Host: nginx.example.com'"

addons_test.go:286: (dbg) Run:  kubectl --context addons-263000 replace --force -f testdata/ingress-dns-example-v1.yaml

addons_test.go:291: (dbg) Run:  out/minikube-darwin-arm64 -p addons-263000 ip

addons_test.go:297: (dbg) Run:  nslookup hello-john.test 192.168.105.2

addons_test.go:297: (dbg) Non-zero exit: nslookup hello-john.test 192.168.105.2: exit status 1 (15.035680916s)

-- stdout --

	;; connection timed out; no servers could be reached

-- /stdout --

addons_test.go:299: failed to nslookup hello-john.test host. args "nslookup hello-john.test 192.168.105.2" : exit status 1

addons_test.go:303: unexpected output from nslookup. stdout: ;; connection timed out; no servers could be reached

stderr: 

addons_test.go:306: (dbg) Run:  out/minikube-darwin-arm64 -p addons-263000 addons disable ingress-dns --alsologtostderr -v=1

addons_test.go:311: (dbg) Run:  out/minikube-darwin-arm64 -p addons-263000 addons disable ingress --alsologtostderr -v=1

addons_test.go:311: (dbg) Done: out/minikube-darwin-arm64 -p addons-263000 addons disable ingress --alsologtostderr -v=1: (7.214449792s)

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p addons-263000 -n addons-263000

helpers_test.go:244: <<< TestAddons/parallel/Ingress FAILED: start of post-mortem logs <<<

helpers_test.go:245: ======>  post-mortem[TestAddons/parallel/Ingress]: minikube logs <======

helpers_test.go:247: (dbg) Run:  out/minikube-darwin-arm64 -p addons-263000 logs -n 25

helpers_test.go:252: TestAddons/parallel/Ingress logs: 

-- stdout --

	==> Audit <==

	|---------|---------------------------------------------------------------------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| Command |                                            Args                                             |       Profile        |  User   | Version |     Start Time      |      End Time       |

	|---------|---------------------------------------------------------------------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| delete  | --all                                                                                       | minikube             | jenkins | v1.32.0 | 07 Mar 24 13:48 PST | 07 Mar 24 13:48 PST |

	| delete  | -p download-only-304000                                                                     | download-only-304000 | jenkins | v1.32.0 | 07 Mar 24 13:48 PST | 07 Mar 24 13:48 PST |

	| delete  | -p download-only-510000                                                                     | download-only-510000 | jenkins | v1.32.0 | 07 Mar 24 13:48 PST | 07 Mar 24 13:48 PST |

	| delete  | -p download-only-973000                                                                     | download-only-973000 | jenkins | v1.32.0 | 07 Mar 24 13:48 PST | 07 Mar 24 13:48 PST |

	| delete  | -p download-only-304000                                                                     | download-only-304000 | jenkins | v1.32.0 | 07 Mar 24 13:48 PST | 07 Mar 24 13:48 PST |

	| start   | --download-only -p                                                                          | binary-mirror-773000 | jenkins | v1.32.0 | 07 Mar 24 13:48 PST |                     |

	|         | binary-mirror-773000                                                                        |                      |         |         |                     |                     |

	|         | --alsologtostderr                                                                           |                      |         |         |                     |                     |

	|         | --binary-mirror                                                                             |                      |         |         |                     |                     |

	|         | http://127.0.0.1:49328                                                                      |                      |         |         |                     |                     |

	|         | --driver=qemu2                                                                              |                      |         |         |                     |                     |

	| delete  | -p binary-mirror-773000                                                                     | binary-mirror-773000 | jenkins | v1.32.0 | 07 Mar 24 13:48 PST | 07 Mar 24 13:48 PST |

	| addons  | enable dashboard -p                                                                         | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:48 PST |                     |

	|         | addons-263000                                                                               |                      |         |         |                     |                     |

	| addons  | disable dashboard -p                                                                        | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:48 PST |                     |

	|         | addons-263000                                                                               |                      |         |         |                     |                     |

	| start   | -p addons-263000 --wait=true                                                                | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:48 PST | 07 Mar 24 13:52 PST |

	|         | --memory=4000 --alsologtostderr                                                             |                      |         |         |                     |                     |

	|         | --addons=registry                                                                           |                      |         |         |                     |                     |

	|         | --addons=metrics-server                                                                     |                      |         |         |                     |                     |

	|         | --addons=volumesnapshots                                                                    |                      |         |         |                     |                     |

	|         | --addons=csi-hostpath-driver                                                                |                      |         |         |                     |                     |

	|         | --addons=gcp-auth                                                                           |                      |         |         |                     |                     |

	|         | --addons=cloud-spanner                                                                      |                      |         |         |                     |                     |

	|         | --addons=inspektor-gadget                                                                   |                      |         |         |                     |                     |

	|         | --addons=storage-provisioner-rancher                                                        |                      |         |         |                     |                     |

	|         | --addons=nvidia-device-plugin                                                               |                      |         |         |                     |                     |

	|         | --addons=yakd --driver=qemu2                                                                |                      |         |         |                     |                     |

	|         |  --addons=ingress                                                                           |                      |         |         |                     |                     |

	|         | --addons=ingress-dns                                                                        |                      |         |         |                     |                     |

	| ip      | addons-263000 ip                                                                            | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:52 PST | 07 Mar 24 13:52 PST |

	| addons  | addons-263000 addons disable                                                                | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:52 PST | 07 Mar 24 13:52 PST |

	|         | registry --alsologtostderr                                                                  |                      |         |         |                     |                     |

	|         | -v=1                                                                                        |                      |         |         |                     |                     |

	| addons  | disable nvidia-device-plugin                                                                | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:52 PST | 07 Mar 24 13:52 PST |

	|         | -p addons-263000                                                                            |                      |         |         |                     |                     |

	| ssh     | addons-263000 ssh cat                                                                       | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:52 PST | 07 Mar 24 13:52 PST |

	|         | /opt/local-path-provisioner/pvc-8066f6fb-2a0d-414c-ad66-0e0e6c420276_default_test-pvc/file1 |                      |         |         |                     |                     |

	| addons  | addons-263000 addons disable                                                                | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:52 PST | 07 Mar 24 13:53 PST |

	|         | storage-provisioner-rancher                                                                 |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1                                                                      |                      |         |         |                     |                     |

	| addons  | addons-263000 addons                                                                        | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:52 PST | 07 Mar 24 13:52 PST |

	|         | disable csi-hostpath-driver                                                                 |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1                                                                      |                      |         |         |                     |                     |

	| addons  | addons-263000 addons                                                                        | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:52 PST | 07 Mar 24 13:52 PST |

	|         | disable volumesnapshots                                                                     |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1                                                                      |                      |         |         |                     |                     |

	| addons  | disable cloud-spanner -p                                                                    | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:53 PST | 07 Mar 24 13:53 PST |

	|         | addons-263000                                                                               |                      |         |         |                     |                     |

	| addons  | enable headlamp                                                                             | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:53 PST | 07 Mar 24 13:53 PST |

	|         | -p addons-263000                                                                            |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1                                                                      |                      |         |         |                     |                     |

	| addons  | addons-263000 addons                                                                        | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:53 PST | 07 Mar 24 13:53 PST |

	|         | disable metrics-server                                                                      |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1                                                                      |                      |         |         |                     |                     |

	| addons  | disable inspektor-gadget -p                                                                 | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:53 PST | 07 Mar 24 13:53 PST |

	|         | addons-263000                                                                               |                      |         |         |                     |                     |

	| ssh     | addons-263000 ssh curl -s                                                                   | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:53 PST | 07 Mar 24 13:53 PST |

	|         | http://127.0.0.1/ -H 'Host:                                                                 |                      |         |         |                     |                     |

	|         | nginx.example.com'                                                                          |                      |         |         |                     |                     |

	| ip      | addons-263000 ip                                                                            | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:53 PST | 07 Mar 24 13:53 PST |

	| addons  | addons-263000 addons disable                                                                | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:53 PST | 07 Mar 24 13:53 PST |

	|         | ingress-dns --alsologtostderr                                                               |                      |         |         |                     |                     |

	|         | -v=1                                                                                        |                      |         |         |                     |                     |

	| addons  | addons-263000 addons disable                                                                | addons-263000        | jenkins | v1.32.0 | 07 Mar 24 13:53 PST | 07 Mar 24 13:53 PST |

	|         | ingress --alsologtostderr -v=1                                                              |                      |         |         |                     |                     |

	|---------|---------------------------------------------------------------------------------------------|----------------------|---------|---------|---------------------|---------------------|

	==> Last Start <==

	Log file created at: 2024/03/07 13:48:51

	Running on machine: MacOS-M1-Agent-1

	Binary: Built with gc go1.22.0 for darwin/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0307 13:48:51.690647    1772 out.go:291] Setting OutFile to fd 1 ...

	I0307 13:48:51.690770    1772 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 13:48:51.690773    1772 out.go:304] Setting ErrFile to fd 2...

	I0307 13:48:51.690776    1772 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 13:48:51.690918    1772 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 13:48:51.691994    1772 out.go:298] Setting JSON to false

	I0307 13:48:51.708231    1772 start.go:129] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":1088,"bootTime":1709847043,"procs":457,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.3.1","kernelVersion":"23.3.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0307 13:48:51.708284    1772 start.go:137] gopshost.Virtualization returned error: not implemented yet

	I0307 13:48:51.712035    1772 out.go:177] * [addons-263000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	I0307 13:48:51.718997    1772 out.go:177]   - MINIKUBE_LOCATION=18320

	I0307 13:48:51.722955    1772 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	I0307 13:48:51.719039    1772 notify.go:220] Checking for updates...

	I0307 13:48:51.729028    1772 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0307 13:48:51.731959    1772 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0307 13:48:51.734960    1772 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	I0307 13:48:51.738003    1772 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0307 13:48:51.741052    1772 driver.go:392] Setting default libvirt URI to qemu:///system

	I0307 13:48:51.744945    1772 out.go:177] * Using the qemu2 driver based on user configuration

	I0307 13:48:51.751989    1772 start.go:297] selected driver: qemu2

	I0307 13:48:51.751995    1772 start.go:901] validating driver "qemu2" against <nil>

	I0307 13:48:51.752002    1772 start.go:912] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0307 13:48:51.754334    1772 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0307 13:48:51.757984    1772 out.go:177] * Automatically selected the socket_vmnet network

	I0307 13:48:51.761087    1772 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0307 13:48:51.761147    1772 cni.go:84] Creating CNI manager for ""

	I0307 13:48:51.761154    1772 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0307 13:48:51.761158    1772 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0307 13:48:51.761198    1772 start.go:340] cluster config:

	{Name:addons-263000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:4000 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:addons-263000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:

docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_c

lient SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 13:48:51.765589    1772 iso.go:125] acquiring lock: {Name:mk9f11fe8095f5aa8805d47835226b5c64c27e19 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0307 13:48:51.774001    1772 out.go:177] * Starting "addons-263000" primary control-plane node in "addons-263000" cluster

	I0307 13:48:51.777833    1772 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0307 13:48:51.777847    1772 preload.go:147] Found local preload: /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4

	I0307 13:48:51.777857    1772 cache.go:56] Caching tarball of preloaded images

	I0307 13:48:51.777913    1772 preload.go:173] Found /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0307 13:48:51.777919    1772 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on docker

	I0307 13:48:51.778150    1772 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/config.json ...

	I0307 13:48:51.778163    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/config.json: {Name:mk981e2c99a568cbf3edd7800e56bb7d9477bf42 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:48:51.778402    1772 start.go:360] acquireMachinesLock for addons-263000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 13:48:51.778561    1772 start.go:364] duration metric: took 153µs to acquireMachinesLock for "addons-263000"

	I0307 13:48:51.778572    1772 start.go:93] Provisioning new machine with config: &{Name:addons-263000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:4000 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{Kube

rnetesVersion:v1.28.4 ClusterName:addons-263000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mou

ntPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 13:48:51.778601    1772 start.go:125] createHost starting for "" (driver="qemu2")

	I0307 13:48:51.783018    1772 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...

	I0307 13:48:52.023896    1772 start.go:159] libmachine.API.Create for "addons-263000" (driver="qemu2")

	I0307 13:48:52.023939    1772 client.go:168] LocalClient.Create starting

	I0307 13:48:52.024119    1772 main.go:141] libmachine: Creating CA: /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem

	I0307 13:48:52.070109    1772 main.go:141] libmachine: Creating client certificate: /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem

	I0307 13:48:52.156957    1772 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso...

	I0307 13:48:52.661525    1772 main.go:141] libmachine: Creating SSH key...

	I0307 13:48:52.856239    1772 main.go:141] libmachine: Creating Disk image...

	I0307 13:48:52.856248    1772 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0307 13:48:52.856509    1772 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/disk.qcow2.raw /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/disk.qcow2

	I0307 13:48:52.879100    1772 main.go:141] libmachine: STDOUT: 

	I0307 13:48:52.879123    1772 main.go:141] libmachine: STDERR: 

	I0307 13:48:52.879172    1772 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/disk.qcow2 +20000M

	I0307 13:48:52.889878    1772 main.go:141] libmachine: STDOUT: Image resized.

	I0307 13:48:52.889898    1772 main.go:141] libmachine: STDERR: 

	I0307 13:48:52.889915    1772 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/disk.qcow2.raw and /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/disk.qcow2

	I0307 13:48:52.889922    1772 main.go:141] libmachine: Starting QEMU VM...

	I0307 13:48:52.889953    1772 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 4000 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/qemu.pid -device virtio-net-pci,netdev=net0,mac=3a:f9:e2:f9:31:e8 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/disk.qcow2

	I0307 13:48:52.947016    1772 main.go:141] libmachine: STDOUT: 

	I0307 13:48:52.947069    1772 main.go:141] libmachine: STDERR: 

	I0307 13:48:52.947073    1772 main.go:141] libmachine: Attempt 0

	I0307 13:48:52.947093    1772 main.go:141] libmachine: Searching for 3a:f9:e2:f9:31:e8 in /var/db/dhcpd_leases ...

	I0307 13:48:52.947142    1772 main.go:141] libmachine: Found 1 entries in /var/db/dhcpd_leases!

	I0307 13:48:52.947162    1772 main.go:141] libmachine: dhcp entry: {Name: IPAddress:192.168.106.2 HWAddress:da:98:b3:c0:e3:d9 ID:1,da:98:b3:c0:e3:d9 Lease:0x65eb8718}

	I0307 13:48:54.949278    1772 main.go:141] libmachine: Attempt 1

	I0307 13:48:54.949358    1772 main.go:141] libmachine: Searching for 3a:f9:e2:f9:31:e8 in /var/db/dhcpd_leases ...

	I0307 13:48:54.949704    1772 main.go:141] libmachine: Found 1 entries in /var/db/dhcpd_leases!

	I0307 13:48:54.949754    1772 main.go:141] libmachine: dhcp entry: {Name: IPAddress:192.168.106.2 HWAddress:da:98:b3:c0:e3:d9 ID:1,da:98:b3:c0:e3:d9 Lease:0x65eb8718}

	I0307 13:48:56.952027    1772 main.go:141] libmachine: Attempt 2

	I0307 13:48:56.952169    1772 main.go:141] libmachine: Searching for 3a:f9:e2:f9:31:e8 in /var/db/dhcpd_leases ...

	I0307 13:48:56.952491    1772 main.go:141] libmachine: Found 1 entries in /var/db/dhcpd_leases!

	I0307 13:48:56.952541    1772 main.go:141] libmachine: dhcp entry: {Name: IPAddress:192.168.106.2 HWAddress:da:98:b3:c0:e3:d9 ID:1,da:98:b3:c0:e3:d9 Lease:0x65eb8718}

	I0307 13:48:58.954669    1772 main.go:141] libmachine: Attempt 3

	I0307 13:48:58.954694    1772 main.go:141] libmachine: Searching for 3a:f9:e2:f9:31:e8 in /var/db/dhcpd_leases ...

	I0307 13:48:58.954788    1772 main.go:141] libmachine: Found 1 entries in /var/db/dhcpd_leases!

	I0307 13:48:58.954814    1772 main.go:141] libmachine: dhcp entry: {Name: IPAddress:192.168.106.2 HWAddress:da:98:b3:c0:e3:d9 ID:1,da:98:b3:c0:e3:d9 Lease:0x65eb8718}

	I0307 13:49:00.956820    1772 main.go:141] libmachine: Attempt 4

	I0307 13:49:00.956831    1772 main.go:141] libmachine: Searching for 3a:f9:e2:f9:31:e8 in /var/db/dhcpd_leases ...

	I0307 13:49:00.956876    1772 main.go:141] libmachine: Found 1 entries in /var/db/dhcpd_leases!

	I0307 13:49:00.956884    1772 main.go:141] libmachine: dhcp entry: {Name: IPAddress:192.168.106.2 HWAddress:da:98:b3:c0:e3:d9 ID:1,da:98:b3:c0:e3:d9 Lease:0x65eb8718}

	I0307 13:49:02.958922    1772 main.go:141] libmachine: Attempt 5

	I0307 13:49:02.958930    1772 main.go:141] libmachine: Searching for 3a:f9:e2:f9:31:e8 in /var/db/dhcpd_leases ...

	I0307 13:49:02.958958    1772 main.go:141] libmachine: Found 1 entries in /var/db/dhcpd_leases!

	I0307 13:49:02.958964    1772 main.go:141] libmachine: dhcp entry: {Name: IPAddress:192.168.106.2 HWAddress:da:98:b3:c0:e3:d9 ID:1,da:98:b3:c0:e3:d9 Lease:0x65eb8718}

	I0307 13:49:04.960976    1772 main.go:141] libmachine: Attempt 6

	I0307 13:49:04.960991    1772 main.go:141] libmachine: Searching for 3a:f9:e2:f9:31:e8 in /var/db/dhcpd_leases ...

	I0307 13:49:04.961053    1772 main.go:141] libmachine: Found 1 entries in /var/db/dhcpd_leases!

	I0307 13:49:04.961073    1772 main.go:141] libmachine: dhcp entry: {Name: IPAddress:192.168.106.2 HWAddress:da:98:b3:c0:e3:d9 ID:1,da:98:b3:c0:e3:d9 Lease:0x65eb8718}

	I0307 13:49:06.963090    1772 main.go:141] libmachine: Attempt 7

	I0307 13:49:06.963115    1772 main.go:141] libmachine: Searching for 3a:f9:e2:f9:31:e8 in /var/db/dhcpd_leases ...

	I0307 13:49:06.963211    1772 main.go:141] libmachine: Found 2 entries in /var/db/dhcpd_leases!

	I0307 13:49:06.963223    1772 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:3a:f9:e2:f9:31:e8 ID:1,3a:f9:e2:f9:31:e8 Lease:0x65eb87d1}

	I0307 13:49:06.963225    1772 main.go:141] libmachine: Found match: 3a:f9:e2:f9:31:e8

	I0307 13:49:06.963234    1772 main.go:141] libmachine: IP: 192.168.105.2

	I0307 13:49:06.963238    1772 main.go:141] libmachine: Waiting for VM to start (ssh -p 22 docker@192.168.105.2)...

	I0307 13:49:08.984478    1772 machine.go:94] provisionDockerMachine start ...

	I0307 13:49:08.986017    1772 main.go:141] libmachine: Using SSH client type: native

	I0307 13:49:08.986526    1772 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100a3da30] 0x100a40290 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0307 13:49:08.986545    1772 main.go:141] libmachine: About to run SSH command:

	hostname

	I0307 13:49:09.053903    1772 main.go:141] libmachine: SSH cmd err, output: <nil>: minikube

	I0307 13:49:09.053947    1772 buildroot.go:166] provisioning hostname "addons-263000"

	I0307 13:49:09.054027    1772 main.go:141] libmachine: Using SSH client type: native

	I0307 13:49:09.054200    1772 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100a3da30] 0x100a40290 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0307 13:49:09.054209    1772 main.go:141] libmachine: About to run SSH command:

	sudo hostname addons-263000 && echo "addons-263000" | sudo tee /etc/hostname

	I0307 13:49:09.116097    1772 main.go:141] libmachine: SSH cmd err, output: <nil>: addons-263000

	I0307 13:49:09.116162    1772 main.go:141] libmachine: Using SSH client type: native

	I0307 13:49:09.116283    1772 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100a3da30] 0x100a40290 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0307 13:49:09.116292    1772 main.go:141] libmachine: About to run SSH command:

			if ! grep -xq '.*\saddons-263000' /etc/hosts; then

				if grep -xq '127.0.1.1\s.*' /etc/hosts; then

					sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 addons-263000/g' /etc/hosts;

				else 

					echo '127.0.1.1 addons-263000' | sudo tee -a /etc/hosts; 

				fi

			fi

	I0307 13:49:09.168249    1772 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	I0307 13:49:09.168269    1772 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/18320-1153/.minikube CaCertPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/18320-1153/.minikube}

	I0307 13:49:09.168281    1772 buildroot.go:174] setting up certificates

	I0307 13:49:09.168290    1772 provision.go:84] configureAuth start

	I0307 13:49:09.168293    1772 provision.go:143] copyHostCerts

	I0307 13:49:09.168411    1772 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/18320-1153/.minikube/ca.pem (1078 bytes)

	I0307 13:49:09.168678    1772 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/18320-1153/.minikube/cert.pem (1123 bytes)

	I0307 13:49:09.168823    1772 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/18320-1153/.minikube/key.pem (1679 bytes)

	I0307 13:49:09.168938    1772 provision.go:117] generating server cert: /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca-key.pem org=jenkins.addons-263000 san=[127.0.0.1 192.168.105.2 addons-263000 localhost minikube]

	I0307 13:49:09.339135    1772 provision.go:177] copyRemoteCerts

	I0307 13:49:09.339201    1772 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker

	I0307 13:49:09.339222    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:09.367085    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)

	I0307 13:49:09.375699    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes)

	I0307 13:49:09.383991    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)

	I0307 13:49:09.395598    1772 provision.go:87] duration metric: took 227.306417ms to configureAuth

	I0307 13:49:09.395606    1772 buildroot.go:189] setting minikube options for container-runtime

	I0307 13:49:09.395692    1772 config.go:182] Loaded profile config "addons-263000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 13:49:09.395724    1772 main.go:141] libmachine: Using SSH client type: native

	I0307 13:49:09.395801    1772 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100a3da30] 0x100a40290 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0307 13:49:09.395805    1772 main.go:141] libmachine: About to run SSH command:

	df --output=fstype / | tail -n 1

	I0307 13:49:09.444765    1772 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs

	I0307 13:49:09.444773    1772 buildroot.go:70] root file system type: tmpfs

	I0307 13:49:09.444828    1772 provision.go:314] Updating docker unit: /lib/systemd/system/docker.service ...

	I0307 13:49:09.444884    1772 main.go:141] libmachine: Using SSH client type: native

	I0307 13:49:09.444983    1772 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100a3da30] 0x100a40290 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0307 13:49:09.445018    1772 main.go:141] libmachine: About to run SSH command:

	sudo mkdir -p /lib/systemd/system && printf %!s(MISSING) "[Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP \$MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	" | sudo tee /lib/systemd/system/docker.service.new

	I0307 13:49:09.496195    1772 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP $MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	I0307 13:49:09.496232    1772 main.go:141] libmachine: Using SSH client type: native

	I0307 13:49:09.496340    1772 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100a3da30] 0x100a40290 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0307 13:49:09.496349    1772 main.go:141] libmachine: About to run SSH command:

	sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }

	I0307 13:49:09.858159    1772 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory

	Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

	I0307 13:49:09.858176    1772 machine.go:97] duration metric: took 873.684042ms to provisionDockerMachine

	I0307 13:49:09.858182    1772 client.go:171] duration metric: took 17.83463725s to LocalClient.Create

	I0307 13:49:09.858196    1772 start.go:167] duration metric: took 17.834704625s to libmachine.API.Create "addons-263000"

	I0307 13:49:09.858200    1772 start.go:293] postStartSetup for "addons-263000" (driver="qemu2")

	I0307 13:49:09.858206    1772 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]

	I0307 13:49:09.858290    1772 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs

	I0307 13:49:09.858300    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:09.883041    1772 ssh_runner.go:195] Run: cat /etc/os-release

	I0307 13:49:09.884410    1772 info.go:137] Remote host: Buildroot 2023.02.9

	I0307 13:49:09.884419    1772 filesync.go:126] Scanning /Users/jenkins/minikube-integration/18320-1153/.minikube/addons for local assets ...

	I0307 13:49:09.884492    1772 filesync.go:126] Scanning /Users/jenkins/minikube-integration/18320-1153/.minikube/files for local assets ...

	I0307 13:49:09.884520    1772 start.go:296] duration metric: took 26.318041ms for postStartSetup

	I0307 13:49:09.884895    1772 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/config.json ...

	I0307 13:49:09.885077    1772 start.go:128] duration metric: took 18.106876208s to createHost

	I0307 13:49:09.885096    1772 main.go:141] libmachine: Using SSH client type: native

	I0307 13:49:09.885173    1772 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100a3da30] 0x100a40290 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0307 13:49:09.885177    1772 main.go:141] libmachine: About to run SSH command:

	date +%!s(MISSING).%!N(MISSING)

	I0307 13:49:09.928081    1772 main.go:141] libmachine: SSH cmd err, output: <nil>: 1709848149.818154752

	I0307 13:49:09.928087    1772 fix.go:216] guest clock: 1709848149.818154752

	I0307 13:49:09.928091    1772 fix.go:229] Guest: 2024-03-07 13:49:09.818154752 -0800 PST Remote: 2024-03-07 13:49:09.885079 -0800 PST m=+18.216039542 (delta=-66.924248ms)

	I0307 13:49:09.928102    1772 fix.go:200] guest clock delta is within tolerance: -66.924248ms

	I0307 13:49:09.928105    1772 start.go:83] releasing machines lock for "addons-263000", held for 18.149944542s

	I0307 13:49:09.928394    1772 ssh_runner.go:195] Run: cat /version.json

	I0307 13:49:09.928405    1772 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/

	I0307 13:49:09.928404    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:09.928420    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:09.950946    1772 ssh_runner.go:195] Run: systemctl --version

	I0307 13:49:09.953389    1772 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"

	W0307 13:49:10.086561    1772 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found

	I0307 13:49:10.086663    1772 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;

	I0307 13:49:10.096697    1772 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)

	I0307 13:49:10.096710    1772 start.go:494] detecting cgroup driver to use...

	I0307 13:49:10.096955    1772 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock

	" | sudo tee /etc/crictl.yaml"

	I0307 13:49:10.107555    1772 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"

	I0307 13:49:10.112726    1772 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"

	I0307 13:49:10.117250    1772 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...

	I0307 13:49:10.117285    1772 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"

	I0307 13:49:10.122167    1772 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0307 13:49:10.126464    1772 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"

	I0307 13:49:10.130674    1772 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0307 13:49:10.134807    1772 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"

	I0307 13:49:10.138551    1772 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"

	I0307 13:49:10.142498    1772 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables

	I0307 13:49:10.146391    1772 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

	I0307 13:49:10.149875    1772 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0307 13:49:10.234313    1772 ssh_runner.go:195] Run: sudo systemctl restart containerd

	I0307 13:49:10.241430    1772 start.go:494] detecting cgroup driver to use...

	I0307 13:49:10.241511    1772 ssh_runner.go:195] Run: sudo systemctl cat docker.service

	I0307 13:49:10.249153    1772 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0307 13:49:10.254573    1772 ssh_runner.go:195] Run: sudo systemctl stop -f containerd

	I0307 13:49:10.260958    1772 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0307 13:49:10.265936    1772 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I0307 13:49:10.271136    1772 ssh_runner.go:195] Run: sudo systemctl stop -f crio

	I0307 13:49:10.311547    1772 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I0307 13:49:10.317705    1772 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock

	" | sudo tee /etc/crictl.yaml"

	I0307 13:49:10.323907    1772 ssh_runner.go:195] Run: which cri-dockerd

	I0307 13:49:10.325517    1772 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d

	I0307 13:49:10.328627    1772 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)

	I0307 13:49:10.334403    1772 ssh_runner.go:195] Run: sudo systemctl unmask docker.service

	I0307 13:49:10.419478    1772 ssh_runner.go:195] Run: sudo systemctl enable docker.socket

	I0307 13:49:10.489958    1772 docker.go:574] configuring docker to use "cgroupfs" as cgroup driver...

	I0307 13:49:10.490023    1772 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)

	I0307 13:49:10.496051    1772 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0307 13:49:10.580368    1772 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0307 13:49:11.735851    1772 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.155491916s)

	I0307 13:49:11.735928    1772 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.socket

	I0307 13:49:11.741649    1772 ssh_runner.go:195] Run: sudo systemctl stop cri-docker.socket

	I0307 13:49:11.748262    1772 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service

	I0307 13:49:11.753739    1772 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket

	I0307 13:49:11.827535    1772 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I0307 13:49:11.890185    1772 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0307 13:49:11.955554    1772 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket

	I0307 13:49:11.962175    1772 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service

	I0307 13:49:11.967427    1772 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0307 13:49:12.032201    1772 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.service

	I0307 13:49:12.055019    1772 start.go:541] Will wait 60s for socket path /var/run/cri-dockerd.sock

	I0307 13:49:12.055112    1772 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock

	I0307 13:49:12.058332    1772 start.go:562] Will wait 60s for crictl version

	I0307 13:49:12.058376    1772 ssh_runner.go:195] Run: which crictl

	I0307 13:49:12.059878    1772 ssh_runner.go:195] Run: sudo /usr/bin/crictl version

	I0307 13:49:12.084561    1772 start.go:578] Version:  0.1.0

	RuntimeName:  docker

	RuntimeVersion:  24.0.7

	RuntimeApiVersion:  v1

	I0307 13:49:12.084624    1772 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0307 13:49:12.096361    1772 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0307 13:49:12.113254    1772 out.go:204] * Preparing Kubernetes v1.28.4 on Docker 24.0.7 ...

	I0307 13:49:12.113402    1772 ssh_runner.go:195] Run: grep 192.168.105.1	host.minikube.internal$ /etc/hosts

	I0307 13:49:12.115092    1772 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.105.1	host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0307 13:49:12.119556    1772 kubeadm.go:877] updating cluster {Name:addons-263000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:4000 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28

.4 ClusterName:addons-263000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.105.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort

:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...

	I0307 13:49:12.119602    1772 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0307 13:49:12.119640    1772 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0307 13:49:12.129157    1772 docker.go:685] Got preloaded images: 

	I0307 13:49:12.129165    1772 docker.go:691] registry.k8s.io/kube-apiserver:v1.28.4 wasn't preloaded

	I0307 13:49:12.129207    1772 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0307 13:49:12.132622    1772 ssh_runner.go:195] Run: which lz4

	I0307 13:49:12.134077    1772 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4

	I0307 13:49:12.135388    1772 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4: Process exited with status 1

	stdout:

	stderr:

	stat: cannot statx '/preloaded.tar.lz4': No such file or directory

	I0307 13:49:12.135400    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4 --> /preloaded.tar.lz4 (357941720 bytes)

	I0307 13:49:13.379019    1772 docker.go:649] duration metric: took 1.245010375s to copy over tarball

	I0307 13:49:13.379075    1772 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4

	I0307 13:49:14.447938    1772 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (1.068872625s)

	I0307 13:49:14.447951    1772 ssh_runner.go:146] rm: /preloaded.tar.lz4

	I0307 13:49:14.464045    1772 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0307 13:49:14.467786    1772 ssh_runner.go:362] scp memory --> /var/lib/docker/image/overlay2/repositories.json (2629 bytes)

	I0307 13:49:14.473838    1772 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0307 13:49:14.561626    1772 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0307 13:49:16.191849    1772 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.630241292s)

	I0307 13:49:16.191936    1772 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0307 13:49:16.198256    1772 docker.go:685] Got preloaded images: -- stdout --

	registry.k8s.io/kube-apiserver:v1.28.4

	registry.k8s.io/kube-scheduler:v1.28.4

	registry.k8s.io/kube-controller-manager:v1.28.4

	registry.k8s.io/kube-proxy:v1.28.4

	registry.k8s.io/etcd:3.5.9-0

	registry.k8s.io/coredns/coredns:v1.10.1

	registry.k8s.io/pause:3.9

	gcr.io/k8s-minikube/storage-provisioner:v5

	-- /stdout --

	I0307 13:49:16.198268    1772 cache_images.go:84] Images are preloaded, skipping loading

	I0307 13:49:16.198274    1772 kubeadm.go:928] updating node { 192.168.105.2 8443 v1.28.4 docker true true} ...

	I0307 13:49:16.198332    1772 kubeadm.go:940] kubelet [Unit]

	Wants=docker.socket

	[Service]

	ExecStart=

	ExecStart=/var/lib/minikube/binaries/v1.28.4/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=addons-263000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.105.2

	[Install]

	 config:

	{KubernetesVersion:v1.28.4 ClusterName:addons-263000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}

	I0307 13:49:16.198391    1772 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}

	I0307 13:49:16.206203    1772 cni.go:84] Creating CNI manager for ""

	I0307 13:49:16.206214    1772 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0307 13:49:16.206220    1772 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16

	I0307 13:49:16.206229    1772 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.105.2 APIServerPort:8443 KubernetesVersion:v1.28.4 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:addons-263000 NodeName:addons-263000 DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.105.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.105.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/ku

bernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///var/run/cri-dockerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}

	I0307 13:49:16.206297    1772 kubeadm.go:187] kubeadm config:

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: InitConfiguration

	localAPIEndpoint:

	  advertiseAddress: 192.168.105.2

	  bindPort: 8443

	bootstrapTokens:

	  - groups:

	      - system:bootstrappers:kubeadm:default-node-token

	    ttl: 24h0m0s

	    usages:

	      - signing

	      - authentication

	nodeRegistration:

	  criSocket: unix:///var/run/cri-dockerd.sock

	  name: "addons-263000"

	  kubeletExtraArgs:

	    node-ip: 192.168.105.2

	  taints: []

	---

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: ClusterConfiguration

	apiServer:

	  certSANs: ["127.0.0.1", "localhost", "192.168.105.2"]

	  extraArgs:

	    enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"

	controllerManager:

	  extraArgs:

	    allocate-node-cidrs: "true"

	    leader-elect: "false"

	scheduler:

	  extraArgs:

	    leader-elect: "false"

	certificatesDir: /var/lib/minikube/certs

	clusterName: mk

	controlPlaneEndpoint: control-plane.minikube.internal:8443

	etcd:

	  local:

	    dataDir: /var/lib/minikube/etcd

	    extraArgs:

	      proxy-refresh-interval: "70000"

	kubernetesVersion: v1.28.4

	networking:

	  dnsDomain: cluster.local

	  podSubnet: "10.244.0.0/16"

	  serviceSubnet: 10.96.0.0/12

	---

	apiVersion: kubelet.config.k8s.io/v1beta1

	kind: KubeletConfiguration

	authentication:

	  x509:

	    clientCAFile: /var/lib/minikube/certs/ca.crt

	cgroupDriver: cgroupfs

	containerRuntimeEndpoint: unix:///var/run/cri-dockerd.sock

	hairpinMode: hairpin-veth

	runtimeRequestTimeout: 15m

	clusterDomain: "cluster.local"

	# disable disk resource management by default

	imageGCHighThresholdPercent: 100

	evictionHard:

	  nodefs.available: "0%!"(MISSING)

	  nodefs.inodesFree: "0%!"(MISSING)

	  imagefs.available: "0%!"(MISSING)

	failSwapOn: false

	staticPodPath: /etc/kubernetes/manifests

	---

	apiVersion: kubeproxy.config.k8s.io/v1alpha1

	kind: KubeProxyConfiguration

	clusterCIDR: "10.244.0.0/16"

	metricsBindAddress: 0.0.0.0:10249

	conntrack:

	  maxPerCore: 0

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"

	  tcpEstablishedTimeout: 0s

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"

	  tcpCloseWaitTimeout: 0s

	I0307 13:49:16.206357    1772 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.4

	I0307 13:49:16.210649    1772 binaries.go:44] Found k8s binaries, skipping transfer

	I0307 13:49:16.210685    1772 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube

	I0307 13:49:16.214516    1772 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (313 bytes)

	I0307 13:49:16.221325    1772 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)

	I0307 13:49:16.227110    1772 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2158 bytes)

	I0307 13:49:16.233155    1772 ssh_runner.go:195] Run: grep 192.168.105.2	control-plane.minikube.internal$ /etc/hosts

	I0307 13:49:16.234520    1772 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.105.2	control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0307 13:49:16.238699    1772 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0307 13:49:16.326067    1772 ssh_runner.go:195] Run: sudo systemctl start kubelet

	I0307 13:49:16.335871    1772 certs.go:68] Setting up /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000 for IP: 192.168.105.2

	I0307 13:49:16.335880    1772 certs.go:194] generating shared ca certs ...

	I0307 13:49:16.335889    1772 certs.go:226] acquiring lock for ca certs: {Name:mkd070f5c39ed755a49409426e8106f4ed85635d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:16.336068    1772 certs.go:240] generating "minikubeCA" ca cert: /Users/jenkins/minikube-integration/18320-1153/.minikube/ca.key

	I0307 13:49:16.384044    1772 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/18320-1153/.minikube/ca.crt ...

	I0307 13:49:16.384054    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/ca.crt: {Name:mkb2fa43b86e56f28611cf39cf6b1e2c2a323232 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:16.384345    1772 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/18320-1153/.minikube/ca.key ...

	I0307 13:49:16.384349    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/ca.key: {Name:mkedd90b99d72ce98dfa64f2cdd371e2ee67886f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:16.384489    1772 certs.go:240] generating "proxyClientCA" ca cert: /Users/jenkins/minikube-integration/18320-1153/.minikube/proxy-client-ca.key

	I0307 13:49:16.497532    1772 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/18320-1153/.minikube/proxy-client-ca.crt ...

	I0307 13:49:16.497539    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/proxy-client-ca.crt: {Name:mk1768fc05e0a57fdc323a669a59da19ac854bda Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:16.497696    1772 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/18320-1153/.minikube/proxy-client-ca.key ...

	I0307 13:49:16.497699    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/proxy-client-ca.key: {Name:mk09f27376294eef33fb3442ff9e0e7e802f1781 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:16.497816    1772 certs.go:256] generating profile certs ...

	I0307 13:49:16.497853    1772 certs.go:363] generating signed profile cert for "minikube-user": /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.key

	I0307 13:49:16.497859    1772 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.crt with IP's: []

	I0307 13:49:16.684173    1772 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.crt ...

	I0307 13:49:16.684179    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.crt: {Name:mka2c972577c2ade47674292d12b392145986842 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:16.684411    1772 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.key ...

	I0307 13:49:16.684414    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.key: {Name:mk1b0f568f3b819c91a4f3dce4fbee7ed40ace1e Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:16.684537    1772 certs.go:363] generating signed profile cert for "minikube": /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.key.898a7cc9

	I0307 13:49:16.684553    1772 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.crt.898a7cc9 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.105.2]

	I0307 13:49:16.916918    1772 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.crt.898a7cc9 ...

	I0307 13:49:16.916938    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.crt.898a7cc9: {Name:mk93bd8f73641fb441ebd53279becce6b7fa258a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:16.917259    1772 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.key.898a7cc9 ...

	I0307 13:49:16.917263    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.key.898a7cc9: {Name:mk5dd3656a95c01f9c46db6e32a2ec7a62c50027 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:16.917387    1772 certs.go:381] copying /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.crt.898a7cc9 -> /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.crt

	I0307 13:49:16.917663    1772 certs.go:385] copying /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.key.898a7cc9 -> /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.key

	I0307 13:49:16.917810    1772 certs.go:363] generating signed profile cert for "aggregator": /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/proxy-client.key

	I0307 13:49:16.917825    1772 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/proxy-client.crt with IP's: []

	I0307 13:49:16.972656    1772 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/proxy-client.crt ...

	I0307 13:49:16.972661    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/proxy-client.crt: {Name:mk61502a61542f87f0e2502e25b2d4a4aaddbea3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:16.972824    1772 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/proxy-client.key ...

	I0307 13:49:16.972827    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/proxy-client.key: {Name:mk3a99bbb1e258c6205e32ae77c51a35fb0aa4e4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:16.973071    1772 certs.go:484] found cert: /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca-key.pem (1679 bytes)

	I0307 13:49:16.973095    1772 certs.go:484] found cert: /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem (1078 bytes)

	I0307 13:49:16.973113    1772 certs.go:484] found cert: /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem (1123 bytes)

	I0307 13:49:16.973130    1772 certs.go:484] found cert: /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/key.pem (1679 bytes)

	I0307 13:49:16.973437    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)

	I0307 13:49:16.984019    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)

	I0307 13:49:16.993336    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)

	I0307 13:49:17.002445    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)

	I0307 13:49:17.013022    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1419 bytes)

	I0307 13:49:17.021377    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)

	I0307 13:49:17.029664    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)

	I0307 13:49:17.038126    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)

	I0307 13:49:17.046612    1772 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/18320-1153/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)

	I0307 13:49:17.055364    1772 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)

	I0307 13:49:17.062339    1772 ssh_runner.go:195] Run: openssl version

	I0307 13:49:17.064576    1772 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"

	I0307 13:49:17.068250    1772 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem

	I0307 13:49:17.069784    1772 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar  7 21:49 /usr/share/ca-certificates/minikubeCA.pem

	I0307 13:49:17.069807    1772 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem

	I0307 13:49:17.071697    1772 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"

	I0307 13:49:17.075576    1772 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt

	I0307 13:49:17.077092    1772 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1

	stdout:

	stderr:

	stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory

	I0307 13:49:17.077115    1772 kubeadm.go:391] StartCluster: {Name:addons-263000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:4000 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4

ClusterName:addons-263000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.105.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0

MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 13:49:17.077183    1772 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}

	I0307 13:49:17.085320    1772 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd

	I0307 13:49:17.088753    1772 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml

	I0307 13:49:17.092071    1772 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf

	I0307 13:49:17.095511    1772 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory

	I0307 13:49:17.095518    1772 kubeadm.go:156] found existing configuration files:

	I0307 13:49:17.095541    1772 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf

	I0307 13:49:17.098940    1772 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/admin.conf: No such file or directory

	I0307 13:49:17.098962    1772 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf

	I0307 13:49:17.102480    1772 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf

	I0307 13:49:17.105681    1772 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/kubelet.conf: No such file or directory

	I0307 13:49:17.105708    1772 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf

	I0307 13:49:17.108908    1772 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf

	I0307 13:49:17.111880    1772 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/controller-manager.conf: No such file or directory

	I0307 13:49:17.111907    1772 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf

	I0307 13:49:17.115310    1772 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf

	I0307 13:49:17.118795    1772 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/scheduler.conf: No such file or directory

	I0307 13:49:17.118820    1772 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf

	I0307 13:49:17.122135    1772 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.4:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"

	I0307 13:49:17.144663    1772 kubeadm.go:309] [init] Using Kubernetes version: v1.28.4

	I0307 13:49:17.144721    1772 kubeadm.go:309] [preflight] Running pre-flight checks

	I0307 13:49:17.197949    1772 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster

	I0307 13:49:17.198019    1772 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection

	I0307 13:49:17.198075    1772 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	I0307 13:49:17.293245    1772 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"

	I0307 13:49:17.306437    1772 out.go:204]   - Generating certificates and keys ...

	I0307 13:49:17.306477    1772 kubeadm.go:309] [certs] Using existing ca certificate authority

	I0307 13:49:17.306508    1772 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk

	I0307 13:49:17.477993    1772 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key

	I0307 13:49:17.549493    1772 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key

	I0307 13:49:17.666345    1772 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key

	I0307 13:49:17.753667    1772 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key

	I0307 13:49:18.109627    1772 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key

	I0307 13:49:18.109694    1772 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [addons-263000 localhost] and IPs [192.168.105.2 127.0.0.1 ::1]

	I0307 13:49:18.207352    1772 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key

	I0307 13:49:18.207419    1772 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [addons-263000 localhost] and IPs [192.168.105.2 127.0.0.1 ::1]

	I0307 13:49:18.289678    1772 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key

	I0307 13:49:18.400435    1772 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key

	I0307 13:49:18.484761    1772 kubeadm.go:309] [certs] Generating "sa" key and public key

	I0307 13:49:18.484790    1772 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	I0307 13:49:18.527573    1772 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file

	I0307 13:49:18.574712    1772 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file

	I0307 13:49:18.667176    1772 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file

	I0307 13:49:18.815839    1772 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file

	I0307 13:49:18.816023    1772 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	I0307 13:49:18.817114    1772 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"

	I0307 13:49:18.820784    1772 out.go:204]   - Booting up control plane ...

	I0307 13:49:18.820831    1772 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"

	I0307 13:49:18.820879    1772 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"

	I0307 13:49:18.820947    1772 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"

	I0307 13:49:18.824696    1772 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	I0307 13:49:18.824748    1772 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	I0307 13:49:18.824768    1772 kubeadm.go:309] [kubelet-start] Starting the kubelet

	I0307 13:49:18.900193    1772 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	I0307 13:49:22.401114    1772 kubeadm.go:309] [apiclient] All control plane components are healthy after 3.500980 seconds

	I0307 13:49:22.401178    1772 kubeadm.go:309] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace

	I0307 13:49:22.408467    1772 kubeadm.go:309] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster

	I0307 13:49:22.917382    1772 kubeadm.go:309] [upload-certs] Skipping phase. Please see --upload-certs

	I0307 13:49:22.917483    1772 kubeadm.go:309] [mark-control-plane] Marking the node addons-263000 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]

	I0307 13:49:23.422255    1772 kubeadm.go:309] [bootstrap-token] Using token: nkhozd.3tkib72dcs4xrktx

	I0307 13:49:23.428429    1772 out.go:204]   - Configuring RBAC rules ...

	I0307 13:49:23.428487    1772 kubeadm.go:309] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles

	I0307 13:49:23.429398    1772 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes

	I0307 13:49:23.433055    1772 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials

	I0307 13:49:23.434156    1772 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token

	I0307 13:49:23.435719    1772 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster

	I0307 13:49:23.437046    1772 kubeadm.go:309] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace

	I0307 13:49:23.440618    1772 kubeadm.go:309] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key

	I0307 13:49:23.612878    1772 kubeadm.go:309] [addons] Applied essential addon: CoreDNS

	I0307 13:49:23.831870    1772 kubeadm.go:309] [addons] Applied essential addon: kube-proxy

	I0307 13:49:23.832269    1772 kubeadm.go:309] 

	I0307 13:49:23.832299    1772 kubeadm.go:309] Your Kubernetes control-plane has initialized successfully!

	I0307 13:49:23.832309    1772 kubeadm.go:309] 

	I0307 13:49:23.832357    1772 kubeadm.go:309] To start using your cluster, you need to run the following as a regular user:

	I0307 13:49:23.832363    1772 kubeadm.go:309] 

	I0307 13:49:23.832380    1772 kubeadm.go:309]   mkdir -p $HOME/.kube

	I0307 13:49:23.832410    1772 kubeadm.go:309]   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

	I0307 13:49:23.832437    1772 kubeadm.go:309]   sudo chown $(id -u):$(id -g) $HOME/.kube/config

	I0307 13:49:23.832440    1772 kubeadm.go:309] 

	I0307 13:49:23.832472    1772 kubeadm.go:309] Alternatively, if you are the root user, you can run:

	I0307 13:49:23.832476    1772 kubeadm.go:309] 

	I0307 13:49:23.832507    1772 kubeadm.go:309]   export KUBECONFIG=/etc/kubernetes/admin.conf

	I0307 13:49:23.832510    1772 kubeadm.go:309] 

	I0307 13:49:23.832543    1772 kubeadm.go:309] You should now deploy a pod network to the cluster.

	I0307 13:49:23.832580    1772 kubeadm.go:309] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

	I0307 13:49:23.832612    1772 kubeadm.go:309]   https://kubernetes.io/docs/concepts/cluster-administration/addons/

	I0307 13:49:23.832615    1772 kubeadm.go:309] 

	I0307 13:49:23.832672    1772 kubeadm.go:309] You can now join any number of control-plane nodes by copying certificate authorities

	I0307 13:49:23.832719    1772 kubeadm.go:309] and service account keys on each node and then running the following as root:

	I0307 13:49:23.832721    1772 kubeadm.go:309] 

	I0307 13:49:23.832780    1772 kubeadm.go:309]   kubeadm join control-plane.minikube.internal:8443 --token nkhozd.3tkib72dcs4xrktx \

	I0307 13:49:23.832841    1772 kubeadm.go:309] 	--discovery-token-ca-cert-hash sha256:dc223c8f61ede9166cd6af749ad18625b70792e87215b933ce92e49a8a2f5e5b \

	I0307 13:49:23.832852    1772 kubeadm.go:309] 	--control-plane 

	I0307 13:49:23.832856    1772 kubeadm.go:309] 

	I0307 13:49:23.832920    1772 kubeadm.go:309] Then you can join any number of worker nodes by running the following on each as root:

	I0307 13:49:23.832927    1772 kubeadm.go:309] 

	I0307 13:49:23.832985    1772 kubeadm.go:309] kubeadm join control-plane.minikube.internal:8443 --token nkhozd.3tkib72dcs4xrktx \

	I0307 13:49:23.833033    1772 kubeadm.go:309] 	--discovery-token-ca-cert-hash sha256:dc223c8f61ede9166cd6af749ad18625b70792e87215b933ce92e49a8a2f5e5b 

	I0307 13:49:23.833113    1772 kubeadm.go:309] 	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	I0307 13:49:23.833122    1772 cni.go:84] Creating CNI manager for ""

	I0307 13:49:23.833132    1772 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0307 13:49:23.837085    1772 out.go:177] * Configuring bridge CNI (Container Networking Interface) ...

	I0307 13:49:23.845100    1772 ssh_runner.go:195] Run: sudo mkdir -p /etc/cni/net.d

	I0307 13:49:23.848920    1772 ssh_runner.go:362] scp memory --> /etc/cni/net.d/1-k8s.conflist (457 bytes)

	I0307 13:49:23.854814    1772 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"

	I0307 13:49:23.854869    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:23.854879    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig label --overwrite nodes addons-263000 minikube.k8s.io/updated_at=2024_03_07T13_49_23_0700 minikube.k8s.io/version=v1.32.0 minikube.k8s.io/commit=3e3656b8cff33aafa60dd2a07a4b34bce666a6a6 minikube.k8s.io/name=addons-263000 minikube.k8s.io/primary=true

	I0307 13:49:23.858918    1772 ops.go:34] apiserver oom_adj: -16

	I0307 13:49:23.915951    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:24.418032    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:24.918058    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:25.418049    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:25.917980    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:26.417653    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:26.916907    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:27.417986    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:27.917956    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:28.417955    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:28.917934    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:29.417903    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:29.917915    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:30.417913    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:30.917901    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:31.417879    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:31.917879    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:32.417871    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:32.917854    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:33.417843    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:33.917802    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:34.417815    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:34.917789    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:35.417767    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:35.917737    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:36.417793    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:36.917721    1772 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0307 13:49:36.952408    1772 kubeadm.go:1106] duration metric: took 13.097878958s to wait for elevateKubeSystemPrivileges

	W0307 13:49:36.952447    1772 kubeadm.go:286] apiserver tunnel failed: apiserver port not set

	I0307 13:49:36.952453    1772 kubeadm.go:393] duration metric: took 19.875784166s to StartCluster

	I0307 13:49:36.952461    1772 settings.go:142] acquiring lock: {Name:mke1b21879aa30c53aba22e595b1eacff0880bb8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:36.952622    1772 settings.go:150] Updating kubeconfig:  /Users/jenkins/minikube-integration/18320-1153/kubeconfig

	I0307 13:49:36.952889    1772 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/kubeconfig: {Name:mk8faa4add6483df163f8e33c2463984a39a544c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 13:49:36.953116    1772 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"

	I0307 13:49:36.953134    1772 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.105.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 13:49:36.956317    1772 out.go:177] * Verifying Kubernetes components...

	I0307 13:49:36.953170    1772 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:true csi-hostpath-driver:true dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:true gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:true ingress-dns:true inspektor-gadget:true istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:true nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:true registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:true volumesnapshots:true yakd:true]

	I0307 13:49:36.956343    1772 addons.go:69] Setting default-storageclass=true in profile "addons-263000"

	I0307 13:49:36.956352    1772 addons.go:69] Setting ingress=true in profile "addons-263000"

	I0307 13:49:36.956357    1772 addons.go:69] Setting csi-hostpath-driver=true in profile "addons-263000"

	I0307 13:49:36.956360    1772 addons.go:234] Setting addon ingress=true in "addons-263000"

	I0307 13:49:36.956373    1772 addons.go:234] Setting addon csi-hostpath-driver=true in "addons-263000"

	I0307 13:49:36.956378    1772 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "addons-263000"

	I0307 13:49:36.956389    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.956396    1772 addons.go:69] Setting storage-provisioner=true in profile "addons-263000"

	I0307 13:49:36.956419    1772 addons.go:69] Setting nvidia-device-plugin=true in profile "addons-263000"

	I0307 13:49:36.956432    1772 addons.go:234] Setting addon nvidia-device-plugin=true in "addons-263000"

	I0307 13:49:36.956443    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.956392    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.956474    1772 addons.go:69] Setting registry=true in profile "addons-263000"

	I0307 13:49:36.956481    1772 addons.go:69] Setting ingress-dns=true in profile "addons-263000"

	I0307 13:49:36.956496    1772 addons.go:234] Setting addon ingress-dns=true in "addons-263000"

	I0307 13:49:36.956506    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.956511    1772 addons.go:234] Setting addon registry=true in "addons-263000"

	I0307 13:49:36.956441    1772 addons.go:69] Setting inspektor-gadget=true in profile "addons-263000"

	I0307 13:49:36.956557    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.956566    1772 addons.go:234] Setting addon inspektor-gadget=true in "addons-263000"

	I0307 13:49:36.956631    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.956700    1772 addons.go:69] Setting metrics-server=true in profile "addons-263000"

	I0307 13:49:36.956711    1772 retry.go:31] will retry after 1.000364067s: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:36.956344    1772 addons.go:69] Setting yakd=true in profile "addons-263000"

	I0307 13:49:36.956744    1772 addons.go:234] Setting addon yakd=true in "addons-263000"

	I0307 13:49:36.956753    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.956718    1772 addons.go:234] Setting addon metrics-server=true in "addons-263000"

	I0307 13:49:36.956769    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.956806    1772 retry.go:31] will retry after 1.153881273s: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:36.956837    1772 retry.go:31] will retry after 839.062794ms: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:36.956351    1772 addons.go:69] Setting cloud-spanner=true in profile "addons-263000"

	I0307 13:49:36.956920    1772 addons.go:234] Setting addon cloud-spanner=true in "addons-263000"

	I0307 13:49:36.953269    1772 config.go:182] Loaded profile config "addons-263000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 13:49:36.956960    1772 retry.go:31] will retry after 837.287826ms: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:36.956447    1772 addons.go:234] Setting addon storage-provisioner=true in "addons-263000"

	I0307 13:49:36.956968    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.956977    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.956982    1772 retry.go:31] will retry after 820.041814ms: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:36.956722    1772 addons.go:69] Setting volumesnapshots=true in profile "addons-263000"

	I0307 13:49:36.957042    1772 retry.go:31] will retry after 907.24251ms: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:36.957047    1772 addons.go:234] Setting addon volumesnapshots=true in "addons-263000"

	I0307 13:49:36.957060    1772 retry.go:31] will retry after 802.655369ms: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:36.957079    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.956725    1772 addons.go:69] Setting storage-provisioner-rancher=true in profile "addons-263000"

	I0307 13:49:36.956349    1772 addons.go:69] Setting gcp-auth=true in profile "addons-263000"

	I0307 13:49:36.963150    1772 mustload.go:65] Loading cluster: addons-263000

	I0307 13:49:36.963159    1772 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0307 13:49:36.957110    1772 addons_storage_classes.go:33] enableOrDisableStorageClasses storage-provisioner-rancher=true on "addons-263000"

	I0307 13:49:36.963221    1772 config.go:182] Loaded profile config "addons-263000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 13:49:36.957173    1772 retry.go:31] will retry after 575.438781ms: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:36.957348    1772 retry.go:31] will retry after 1.022295219s: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:36.957366    1772 retry.go:31] will retry after 827.696037ms: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:36.958780    1772 addons.go:234] Setting addon default-storageclass=true in "addons-263000"

	I0307 13:49:36.963241    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.966216    1772 out.go:177]   - Using image gcr.io/k8s-minikube/minikube-ingress-dns:0.0.2

	I0307 13:49:36.963481    1772 retry.go:31] will retry after 847.749927ms: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:36.964153    1772 addons.go:234] Setting addon storage-provisioner-rancher=true in "addons-263000"

	I0307 13:49:36.964348    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.970170    1772 addons.go:426] installing /etc/kubernetes/addons/ingress-dns-pod.yaml

	I0307 13:49:36.970176    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ingress-dns-pod.yaml (2442 bytes)

	I0307 13:49:36.970185    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:36.970237    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:36.975162    1772 out.go:177]   - Using image docker.io/busybox:stable

	I0307 13:49:36.984093    1772 out.go:177]   - Using image docker.io/rancher/local-path-provisioner:v0.0.22

	I0307 13:49:36.990186    1772 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner-rancher.yaml

	I0307 13:49:36.990196    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner-rancher.yaml (3113 bytes)

	I0307 13:49:36.990205    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:37.018442    1772 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^        forward . \/etc\/resolv.conf.*/i \        hosts {\n           192.168.105.1 host.minikube.internal\n           fallthrough\n        }' -e '/^        errors *$/i \        log' | sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"

	I0307 13:49:37.083951    1772 ssh_runner.go:195] Run: sudo systemctl start kubelet

	I0307 13:49:37.141815    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	I0307 13:49:37.163931    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storage-provisioner-rancher.yaml

	I0307 13:49:37.545249    1772 out.go:177]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5

	I0307 13:49:37.549134    1772 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml

	I0307 13:49:37.549140    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)

	I0307 13:49:37.549150    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:37.708607    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml

	I0307 13:49:37.765221    1772 out.go:177]   - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0

	I0307 13:49:37.769175    1772 out.go:177]   - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0

	I0307 13:49:37.773120    1772 out.go:177]   - Using image registry.k8s.io/ingress-nginx/controller:v1.10.0

	I0307 13:49:37.777240    1772 addons.go:426] installing /etc/kubernetes/addons/ingress-deploy.yaml

	I0307 13:49:37.777248    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ingress-deploy.yaml (16078 bytes)

	I0307 13:49:37.777259    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:37.777491    1772 start.go:948] {"host.minikube.internal": 192.168.105.1} host record injected into CoreDNS's ConfigMap

	I0307 13:49:37.777898    1772 node_ready.go:35] waiting up to 6m0s for node "addons-263000" to be "Ready" ...

	I0307 13:49:37.782017    1772 out.go:177]   - Using image docker.io/registry:2.8.3

	I0307 13:49:37.786141    1772 out.go:177]   - Using image gcr.io/k8s-minikube/kube-registry-proxy:0.0.5

	I0307 13:49:37.790242    1772 addons.go:426] installing /etc/kubernetes/addons/registry-rc.yaml

	I0307 13:49:37.790250    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/registry-rc.yaml (798 bytes)

	I0307 13:49:37.790260    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:37.790785    1772 node_ready.go:49] node "addons-263000" has status "Ready":"True"

	I0307 13:49:37.790804    1772 node_ready.go:38] duration metric: took 12.88475ms for node "addons-263000" to be "Ready" ...

	I0307 13:49:37.790822    1772 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I0307 13:49:37.795059    1772 out.go:177]   - Using image registry.k8s.io/sig-storage/snapshot-controller:v6.1.0

	I0307 13:49:37.799233    1772 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml

	I0307 13:49:37.799249    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml (934 bytes)

	I0307 13:49:37.799265    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:37.799641    1772 retry.go:31] will retry after 2.086535579s: connect: dial unix /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/monitor: connect: connection refused

	I0307 13:49:37.803156    1772 out.go:177]   - Using image registry.k8s.io/metrics-server/metrics-server:v0.7.0

	I0307 13:49:37.801469    1772 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace to be "Ready" ...

	I0307 13:49:37.807196    1772 addons.go:426] installing /etc/kubernetes/addons/metrics-apiservice.yaml

	I0307 13:49:37.807207    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)

	I0307 13:49:37.807217    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:37.818820    1772 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml

	I0307 13:49:37.818851    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)

	I0307 13:49:37.818859    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:37.868190    1772 out.go:177]   - Using image docker.io/marcnuri/yakd:0.0.4

	I0307 13:49:37.874183    1772 addons.go:426] installing /etc/kubernetes/addons/yakd-ns.yaml

	I0307 13:49:37.874194    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/yakd-ns.yaml (171 bytes)

	I0307 13:49:37.874204    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:37.943706    1772 addons.go:426] installing /etc/kubernetes/addons/metrics-server-deployment.yaml

	I0307 13:49:37.943716    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1907 bytes)

	I0307 13:49:37.956336    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml

	I0307 13:49:37.962165    1772 out.go:177]   - Using image nvcr.io/nvidia/k8s-device-plugin:v0.14.5

	I0307 13:49:37.965074    1772 addons.go:426] installing /etc/kubernetes/addons/nvidia-device-plugin.yaml

	I0307 13:49:37.965080    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/nvidia-device-plugin.yaml (1966 bytes)

	I0307 13:49:37.965090    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:37.971609    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml

	I0307 13:49:37.990147    1772 out.go:177]   - Using image gcr.io/cloud-spanner-emulator/emulator:1.5.14

	I0307 13:49:37.994203    1772 addons.go:426] installing /etc/kubernetes/addons/deployment.yaml

	I0307 13:49:37.994214    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/deployment.yaml (1004 bytes)

	I0307 13:49:37.994224    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:37.994468    1772 addons.go:426] installing /etc/kubernetes/addons/registry-svc.yaml

	I0307 13:49:37.994473    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/registry-svc.yaml (398 bytes)

	I0307 13:49:37.996650    1772 addons.go:426] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml

	I0307 13:49:37.996658    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml (6471 bytes)

	I0307 13:49:38.000809    1772 addons.go:426] installing /etc/kubernetes/addons/metrics-server-rbac.yaml

	I0307 13:49:38.000817    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)

	I0307 13:49:38.025826    1772 addons.go:426] installing /etc/kubernetes/addons/registry-proxy.yaml

	I0307 13:49:38.025835    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/registry-proxy.yaml (947 bytes)

	I0307 13:49:38.031777    1772 addons.go:426] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml

	I0307 13:49:38.031787    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml (23126 bytes)

	I0307 13:49:38.055094    1772 addons.go:426] installing /etc/kubernetes/addons/metrics-server-service.yaml

	I0307 13:49:38.055107    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)

	I0307 13:49:38.055207    1772 addons.go:426] installing /etc/kubernetes/addons/yakd-sa.yaml

	I0307 13:49:38.055216    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/yakd-sa.yaml (247 bytes)

	I0307 13:49:38.076920    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/registry-rc.yaml -f /etc/kubernetes/addons/registry-svc.yaml -f /etc/kubernetes/addons/registry-proxy.yaml

	I0307 13:49:38.078283    1772 addons.go:426] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml

	I0307 13:49:38.078289    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml (19582 bytes)

	I0307 13:49:38.099234    1772 addons.go:426] installing /etc/kubernetes/addons/yakd-crb.yaml

	I0307 13:49:38.099247    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/yakd-crb.yaml (422 bytes)

	I0307 13:49:38.103733    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/deployment.yaml

	I0307 13:49:38.116269    1772 out.go:177]   - Using image registry.k8s.io/sig-storage/hostpathplugin:v1.9.0

	I0307 13:49:38.114030    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml

	I0307 13:49:38.114120    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/nvidia-device-plugin.yaml

	I0307 13:49:38.123104    1772 out.go:177]   - Using image registry.k8s.io/sig-storage/livenessprobe:v2.8.0

	I0307 13:49:38.119735    1772 addons.go:426] installing /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml

	I0307 13:49:38.123120    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml (3545 bytes)

	I0307 13:49:38.131100    1772 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-resizer:v1.6.0

	I0307 13:49:38.135096    1772 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0

	I0307 13:49:38.143047    1772 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-provisioner:v3.3.0

	I0307 13:49:38.150989    1772 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-attacher:v4.0.0

	I0307 13:49:38.159092    1772 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.7.0

	I0307 13:49:38.163104    1772 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.6.0

	I0307 13:49:38.167198    1772 addons.go:426] installing /etc/kubernetes/addons/rbac-external-attacher.yaml

	I0307 13:49:38.167210    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-attacher.yaml (3073 bytes)

	I0307 13:49:38.167221    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:38.182496    1772 addons.go:426] installing /etc/kubernetes/addons/yakd-svc.yaml

	I0307 13:49:38.182506    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/yakd-svc.yaml (412 bytes)

	I0307 13:49:38.190165    1772 addons.go:426] installing /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0307 13:49:38.190174    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml (1475 bytes)

	I0307 13:49:38.253518    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0307 13:49:38.280546    1772 kapi.go:248] "coredns" deployment in "kube-system" namespace and "addons-263000" context rescaled to 1 replicas

	I0307 13:49:38.327497    1772 addons.go:426] installing /etc/kubernetes/addons/yakd-dp.yaml

	I0307 13:49:38.327507    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/yakd-dp.yaml (2017 bytes)

	I0307 13:49:38.386695    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/yakd-ns.yaml -f /etc/kubernetes/addons/yakd-sa.yaml -f /etc/kubernetes/addons/yakd-crb.yaml -f /etc/kubernetes/addons/yakd-svc.yaml -f /etc/kubernetes/addons/yakd-dp.yaml

	I0307 13:49:38.437897    1772 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storage-provisioner-rancher.yaml: (1.2739775s)

	I0307 13:49:38.518376    1772 addons.go:426] installing /etc/kubernetes/addons/rbac-hostpath.yaml

	I0307 13:49:38.518388    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-hostpath.yaml (4266 bytes)

	I0307 13:49:38.562626    1772 addons.go:426] installing /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml

	I0307 13:49:38.562639    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml (3038 bytes)

	I0307 13:49:38.615105    1772 addons.go:426] installing /etc/kubernetes/addons/rbac-external-provisioner.yaml

	I0307 13:49:38.615116    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-provisioner.yaml (4442 bytes)

	I0307 13:49:38.690932    1772 addons.go:426] installing /etc/kubernetes/addons/rbac-external-resizer.yaml

	I0307 13:49:38.690945    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-resizer.yaml (2943 bytes)

	I0307 13:49:38.757629    1772 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (1.049029875s)

	I0307 13:49:38.758617    1772 addons.go:426] installing /etc/kubernetes/addons/rbac-external-snapshotter.yaml

	I0307 13:49:38.758623    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-snapshotter.yaml (3149 bytes)

	I0307 13:49:38.843513    1772 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-attacher.yaml

	I0307 13:49:38.843525    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-attacher.yaml (2143 bytes)

	I0307 13:49:38.904184    1772 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml

	I0307 13:49:38.904197    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml (1274 bytes)

	I0307 13:49:38.993842    1772 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-plugin.yaml

	I0307 13:49:38.993852    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-plugin.yaml (8201 bytes)

	I0307 13:49:39.106181    1772 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-resizer.yaml

	I0307 13:49:39.106192    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-resizer.yaml (2191 bytes)

	I0307 13:49:39.119439    1772 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-storageclass.yaml

	I0307 13:49:39.119451    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-storageclass.yaml (846 bytes)

	I0307 13:49:39.126998    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml

	I0307 13:49:39.806826    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:49:39.893236    1772 out.go:177]   - Using image ghcr.io/inspektor-gadget/inspektor-gadget:v0.25.1

	I0307 13:49:39.899189    1772 addons.go:426] installing /etc/kubernetes/addons/ig-namespace.yaml

	I0307 13:49:39.899201    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-namespace.yaml (55 bytes)

	I0307 13:49:39.899212    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:40.041657    1772 addons.go:426] installing /etc/kubernetes/addons/ig-serviceaccount.yaml

	I0307 13:49:40.041669    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-serviceaccount.yaml (80 bytes)

	I0307 13:49:40.100285    1772 addons.go:426] installing /etc/kubernetes/addons/ig-role.yaml

	I0307 13:49:40.100297    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-role.yaml (210 bytes)

	I0307 13:49:40.158718    1772 addons.go:426] installing /etc/kubernetes/addons/ig-rolebinding.yaml

	I0307 13:49:40.158729    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-rolebinding.yaml (244 bytes)

	I0307 13:49:40.242814    1772 addons.go:426] installing /etc/kubernetes/addons/ig-clusterrole.yaml

	I0307 13:49:40.242826    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-clusterrole.yaml (1485 bytes)

	I0307 13:49:40.283371    1772 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/registry-rc.yaml -f /etc/kubernetes/addons/registry-svc.yaml -f /etc/kubernetes/addons/registry-proxy.yaml: (2.206475834s)

	I0307 13:49:40.283381    1772 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/deployment.yaml: (2.179686542s)

	I0307 13:49:40.283412    1772 addons.go:470] Verifying addon registry=true in "addons-263000"

	I0307 13:49:40.283419    1772 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (2.167185875s)

	I0307 13:49:40.283424    1772 addons.go:470] Verifying addon metrics-server=true in "addons-263000"

	I0307 13:49:40.287219    1772 out.go:177] * Verifying registry addon...

	I0307 13:49:40.283465    1772 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/nvidia-device-plugin.yaml: (2.164505333s)

	I0307 13:49:40.283511    1772 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: (2.03002525s)

	I0307 13:49:40.283523    1772 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/yakd-ns.yaml -f /etc/kubernetes/addons/yakd-sa.yaml -f /etc/kubernetes/addons/yakd-crb.yaml -f /etc/kubernetes/addons/yakd-svc.yaml -f /etc/kubernetes/addons/yakd-dp.yaml: (1.896860125s)

	I0307 13:49:40.284651    1772 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml: (2.313082958s)

	I0307 13:49:40.290577    1772 addons.go:426] installing /etc/kubernetes/addons/ig-clusterrolebinding.yaml

	I0307 13:49:40.295365    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-clusterrolebinding.yaml (274 bytes)

	W0307 13:49:40.295411    1772 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1

	stdout:

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created

	serviceaccount/snapshot-controller created

	clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created

	clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created

	role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	deployment.apps/snapshot-controller created

	stderr:

	error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"

	ensure CRDs are installed first

	I0307 13:49:40.295429    1772 retry.go:31] will retry after 309.817702ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1

	stdout:

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created

	serviceaccount/snapshot-controller created

	clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created

	clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created

	role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	deployment.apps/snapshot-controller created

	stderr:

	error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"

	ensure CRDs are installed first

	I0307 13:49:40.295432    1772 addons.go:470] Verifying addon ingress=true in "addons-263000"

	I0307 13:49:40.303257    1772 out.go:177] * Verifying ingress addon...

	I0307 13:49:40.295900    1772 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=registry" in ns "kube-system" ...

	I0307 13:49:40.306568    1772 kapi.go:75] Waiting for pod with label "app.kubernetes.io/name=ingress-nginx" in ns "ingress-nginx" ...

	I0307 13:49:40.310309    1772 out.go:177] * To access YAKD - Kubernetes Dashboard, wait for Pod to be ready and run the following command:

		minikube -p addons-263000 service yakd-dashboard -n yakd-dashboard

	I0307 13:49:40.315905    1772 kapi.go:86] Found 3 Pods for label selector app.kubernetes.io/name=ingress-nginx

	I0307 13:49:40.315917    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:40.321162    1772 kapi.go:86] Found 2 Pods for label selector kubernetes.io/minikube-addons=registry

	I0307 13:49:40.321170    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:40.332648    1772 addons.go:426] installing /etc/kubernetes/addons/ig-crd.yaml

	I0307 13:49:40.332660    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-crd.yaml (5216 bytes)

	I0307 13:49:40.348876    1772 addons.go:426] installing /etc/kubernetes/addons/ig-daemonset.yaml

	I0307 13:49:40.348885    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-daemonset.yaml (7735 bytes)

	I0307 13:49:40.370284    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/ig-namespace.yaml -f /etc/kubernetes/addons/ig-serviceaccount.yaml -f /etc/kubernetes/addons/ig-role.yaml -f /etc/kubernetes/addons/ig-rolebinding.yaml -f /etc/kubernetes/addons/ig-clusterrole.yaml -f /etc/kubernetes/addons/ig-clusterrolebinding.yaml -f /etc/kubernetes/addons/ig-crd.yaml -f /etc/kubernetes/addons/ig-daemonset.yaml

	I0307 13:49:40.607378    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply --force -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0307 13:49:40.818661    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:40.819560    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:41.121998    1772 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml: (1.995020917s)

	I0307 13:49:41.122018    1772 addons.go:470] Verifying addon csi-hostpath-driver=true in "addons-263000"

	I0307 13:49:41.131211    1772 out.go:177] * Verifying csi-hostpath-driver addon...

	I0307 13:49:41.137668    1772 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=csi-hostpath-driver" in ns "kube-system" ...

	I0307 13:49:41.148768    1772 kapi.go:86] Found 3 Pods for label selector kubernetes.io/minikube-addons=csi-hostpath-driver

	I0307 13:49:41.148776    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:41.312534    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:41.312957    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:41.642154    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:41.812578    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:41.813201    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:42.142331    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:42.308269    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:49:42.312780    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:42.315190    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:42.642798    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:42.812595    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:42.812781    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:43.142345    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:43.313580    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:43.313621    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:43.577170    1772 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_application_credentials.json (162 bytes)

	I0307 13:49:43.577186    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:43.608517    1772 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_cloud_project (12 bytes)

	I0307 13:49:43.617885    1772 addons.go:234] Setting addon gcp-auth=true in "addons-263000"

	I0307 13:49:43.617914    1772 host.go:66] Checking if "addons-263000" exists ...

	I0307 13:49:43.618747    1772 ssh_runner.go:195] Run: cat /var/lib/minikube/google_application_credentials.json

	I0307 13:49:43.618755    1772 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/addons-263000/id_rsa Username:docker}

	I0307 13:49:43.641288    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:43.652124    1772 out.go:177]   - Using image gcr.io/k8s-minikube/gcp-auth-webhook:v0.1.1

	I0307 13:49:43.656170    1772 out.go:177]   - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0

	I0307 13:49:43.660102    1772 addons.go:426] installing /etc/kubernetes/addons/gcp-auth-ns.yaml

	I0307 13:49:43.660108    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-ns.yaml (700 bytes)

	I0307 13:49:43.666624    1772 addons.go:426] installing /etc/kubernetes/addons/gcp-auth-service.yaml

	I0307 13:49:43.666631    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-service.yaml (788 bytes)

	I0307 13:49:43.673345    1772 addons.go:426] installing /etc/kubernetes/addons/gcp-auth-webhook.yaml

	I0307 13:49:43.673352    1772 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-webhook.yaml (5421 bytes)

	I0307 13:49:43.680750    1772 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/gcp-auth-ns.yaml -f /etc/kubernetes/addons/gcp-auth-service.yaml -f /etc/kubernetes/addons/gcp-auth-webhook.yaml

	I0307 13:49:43.837958    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:43.838068    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:44.142262    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:44.184595    1772 addons.go:470] Verifying addon gcp-auth=true in "addons-263000"

	I0307 13:49:44.188723    1772 out.go:177] * Verifying gcp-auth addon...

	I0307 13:49:44.196073    1772 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=gcp-auth" in ns "gcp-auth" ...

	I0307 13:49:44.206928    1772 kapi.go:86] Found 1 Pods for label selector kubernetes.io/minikube-addons=gcp-auth

	I0307 13:49:44.206938    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:44.312606    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:44.312832    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:44.644418    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:44.700039    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:44.807951    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:49:44.812887    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:44.812889    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:45.142745    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:45.199790    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:45.313548    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:45.313780    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:45.643517    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:45.697971    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:45.813294    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:45.813421    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:46.142458    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:46.199404    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:46.312648    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:46.312661    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:46.640524    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:46.698510    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:46.812546    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:46.812668    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:47.142197    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:47.199421    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:47.307535    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:49:47.312537    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:47.313038    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:47.642542    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:47.700024    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:47.813169    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:47.813236    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:48.142447    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:48.199738    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:48.312626    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:48.313184    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:48.642435    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:48.699962    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:48.813339    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:48.813482    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:49.142530    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:49.199731    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:49.307954    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:49:49.312696    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:49.313328    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:49.643601    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:49.699625    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:49.812836    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:49.813120    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:50.142616    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:50.199973    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:50.312511    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:50.312561    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:50.642335    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:50.699346    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:50.813819    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:50.816055    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:51.144310    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:51.200914    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:51.313000    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:51.313336    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:51.642586    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:51.698870    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:51.807645    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:49:51.813633    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:51.813688    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:52.141868    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:52.200500    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:52.313334    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:52.314010    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:52.643052    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:52.700179    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:52.815489    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:52.815506    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:53.144042    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:53.198681    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:53.313519    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:53.313667    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:53.642423    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:53.699937    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:53.807908    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:49:53.812552    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:53.812821    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:54.142340    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:54.199864    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:54.312905    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:54.314025    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:54.642411    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:54.699802    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:54.812603    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:54.812678    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:55.142024    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:55.197868    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:55.312833    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:55.312838    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:55.642781    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:55.699465    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:55.812954    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:55.813109    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:56.142220    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:56.199251    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:56.307818    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:49:56.312615    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:56.312750    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:56.641389    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:56.698572    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:56.812913    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:56.813029    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:57.144512    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:57.199805    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:57.313162    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:57.313285    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:57.642404    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:57.699328    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:57.812215    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:57.812280    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:58.141875    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:58.199268    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:58.312297    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:58.312508    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:58.642075    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:58.699339    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:58.807494    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:49:58.812246    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:58.812558    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:59.142258    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:59.199196    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:59.312246    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:59.312356    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:49:59.642145    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:49:59.699330    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:49:59.812219    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:49:59.812578    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:00.142140    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:00.199466    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:00.312380    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:00.312380    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:00.642086    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:00.699270    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:00.812300    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:00.812516    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:01.142103    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:01.199034    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:01.307764    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:50:01.312047    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:01.312743    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:01.642527    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:01.697938    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:01.812212    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:01.812763    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:02.141768    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:02.197190    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:02.312193    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:02.312513    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:02.641942    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:02.699154    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:02.812668    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:02.812861    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:03.142415    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:03.199296    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:03.307754    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:50:03.312093    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:03.312880    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:03.642257    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:03.698407    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:03.812888    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:03.813033    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:04.142204    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:04.198980    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:04.312000    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:04.312625    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:04.640981    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:04.699085    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:04.812334    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:04.812338    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:05.142171    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:05.199294    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:05.312137    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:05.312155    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:05.641963    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:05.699054    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:05.807324    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:50:05.812119    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:05.812259    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:06.140721    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:06.199070    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:06.312847    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:06.313146    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:06.641910    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:06.697992    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:06.812912    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:06.813003    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:07.141710    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:07.199012    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:07.312920    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:07.313176    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:07.642590    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:07.700052    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:07.807641    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:50:07.812663    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:07.812790    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:08.142013    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:08.199440    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:08.312314    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:08.312483    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:08.641917    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:08.699251    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:08.812152    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:08.812225    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:09.142160    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:09.199073    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:09.311812    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:09.312291    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:09.642095    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:09.699045    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:09.811838    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:09.812426    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:10.141251    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:10.199528    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:10.307171    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:50:10.312275    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:10.312930    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:10.641606    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:10.698887    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:10.811927    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:10.811964    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:11.143815    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:11.199096    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:11.314159    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:11.314368    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:11.642276    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:11.697630    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:11.812191    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:11.813143    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:12.141630    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:12.197301    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:12.307384    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:50:12.311862    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:12.312902    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:12.641934    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:12.699448    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:12.812134    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:12.812167    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:13.141883    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:13.199544    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:13.311842    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:13.312039    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:13.641865    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:13.699504    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:13.812170    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:13.812639    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:14.142279    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:14.198967    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:14.308093    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:50:14.312019    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:14.312143    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:14.641919    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:14.699031    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:14.811648    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:14.812221    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:15.143518    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:15.199265    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:15.313979    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:15.314146    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:15.641879    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:15.698938    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:15.813242    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:15.813381    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:16.141809    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:16.199029    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:16.309335    1772 pod_ready.go:102] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"False"

	I0307 13:50:16.312021    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:16.312210    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:16.641606    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:16.698181    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:16.812909    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:16.813110    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:17.141704    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:17.198866    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:17.306978    1772 pod_ready.go:92] pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace has status "Ready":"True"

	I0307 13:50:17.306990    1772 pod_ready.go:81] duration metric: took 39.504705709s for pod "coredns-5dd5756b68-j8npl" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.306995    1772 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5dd5756b68-w7qrt" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.307873    1772 pod_ready.go:97] error getting pod "coredns-5dd5756b68-w7qrt" in "kube-system" namespace (skipping!): pods "coredns-5dd5756b68-w7qrt" not found

	I0307 13:50:17.307879    1772 pod_ready.go:81] duration metric: took 881.458µs for pod "coredns-5dd5756b68-w7qrt" in "kube-system" namespace to be "Ready" ...

	E0307 13:50:17.307883    1772 pod_ready.go:66] WaitExtra: waitPodCondition: error getting pod "coredns-5dd5756b68-w7qrt" in "kube-system" namespace (skipping!): pods "coredns-5dd5756b68-w7qrt" not found

	I0307 13:50:17.307886    1772 pod_ready.go:78] waiting up to 6m0s for pod "etcd-addons-263000" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.310432    1772 pod_ready.go:92] pod "etcd-addons-263000" in "kube-system" namespace has status "Ready":"True"

	I0307 13:50:17.310440    1772 pod_ready.go:81] duration metric: took 2.552083ms for pod "etcd-addons-263000" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.310445    1772 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-addons-263000" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.312837    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:17.313018    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:17.313855    1772 pod_ready.go:92] pod "kube-apiserver-addons-263000" in "kube-system" namespace has status "Ready":"True"

	I0307 13:50:17.313862    1772 pod_ready.go:81] duration metric: took 3.414417ms for pod "kube-apiserver-addons-263000" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.313866    1772 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-addons-263000" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.316076    1772 pod_ready.go:92] pod "kube-controller-manager-addons-263000" in "kube-system" namespace has status "Ready":"True"

	I0307 13:50:17.316081    1772 pod_ready.go:81] duration metric: took 2.212208ms for pod "kube-controller-manager-addons-263000" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.316086    1772 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-9p6xr" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.507976    1772 pod_ready.go:92] pod "kube-proxy-9p6xr" in "kube-system" namespace has status "Ready":"True"

	I0307 13:50:17.507985    1772 pod_ready.go:81] duration metric: took 191.899625ms for pod "kube-proxy-9p6xr" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.507990    1772 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-addons-263000" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.641655    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:17.698560    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:17.814050    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:17.814130    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:17.907843    1772 pod_ready.go:92] pod "kube-scheduler-addons-263000" in "kube-system" namespace has status "Ready":"True"

	I0307 13:50:17.907849    1772 pod_ready.go:81] duration metric: took 399.864167ms for pod "kube-scheduler-addons-263000" in "kube-system" namespace to be "Ready" ...

	I0307 13:50:17.907852    1772 pod_ready.go:38] duration metric: took 40.117920583s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I0307 13:50:17.907862    1772 api_server.go:52] waiting for apiserver process to appear ...

	I0307 13:50:17.907924    1772 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I0307 13:50:17.914755    1772 api_server.go:72] duration metric: took 40.962525666s to wait for apiserver process to appear ...

	I0307 13:50:17.914772    1772 api_server.go:88] waiting for apiserver healthz status ...

	I0307 13:50:17.914780    1772 api_server.go:253] Checking apiserver healthz at https://192.168.105.2:8443/healthz ...

	I0307 13:50:17.918032    1772 api_server.go:279] https://192.168.105.2:8443/healthz returned 200:

	ok

	I0307 13:50:17.918789    1772 api_server.go:141] control plane version: v1.28.4

	I0307 13:50:17.918797    1772 api_server.go:131] duration metric: took 4.0225ms to wait for apiserver health ...

	I0307 13:50:17.918800    1772 system_pods.go:43] waiting for kube-system pods to appear ...

	I0307 13:50:18.113392    1772 system_pods.go:59] 17 kube-system pods found

	I0307 13:50:18.113402    1772 system_pods.go:61] "coredns-5dd5756b68-j8npl" [37107ddd-61d6-437b-a460-6cf9f1956154] Running

	I0307 13:50:18.113406    1772 system_pods.go:61] "csi-hostpath-attacher-0" [8b16e86b-465e-4265-a784-fedb7b43a8ba] Pending / Ready:ContainersNotReady (containers with unready status: [csi-attacher]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-attacher])

	I0307 13:50:18.113409    1772 system_pods.go:61] "csi-hostpath-resizer-0" [b2b78271-c060-45f0-8d99-86d93c16eef1] Pending / Ready:ContainersNotReady (containers with unready status: [csi-resizer]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-resizer])

	I0307 13:50:18.113412    1772 system_pods.go:61] "csi-hostpathplugin-snl6t" [77d28788-0230-4ee2-b860-cc6a2c27b3fa] Pending / Ready:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter])

	I0307 13:50:18.113414    1772 system_pods.go:61] "etcd-addons-263000" [1f4daba8-05c0-4b26-9a93-eb1ad00b4998] Running

	I0307 13:50:18.113417    1772 system_pods.go:61] "kube-apiserver-addons-263000" [a4879d19-764e-46be-8660-751c0baffdf2] Running

	I0307 13:50:18.113419    1772 system_pods.go:61] "kube-controller-manager-addons-263000" [4a0a5f02-a514-4269-8f9e-f4db0899f008] Running

	I0307 13:50:18.113421    1772 system_pods.go:61] "kube-ingress-dns-minikube" [2b265f89-7503-4f45-b8e5-edaee7812238] Running / Ready:ContainersNotReady (containers with unready status: [minikube-ingress-dns]) / ContainersReady:ContainersNotReady (containers with unready status: [minikube-ingress-dns])

	I0307 13:50:18.113423    1772 system_pods.go:61] "kube-proxy-9p6xr" [ffe6c954-bd6f-464d-a2b0-f679e2a1b0ba] Running

	I0307 13:50:18.113425    1772 system_pods.go:61] "kube-scheduler-addons-263000" [3c222db7-df35-4000-b35e-ef7c75f5b1e3] Running

	I0307 13:50:18.113427    1772 system_pods.go:61] "metrics-server-69cf46c98-bq5l7" [3cbd03d0-e96e-4a05-b38e-f0a9aa3b5fb7] Running

	I0307 13:50:18.113428    1772 system_pods.go:61] "nvidia-device-plugin-daemonset-lkgxm" [f2dc7d70-d24c-4ff8-b079-b1dd168beae6] Running

	I0307 13:50:18.113431    1772 system_pods.go:61] "registry-proxy-rcw2g" [631fd6f8-a701-47ff-ba2e-767f14a10490] Pending / Ready:ContainersNotReady (containers with unready status: [registry-proxy]) / ContainersReady:ContainersNotReady (containers with unready status: [registry-proxy])

	I0307 13:50:18.113433    1772 system_pods.go:61] "registry-zp2kw" [23c4da1e-1600-4bf6-bd46-b55a29d69d26] Running

	I0307 13:50:18.113436    1772 system_pods.go:61] "snapshot-controller-58dbcc7b99-629zs" [7d07b993-0203-4c69-be76-9b1a145a5731] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0307 13:50:18.113439    1772 system_pods.go:61] "snapshot-controller-58dbcc7b99-rzhl4" [6e97847d-f38a-4e7c-bf2e-b8c0cf07d5cd] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0307 13:50:18.113441    1772 system_pods.go:61] "storage-provisioner" [e1a380a1-1895-4a32-b3c4-f2c8d3dc2a88] Running

	I0307 13:50:18.113444    1772 system_pods.go:74] duration metric: took 194.644625ms to wait for pod list to return data ...

	I0307 13:50:18.113448    1772 default_sa.go:34] waiting for default service account to be created ...

	I0307 13:50:18.141342    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:18.197535    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:18.305819    1772 default_sa.go:45] found service account: "default"

	I0307 13:50:18.305830    1772 default_sa.go:55] duration metric: took 192.383ms for default service account to be created ...

	I0307 13:50:18.305834    1772 system_pods.go:116] waiting for k8s-apps to be running ...

	I0307 13:50:18.312624    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:18.313377    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:18.515954    1772 system_pods.go:86] 17 kube-system pods found

	I0307 13:50:18.515965    1772 system_pods.go:89] "coredns-5dd5756b68-j8npl" [37107ddd-61d6-437b-a460-6cf9f1956154] Running

	I0307 13:50:18.515970    1772 system_pods.go:89] "csi-hostpath-attacher-0" [8b16e86b-465e-4265-a784-fedb7b43a8ba] Pending / Ready:ContainersNotReady (containers with unready status: [csi-attacher]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-attacher])

	I0307 13:50:18.515973    1772 system_pods.go:89] "csi-hostpath-resizer-0" [b2b78271-c060-45f0-8d99-86d93c16eef1] Pending / Ready:ContainersNotReady (containers with unready status: [csi-resizer]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-resizer])

	I0307 13:50:18.515976    1772 system_pods.go:89] "csi-hostpathplugin-snl6t" [77d28788-0230-4ee2-b860-cc6a2c27b3fa] Pending / Ready:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter])

	I0307 13:50:18.515979    1772 system_pods.go:89] "etcd-addons-263000" [1f4daba8-05c0-4b26-9a93-eb1ad00b4998] Running

	I0307 13:50:18.515982    1772 system_pods.go:89] "kube-apiserver-addons-263000" [a4879d19-764e-46be-8660-751c0baffdf2] Running

	I0307 13:50:18.515984    1772 system_pods.go:89] "kube-controller-manager-addons-263000" [4a0a5f02-a514-4269-8f9e-f4db0899f008] Running

	I0307 13:50:18.515986    1772 system_pods.go:89] "kube-ingress-dns-minikube" [2b265f89-7503-4f45-b8e5-edaee7812238] Running / Ready:ContainersNotReady (containers with unready status: [minikube-ingress-dns]) / ContainersReady:ContainersNotReady (containers with unready status: [minikube-ingress-dns])

	I0307 13:50:18.515988    1772 system_pods.go:89] "kube-proxy-9p6xr" [ffe6c954-bd6f-464d-a2b0-f679e2a1b0ba] Running

	I0307 13:50:18.515990    1772 system_pods.go:89] "kube-scheduler-addons-263000" [3c222db7-df35-4000-b35e-ef7c75f5b1e3] Running

	I0307 13:50:18.515991    1772 system_pods.go:89] "metrics-server-69cf46c98-bq5l7" [3cbd03d0-e96e-4a05-b38e-f0a9aa3b5fb7] Running

	I0307 13:50:18.515993    1772 system_pods.go:89] "nvidia-device-plugin-daemonset-lkgxm" [f2dc7d70-d24c-4ff8-b079-b1dd168beae6] Running

	I0307 13:50:18.515995    1772 system_pods.go:89] "registry-proxy-rcw2g" [631fd6f8-a701-47ff-ba2e-767f14a10490] Pending / Ready:ContainersNotReady (containers with unready status: [registry-proxy]) / ContainersReady:ContainersNotReady (containers with unready status: [registry-proxy])

	I0307 13:50:18.516008    1772 system_pods.go:89] "registry-zp2kw" [23c4da1e-1600-4bf6-bd46-b55a29d69d26] Running

	I0307 13:50:18.516012    1772 system_pods.go:89] "snapshot-controller-58dbcc7b99-629zs" [7d07b993-0203-4c69-be76-9b1a145a5731] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0307 13:50:18.516016    1772 system_pods.go:89] "snapshot-controller-58dbcc7b99-rzhl4" [6e97847d-f38a-4e7c-bf2e-b8c0cf07d5cd] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0307 13:50:18.516018    1772 system_pods.go:89] "storage-provisioner" [e1a380a1-1895-4a32-b3c4-f2c8d3dc2a88] Running

	I0307 13:50:18.516022    1772 system_pods.go:126] duration metric: took 210.190041ms to wait for k8s-apps to be running ...

	I0307 13:50:18.516026    1772 system_svc.go:44] waiting for kubelet service to be running ....

	I0307 13:50:18.516093    1772 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I0307 13:50:18.522749    1772 system_svc.go:56] duration metric: took 6.721917ms WaitForService to wait for kubelet

	I0307 13:50:18.522756    1772 kubeadm.go:576] duration metric: took 41.570541291s to wait for: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0307 13:50:18.522765    1772 node_conditions.go:102] verifying NodePressure condition ...

	I0307 13:50:18.641817    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:18.698877    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:18.707058    1772 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki

	I0307 13:50:18.707066    1772 node_conditions.go:123] node cpu capacity is 2

	I0307 13:50:18.707083    1772 node_conditions.go:105] duration metric: took 184.318083ms to run NodePressure ...

	I0307 13:50:18.707092    1772 start.go:240] waiting for startup goroutines ...

	I0307 13:50:18.813313    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:18.813417    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:19.141828    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:19.198880    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:19.313476    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:19.313548    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:19.640890    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:19.698825    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:19.813786    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:19.813941    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:20.140336    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:20.198993    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:20.313262    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:20.313973    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:20.641520    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:20.699048    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:20.813625    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:20.813743    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:21.141660    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:21.199043    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:21.314489    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:21.314576    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:21.641907    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:21.698138    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:21.815758    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:21.815914    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:22.141546    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:22.199049    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:22.314083    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:22.314264    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:22.641409    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:22.699049    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:22.814348    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:22.814548    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:23.141541    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:23.199262    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:23.313558    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:23.313613    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:23.641730    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:23.698968    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:23.814494    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:23.814499    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:24.210231    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:24.211383    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:24.313884    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0307 13:50:24.314028    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:24.642473    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:24.696800    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:24.813450    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:24.813892    1772 kapi.go:107] duration metric: took 44.518987333s to wait for kubernetes.io/minikube-addons=registry ...

	I0307 13:50:25.141451    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:25.198461    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:25.313569    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:25.641831    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:25.699031    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:25.813581    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:26.141454    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:26.199084    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:26.313817    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:26.641816    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:26.697703    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:26.813717    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:27.141397    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:27.198855    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:27.394017    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:27.641711    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:27.698899    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:27.813517    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:28.140820    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:28.198482    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:28.313336    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:28.641760    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:28.698509    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:28.813139    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:29.141546    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:29.198543    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:29.313284    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:29.641860    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:29.698521    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:29.813261    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:30.141046    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:30.198507    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:30.313128    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:30.641829    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:30.698575    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:30.813334    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:31.141389    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:31.199011    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:31.313442    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:31.641325    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:31.697220    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:31.813021    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:32.141691    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:32.198789    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:32.312810    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:32.641191    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:32.698665    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:32.813069    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:33.141176    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:33.198297    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:33.312422    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:33.641665    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:33.698464    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:33.812974    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:34.142169    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:34.197715    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:34.313075    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:34.643261    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:34.698643    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:34.813216    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:35.141505    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:35.198273    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:35.313066    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:35.640636    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:35.698598    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:35.812482    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:36.141781    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:36.198460    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:36.313025    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:36.641145    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:36.697622    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:36.813240    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:37.141089    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:37.198309    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:37.313206    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:37.641387    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:37.698917    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:37.813613    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:38.141325    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:38.199397    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:38.311949    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:38.641292    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:38.698487    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:38.813044    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:39.141622    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:39.198552    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:39.313299    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:39.641167    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:39.698200    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:39.812917    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:40.141287    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:40.198260    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:40.312990    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:40.640606    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:40.698366    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:40.812915    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:41.141020    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:41.198562    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:41.313255    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:41.641007    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:41.697486    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:41.812794    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:42.141012    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:42.198718    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:42.313367    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:42.641665    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:42.698583    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:42.812963    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:43.141479    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:43.198327    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:43.313202    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:43.640293    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:43.698385    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:43.812894    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:44.140662    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:44.198355    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:44.313670    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:44.641237    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:44.698187    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:44.812982    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:45.140085    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:45.198246    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:45.313158    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:45.641200    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:45.698396    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:45.813008    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:46.141209    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:46.198442    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:46.310851    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:46.641664    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:46.696865    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:46.813291    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:47.141206    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:47.198713    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:47.311352    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:47.640902    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:47.698059    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:47.812986    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:48.145629    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:48.198354    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:48.313010    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:48.641127    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:48.698358    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:48.812886    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:49.139233    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:49.198253    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:49.313371    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:49.641124    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:49.698327    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:49.812719    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:50.139541    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:50.198261    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:50.313184    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:50.641732    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:50.697993    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:50.812924    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:51.141212    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:51.198524    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:51.312606    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:51.640342    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:51.697408    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:51.813060    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:52.140573    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:52.198476    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:52.313170    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:52.642538    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:52.698282    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:52.813086    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:53.141014    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:53.198324    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:53.313131    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:53.640955    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:53.698254    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:53.812984    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:54.141279    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:54.198532    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:54.313198    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:54.640905    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:54.699822    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:54.813148    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:55.141015    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:55.198182    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:55.312712    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:55.640671    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:55.698073    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:55.812883    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:56.141329    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:56.200874    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:56.312702    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:56.640787    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:56.696937    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:56.812505    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:57.141297    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:57.197737    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:57.312938    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:57.640857    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:57.697945    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:57.812938    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:58.140581    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:58.197850    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:58.312796    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:58.641290    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:58.698060    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:58.812864    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:59.140795    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:59.198177    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:59.313955    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:50:59.641301    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:50:59.698944    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:50:59.812876    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:00.140693    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:00.197874    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:00.312594    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:00.641328    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:00.698266    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:00.812789    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:01.141191    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:01.198252    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:01.312546    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:01.640865    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:01.697149    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:01.812634    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:02.142211    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:02.198013    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:02.312583    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:02.640684    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:02.697953    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:02.813145    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:03.140478    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:03.198306    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:03.312578    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:03.641511    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:03.698018    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:03.812585    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:04.141278    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:04.198156    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:04.312654    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:04.640453    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:04.697921    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:04.812630    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:05.140283    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:05.198019    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:05.313136    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:05.640650    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:05.698258    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:05.813268    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:06.140558    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:06.198157    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:06.312713    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:06.640652    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:06.697279    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:06.812764    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:07.141040    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:07.327312    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:07.327551    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:07.641887    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:07.698250    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:07.812468    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:08.140741    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:08.198097    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:08.312753    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:08.640616    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:08.697964    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:08.814076    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:09.140546    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:09.197837    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:09.312393    1772 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0307 13:51:09.640522    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:09.697700    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:09.810938    1772 kapi.go:107] duration metric: took 1m29.506368958s to wait for app.kubernetes.io/name=ingress-nginx ...

	I0307 13:51:10.140254    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:10.198331    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:10.640887    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:10.697530    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:11.141025    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:11.198004    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:11.640680    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:11.696585    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:12.140701    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:12.196769    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:12.640223    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:12.698580    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:13.140593    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:13.196952    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:13.639225    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:13.697552    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:14.141751    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:14.198031    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:14.638965    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:14.697507    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:15.140866    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:15.197631    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:15.642067    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:15.697519    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:16.140659    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:16.197683    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:16.640429    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:16.696731    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:17.140308    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:17.197624    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:17.640234    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:17.697718    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:18.139998    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:18.196570    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:18.639985    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:18.697757    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:19.140678    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:19.197371    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:19.640422    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:19.697447    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:20.140434    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:20.197436    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:20.640200    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:20.697659    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:21.140347    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:21.197393    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:21.640390    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:21.696115    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:22.140063    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:22.197475    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:22.640318    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:22.697321    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:23.140622    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:23.195506    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:23.640403    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:23.697268    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:24.140335    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:24.197459    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:24.639985    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:24.697352    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:25.140244    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:25.197348    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:25.640335    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:25.697593    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:26.140275    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:26.199076    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:26.639854    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:26.696316    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:27.140250    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:27.198093    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:27.640071    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:27.697601    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:28.140547    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:28.197299    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:28.640753    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:28.697171    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:29.140389    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:29.197351    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:29.639998    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:29.697315    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:30.140202    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:30.197370    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:30.640192    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0307 13:51:30.697615    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:31.140138    1772 kapi.go:107] duration metric: took 1m50.004925s to wait for kubernetes.io/minikube-addons=csi-hostpath-driver ...

	I0307 13:51:31.197313    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:31.696650    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:32.197410    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:32.697692    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:33.197554    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:33.697477    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:34.197631    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:34.697758    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:35.197500    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:35.697547    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:36.197587    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:36.696416    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:37.198123    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:37.697180    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:38.196931    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:38.697359    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:39.197553    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:39.697244    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:40.197059    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:40.697216    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:41.197487    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:41.696133    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:42.197231    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:42.697069    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:43.197391    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:43.697047    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:44.197377    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:44.697841    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:45.197479    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:45.697283    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:46.197418    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:46.695800    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:47.197367    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:47.696986    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:48.197216    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:48.697317    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:49.196438    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:49.697435    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:50.197133    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:50.696095    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:51.197381    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:51.695519    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:52.197073    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:52.696793    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:53.197041    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:53.697053    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:54.197225    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:54.696937    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:55.196379    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:55.696950    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:56.196760    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:56.695519    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:57.197145    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:57.697267    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:58.197086    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:58.697047    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:59.197261    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:51:59.696913    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:00.198027    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:00.696933    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:01.196982    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:01.695873    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:02.196924    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:02.696571    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:03.196877    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:03.697028    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:04.196702    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:04.696952    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:05.196734    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:05.696877    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:06.196952    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:06.695248    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:07.196720    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:07.696577    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:08.196774    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:08.697069    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:09.196795    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:09.696700    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:10.196761    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:10.696766    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:11.196482    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:11.695181    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:12.196812    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:12.696712    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:13.196894    1772 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0307 13:52:13.696418    1772 kapi.go:107] duration metric: took 2m29.503683666s to wait for kubernetes.io/minikube-addons=gcp-auth ...

	I0307 13:52:13.700309    1772 out.go:177] * Your GCP credentials will now be mounted into every pod created in the addons-263000 cluster.

	I0307 13:52:13.703514    1772 out.go:177] * If you don't want your credentials mounted into a specific pod, add a label with the `gcp-auth-skip-secret` key to your pod configuration.

	I0307 13:52:13.707549    1772 out.go:177] * If you want existing pods to be mounted with credentials, either recreate them or rerun addons enable with --refresh.

	I0307 13:52:13.711554    1772 out.go:177] * Enabled addons: ingress-dns, storage-provisioner-rancher, storage-provisioner, default-storageclass, cloud-spanner, metrics-server, nvidia-device-plugin, yakd, inspektor-gadget, volumesnapshots, registry, ingress, csi-hostpath-driver, gcp-auth

	I0307 13:52:13.715490    1772 addons.go:505] duration metric: took 2m36.765832s for enable addons: enabled=[ingress-dns storage-provisioner-rancher storage-provisioner default-storageclass cloud-spanner metrics-server nvidia-device-plugin yakd inspektor-gadget volumesnapshots registry ingress csi-hostpath-driver gcp-auth]

	I0307 13:52:13.715504    1772 start.go:245] waiting for cluster config update ...

	I0307 13:52:13.715524    1772 start.go:254] writing updated cluster config ...

	I0307 13:52:13.715903    1772 ssh_runner.go:195] Run: rm -f paused

	I0307 13:52:13.865016    1772 start.go:600] kubectl: 1.29.1, cluster: 1.28.4 (minor skew: 1)

	I0307 13:52:13.866718    1772 out.go:177] * Done! kubectl is now configured to use "addons-263000" cluster and "default" namespace by default

	==> Docker <==

	Mar 07 21:53:45 addons-263000 dockerd[1106]: time="2024-03-07T21:53:45.509516251Z" level=info msg="ignoring event" container=bad4caf1a1284e8d7ae753d52f99f3fdbce168352d03b484a7d82152191b81a3 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Mar 07 21:53:45 addons-263000 dockerd[1112]: time="2024-03-07T21:53:45.509613400Z" level=info msg="shim disconnected" id=bad4caf1a1284e8d7ae753d52f99f3fdbce168352d03b484a7d82152191b81a3 namespace=moby

	Mar 07 21:53:45 addons-263000 dockerd[1112]: time="2024-03-07T21:53:45.509653868Z" level=warning msg="cleaning up after shim disconnected" id=bad4caf1a1284e8d7ae753d52f99f3fdbce168352d03b484a7d82152191b81a3 namespace=moby

	Mar 07 21:53:45 addons-263000 dockerd[1112]: time="2024-03-07T21:53:45.509658161Z" level=info msg="cleaning up dead shim" namespace=moby

	Mar 07 21:53:46 addons-263000 dockerd[1112]: time="2024-03-07T21:53:46.314312360Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Mar 07 21:53:46 addons-263000 dockerd[1112]: time="2024-03-07T21:53:46.314343326Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Mar 07 21:53:46 addons-263000 dockerd[1112]: time="2024-03-07T21:53:46.314363123Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Mar 07 21:53:46 addons-263000 dockerd[1112]: time="2024-03-07T21:53:46.314389962Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Mar 07 21:53:46 addons-263000 dockerd[1106]: time="2024-03-07T21:53:46.340590570Z" level=info msg="ignoring event" container=2cf20bdb1244ab4fec99a3d7045d6d2f55635c9ff7ff058f9c4e1355ec520b92 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Mar 07 21:53:46 addons-263000 dockerd[1112]: time="2024-03-07T21:53:46.340716809Z" level=info msg="shim disconnected" id=2cf20bdb1244ab4fec99a3d7045d6d2f55635c9ff7ff058f9c4e1355ec520b92 namespace=moby

	Mar 07 21:53:46 addons-263000 dockerd[1112]: time="2024-03-07T21:53:46.340749775Z" level=warning msg="cleaning up after shim disconnected" id=2cf20bdb1244ab4fec99a3d7045d6d2f55635c9ff7ff058f9c4e1355ec520b92 namespace=moby

	Mar 07 21:53:46 addons-263000 dockerd[1112]: time="2024-03-07T21:53:46.340754026Z" level=info msg="cleaning up dead shim" namespace=moby

	Mar 07 21:53:48 addons-263000 dockerd[1106]: time="2024-03-07T21:53:48.769840295Z" level=info msg="ignoring event" container=1fc08b479096239ce3bf7f45dee743c13672f31cc2ccfad6073bf57d57c71179 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Mar 07 21:53:48 addons-263000 dockerd[1112]: time="2024-03-07T21:53:48.769938111Z" level=info msg="shim disconnected" id=1fc08b479096239ce3bf7f45dee743c13672f31cc2ccfad6073bf57d57c71179 namespace=moby

	Mar 07 21:53:48 addons-263000 dockerd[1112]: time="2024-03-07T21:53:48.769969285Z" level=warning msg="cleaning up after shim disconnected" id=1fc08b479096239ce3bf7f45dee743c13672f31cc2ccfad6073bf57d57c71179 namespace=moby

	Mar 07 21:53:48 addons-263000 dockerd[1112]: time="2024-03-07T21:53:48.769973161Z" level=info msg="cleaning up dead shim" namespace=moby

	Mar 07 21:53:52 addons-263000 dockerd[1106]: time="2024-03-07T21:53:52.549940342Z" level=info msg="Container failed to exit within 2s of signal 15 - using the force" container=e498100ade0bcb9163922886c2a68e33e8da7627588fdeb9fb0667dd2055d881

	Mar 07 21:53:52 addons-263000 dockerd[1112]: time="2024-03-07T21:53:52.590504095Z" level=info msg="shim disconnected" id=e498100ade0bcb9163922886c2a68e33e8da7627588fdeb9fb0667dd2055d881 namespace=moby

	Mar 07 21:53:52 addons-263000 dockerd[1106]: time="2024-03-07T21:53:52.590696100Z" level=info msg="ignoring event" container=e498100ade0bcb9163922886c2a68e33e8da7627588fdeb9fb0667dd2055d881 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Mar 07 21:53:52 addons-263000 dockerd[1112]: time="2024-03-07T21:53:52.590758156Z" level=warning msg="cleaning up after shim disconnected" id=e498100ade0bcb9163922886c2a68e33e8da7627588fdeb9fb0667dd2055d881 namespace=moby

	Mar 07 21:53:52 addons-263000 dockerd[1112]: time="2024-03-07T21:53:52.590769617Z" level=info msg="cleaning up dead shim" namespace=moby

	Mar 07 21:53:52 addons-263000 dockerd[1112]: time="2024-03-07T21:53:52.674621986Z" level=info msg="shim disconnected" id=b24690213955db316e851f961e70b9e36f39c3ced9fc2859d60d9a4f5ac6fee1 namespace=moby

	Mar 07 21:53:52 addons-263000 dockerd[1112]: time="2024-03-07T21:53:52.674651327Z" level=warning msg="cleaning up after shim disconnected" id=b24690213955db316e851f961e70b9e36f39c3ced9fc2859d60d9a4f5ac6fee1 namespace=moby

	Mar 07 21:53:52 addons-263000 dockerd[1112]: time="2024-03-07T21:53:52.674655786Z" level=info msg="cleaning up dead shim" namespace=moby

	Mar 07 21:53:52 addons-263000 dockerd[1106]: time="2024-03-07T21:53:52.674742974Z" level=info msg="ignoring event" container=b24690213955db316e851f961e70b9e36f39c3ced9fc2859d60d9a4f5ac6fee1 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	==> container status <==

	CONTAINER           IMAGE                                                                                                                        CREATED              STATE               NAME                      ATTEMPT             POD ID              POD

	2cf20bdb1244a       dd1b12fcb6097                                                                                                                10 seconds ago       Exited              hello-world-app           1                   2c525b2638653       hello-world-app-5d77478584-4zz98

	37688c6408255       nginx@sha256:6a2f8b28e45c4adea04ec207a251fd4a2df03ddc930f782af51e315ebc76e9a9                                                30 seconds ago       Running             nginx                     0                   0d825bf869743       nginx

	5c1f143322ea5       ghcr.io/headlamp-k8s/headlamp@sha256:94e00732e1b43057a9135dafc7483781aea4a73a26cec449ed19f4d8794308d5                        48 seconds ago       Running             headlamp                  0                   808949d779918       headlamp-5485c556b-v68g9

	5b4739a5a4674       gcr.io/k8s-minikube/gcp-auth-webhook@sha256:01b0de782aa30e7fc91ac5a91b5cc35e95e9679dee7ef07af06457b471f88f32                 About a minute ago   Running             gcp-auth                  0                   3bae46834bde9       gcp-auth-5f6b4f85fd-vfhrm

	d815d0b9caebc       1a024e390dd05                                                                                                                3 minutes ago        Exited              patch                     1                   69ea3c9aec884       ingress-nginx-admission-patch-ww9gk

	1446a72351f33       registry.k8s.io/ingress-nginx/kube-webhook-certgen@sha256:44d1d0e9f19c63f58b380c5fddaca7cf22c7cee564adeff365225a5df5ef3334   3 minutes ago        Exited              create                    0                   1935d09651253       ingress-nginx-admission-create-rzbv2

	8fcbeb5b7275a       marcnuri/yakd@sha256:a3f540278e4c11373e15605311851dd9c64d208f4d63e727bccc0e39f9329310                                        3 minutes ago        Running             yakd                      0                   b1764ef93bd63       yakd-dashboard-9947fc6bf-f6gpj

	6715bc4652b84       ba04bb24b9575                                                                                                                4 minutes ago        Running             storage-provisioner       0                   35af17894dade       storage-provisioner

	9a0dc72a6e29c       97e04611ad434                                                                                                                4 minutes ago        Running             coredns                   0                   31aa5a064bb41       coredns-5dd5756b68-j8npl

	014d925edadde       3ca3ca488cf13                                                                                                                4 minutes ago        Running             kube-proxy                0                   70196dfffd158       kube-proxy-9p6xr

	f7b008820594c       04b4c447bb9d4                                                                                                                4 minutes ago        Running             kube-apiserver            0                   21b21c6f9db0c       kube-apiserver-addons-263000

	6025cd5ea8e30       05c284c929889                                                                                                                4 minutes ago        Running             kube-scheduler            0                   ad6bc0e8ffbc0       kube-scheduler-addons-263000

	269e89325c468       9cdd6470f48c8                                                                                                                4 minutes ago        Running             etcd                      0                   a53903a91e7bb       etcd-addons-263000

	932e242a08ce3       9961cbceaf234                                                                                                                4 minutes ago        Running             kube-controller-manager   0                   91453d4b9b4f8       kube-controller-manager-addons-263000

	==> coredns [9a0dc72a6e29] <==

	[INFO] 10.244.0.20:60912 - 54993 "AAAA IN hello-world-app.default.svc.cluster.local.svc.cluster.local. udp 77 false 512" NXDOMAIN qr,aa,rd 170 0.000030758s

	[INFO] 10.244.0.20:60912 - 2759 "A IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000042302s

	[INFO] 10.244.0.20:60912 - 22341 "AAAA IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000036426s

	[INFO] 10.244.0.20:60912 - 24183 "A IN hello-world-app.default.svc.cluster.local. udp 59 false 512" NOERROR qr,aa,rd 116 0.000084896s

	[INFO] 10.244.0.20:48193 - 48289 "A IN hello-world-app.default.svc.cluster.local.ingress-nginx.svc.cluster.local. udp 91 false 512" NXDOMAIN qr,aa,rd 184 0.000073976s

	[INFO] 10.244.0.20:48193 - 7677 "AAAA IN hello-world-app.default.svc.cluster.local.ingress-nginx.svc.cluster.local. udp 91 false 512" NXDOMAIN qr,aa,rd 184 0.000048179s

	[INFO] 10.244.0.20:48193 - 62701 "A IN hello-world-app.default.svc.cluster.local.svc.cluster.local. udp 77 false 512" NXDOMAIN qr,aa,rd 170 0.000020005s

	[INFO] 10.244.0.20:48193 - 15502 "AAAA IN hello-world-app.default.svc.cluster.local.svc.cluster.local. udp 77 false 512" NXDOMAIN qr,aa,rd 170 0.000013045s

	[INFO] 10.244.0.20:48193 - 11222 "A IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000024589s

	[INFO] 10.244.0.20:48193 - 40516 "AAAA IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000016963s

	[INFO] 10.244.0.20:48193 - 8896 "A IN hello-world-app.default.svc.cluster.local. udp 59 false 512" NOERROR qr,aa,rd 116 0.00005318s

	[INFO] 10.244.0.20:38687 - 31591 "A IN hello-world-app.default.svc.cluster.local.ingress-nginx.svc.cluster.local. udp 91 false 512" NXDOMAIN qr,aa,rd 184 0.002772354s

	[INFO] 10.244.0.20:38687 - 29022 "AAAA IN hello-world-app.default.svc.cluster.local.ingress-nginx.svc.cluster.local. udp 91 false 512" NXDOMAIN qr,aa,rd 184 0.000034092s

	[INFO] 10.244.0.20:38687 - 6027 "A IN hello-world-app.default.svc.cluster.local.svc.cluster.local. udp 77 false 512" NXDOMAIN qr,aa,rd 170 0.000038802s

	[INFO] 10.244.0.20:44948 - 58337 "A IN hello-world-app.default.svc.cluster.local.ingress-nginx.svc.cluster.local. udp 91 false 512" NXDOMAIN qr,aa,rd 184 0.000015754s

	[INFO] 10.244.0.20:38687 - 48726 "AAAA IN hello-world-app.default.svc.cluster.local.svc.cluster.local. udp 77 false 512" NXDOMAIN qr,aa,rd 170 0.0000333s

	[INFO] 10.244.0.20:38687 - 52873 "A IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000012211s

	[INFO] 10.244.0.20:44948 - 13061 "AAAA IN hello-world-app.default.svc.cluster.local.ingress-nginx.svc.cluster.local. udp 91 false 512" NXDOMAIN qr,aa,rd 184 0.000020463s

	[INFO] 10.244.0.20:44948 - 5524 "A IN hello-world-app.default.svc.cluster.local.svc.cluster.local. udp 77 false 512" NXDOMAIN qr,aa,rd 170 0.00001067s

	[INFO] 10.244.0.20:38687 - 41367 "AAAA IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.0000328s

	[INFO] 10.244.0.20:44948 - 41597 "AAAA IN hello-world-app.default.svc.cluster.local.svc.cluster.local. udp 77 false 512" NXDOMAIN qr,aa,rd 170 0.000024632s

	[INFO] 10.244.0.20:38687 - 55351 "A IN hello-world-app.default.svc.cluster.local. udp 59 false 512" NOERROR qr,aa,rd 116 0.000012628s

	[INFO] 10.244.0.20:44948 - 43468 "A IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000010753s

	[INFO] 10.244.0.20:44948 - 36478 "AAAA IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000010086s

	[INFO] 10.244.0.20:44948 - 48154 "A IN hello-world-app.default.svc.cluster.local. udp 59 false 512" NOERROR qr,aa,rd 116 0.000009836s

	==> describe nodes <==

	Name:               addons-263000

	Roles:              control-plane

	Labels:             beta.kubernetes.io/arch=arm64

	                    beta.kubernetes.io/os=linux

	                    kubernetes.io/arch=arm64

	                    kubernetes.io/hostname=addons-263000

	                    kubernetes.io/os=linux

	                    minikube.k8s.io/commit=3e3656b8cff33aafa60dd2a07a4b34bce666a6a6

	                    minikube.k8s.io/name=addons-263000

	                    minikube.k8s.io/primary=true

	                    minikube.k8s.io/updated_at=2024_03_07T13_49_23_0700

	                    minikube.k8s.io/version=v1.32.0

	                    node-role.kubernetes.io/control-plane=

	                    node.kubernetes.io/exclude-from-external-load-balancers=

	                    topology.hostpath.csi/node=addons-263000

	Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock

	                    node.alpha.kubernetes.io/ttl: 0

	                    volumes.kubernetes.io/controller-managed-attach-detach: true

	CreationTimestamp:  Thu, 07 Mar 2024 21:49:21 +0000

	Taints:             <none>

	Unschedulable:      false

	Lease:

	  HolderIdentity:  addons-263000

	  AcquireTime:     <unset>

	  RenewTime:       Thu, 07 Mar 2024 21:53:47 +0000

	Conditions:

	  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message

	  ----             ------  -----------------                 ------------------                ------                       -------

	  MemoryPressure   False   Thu, 07 Mar 2024 21:53:28 +0000   Thu, 07 Mar 2024 21:49:20 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available

	  DiskPressure     False   Thu, 07 Mar 2024 21:53:28 +0000   Thu, 07 Mar 2024 21:49:20 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure

	  PIDPressure      False   Thu, 07 Mar 2024 21:53:28 +0000   Thu, 07 Mar 2024 21:49:20 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available

	  Ready            True    Thu, 07 Mar 2024 21:53:28 +0000   Thu, 07 Mar 2024 21:49:27 +0000   KubeletReady                 kubelet is posting ready status

	Addresses:

	  InternalIP:  192.168.105.2

	  Hostname:    addons-263000

	Capacity:

	  cpu:                2

	  ephemeral-storage:  17734596Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3904752Ki

	  pods:               110

	Allocatable:

	  cpu:                2

	  ephemeral-storage:  17734596Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3904752Ki

	  pods:               110

	System Info:

	  Machine ID:                 f16d43b128924b46895223ee9f0beb8e

	  System UUID:                f16d43b128924b46895223ee9f0beb8e

	  Boot ID:                    2fb46d85-0753-40f6-8d1f-f2faa99712ce

	  Kernel Version:             5.10.207

	  OS Image:                   Buildroot 2023.02.9

	  Operating System:           linux

	  Architecture:               arm64

	  Container Runtime Version:  docker://24.0.7

	  Kubelet Version:            v1.28.4

	  Kube-Proxy Version:         v1.28.4

	PodCIDR:                      10.244.0.0/24

	PodCIDRs:                     10.244.0.0/24

	Non-terminated Pods:          (12 in total)

	  Namespace                   Name                                     CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age

	  ---------                   ----                                     ------------  ----------  ---------------  -------------  ---

	  default                     hello-world-app-5d77478584-4zz98         0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         23s

	  default                     nginx                                    0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         33s

	  gcp-auth                    gcp-auth-5f6b4f85fd-vfhrm                0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         4m12s

	  headlamp                    headlamp-5485c556b-v68g9                 0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         52s

	  kube-system                 coredns-5dd5756b68-j8npl                 100m (5%!)(MISSING)     0 (0%!)(MISSING)      70Mi (1%!)(MISSING)        170Mi (4%!)(MISSING)     4m20s

	  kube-system                 etcd-addons-263000                       100m (5%!)(MISSING)     0 (0%!)(MISSING)      100Mi (2%!)(MISSING)       0 (0%!)(MISSING)         4m33s

	  kube-system                 kube-apiserver-addons-263000             250m (12%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         4m33s

	  kube-system                 kube-controller-manager-addons-263000    200m (10%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         4m33s

	  kube-system                 kube-proxy-9p6xr                         0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         4m20s

	  kube-system                 kube-scheduler-addons-263000             100m (5%!)(MISSING)     0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         4m33s

	  kube-system                 storage-provisioner                      0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         4m18s

	  yakd-dashboard              yakd-dashboard-9947fc6bf-f6gpj           0 (0%!)(MISSING)        0 (0%!)(MISSING)      128Mi (3%!)(MISSING)       256Mi (6%!)(MISSING)     4m17s

	Allocated resources:

	  (Total limits may be over 100 percent, i.e., overcommitted.)

	  Resource           Requests    Limits

	  --------           --------    ------

	  cpu                750m (37%!)(MISSING)  0 (0%!)(MISSING)

	  memory             298Mi (7%!)(MISSING)  426Mi (11%!)(MISSING)

	  ephemeral-storage  0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-1Gi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-2Mi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-32Mi     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-64Ki     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	Events:

	  Type    Reason                   Age                    From             Message

	  ----    ------                   ----                   ----             -------

	  Normal  Starting                 4m19s                  kube-proxy       

	  Normal  Starting                 4m38s                  kubelet          Starting kubelet.

	  Normal  NodeAllocatableEnforced  4m38s                  kubelet          Updated Node Allocatable limit across pods

	  Normal  NodeHasSufficientMemory  4m37s (x8 over 4m38s)  kubelet          Node addons-263000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    4m37s (x8 over 4m38s)  kubelet          Node addons-263000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     4m37s (x7 over 4m38s)  kubelet          Node addons-263000 status is now: NodeHasSufficientPID

	  Normal  Starting                 4m33s                  kubelet          Starting kubelet.

	  Normal  NodeAllocatableEnforced  4m33s                  kubelet          Updated Node Allocatable limit across pods

	  Normal  NodeHasSufficientMemory  4m33s                  kubelet          Node addons-263000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    4m33s                  kubelet          Node addons-263000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     4m33s                  kubelet          Node addons-263000 status is now: NodeHasSufficientPID

	  Normal  NodeReady                4m29s                  kubelet          Node addons-263000 status is now: NodeReady

	  Normal  RegisteredNode           4m21s                  node-controller  Node addons-263000 event: Registered Node addons-263000 in Controller

	==> dmesg <==

	[  +5.599514] kauditd_printk_skb: 22 callbacks suppressed

	[  +5.241977] kauditd_printk_skb: 7 callbacks suppressed

	[Mar 7 21:50] kauditd_printk_skb: 3 callbacks suppressed

	[  +5.075072] kauditd_printk_skb: 2 callbacks suppressed

	[ +17.154597] kauditd_printk_skb: 2 callbacks suppressed

	[  +5.261797] kauditd_printk_skb: 6 callbacks suppressed

	[  +6.529116] kauditd_printk_skb: 25 callbacks suppressed

	[ +10.213718] kauditd_printk_skb: 13 callbacks suppressed

	[  +5.192905] kauditd_printk_skb: 19 callbacks suppressed

	[Mar 7 21:51] kauditd_printk_skb: 6 callbacks suppressed

	[  +8.372526] kauditd_printk_skb: 16 callbacks suppressed

	[  +7.042859] kauditd_printk_skb: 2 callbacks suppressed

	[  +6.683679] kauditd_printk_skb: 5 callbacks suppressed

	[Mar 7 21:52] kauditd_printk_skb: 7 callbacks suppressed

	[ +17.009562] kauditd_printk_skb: 9 callbacks suppressed

	[ +10.513593] kauditd_printk_skb: 15 callbacks suppressed

	[ +12.079361] kauditd_printk_skb: 27 callbacks suppressed

	[  +5.488254] kauditd_printk_skb: 32 callbacks suppressed

	[  +5.068701] kauditd_printk_skb: 42 callbacks suppressed

	[  +5.714769] kauditd_printk_skb: 6 callbacks suppressed

	[Mar 7 21:53] kauditd_printk_skb: 8 callbacks suppressed

	[  +5.177949] kauditd_printk_skb: 12 callbacks suppressed

	[ +16.879872] kauditd_printk_skb: 22 callbacks suppressed

	[ +19.769926] kauditd_printk_skb: 4 callbacks suppressed

	[  +5.003269] kauditd_printk_skb: 9 callbacks suppressed

	==> etcd [269e89325c46] <==

	{"level":"info","ts":"2024-03-07T21:49:19.711664Z","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"/var/lib/minikube/etcd/member/wal","suffix":"wal","max":5,"interval":"30s"}

	{"level":"info","ts":"2024-03-07T21:49:19.711819Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"c46d288d2fcb0590 switched to configuration voters=(14154013790752671120)"}

	{"level":"info","ts":"2024-03-07T21:49:19.712008Z","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"6e03e7863b4f9c54","local-member-id":"c46d288d2fcb0590","added-peer-id":"c46d288d2fcb0590","added-peer-peer-urls":["https://192.168.105.2:2380"]}

	{"level":"info","ts":"2024-03-07T21:49:20.500491Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"c46d288d2fcb0590 is starting a new election at term 1"}

	{"level":"info","ts":"2024-03-07T21:49:20.500548Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"c46d288d2fcb0590 became pre-candidate at term 1"}

	{"level":"info","ts":"2024-03-07T21:49:20.500567Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"c46d288d2fcb0590 received MsgPreVoteResp from c46d288d2fcb0590 at term 1"}

	{"level":"info","ts":"2024-03-07T21:49:20.500578Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"c46d288d2fcb0590 became candidate at term 2"}

	{"level":"info","ts":"2024-03-07T21:49:20.500587Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"c46d288d2fcb0590 received MsgVoteResp from c46d288d2fcb0590 at term 2"}

	{"level":"info","ts":"2024-03-07T21:49:20.500594Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"c46d288d2fcb0590 became leader at term 2"}

	{"level":"info","ts":"2024-03-07T21:49:20.500601Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: c46d288d2fcb0590 elected leader c46d288d2fcb0590 at term 2"}

	{"level":"info","ts":"2024-03-07T21:49:20.501509Z","caller":"etcdserver/server.go:2062","msg":"published local member to cluster through raft","local-member-id":"c46d288d2fcb0590","local-member-attributes":"{Name:addons-263000 ClientURLs:[https://192.168.105.2:2379]}","request-path":"/0/members/c46d288d2fcb0590/attributes","cluster-id":"6e03e7863b4f9c54","publish-timeout":"7s"}

	{"level":"info","ts":"2024-03-07T21:49:20.501527Z","caller":"etcdserver/server.go:2571","msg":"setting up initial cluster version using v2 API","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-07T21:49:20.501627Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2024-03-07T21:49:20.501923Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"6e03e7863b4f9c54","local-member-id":"c46d288d2fcb0590","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-07T21:49:20.501956Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-07T21:49:20.501966Z","caller":"etcdserver/server.go:2595","msg":"cluster version is updated","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-07T21:49:20.501993Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}

	{"level":"info","ts":"2024-03-07T21:49:20.502Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}

	{"level":"info","ts":"2024-03-07T21:49:20.502004Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2024-03-07T21:49:20.502451Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.105.2:2379"}

	{"level":"info","ts":"2024-03-07T21:49:20.502571Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"127.0.0.1:2379"}

	{"level":"info","ts":"2024-03-07T21:50:27.547507Z","caller":"traceutil/trace.go:171","msg":"trace[1724346099] transaction","detail":"{read_only:false; response_revision:979; number_of_response:1; }","duration":"100.246349ms","start":"2024-03-07T21:50:27.447252Z","end":"2024-03-07T21:50:27.547498Z","steps":["trace[1724346099] 'process raft request'  (duration: 100.019912ms)"],"step_count":1}

	{"level":"info","ts":"2024-03-07T21:51:07.487786Z","caller":"traceutil/trace.go:171","msg":"trace[579090319] linearizableReadLoop","detail":"{readStateIndex:1165; appliedIndex:1164; }","duration":"128.734475ms","start":"2024-03-07T21:51:07.359044Z","end":"2024-03-07T21:51:07.487778Z","steps":["trace[579090319] 'read index received'  (duration: 128.68821ms)","trace[579090319] 'applied index is now lower than readState.Index'  (duration: 45.932µs)"],"step_count":2}

	{"level":"warn","ts":"2024-03-07T21:51:07.487874Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"128.8343ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/gcp-auth/\" range_end:\"/registry/pods/gcp-auth0\" ","response":"range_response_count:3 size:10918"}

	{"level":"info","ts":"2024-03-07T21:51:07.487887Z","caller":"traceutil/trace.go:171","msg":"trace[979306414] range","detail":"{range_begin:/registry/pods/gcp-auth/; range_end:/registry/pods/gcp-auth0; response_count:3; response_revision:1131; }","duration":"128.856598ms","start":"2024-03-07T21:51:07.359028Z","end":"2024-03-07T21:51:07.487884Z","steps":["trace[979306414] 'agreement among raft nodes before linearized reading'  (duration: 128.795537ms)"],"step_count":1}

	==> gcp-auth [5b4739a5a467] <==

	2024/03/07 21:52:13 GCP Auth Webhook started!

	2024/03/07 21:52:19 Ready to marshal response ...

	2024/03/07 21:52:19 Ready to write response ...

	2024/03/07 21:52:24 Ready to marshal response ...

	2024/03/07 21:52:24 Ready to write response ...

	2024/03/07 21:52:42 Ready to marshal response ...

	2024/03/07 21:52:42 Ready to write response ...

	2024/03/07 21:52:42 Ready to marshal response ...

	2024/03/07 21:52:42 Ready to write response ...

	2024/03/07 21:52:42 Ready to marshal response ...

	2024/03/07 21:52:42 Ready to write response ...

	2024/03/07 21:52:51 Ready to marshal response ...

	2024/03/07 21:52:51 Ready to write response ...

	2024/03/07 21:53:04 Ready to marshal response ...

	2024/03/07 21:53:04 Ready to write response ...

	2024/03/07 21:53:04 Ready to marshal response ...

	2024/03/07 21:53:04 Ready to write response ...

	2024/03/07 21:53:04 Ready to marshal response ...

	2024/03/07 21:53:04 Ready to write response ...

	2024/03/07 21:53:23 Ready to marshal response ...

	2024/03/07 21:53:23 Ready to write response ...

	2024/03/07 21:53:33 Ready to marshal response ...

	2024/03/07 21:53:33 Ready to write response ...

	==> kernel <==

	 21:53:57 up 4 min,  0 users,  load average: 0.59, 0.66, 0.34

	Linux addons-263000 5.10.207 #1 SMP PREEMPT Thu Feb 22 23:40:42 UTC 2024 aarch64 GNU/Linux

	PRETTY_NAME="Buildroot 2023.02.9"

	==> kube-apiserver [f7b008820594] <==

	I0307 21:52:58.379071       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0307 21:52:58.379089       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0307 21:52:58.387129       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0307 21:52:58.387150       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0307 21:52:58.395107       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0307 21:52:58.395121       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0307 21:52:58.399130       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0307 21:52:58.399150       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0307 21:52:58.402525       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0307 21:52:58.402544       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0307 21:52:58.409889       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0307 21:52:58.409964       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0307 21:52:58.412559       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0307 21:52:58.412579       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	W0307 21:52:59.395391       1 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io

	W0307 21:52:59.412799       1 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io

	W0307 21:52:59.419514       1 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io

	I0307 21:53:04.167128       1 alloc.go:330] "allocated clusterIPs" service="headlamp/headlamp" clusterIPs={"IPv4":"10.108.2.99"}

	E0307 21:53:07.602169       1 authentication.go:73] "Unable to authenticate the request" err="[invalid bearer token, serviceaccounts \"local-path-provisioner-service-account\" not found]"

	I0307 21:53:23.236256       1 controller.go:624] quota admission added evaluator for: ingresses.networking.k8s.io

	I0307 21:53:23.328517       1 alloc.go:330] "allocated clusterIPs" service="default/nginx" clusterIPs={"IPv4":"10.102.203.116"}

	I0307 21:53:25.646314       1 handler.go:232] Adding GroupVersion gadget.kinvolk.io v1alpha1 to ResourceManager

	I0307 21:53:25.652437       1 handler.go:232] Adding GroupVersion gadget.kinvolk.io v1alpha1 to ResourceManager

	W0307 21:53:26.658818       1 cacher.go:171] Terminating all watchers from cacher traces.gadget.kinvolk.io

	I0307 21:53:33.537927       1 alloc.go:330] "allocated clusterIPs" service="default/hello-world-app" clusterIPs={"IPv4":"10.98.208.183"}

	==> kube-controller-manager [932e242a08ce] <==

	W0307 21:53:33.227475       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0307 21:53:33.227491       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	I0307 21:53:33.467797       1 event.go:307] "Event occurred" object="default/hello-world-app" fieldPath="" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set hello-world-app-5d77478584 to 1"

	I0307 21:53:33.473542       1 event.go:307] "Event occurred" object="default/hello-world-app-5d77478584" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: hello-world-app-5d77478584-4zz98"

	I0307 21:53:33.479689       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="12.162473ms"

	I0307 21:53:33.486012       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="6.298811ms"

	I0307 21:53:33.493589       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="7.547746ms"

	I0307 21:53:33.493694       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="83.271µs"

	I0307 21:53:35.679535       1 namespace_controller.go:182] "Namespace has been deleted" namespace="gadget"

	I0307 21:53:36.151175       1 shared_informer.go:311] Waiting for caches to sync for resource quota

	I0307 21:53:36.151239       1 shared_informer.go:318] Caches are synced for resource quota

	I0307 21:53:36.568576       1 shared_informer.go:311] Waiting for caches to sync for garbage collector

	I0307 21:53:36.568602       1 shared_informer.go:318] Caches are synced for garbage collector

	W0307 21:53:37.451976       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0307 21:53:37.452001       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0307 21:53:40.552059       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0307 21:53:40.552084       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0307 21:53:43.143135       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0307 21:53:43.143158       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	I0307 21:53:46.291626       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="26.84µs"

	I0307 21:53:47.308128       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="23.755µs"

	I0307 21:53:48.309125       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="20.546µs"

	I0307 21:53:49.532352       1 job_controller.go:562] "enqueueing job" key="ingress-nginx/ingress-nginx-admission-create"

	I0307 21:53:49.533588       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="ingress-nginx/ingress-nginx-controller-76dc478dd8" duration="1.792µs"

	I0307 21:53:49.535902       1 job_controller.go:562] "enqueueing job" key="ingress-nginx/ingress-nginx-admission-patch"

	==> kube-proxy [014d925edadd] <==

	I0307 21:49:37.857215       1 server_others.go:69] "Using iptables proxy"

	I0307 21:49:37.872819       1 node.go:141] Successfully retrieved node IP: 192.168.105.2

	I0307 21:49:37.935169       1 server_others.go:121] "No iptables support for family" ipFamily="IPv6"

	I0307 21:49:37.935189       1 server.go:634] "kube-proxy running in single-stack mode" ipFamily="IPv4"

	I0307 21:49:37.935974       1 server_others.go:152] "Using iptables Proxier"

	I0307 21:49:37.936000       1 proxier.go:251] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses"

	I0307 21:49:37.936077       1 server.go:846] "Version info" version="v1.28.4"

	I0307 21:49:37.936083       1 server.go:848] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0307 21:49:37.936730       1 config.go:188] "Starting service config controller"

	I0307 21:49:37.936734       1 shared_informer.go:311] Waiting for caches to sync for service config

	I0307 21:49:37.936742       1 config.go:97] "Starting endpoint slice config controller"

	I0307 21:49:37.936743       1 shared_informer.go:311] Waiting for caches to sync for endpoint slice config

	I0307 21:49:37.936909       1 config.go:315] "Starting node config controller"

	I0307 21:49:37.936911       1 shared_informer.go:311] Waiting for caches to sync for node config

	I0307 21:49:38.037089       1 shared_informer.go:318] Caches are synced for node config

	I0307 21:49:38.037108       1 shared_informer.go:318] Caches are synced for service config

	I0307 21:49:38.037120       1 shared_informer.go:318] Caches are synced for endpoint slice config

	==> kube-scheduler [6025cd5ea8e3] <==

	W0307 21:49:21.107917       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	E0307 21:49:21.107920       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	W0307 21:49:21.107931       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	E0307 21:49:21.107934       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	W0307 21:49:21.107969       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	E0307 21:49:21.107976       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	W0307 21:49:21.107998       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope

	E0307 21:49:21.108004       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope

	W0307 21:49:21.108369       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	E0307 21:49:21.108405       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	W0307 21:49:21.108534       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	E0307 21:49:21.108545       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	W0307 21:49:21.109109       1 reflector.go:535] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	E0307 21:49:21.109144       1 reflector.go:147] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	W0307 21:49:22.014433       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	E0307 21:49:22.014449       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	W0307 21:49:22.058205       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	E0307 21:49:22.058273       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	W0307 21:49:22.077237       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope

	E0307 21:49:22.077256       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope

	W0307 21:49:22.147745       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	E0307 21:49:22.147889       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	W0307 21:49:22.151423       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	E0307 21:49:22.151435       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	I0307 21:49:22.502873       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	==> kubelet <==

	Mar 07 21:53:39 addons-263000 kubelet[2390]: E0307 21:53:39.565300    2390 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"minikube-ingress-dns\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=minikube-ingress-dns pod=kube-ingress-dns-minikube_kube-system(2b265f89-7503-4f45-b8e5-edaee7812238)\"" pod="kube-system/kube-ingress-dns-minikube" podUID="2b265f89-7503-4f45-b8e5-edaee7812238"

	Mar 07 21:53:46 addons-263000 kubelet[2390]: I0307 21:53:46.284678    2390 scope.go:117] "RemoveContainer" containerID="bad4caf1a1284e8d7ae753d52f99f3fdbce168352d03b484a7d82152191b81a3"

	Mar 07 21:53:47 addons-263000 kubelet[2390]: I0307 21:53:47.292272    2390 scope.go:117] "RemoveContainer" containerID="bad4caf1a1284e8d7ae753d52f99f3fdbce168352d03b484a7d82152191b81a3"

	Mar 07 21:53:47 addons-263000 kubelet[2390]: I0307 21:53:47.292447    2390 scope.go:117] "RemoveContainer" containerID="2cf20bdb1244ab4fec99a3d7045d6d2f55635c9ff7ff058f9c4e1355ec520b92"

	Mar 07 21:53:47 addons-263000 kubelet[2390]: E0307 21:53:47.292564    2390 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"hello-world-app\" with CrashLoopBackOff: \"back-off 10s restarting failed container=hello-world-app pod=hello-world-app-5d77478584-4zz98_default(43c128c1-77c9-4fde-af18-22a07a3256e5)\"" pod="default/hello-world-app-5d77478584-4zz98" podUID="43c128c1-77c9-4fde-af18-22a07a3256e5"

	Mar 07 21:53:48 addons-263000 kubelet[2390]: I0307 21:53:48.300069    2390 scope.go:117] "RemoveContainer" containerID="2cf20bdb1244ab4fec99a3d7045d6d2f55635c9ff7ff058f9c4e1355ec520b92"

	Mar 07 21:53:48 addons-263000 kubelet[2390]: E0307 21:53:48.300190    2390 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"hello-world-app\" with CrashLoopBackOff: \"back-off 10s restarting failed container=hello-world-app pod=hello-world-app-5d77478584-4zz98_default(43c128c1-77c9-4fde-af18-22a07a3256e5)\"" pod="default/hello-world-app-5d77478584-4zz98" podUID="43c128c1-77c9-4fde-af18-22a07a3256e5"

	Mar 07 21:53:48 addons-263000 kubelet[2390]: I0307 21:53:48.800742    2390 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqhlp\" (UniqueName: \"kubernetes.io/projected/2b265f89-7503-4f45-b8e5-edaee7812238-kube-api-access-bqhlp\") pod \"2b265f89-7503-4f45-b8e5-edaee7812238\" (UID: \"2b265f89-7503-4f45-b8e5-edaee7812238\") "

	Mar 07 21:53:48 addons-263000 kubelet[2390]: I0307 21:53:48.803595    2390 operation_generator.go:882] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b265f89-7503-4f45-b8e5-edaee7812238-kube-api-access-bqhlp" (OuterVolumeSpecName: "kube-api-access-bqhlp") pod "2b265f89-7503-4f45-b8e5-edaee7812238" (UID: "2b265f89-7503-4f45-b8e5-edaee7812238"). InnerVolumeSpecName "kube-api-access-bqhlp". PluginName "kubernetes.io/projected", VolumeGidValue ""

	Mar 07 21:53:48 addons-263000 kubelet[2390]: I0307 21:53:48.901812    2390 reconciler_common.go:300] "Volume detached for volume \"kube-api-access-bqhlp\" (UniqueName: \"kubernetes.io/projected/2b265f89-7503-4f45-b8e5-edaee7812238-kube-api-access-bqhlp\") on node \"addons-263000\" DevicePath \"\""

	Mar 07 21:53:49 addons-263000 kubelet[2390]: I0307 21:53:49.306071    2390 scope.go:117] "RemoveContainer" containerID="438e1a87ca8b0b51200ffed80eb9e8d4b5fd0b19555db837dca2c41c8b440937"

	Mar 07 21:53:49 addons-263000 kubelet[2390]: I0307 21:53:49.569437    2390 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b265f89-7503-4f45-b8e5-edaee7812238" path="/var/lib/kubelet/pods/2b265f89-7503-4f45-b8e5-edaee7812238/volumes"

	Mar 07 21:53:49 addons-263000 kubelet[2390]: I0307 21:53:49.569706    2390 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75eabb00-9888-4a70-8131-4ee72f1c329e" path="/var/lib/kubelet/pods/75eabb00-9888-4a70-8131-4ee72f1c329e/volumes"

	Mar 07 21:53:49 addons-263000 kubelet[2390]: I0307 21:53:49.569862    2390 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7927b1b8-42ee-432d-8827-4e6765d8136c" path="/var/lib/kubelet/pods/7927b1b8-42ee-432d-8827-4e6765d8136c/volumes"

	Mar 07 21:53:52 addons-263000 kubelet[2390]: I0307 21:53:52.824292    2390 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb3b7868-d91d-4de7-85fc-1269827c053a-webhook-cert\") pod \"eb3b7868-d91d-4de7-85fc-1269827c053a\" (UID: \"eb3b7868-d91d-4de7-85fc-1269827c053a\") "

	Mar 07 21:53:52 addons-263000 kubelet[2390]: I0307 21:53:52.824326    2390 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mghl8\" (UniqueName: \"kubernetes.io/projected/eb3b7868-d91d-4de7-85fc-1269827c053a-kube-api-access-mghl8\") pod \"eb3b7868-d91d-4de7-85fc-1269827c053a\" (UID: \"eb3b7868-d91d-4de7-85fc-1269827c053a\") "

	Mar 07 21:53:52 addons-263000 kubelet[2390]: I0307 21:53:52.825094    2390 operation_generator.go:882] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb3b7868-d91d-4de7-85fc-1269827c053a-kube-api-access-mghl8" (OuterVolumeSpecName: "kube-api-access-mghl8") pod "eb3b7868-d91d-4de7-85fc-1269827c053a" (UID: "eb3b7868-d91d-4de7-85fc-1269827c053a"). InnerVolumeSpecName "kube-api-access-mghl8". PluginName "kubernetes.io/projected", VolumeGidValue ""

	Mar 07 21:53:52 addons-263000 kubelet[2390]: I0307 21:53:52.825143    2390 operation_generator.go:882] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb3b7868-d91d-4de7-85fc-1269827c053a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "eb3b7868-d91d-4de7-85fc-1269827c053a" (UID: "eb3b7868-d91d-4de7-85fc-1269827c053a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue ""

	Mar 07 21:53:52 addons-263000 kubelet[2390]: I0307 21:53:52.924373    2390 reconciler_common.go:300] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb3b7868-d91d-4de7-85fc-1269827c053a-webhook-cert\") on node \"addons-263000\" DevicePath \"\""

	Mar 07 21:53:52 addons-263000 kubelet[2390]: I0307 21:53:52.924387    2390 reconciler_common.go:300] "Volume detached for volume \"kube-api-access-mghl8\" (UniqueName: \"kubernetes.io/projected/eb3b7868-d91d-4de7-85fc-1269827c053a-kube-api-access-mghl8\") on node \"addons-263000\" DevicePath \"\""

	Mar 07 21:53:53 addons-263000 kubelet[2390]: I0307 21:53:53.328952    2390 scope.go:117] "RemoveContainer" containerID="e498100ade0bcb9163922886c2a68e33e8da7627588fdeb9fb0667dd2055d881"

	Mar 07 21:53:53 addons-263000 kubelet[2390]: I0307 21:53:53.336132    2390 scope.go:117] "RemoveContainer" containerID="e498100ade0bcb9163922886c2a68e33e8da7627588fdeb9fb0667dd2055d881"

	Mar 07 21:53:53 addons-263000 kubelet[2390]: E0307 21:53:53.336420    2390 remote_runtime.go:432] "ContainerStatus from runtime service failed" err="rpc error: code = Unknown desc = Error response from daemon: No such container: e498100ade0bcb9163922886c2a68e33e8da7627588fdeb9fb0667dd2055d881" containerID="e498100ade0bcb9163922886c2a68e33e8da7627588fdeb9fb0667dd2055d881"

	Mar 07 21:53:53 addons-263000 kubelet[2390]: I0307 21:53:53.336441    2390 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"docker","ID":"e498100ade0bcb9163922886c2a68e33e8da7627588fdeb9fb0667dd2055d881"} err="failed to get container status \"e498100ade0bcb9163922886c2a68e33e8da7627588fdeb9fb0667dd2055d881\": rpc error: code = Unknown desc = Error response from daemon: No such container: e498100ade0bcb9163922886c2a68e33e8da7627588fdeb9fb0667dd2055d881"

	Mar 07 21:53:53 addons-263000 kubelet[2390]: I0307 21:53:53.567435    2390 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb3b7868-d91d-4de7-85fc-1269827c053a" path="/var/lib/kubelet/pods/eb3b7868-d91d-4de7-85fc-1269827c053a/volumes"

	==> storage-provisioner [6715bc4652b8] <==

	I0307 21:49:39.319777       1 storage_provisioner.go:116] Initializing the minikube storage provisioner...

	I0307 21:49:39.326528       1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!

	I0307 21:49:39.326549       1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...

	I0307 21:49:39.333845       1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath

	I0307 21:49:39.333900       1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_addons-263000_c9051ca7-fdeb-4ad3-a0c2-7c1fc288699d!

	I0307 21:49:39.334275       1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"2593583d-60f1-4a0c-8607-de69c9c47787", APIVersion:"v1", ResourceVersion:"554", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' addons-263000_c9051ca7-fdeb-4ad3-a0c2-7c1fc288699d became leader

	I0307 21:49:39.435210       1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_addons-263000_c9051ca7-fdeb-4ad3-a0c2-7c1fc288699d!

-- /stdout --

helpers_test.go:254: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.APIServer}} -p addons-263000 -n addons-263000

helpers_test.go:261: (dbg) Run:  kubectl --context addons-263000 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running

helpers_test.go:285: <<< TestAddons/parallel/Ingress FAILED: end of post-mortem logs <<<

helpers_test.go:286: ---------------------/post-mortem---------------------------------

--- FAIL: TestAddons/parallel/Ingress (34.45s)

=== RUN   TestCertOptions

=== PAUSE TestCertOptions

=== CONT  TestCertOptions

cert_options_test.go:49: (dbg) Run:  out/minikube-darwin-arm64 start -p cert-options-995000 --memory=2048 --apiserver-ips=127.0.0.1 --apiserver-ips=192.168.15.15 --apiserver-names=localhost --apiserver-names=www.google.com --apiserver-port=8555 --driver=qemu2 

cert_options_test.go:49: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p cert-options-995000 --memory=2048 --apiserver-ips=127.0.0.1 --apiserver-ips=192.168.15.15 --apiserver-names=localhost --apiserver-names=www.google.com --apiserver-port=8555 --driver=qemu2 : exit status 80 (9.9500105s)

-- stdout --

	* [cert-options-995000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting "cert-options-995000" primary control-plane node in "cert-options-995000" cluster

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "cert-options-995000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p cert-options-995000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

cert_options_test.go:51: failed to start minikube with args: "out/minikube-darwin-arm64 start -p cert-options-995000 --memory=2048 --apiserver-ips=127.0.0.1 --apiserver-ips=192.168.15.15 --apiserver-names=localhost --apiserver-names=www.google.com --apiserver-port=8555 --driver=qemu2 " : exit status 80

cert_options_test.go:60: (dbg) Run:  out/minikube-darwin-arm64 -p cert-options-995000 ssh "openssl x509 -text -noout -in /var/lib/minikube/certs/apiserver.crt"

cert_options_test.go:60: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p cert-options-995000 ssh "openssl x509 -text -noout -in /var/lib/minikube/certs/apiserver.crt": exit status 83 (82.564417ms)

-- stdout --

	* The control-plane node cert-options-995000 host is not running: state=Stopped

	  To start a cluster, run: "minikube start -p cert-options-995000"

-- /stdout --

cert_options_test.go:62: failed to read apiserver cert inside minikube. args "out/minikube-darwin-arm64 -p cert-options-995000 ssh \"openssl x509 -text -noout -in /var/lib/minikube/certs/apiserver.crt\"": exit status 83

cert_options_test.go:69: apiserver cert does not include 127.0.0.1 in SAN.

cert_options_test.go:69: apiserver cert does not include 192.168.15.15 in SAN.

cert_options_test.go:69: apiserver cert does not include localhost in SAN.

cert_options_test.go:69: apiserver cert does not include www.google.com in SAN.

cert_options_test.go:88: (dbg) Run:  kubectl --context cert-options-995000 config view

cert_options_test.go:93: Kubeconfig apiserver server port incorrect. Output of 

'kubectl config view' = "\n-- stdout --\n\tapiVersion: v1\n\tclusters: null\n\tcontexts: null\n\tcurrent-context: \"\"\n\tkind: Config\n\tpreferences: {}\n\tusers: null\n\n-- /stdout --"

cert_options_test.go:100: (dbg) Run:  out/minikube-darwin-arm64 ssh -p cert-options-995000 -- "sudo cat /etc/kubernetes/admin.conf"

cert_options_test.go:100: (dbg) Non-zero exit: out/minikube-darwin-arm64 ssh -p cert-options-995000 -- "sudo cat /etc/kubernetes/admin.conf": exit status 83 (42.290667ms)

-- stdout --

	* The control-plane node cert-options-995000 host is not running: state=Stopped

	  To start a cluster, run: "minikube start -p cert-options-995000"

-- /stdout --

cert_options_test.go:102: failed to SSH to minikube with args: "out/minikube-darwin-arm64 ssh -p cert-options-995000 -- \"sudo cat /etc/kubernetes/admin.conf\"" : exit status 83

cert_options_test.go:106: Internal minikube kubeconfig (admin.conf) does not contains the right api port. 

-- stdout --

	* The control-plane node cert-options-995000 host is not running: state=Stopped

	  To start a cluster, run: "minikube start -p cert-options-995000"

-- /stdout --

cert_options_test.go:109: *** TestCertOptions FAILED at 2024-03-07 14:37:56.348016 -0800 PST m=+3088.843454585

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p cert-options-995000 -n cert-options-995000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p cert-options-995000 -n cert-options-995000: exit status 7 (32.449834ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "cert-options-995000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "cert-options-995000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p cert-options-995000

--- FAIL: TestCertOptions (10.24s)

=== RUN   TestCertExpiration

=== PAUSE TestCertExpiration

=== CONT  TestCertExpiration

cert_options_test.go:123: (dbg) Run:  out/minikube-darwin-arm64 start -p cert-expiration-072000 --memory=2048 --cert-expiration=3m --driver=qemu2 

cert_options_test.go:123: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p cert-expiration-072000 --memory=2048 --cert-expiration=3m --driver=qemu2 : exit status 80 (10.03582775s)

-- stdout --

	* [cert-expiration-072000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting "cert-expiration-072000" primary control-plane node in "cert-expiration-072000" cluster

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "cert-expiration-072000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p cert-expiration-072000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

cert_options_test.go:125: failed to start minikube with args: "out/minikube-darwin-arm64 start -p cert-expiration-072000 --memory=2048 --cert-expiration=3m --driver=qemu2 " : exit status 80

cert_options_test.go:131: (dbg) Run:  out/minikube-darwin-arm64 start -p cert-expiration-072000 --memory=2048 --cert-expiration=8760h --driver=qemu2 

cert_options_test.go:131: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p cert-expiration-072000 --memory=2048 --cert-expiration=8760h --driver=qemu2 : exit status 80 (5.202055166s)

-- stdout --

	* [cert-expiration-072000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on existing profile

	* Starting "cert-expiration-072000" primary control-plane node in "cert-expiration-072000" cluster

	* Restarting existing qemu2 VM for "cert-expiration-072000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Restarting existing qemu2 VM for "cert-expiration-072000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p cert-expiration-072000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

cert_options_test.go:133: failed to start minikube after cert expiration: "out/minikube-darwin-arm64 start -p cert-expiration-072000 --memory=2048 --cert-expiration=8760h --driver=qemu2 " : exit status 80

cert_options_test.go:136: minikube start output did not warn about expired certs: 

-- stdout --

	* [cert-expiration-072000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on existing profile

	* Starting "cert-expiration-072000" primary control-plane node in "cert-expiration-072000" cluster

	* Restarting existing qemu2 VM for "cert-expiration-072000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Restarting existing qemu2 VM for "cert-expiration-072000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p cert-expiration-072000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

cert_options_test.go:138: *** TestCertExpiration FAILED at 2024-03-07 14:40:56.309868 -0800 PST m=+3268.804961085

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p cert-expiration-072000 -n cert-expiration-072000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p cert-expiration-072000 -n cert-expiration-072000: exit status 7 (41.346042ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "cert-expiration-072000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "cert-expiration-072000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p cert-expiration-072000

--- FAIL: TestCertExpiration (195.39s)

=== RUN   TestDockerFlags

=== PAUSE TestDockerFlags

=== CONT  TestDockerFlags

docker_test.go:51: (dbg) Run:  out/minikube-darwin-arm64 start -p docker-flags-655000 --cache-images=false --memory=2048 --install-addons=false --wait=false --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug --docker-opt=icc=true --alsologtostderr -v=5 --driver=qemu2 

docker_test.go:51: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p docker-flags-655000 --cache-images=false --memory=2048 --install-addons=false --wait=false --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug --docker-opt=icc=true --alsologtostderr -v=5 --driver=qemu2 : exit status 80 (10.018819875s)

-- stdout --

	* [docker-flags-655000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting "docker-flags-655000" primary control-plane node in "docker-flags-655000" cluster

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "docker-flags-655000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0307 14:37:35.987766    4001 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:37:35.987877    4001 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:37:35.987881    4001 out.go:304] Setting ErrFile to fd 2...

	I0307 14:37:35.987883    4001 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:37:35.988007    4001 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:37:35.989062    4001 out.go:298] Setting JSON to false

	I0307 14:37:36.005325    4001 start.go:129] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":4012,"bootTime":1709847043,"procs":473,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.3.1","kernelVersion":"23.3.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0307 14:37:36.005402    4001 start.go:137] gopshost.Virtualization returned error: not implemented yet

	I0307 14:37:36.011430    4001 out.go:177] * [docker-flags-655000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	I0307 14:37:36.019439    4001 out.go:177]   - MINIKUBE_LOCATION=18320

	I0307 14:37:36.019568    4001 notify.go:220] Checking for updates...

	I0307 14:37:36.024429    4001 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	I0307 14:37:36.027330    4001 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0307 14:37:36.030386    4001 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0307 14:37:36.033473    4001 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	I0307 14:37:36.034896    4001 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0307 14:37:36.038770    4001 config.go:182] Loaded profile config "force-systemd-flag-843000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:37:36.038835    4001 config.go:182] Loaded profile config "multinode-172000-m01": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:37:36.038885    4001 driver.go:392] Setting default libvirt URI to qemu:///system

	I0307 14:37:36.043391    4001 out.go:177] * Using the qemu2 driver based on user configuration

	I0307 14:37:36.049395    4001 start.go:297] selected driver: qemu2

	I0307 14:37:36.049401    4001 start.go:901] validating driver "qemu2" against <nil>

	I0307 14:37:36.049409    4001 start.go:912] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0307 14:37:36.051657    4001 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0307 14:37:36.055414    4001 out.go:177] * Automatically selected the socket_vmnet network

	I0307 14:37:36.058485    4001 start_flags.go:942] Waiting for no components: map[apiserver:false apps_running:false default_sa:false extra:false kubelet:false node_ready:false system_pods:false]

	I0307 14:37:36.058520    4001 cni.go:84] Creating CNI manager for ""

	I0307 14:37:36.058528    4001 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0307 14:37:36.058533    4001 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0307 14:37:36.058566    4001 start.go:340] cluster config:

	{Name:docker-flags-655000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[FOO=BAR BAZ=BAT] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[debug icc=true] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:docker-flags-655000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[]

DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:false EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:false apps_running:false default_sa:false extra:false kubelet:false node_ready:false system_pods:false] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMn

etClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 14:37:36.063062    4001 iso.go:125] acquiring lock: {Name:mk9f11fe8095f5aa8805d47835226b5c64c27e19 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0307 14:37:36.071338    4001 out.go:177] * Starting "docker-flags-655000" primary control-plane node in "docker-flags-655000" cluster

	I0307 14:37:36.075380    4001 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0307 14:37:36.075404    4001 preload.go:147] Found local preload: /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4

	I0307 14:37:36.075417    4001 cache.go:56] Caching tarball of preloaded images

	I0307 14:37:36.075491    4001 preload.go:173] Found /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0307 14:37:36.075497    4001 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on docker

	I0307 14:37:36.075576    4001 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/docker-flags-655000/config.json ...

	I0307 14:37:36.075589    4001 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/docker-flags-655000/config.json: {Name:mkf5a519d629fa4714eb34332842d1349581ced1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 14:37:36.075818    4001 start.go:360] acquireMachinesLock for docker-flags-655000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:37:36.075851    4001 start.go:364] duration metric: took 27µs to acquireMachinesLock for "docker-flags-655000"

	I0307 14:37:36.075862    4001 start.go:93] Provisioning new machine with config: &{Name:docker-flags-655000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[FOO=BAR BAZ=BAT] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[debug icc=true] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey

: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:docker-flags-655000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:false EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:false apps_running:false default_sa:false extra:false kubelet:false node_ready:false system_pods:false] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:dock

er MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 14:37:36.075896    4001 start.go:125] createHost starting for "" (driver="qemu2")

	I0307 14:37:36.084409    4001 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0307 14:37:36.101994    4001 start.go:159] libmachine.API.Create for "docker-flags-655000" (driver="qemu2")

	I0307 14:37:36.102025    4001 client.go:168] LocalClient.Create starting

	I0307 14:37:36.102094    4001 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem

	I0307 14:37:36.102125    4001 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:36.102135    4001 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:36.102185    4001 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem

	I0307 14:37:36.102207    4001 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:36.102213    4001 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:36.102574    4001 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso...

	I0307 14:37:36.244284    4001 main.go:141] libmachine: Creating SSH key...

	I0307 14:37:36.433240    4001 main.go:141] libmachine: Creating Disk image...

	I0307 14:37:36.433250    4001 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0307 14:37:36.433426    4001 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2.raw /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2

	I0307 14:37:36.445849    4001 main.go:141] libmachine: STDOUT: 

	I0307 14:37:36.445865    4001 main.go:141] libmachine: STDERR: 

	I0307 14:37:36.445918    4001 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2 +20000M

	I0307 14:37:36.456496    4001 main.go:141] libmachine: STDOUT: Image resized.

	I0307 14:37:36.456514    4001 main.go:141] libmachine: STDERR: 

	I0307 14:37:36.456530    4001 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2.raw and /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2

	I0307 14:37:36.456537    4001 main.go:141] libmachine: Starting QEMU VM...

	I0307 14:37:36.456580    4001 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/qemu.pid -device virtio-net-pci,netdev=net0,mac=ae:21:0a:15:61:61 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2

	I0307 14:37:36.458308    4001 main.go:141] libmachine: STDOUT: 

	I0307 14:37:36.458324    4001 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:37:36.458354    4001 client.go:171] duration metric: took 356.311584ms to LocalClient.Create

	I0307 14:37:38.460570    4001 start.go:128] duration metric: took 2.384648125s to createHost

	I0307 14:37:38.460694    4001 start.go:83] releasing machines lock for "docker-flags-655000", held for 2.384825958s

	W0307 14:37:38.460753    4001 start.go:713] error starting host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:38.475817    4001 out.go:177] * Deleting "docker-flags-655000" in qemu2 ...

	W0307 14:37:38.511118    4001 out.go:239] ! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:38.511150    4001 start.go:728] Will try again in 5 seconds ...

	I0307 14:37:43.513347    4001 start.go:360] acquireMachinesLock for docker-flags-655000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:37:43.513692    4001 start.go:364] duration metric: took 256.917µs to acquireMachinesLock for "docker-flags-655000"

	I0307 14:37:43.513756    4001 start.go:93] Provisioning new machine with config: &{Name:docker-flags-655000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[FOO=BAR BAZ=BAT] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[debug icc=true] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey

: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:docker-flags-655000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:false EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:false apps_running:false default_sa:false extra:false kubelet:false node_ready:false system_pods:false] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:dock

er MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 14:37:43.514006    4001 start.go:125] createHost starting for "" (driver="qemu2")

	I0307 14:37:43.537257    4001 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0307 14:37:43.585845    4001 start.go:159] libmachine.API.Create for "docker-flags-655000" (driver="qemu2")

	I0307 14:37:43.585927    4001 client.go:168] LocalClient.Create starting

	I0307 14:37:43.586153    4001 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem

	I0307 14:37:43.586243    4001 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:43.586262    4001 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:43.586373    4001 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem

	I0307 14:37:43.586426    4001 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:43.586445    4001 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:43.587451    4001 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso...

	I0307 14:37:43.764698    4001 main.go:141] libmachine: Creating SSH key...

	I0307 14:37:43.905949    4001 main.go:141] libmachine: Creating Disk image...

	I0307 14:37:43.905955    4001 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0307 14:37:43.906149    4001 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2.raw /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2

	I0307 14:37:43.918673    4001 main.go:141] libmachine: STDOUT: 

	I0307 14:37:43.918694    4001 main.go:141] libmachine: STDERR: 

	I0307 14:37:43.918738    4001 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2 +20000M

	I0307 14:37:43.929476    4001 main.go:141] libmachine: STDOUT: Image resized.

	I0307 14:37:43.929497    4001 main.go:141] libmachine: STDERR: 

	I0307 14:37:43.929507    4001 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2.raw and /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2

	I0307 14:37:43.929522    4001 main.go:141] libmachine: Starting QEMU VM...

	I0307 14:37:43.929560    4001 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/qemu.pid -device virtio-net-pci,netdev=net0,mac=36:c4:52:b4:0a:d2 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/docker-flags-655000/disk.qcow2

	I0307 14:37:43.931352    4001 main.go:141] libmachine: STDOUT: 

	I0307 14:37:43.931372    4001 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:37:43.931389    4001 client.go:171] duration metric: took 345.455542ms to LocalClient.Create

	I0307 14:37:45.933564    4001 start.go:128] duration metric: took 2.419525417s to createHost

	I0307 14:37:45.933663    4001 start.go:83] releasing machines lock for "docker-flags-655000", held for 2.419943625s

	W0307 14:37:45.934125    4001 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p docker-flags-655000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p docker-flags-655000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:45.943799    4001 out.go:177] 

	W0307 14:37:45.948815    4001 out.go:239] X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0307 14:37:45.948840    4001 out.go:239] * 

	* 

	W0307 14:37:45.951475    4001 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0307 14:37:45.960817    4001 out.go:177] 

** /stderr **

docker_test.go:53: failed to start minikube with args: "out/minikube-darwin-arm64 start -p docker-flags-655000 --cache-images=false --memory=2048 --install-addons=false --wait=false --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug --docker-opt=icc=true --alsologtostderr -v=5 --driver=qemu2 " : exit status 80

docker_test.go:56: (dbg) Run:  out/minikube-darwin-arm64 -p docker-flags-655000 ssh "sudo systemctl show docker --property=Environment --no-pager"

docker_test.go:56: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p docker-flags-655000 ssh "sudo systemctl show docker --property=Environment --no-pager": exit status 83 (79.136625ms)

-- stdout --

	* The control-plane node docker-flags-655000 host is not running: state=Stopped

	  To start a cluster, run: "minikube start -p docker-flags-655000"

-- /stdout --

docker_test.go:58: failed to 'systemctl show docker' inside minikube. args "out/minikube-darwin-arm64 -p docker-flags-655000 ssh \"sudo systemctl show docker --property=Environment --no-pager\"": exit status 83

docker_test.go:63: expected env key/value "FOO=BAR" to be passed to minikube's docker and be included in: *"* The control-plane node docker-flags-655000 host is not running: state=Stopped\n  To start a cluster, run: \"minikube start -p docker-flags-655000\"\n"*.

docker_test.go:63: expected env key/value "BAZ=BAT" to be passed to minikube's docker and be included in: *"* The control-plane node docker-flags-655000 host is not running: state=Stopped\n  To start a cluster, run: \"minikube start -p docker-flags-655000\"\n"*.

docker_test.go:67: (dbg) Run:  out/minikube-darwin-arm64 -p docker-flags-655000 ssh "sudo systemctl show docker --property=ExecStart --no-pager"

docker_test.go:67: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p docker-flags-655000 ssh "sudo systemctl show docker --property=ExecStart --no-pager": exit status 83 (46.707292ms)

-- stdout --

	* The control-plane node docker-flags-655000 host is not running: state=Stopped

	  To start a cluster, run: "minikube start -p docker-flags-655000"

-- /stdout --

docker_test.go:69: failed on the second 'systemctl show docker' inside minikube. args "out/minikube-darwin-arm64 -p docker-flags-655000 ssh \"sudo systemctl show docker --property=ExecStart --no-pager\"": exit status 83

docker_test.go:73: expected "out/minikube-darwin-arm64 -p docker-flags-655000 ssh \"sudo systemctl show docker --property=ExecStart --no-pager\"" output to have include *--debug* . output: "* The control-plane node docker-flags-655000 host is not running: state=Stopped\n  To start a cluster, run: \"minikube start -p docker-flags-655000\"\n"

panic.go:626: *** TestDockerFlags FAILED at 2024-03-07 14:37:46.104922 -0800 PST m=+3078.600380418

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p docker-flags-655000 -n docker-flags-655000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p docker-flags-655000 -n docker-flags-655000: exit status 7 (31.234958ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "docker-flags-655000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "docker-flags-655000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p docker-flags-655000

--- FAIL: TestDockerFlags (10.28s)

=== RUN   TestForceSystemdFlag

=== PAUSE TestForceSystemdFlag

=== CONT  TestForceSystemdFlag

docker_test.go:91: (dbg) Run:  out/minikube-darwin-arm64 start -p force-systemd-flag-843000 --memory=2048 --force-systemd --alsologtostderr -v=5 --driver=qemu2 

docker_test.go:91: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p force-systemd-flag-843000 --memory=2048 --force-systemd --alsologtostderr -v=5 --driver=qemu2 : exit status 80 (9.819821625s)

-- stdout --

	* [force-systemd-flag-843000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting "force-systemd-flag-843000" primary control-plane node in "force-systemd-flag-843000" cluster

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "force-systemd-flag-843000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0307 14:37:31.055231    3977 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:37:31.055339    3977 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:37:31.055342    3977 out.go:304] Setting ErrFile to fd 2...

	I0307 14:37:31.055344    3977 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:37:31.055465    3977 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:37:31.056546    3977 out.go:298] Setting JSON to false

	I0307 14:37:31.072571    3977 start.go:129] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":4008,"bootTime":1709847043,"procs":475,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.3.1","kernelVersion":"23.3.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0307 14:37:31.072631    3977 start.go:137] gopshost.Virtualization returned error: not implemented yet

	I0307 14:37:31.079567    3977 out.go:177] * [force-systemd-flag-843000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	I0307 14:37:31.086525    3977 out.go:177]   - MINIKUBE_LOCATION=18320

	I0307 14:37:31.090504    3977 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	I0307 14:37:31.086621    3977 notify.go:220] Checking for updates...

	I0307 14:37:31.093532    3977 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0307 14:37:31.096481    3977 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0307 14:37:31.099487    3977 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	I0307 14:37:31.102458    3977 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0307 14:37:31.105732    3977 config.go:182] Loaded profile config "force-systemd-env-041000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:37:31.105800    3977 config.go:182] Loaded profile config "multinode-172000-m01": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:37:31.105856    3977 driver.go:392] Setting default libvirt URI to qemu:///system

	I0307 14:37:31.110442    3977 out.go:177] * Using the qemu2 driver based on user configuration

	I0307 14:37:31.116425    3977 start.go:297] selected driver: qemu2

	I0307 14:37:31.116430    3977 start.go:901] validating driver "qemu2" against <nil>

	I0307 14:37:31.116435    3977 start.go:912] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0307 14:37:31.118604    3977 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0307 14:37:31.121452    3977 out.go:177] * Automatically selected the socket_vmnet network

	I0307 14:37:31.124560    3977 start_flags.go:929] Wait components to verify : map[apiserver:true system_pods:true]

	I0307 14:37:31.124600    3977 cni.go:84] Creating CNI manager for ""

	I0307 14:37:31.124608    3977 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0307 14:37:31.124615    3977 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0307 14:37:31.124645    3977 start.go:340] cluster config:

	{Name:force-systemd-flag-843000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:force-systemd-flag-843000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster

.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet Static

IP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 14:37:31.128975    3977 iso.go:125] acquiring lock: {Name:mk9f11fe8095f5aa8805d47835226b5c64c27e19 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0307 14:37:31.136443    3977 out.go:177] * Starting "force-systemd-flag-843000" primary control-plane node in "force-systemd-flag-843000" cluster

	I0307 14:37:31.140531    3977 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0307 14:37:31.140549    3977 preload.go:147] Found local preload: /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4

	I0307 14:37:31.140558    3977 cache.go:56] Caching tarball of preloaded images

	I0307 14:37:31.140630    3977 preload.go:173] Found /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0307 14:37:31.140637    3977 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on docker

	I0307 14:37:31.140712    3977 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/force-systemd-flag-843000/config.json ...

	I0307 14:37:31.140728    3977 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/force-systemd-flag-843000/config.json: {Name:mk5d7339638b7e1a17a712057eedb32e6d0f1ddc Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 14:37:31.140982    3977 start.go:360] acquireMachinesLock for force-systemd-flag-843000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:37:31.141021    3977 start.go:364] duration metric: took 27.833µs to acquireMachinesLock for "force-systemd-flag-843000"

	I0307 14:37:31.141034    3977 start.go:93] Provisioning new machine with config: &{Name:force-systemd-flag-843000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kubernetes

Config:{KubernetesVersion:v1.28.4 ClusterName:force-systemd-flag-843000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror

: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 14:37:31.141092    3977 start.go:125] createHost starting for "" (driver="qemu2")

	I0307 14:37:31.149457    3977 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0307 14:37:31.167251    3977 start.go:159] libmachine.API.Create for "force-systemd-flag-843000" (driver="qemu2")

	I0307 14:37:31.167284    3977 client.go:168] LocalClient.Create starting

	I0307 14:37:31.167364    3977 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem

	I0307 14:37:31.167400    3977 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:31.167409    3977 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:31.167455    3977 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem

	I0307 14:37:31.167477    3977 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:31.167487    3977 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:31.167857    3977 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso...

	I0307 14:37:31.307970    3977 main.go:141] libmachine: Creating SSH key...

	I0307 14:37:31.390384    3977 main.go:141] libmachine: Creating Disk image...

	I0307 14:37:31.390389    3977 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0307 14:37:31.390570    3977 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2.raw /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2

	I0307 14:37:31.403043    3977 main.go:141] libmachine: STDOUT: 

	I0307 14:37:31.403067    3977 main.go:141] libmachine: STDERR: 

	I0307 14:37:31.403121    3977 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2 +20000M

	I0307 14:37:31.414522    3977 main.go:141] libmachine: STDOUT: Image resized.

	I0307 14:37:31.414538    3977 main.go:141] libmachine: STDERR: 

	I0307 14:37:31.414547    3977 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2.raw and /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2

	I0307 14:37:31.414554    3977 main.go:141] libmachine: Starting QEMU VM...

	I0307 14:37:31.414580    3977 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/qemu.pid -device virtio-net-pci,netdev=net0,mac=1a:a5:10:f9:2d:f8 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2

	I0307 14:37:31.416371    3977 main.go:141] libmachine: STDOUT: 

	I0307 14:37:31.416399    3977 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:37:31.416417    3977 client.go:171] duration metric: took 249.125792ms to LocalClient.Create

	I0307 14:37:33.418646    3977 start.go:128] duration metric: took 2.27749175s to createHost

	I0307 14:37:33.418712    3977 start.go:83] releasing machines lock for "force-systemd-flag-843000", held for 2.277666833s

	W0307 14:37:33.418760    3977 start.go:713] error starting host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:33.444928    3977 out.go:177] * Deleting "force-systemd-flag-843000" in qemu2 ...

	W0307 14:37:33.464233    3977 out.go:239] ! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:33.464258    3977 start.go:728] Will try again in 5 seconds ...

	I0307 14:37:38.466482    3977 start.go:360] acquireMachinesLock for force-systemd-flag-843000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:37:38.466960    3977 start.go:364] duration metric: took 366.542µs to acquireMachinesLock for "force-systemd-flag-843000"

	I0307 14:37:38.467056    3977 start.go:93] Provisioning new machine with config: &{Name:force-systemd-flag-843000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kubernetes

Config:{KubernetesVersion:v1.28.4 ClusterName:force-systemd-flag-843000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror

: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 14:37:38.467319    3977 start.go:125] createHost starting for "" (driver="qemu2")

	I0307 14:37:38.486711    3977 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0307 14:37:38.534823    3977 start.go:159] libmachine.API.Create for "force-systemd-flag-843000" (driver="qemu2")

	I0307 14:37:38.534866    3977 client.go:168] LocalClient.Create starting

	I0307 14:37:38.534987    3977 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem

	I0307 14:37:38.535039    3977 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:38.535053    3977 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:38.535113    3977 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem

	I0307 14:37:38.535153    3977 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:38.535166    3977 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:38.535694    3977 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso...

	I0307 14:37:38.690126    3977 main.go:141] libmachine: Creating SSH key...

	I0307 14:37:38.766001    3977 main.go:141] libmachine: Creating Disk image...

	I0307 14:37:38.766007    3977 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0307 14:37:38.766200    3977 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2.raw /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2

	I0307 14:37:38.778334    3977 main.go:141] libmachine: STDOUT: 

	I0307 14:37:38.778353    3977 main.go:141] libmachine: STDERR: 

	I0307 14:37:38.778402    3977 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2 +20000M

	I0307 14:37:38.788966    3977 main.go:141] libmachine: STDOUT: Image resized.

	I0307 14:37:38.788991    3977 main.go:141] libmachine: STDERR: 

	I0307 14:37:38.789003    3977 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2.raw and /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2

	I0307 14:37:38.789007    3977 main.go:141] libmachine: Starting QEMU VM...

	I0307 14:37:38.789051    3977 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/qemu.pid -device virtio-net-pci,netdev=net0,mac=76:2a:54:79:28:8b -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-flag-843000/disk.qcow2

	I0307 14:37:38.790811    3977 main.go:141] libmachine: STDOUT: 

	I0307 14:37:38.790828    3977 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:37:38.790839    3977 client.go:171] duration metric: took 255.967125ms to LocalClient.Create

	I0307 14:37:40.793082    3977 start.go:128] duration metric: took 2.325731291s to createHost

	I0307 14:37:40.793139    3977 start.go:83] releasing machines lock for "force-systemd-flag-843000", held for 2.326146583s

	W0307 14:37:40.793438    3977 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p force-systemd-flag-843000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p force-systemd-flag-843000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:40.810198    3977 out.go:177] 

	W0307 14:37:40.817086    3977 out.go:239] X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0307 14:37:40.817129    3977 out.go:239] * 

	* 

	W0307 14:37:40.819813    3977 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0307 14:37:40.831010    3977 out.go:177] 

** /stderr **

docker_test.go:93: failed to start minikube with args: "out/minikube-darwin-arm64 start -p force-systemd-flag-843000 --memory=2048 --force-systemd --alsologtostderr -v=5 --driver=qemu2 " : exit status 80

docker_test.go:110: (dbg) Run:  out/minikube-darwin-arm64 -p force-systemd-flag-843000 ssh "docker info --format {{.CgroupDriver}}"

docker_test.go:110: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p force-systemd-flag-843000 ssh "docker info --format {{.CgroupDriver}}": exit status 83 (81.411167ms)

-- stdout --

	* The control-plane node force-systemd-flag-843000 host is not running: state=Stopped

	  To start a cluster, run: "minikube start -p force-systemd-flag-843000"

-- /stdout --

docker_test.go:112: failed to get docker cgroup driver. args "out/minikube-darwin-arm64 -p force-systemd-flag-843000 ssh \"docker info --format {{.CgroupDriver}}\"": exit status 83

docker_test.go:106: *** TestForceSystemdFlag FAILED at 2024-03-07 14:37:40.930339 -0800 PST m=+3073.425807001

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p force-systemd-flag-843000 -n force-systemd-flag-843000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p force-systemd-flag-843000 -n force-systemd-flag-843000: exit status 7 (36.621666ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "force-systemd-flag-843000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "force-systemd-flag-843000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p force-systemd-flag-843000

--- FAIL: TestForceSystemdFlag (10.04s)

=== RUN   TestForceSystemdEnv

=== PAUSE TestForceSystemdEnv

=== CONT  TestForceSystemdEnv

docker_test.go:155: (dbg) Run:  out/minikube-darwin-arm64 start -p force-systemd-env-041000 --memory=2048 --alsologtostderr -v=5 --driver=qemu2 

docker_test.go:155: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p force-systemd-env-041000 --memory=2048 --alsologtostderr -v=5 --driver=qemu2 : exit status 80 (9.794784833s)

-- stdout --

	* [force-systemd-env-041000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=true

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting "force-systemd-env-041000" primary control-plane node in "force-systemd-env-041000" cluster

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "force-systemd-env-041000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0307 14:37:25.973720    3941 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:37:25.973835    3941 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:37:25.973838    3941 out.go:304] Setting ErrFile to fd 2...

	I0307 14:37:25.973840    3941 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:37:25.973985    3941 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:37:25.975291    3941 out.go:298] Setting JSON to false

	I0307 14:37:25.992655    3941 start.go:129] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":4002,"bootTime":1709847043,"procs":470,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.3.1","kernelVersion":"23.3.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0307 14:37:25.992724    3941 start.go:137] gopshost.Virtualization returned error: not implemented yet

	I0307 14:37:25.997202    3941 out.go:177] * [force-systemd-env-041000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	I0307 14:37:26.010162    3941 out.go:177]   - MINIKUBE_LOCATION=18320

	I0307 14:37:26.005193    3941 notify.go:220] Checking for updates...

	I0307 14:37:26.018110    3941 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	I0307 14:37:26.026098    3941 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0307 14:37:26.033974    3941 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0307 14:37:26.042097    3941 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	I0307 14:37:26.049049    3941 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=true

	I0307 14:37:26.053464    3941 config.go:182] Loaded profile config "multinode-172000-m01": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:37:26.053517    3941 driver.go:392] Setting default libvirt URI to qemu:///system

	I0307 14:37:26.058105    3941 out.go:177] * Using the qemu2 driver based on user configuration

	I0307 14:37:26.065118    3941 start.go:297] selected driver: qemu2

	I0307 14:37:26.065123    3941 start.go:901] validating driver "qemu2" against <nil>

	I0307 14:37:26.065129    3941 start.go:912] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0307 14:37:26.067550    3941 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0307 14:37:26.071105    3941 out.go:177] * Automatically selected the socket_vmnet network

	I0307 14:37:26.075195    3941 start_flags.go:929] Wait components to verify : map[apiserver:true system_pods:true]

	I0307 14:37:26.075232    3941 cni.go:84] Creating CNI manager for ""

	I0307 14:37:26.075244    3941 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0307 14:37:26.075250    3941 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0307 14:37:26.075274    3941 start.go:340] cluster config:

	{Name:force-systemd-env-041000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:force-systemd-env-041000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.l

ocal ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP

: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 14:37:26.079755    3941 iso.go:125] acquiring lock: {Name:mk9f11fe8095f5aa8805d47835226b5c64c27e19 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0307 14:37:26.087124    3941 out.go:177] * Starting "force-systemd-env-041000" primary control-plane node in "force-systemd-env-041000" cluster

	I0307 14:37:26.091087    3941 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0307 14:37:26.091103    3941 preload.go:147] Found local preload: /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4

	I0307 14:37:26.091113    3941 cache.go:56] Caching tarball of preloaded images

	I0307 14:37:26.091171    3941 preload.go:173] Found /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0307 14:37:26.091177    3941 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on docker

	I0307 14:37:26.091252    3941 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/force-systemd-env-041000/config.json ...

	I0307 14:37:26.091263    3941 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/force-systemd-env-041000/config.json: {Name:mk1d7ba6743fdd12eb3fa8009c9540e35ba70c80 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 14:37:26.091472    3941 start.go:360] acquireMachinesLock for force-systemd-env-041000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:37:26.091504    3941 start.go:364] duration metric: took 25.459µs to acquireMachinesLock for "force-systemd-env-041000"

	I0307 14:37:26.091514    3941 start.go:93] Provisioning new machine with config: &{Name:force-systemd-env-041000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesC

onfig:{KubernetesVersion:v1.28.4 ClusterName:force-systemd-env-041000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror:

DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 14:37:26.091545    3941 start.go:125] createHost starting for "" (driver="qemu2")

	I0307 14:37:26.099162    3941 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0307 14:37:26.115893    3941 start.go:159] libmachine.API.Create for "force-systemd-env-041000" (driver="qemu2")

	I0307 14:37:26.115928    3941 client.go:168] LocalClient.Create starting

	I0307 14:37:26.115996    3941 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem

	I0307 14:37:26.116022    3941 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:26.116032    3941 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:26.116078    3941 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem

	I0307 14:37:26.116102    3941 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:26.116108    3941 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:26.116435    3941 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso...

	I0307 14:37:26.259028    3941 main.go:141] libmachine: Creating SSH key...

	I0307 14:37:26.314087    3941 main.go:141] libmachine: Creating Disk image...

	I0307 14:37:26.314097    3941 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0307 14:37:26.314266    3941 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2.raw /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2

	I0307 14:37:26.327295    3941 main.go:141] libmachine: STDOUT: 

	I0307 14:37:26.327317    3941 main.go:141] libmachine: STDERR: 

	I0307 14:37:26.327380    3941 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2 +20000M

	I0307 14:37:26.339212    3941 main.go:141] libmachine: STDOUT: Image resized.

	I0307 14:37:26.339230    3941 main.go:141] libmachine: STDERR: 

	I0307 14:37:26.339254    3941 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2.raw and /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2

	I0307 14:37:26.339260    3941 main.go:141] libmachine: Starting QEMU VM...

	I0307 14:37:26.339292    3941 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/qemu.pid -device virtio-net-pci,netdev=net0,mac=ae:b5:97:d2:02:71 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2

	I0307 14:37:26.341146    3941 main.go:141] libmachine: STDOUT: 

	I0307 14:37:26.341172    3941 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:37:26.341192    3941 client.go:171] duration metric: took 225.25825ms to LocalClient.Create

	I0307 14:37:28.343417    3941 start.go:128] duration metric: took 2.251835625s to createHost

	I0307 14:37:28.343547    3941 start.go:83] releasing machines lock for "force-systemd-env-041000", held for 2.252015709s

	W0307 14:37:28.343644    3941 start.go:713] error starting host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:28.350725    3941 out.go:177] * Deleting "force-systemd-env-041000" in qemu2 ...

	W0307 14:37:28.381528    3941 out.go:239] ! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:28.381566    3941 start.go:728] Will try again in 5 seconds ...

	I0307 14:37:33.383801    3941 start.go:360] acquireMachinesLock for force-systemd-env-041000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:37:33.418794    3941 start.go:364] duration metric: took 34.871292ms to acquireMachinesLock for "force-systemd-env-041000"

	I0307 14:37:33.418946    3941 start.go:93] Provisioning new machine with config: &{Name:force-systemd-env-041000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesC

onfig:{KubernetesVersion:v1.28.4 ClusterName:force-systemd-env-041000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror:

DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 14:37:33.419182    3941 start.go:125] createHost starting for "" (driver="qemu2")

	I0307 14:37:33.433846    3941 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0307 14:37:33.482574    3941 start.go:159] libmachine.API.Create for "force-systemd-env-041000" (driver="qemu2")

	I0307 14:37:33.482624    3941 client.go:168] LocalClient.Create starting

	I0307 14:37:33.482792    3941 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem

	I0307 14:37:33.482858    3941 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:33.482877    3941 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:33.482945    3941 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem

	I0307 14:37:33.482986    3941 main.go:141] libmachine: Decoding PEM data...

	I0307 14:37:33.482997    3941 main.go:141] libmachine: Parsing certificate...

	I0307 14:37:33.483524    3941 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso...

	I0307 14:37:33.639060    3941 main.go:141] libmachine: Creating SSH key...

	I0307 14:37:33.664648    3941 main.go:141] libmachine: Creating Disk image...

	I0307 14:37:33.664653    3941 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0307 14:37:33.664832    3941 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2.raw /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2

	I0307 14:37:33.677058    3941 main.go:141] libmachine: STDOUT: 

	I0307 14:37:33.677081    3941 main.go:141] libmachine: STDERR: 

	I0307 14:37:33.677132    3941 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2 +20000M

	I0307 14:37:33.688094    3941 main.go:141] libmachine: STDOUT: Image resized.

	I0307 14:37:33.688131    3941 main.go:141] libmachine: STDERR: 

	I0307 14:37:33.688139    3941 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2.raw and /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2

	I0307 14:37:33.688144    3941 main.go:141] libmachine: Starting QEMU VM...

	I0307 14:37:33.688179    3941 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/qemu.pid -device virtio-net-pci,netdev=net0,mac=aa:c1:87:8f:4b:02 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/force-systemd-env-041000/disk.qcow2

	I0307 14:37:33.690038    3941 main.go:141] libmachine: STDOUT: 

	I0307 14:37:33.690054    3941 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:37:33.690066    3941 client.go:171] duration metric: took 207.436083ms to LocalClient.Create

	I0307 14:37:35.692281    3941 start.go:128] duration metric: took 2.273063s to createHost

	I0307 14:37:35.692358    3941 start.go:83] releasing machines lock for "force-systemd-env-041000", held for 2.273535625s

	W0307 14:37:35.692742    3941 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p force-systemd-env-041000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p force-systemd-env-041000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:37:35.705368    3941 out.go:177] 

	W0307 14:37:35.709406    3941 out.go:239] X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0307 14:37:35.709455    3941 out.go:239] * 

	* 

	W0307 14:37:35.712382    3941 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0307 14:37:35.722115    3941 out.go:177] 

** /stderr **

docker_test.go:157: failed to start minikube with args: "out/minikube-darwin-arm64 start -p force-systemd-env-041000 --memory=2048 --alsologtostderr -v=5 --driver=qemu2 " : exit status 80

docker_test.go:110: (dbg) Run:  out/minikube-darwin-arm64 -p force-systemd-env-041000 ssh "docker info --format {{.CgroupDriver}}"

docker_test.go:110: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p force-systemd-env-041000 ssh "docker info --format {{.CgroupDriver}}": exit status 83 (78.418167ms)

-- stdout --

	* The control-plane node force-systemd-env-041000 host is not running: state=Stopped

	  To start a cluster, run: "minikube start -p force-systemd-env-041000"

-- /stdout --

docker_test.go:112: failed to get docker cgroup driver. args "out/minikube-darwin-arm64 -p force-systemd-env-041000 ssh \"docker info --format {{.CgroupDriver}}\"": exit status 83

docker_test.go:166: *** TestForceSystemdEnv FAILED at 2024-03-07 14:37:35.818501 -0800 PST m=+3068.313979793

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p force-systemd-env-041000 -n force-systemd-env-041000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p force-systemd-env-041000 -n force-systemd-env-041000: exit status 7 (34.990291ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "force-systemd-env-041000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "force-systemd-env-041000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p force-systemd-env-041000

--- FAIL: TestForceSystemdEnv (10.01s)

=== RUN   TestFunctional/parallel/ServiceCmdConnect

=== PAUSE TestFunctional/parallel/ServiceCmdConnect

=== CONT  TestFunctional/parallel/ServiceCmdConnect

functional_test.go:1623: (dbg) Run:  kubectl --context functional-836000 create deployment hello-node-connect --image=registry.k8s.io/echoserver-arm:1.8

functional_test.go:1631: (dbg) Run:  kubectl --context functional-836000 expose deployment hello-node-connect --type=NodePort --port=8080

functional_test.go:1636: (dbg) TestFunctional/parallel/ServiceCmdConnect: waiting 10m0s for pods matching "app=hello-node-connect" in namespace "default" ...

helpers_test.go:344: "hello-node-connect-7799dfb7c6-gfnrd" [ec5c5794-652a-4971-b709-4333f321052d] Pending / Ready:ContainersNotReady (containers with unready status: [echoserver-arm]) / ContainersReady:ContainersNotReady (containers with unready status: [echoserver-arm])

helpers_test.go:344: "hello-node-connect-7799dfb7c6-gfnrd" [ec5c5794-652a-4971-b709-4333f321052d] Running / Ready:ContainersNotReady (containers with unready status: [echoserver-arm]) / ContainersReady:ContainersNotReady (containers with unready status: [echoserver-arm])

functional_test.go:1636: (dbg) TestFunctional/parallel/ServiceCmdConnect: app=hello-node-connect healthy within 14.003572292s

functional_test.go:1645: (dbg) Run:  out/minikube-darwin-arm64 -p functional-836000 service hello-node-connect --url

functional_test.go:1651: found endpoint for hello-node-connect: http://192.168.105.4:32424

functional_test.go:1657: error fetching http://192.168.105.4:32424: Get "http://192.168.105.4:32424": dial tcp 192.168.105.4:32424: connect: connection refused

functional_test.go:1657: error fetching http://192.168.105.4:32424: Get "http://192.168.105.4:32424": dial tcp 192.168.105.4:32424: connect: connection refused

functional_test.go:1657: error fetching http://192.168.105.4:32424: Get "http://192.168.105.4:32424": dial tcp 192.168.105.4:32424: connect: connection refused

functional_test.go:1657: error fetching http://192.168.105.4:32424: Get "http://192.168.105.4:32424": dial tcp 192.168.105.4:32424: connect: connection refused

functional_test.go:1657: error fetching http://192.168.105.4:32424: Get "http://192.168.105.4:32424": dial tcp 192.168.105.4:32424: connect: connection refused

functional_test.go:1657: error fetching http://192.168.105.4:32424: Get "http://192.168.105.4:32424": dial tcp 192.168.105.4:32424: connect: connection refused

functional_test.go:1657: error fetching http://192.168.105.4:32424: Get "http://192.168.105.4:32424": dial tcp 192.168.105.4:32424: connect: connection refused

functional_test.go:1677: failed to fetch http://192.168.105.4:32424: Get "http://192.168.105.4:32424": dial tcp 192.168.105.4:32424: connect: connection refused

functional_test.go:1594: service test failed - dumping debug information

functional_test.go:1595: -----------------------service failure post-mortem--------------------------------

functional_test.go:1598: (dbg) Run:  kubectl --context functional-836000 describe po hello-node-connect

functional_test.go:1602: hello-node pod describe:

Name:             hello-node-connect-7799dfb7c6-gfnrd

Namespace:        default

Priority:         0

Service Account:  default

Node:             functional-836000/192.168.105.4

Start Time:       Thu, 07 Mar 2024 13:58:28 -0800

Labels:           app=hello-node-connect

pod-template-hash=7799dfb7c6

Annotations:      <none>

Status:           Running

IP:               10.244.0.8

IPs:

IP:           10.244.0.8

Controlled By:  ReplicaSet/hello-node-connect-7799dfb7c6

Containers:

echoserver-arm:

Container ID:   docker://27ae7bcacdb2e20876c2ff4fca003a402e7f8657ab05a11d21238d1f11eabf2b

Image:          registry.k8s.io/echoserver-arm:1.8

Image ID:       docker-pullable://registry.k8s.io/echoserver-arm@sha256:b33d4cdf6ed097f4e9b77b135d83a596ab73c6268b0342648818eb85f5edfdb5

Port:           <none>

Host Port:      <none>

State:          Waiting

Reason:       CrashLoopBackOff

Last State:     Terminated

Reason:       Error

Exit Code:    1

Started:      Thu, 07 Mar 2024 13:58:52 -0800

Finished:     Thu, 07 Mar 2024 13:58:52 -0800

Ready:          False

Restart Count:  2

Environment:    <none>

Mounts:

/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-qnk9k (ro)

Conditions:

Type              Status

Initialized       True 

Ready             False 

ContainersReady   False 

PodScheduled      True 

Volumes:

kube-api-access-qnk9k:

Type:                    Projected (a volume that contains injected data from multiple sources)

TokenExpirationSeconds:  3607

ConfigMapName:           kube-root-ca.crt

ConfigMapOptional:       <nil>

DownwardAPI:             true

QoS Class:                   BestEffort

Node-Selectors:              <none>

Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s

node.kubernetes.io/unreachable:NoExecute op=Exists for 300s

Events:

Type     Reason     Age                From               Message

----     ------     ----               ----               -------

Normal   Scheduled  38s                default-scheduler  Successfully assigned default/hello-node-connect-7799dfb7c6-gfnrd to functional-836000

Normal   Pulling    38s                kubelet            Pulling image "registry.k8s.io/echoserver-arm:1.8"

Normal   Pulled     31s                kubelet            Successfully pulled image "registry.k8s.io/echoserver-arm:1.8" in 6.434s (6.434s including waiting)

Normal   Created    14s (x3 over 31s)  kubelet            Created container echoserver-arm

Normal   Started    14s (x3 over 31s)  kubelet            Started container echoserver-arm

Normal   Pulled     14s (x2 over 30s)  kubelet            Container image "registry.k8s.io/echoserver-arm:1.8" already present on machine

Warning  BackOff    3s (x4 over 29s)   kubelet            Back-off restarting failed container echoserver-arm in pod hello-node-connect-7799dfb7c6-gfnrd_default(ec5c5794-652a-4971-b709-4333f321052d)

functional_test.go:1604: (dbg) Run:  kubectl --context functional-836000 logs -l app=hello-node-connect

functional_test.go:1608: hello-node logs:

exec /usr/sbin/nginx: exec format error

functional_test.go:1610: (dbg) Run:  kubectl --context functional-836000 describe svc hello-node-connect

functional_test.go:1614: hello-node svc describe:

Name:                     hello-node-connect

Namespace:                default

Labels:                   app=hello-node-connect

Annotations:              <none>

Selector:                 app=hello-node-connect

Type:                     NodePort

IP Family Policy:         SingleStack

IP Families:              IPv4

IP:                       10.110.89.210

IPs:                      10.110.89.210

Port:                     <unset>  8080/TCP

TargetPort:               8080/TCP

NodePort:                 <unset>  32424/TCP

Endpoints:                

Session Affinity:         None

External Traffic Policy:  Cluster

Events:                   <none>

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p functional-836000 -n functional-836000

helpers_test.go:244: <<< TestFunctional/parallel/ServiceCmdConnect FAILED: start of post-mortem logs <<<

helpers_test.go:245: ======>  post-mortem[TestFunctional/parallel/ServiceCmdConnect]: minikube logs <======

helpers_test.go:247: (dbg) Run:  out/minikube-darwin-arm64 -p functional-836000 logs -n 25

helpers_test.go:252: TestFunctional/parallel/ServiceCmdConnect logs: 

-- stdout --

	==> Audit <==

	|-----------|----------------------------------------------------------------------------------------------------------------------|-------------------|---------|---------|---------------------|---------------------|

	|  Command  |                                                         Args                                                         |      Profile      |  User   | Version |     Start Time      |      End Time       |

	|-----------|----------------------------------------------------------------------------------------------------------------------|-------------------|---------|---------|---------------------|---------------------|

	| ssh       | functional-836000 ssh stat                                                                                           | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:58 PST | 07 Mar 24 13:58 PST |

	|           | /mount-9p/created-by-pod                                                                                             |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh sudo                                                                                           | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:58 PST | 07 Mar 24 13:58 PST |

	|           | umount -f /mount-9p                                                                                                  |                   |         |         |                     |                     |

	| mount     | -p functional-836000                                                                                                 | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:58 PST |                     |

	|           | /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/TestFunctionalparallelMountCmdspecific-port3997279170/001:/mount-9p |                   |         |         |                     |                     |

	|           | --alsologtostderr -v=1 --port 46464                                                                                  |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:58 PST |                     |

	|           | -T /mount-9p | grep 9p                                                                                               |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST | 07 Mar 24 13:59 PST |

	|           | -T /mount-9p | grep 9p                                                                                               |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh -- ls                                                                                          | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST | 07 Mar 24 13:59 PST |

	|           | -la /mount-9p                                                                                                        |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh sudo                                                                                           | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | umount -f /mount-9p                                                                                                  |                   |         |         |                     |                     |

	| mount     | -p functional-836000                                                                                                 | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/TestFunctionalparallelMountCmdVerifyCleanup2577347136/001:/mount2   |                   |         |         |                     |                     |

	|           | --alsologtostderr -v=1                                                                                               |                   |         |         |                     |                     |

	| mount     | -p functional-836000                                                                                                 | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/TestFunctionalparallelMountCmdVerifyCleanup2577347136/001:/mount3   |                   |         |         |                     |                     |

	|           | --alsologtostderr -v=1                                                                                               |                   |         |         |                     |                     |

	| mount     | -p functional-836000                                                                                                 | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/TestFunctionalparallelMountCmdVerifyCleanup2577347136/001:/mount1   |                   |         |         |                     |                     |

	|           | --alsologtostderr -v=1                                                                                               |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | -T /mount1                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST | 07 Mar 24 13:59 PST |

	|           | -T /mount1                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST | 07 Mar 24 13:59 PST |

	|           | -T /mount2                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | -T /mount3                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST | 07 Mar 24 13:59 PST |

	|           | -T /mount1                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST | 07 Mar 24 13:59 PST |

	|           | -T /mount2                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | -T /mount3                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST | 07 Mar 24 13:59 PST |

	|           | -T /mount1                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST | 07 Mar 24 13:59 PST |

	|           | -T /mount2                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-836000 ssh findmnt                                                                                        | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST | 07 Mar 24 13:59 PST |

	|           | -T /mount3                                                                                                           |                   |         |         |                     |                     |

	| mount     | -p functional-836000                                                                                                 | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | --kill=true                                                                                                          |                   |         |         |                     |                     |

	| start     | -p functional-836000                                                                                                 | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | --dry-run --memory                                                                                                   |                   |         |         |                     |                     |

	|           | 250MB --alsologtostderr                                                                                              |                   |         |         |                     |                     |

	|           | --driver=qemu2                                                                                                       |                   |         |         |                     |                     |

	| start     | -p functional-836000                                                                                                 | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | --dry-run --memory                                                                                                   |                   |         |         |                     |                     |

	|           | 250MB --alsologtostderr                                                                                              |                   |         |         |                     |                     |

	|           | --driver=qemu2                                                                                                       |                   |         |         |                     |                     |

	| start     | -p functional-836000 --dry-run                                                                                       | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | --alsologtostderr -v=1                                                                                               |                   |         |         |                     |                     |

	|           | --driver=qemu2                                                                                                       |                   |         |         |                     |                     |

	| dashboard | --url --port 36195                                                                                                   | functional-836000 | jenkins | v1.32.0 | 07 Mar 24 13:59 PST |                     |

	|           | -p functional-836000                                                                                                 |                   |         |         |                     |                     |

	|           | --alsologtostderr -v=1                                                                                               |                   |         |         |                     |                     |

	|-----------|----------------------------------------------------------------------------------------------------------------------|-------------------|---------|---------|---------------------|---------------------|

	==> Last Start <==

	Log file created at: 2024/03/07 13:59:03

	Running on machine: MacOS-M1-Agent-1

	Binary: Built with gc go1.22.0 for darwin/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0307 13:59:03.594546    2396 out.go:291] Setting OutFile to fd 1 ...

	I0307 13:59:03.594919    2396 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 13:59:03.594925    2396 out.go:304] Setting ErrFile to fd 2...

	I0307 13:59:03.594928    2396 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 13:59:03.595107    2396 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 13:59:03.596485    2396 out.go:298] Setting JSON to false

	I0307 13:59:03.614010    2396 start.go:129] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":1700,"bootTime":1709847043,"procs":462,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.3.1","kernelVersion":"23.3.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0307 13:59:03.614072    2396 start.go:137] gopshost.Virtualization returned error: not implemented yet

	I0307 13:59:03.619306    2396 out.go:177] * [functional-836000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	I0307 13:59:03.626410    2396 out.go:177]   - MINIKUBE_LOCATION=18320

	I0307 13:59:03.626462    2396 notify.go:220] Checking for updates...

	I0307 13:59:03.633409    2396 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	I0307 13:59:03.636389    2396 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0307 13:59:03.639433    2396 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0307 13:59:03.646436    2396 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	I0307 13:59:03.653358    2396 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0307 13:59:03.657617    2396 config.go:182] Loaded profile config "functional-836000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 13:59:03.657870    2396 driver.go:392] Setting default libvirt URI to qemu:///system

	I0307 13:59:03.662269    2396 out.go:177] * Using the qemu2 driver based on existing profile

	I0307 13:59:03.669383    2396 start.go:297] selected driver: qemu2

	I0307 13:59:03.669389    2396 start.go:901] validating driver "qemu2" against &{Name:functional-836000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:4000 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8441 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{Kub

ernetesVersion:v1.28.4 ClusterName:functional-836000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:apiserver Key:enable-admission-plugins Value:NamespaceAutoProvision}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.105.4 Port:8441 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpirat

ion:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 13:59:03.669451    2396 start.go:912] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0307 13:59:03.671647    2396 cni.go:84] Creating CNI manager for ""

	I0307 13:59:03.671665    2396 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0307 13:59:03.671708    2396 start.go:340] cluster config:

	{Name:functional-836000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:4000 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8441 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:functional-836000 Namespace:default APIServe

rHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:apiserver Key:enable-admission-plugins Value:NamespaceAutoProvision}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.105.4 Port:8441 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2

000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 13:59:03.686377    2396 out.go:177] * dry-run validation complete!

	==> Docker <==

	Mar 07 21:58:57 functional-836000 dockerd[7034]: time="2024-03-07T21:58:57.868871090Z" level=info msg="ignoring event" container=4b3363a4604a9e6cbdd8259e99fa7fe9d91fa75e22d356e896b8b531f4b5da78 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Mar 07 21:58:57 functional-836000 dockerd[7040]: time="2024-03-07T21:58:57.868982465Z" level=warning msg="cleaning up after shim disconnected" id=4b3363a4604a9e6cbdd8259e99fa7fe9d91fa75e22d356e896b8b531f4b5da78 namespace=moby

	Mar 07 21:58:57 functional-836000 dockerd[7040]: time="2024-03-07T21:58:57.868992548Z" level=info msg="cleaning up dead shim" namespace=moby

	Mar 07 21:58:57 functional-836000 dockerd[7040]: time="2024-03-07T21:58:57.872930749Z" level=warning msg="cleanup warnings time=\"2024-03-07T21:58:57Z\" level=warning msg=\"failed to remove runc container\" error=\"runc did not terminate successfully: exit status 255: \" runtime=io.containerd.runc.v2\n" namespace=moby

	Mar 07 21:58:59 functional-836000 dockerd[7034]: time="2024-03-07T21:58:59.392947051Z" level=info msg="ignoring event" container=2632c5b685031f404350439dcc1c02fe4247869572b8a8413d7c73b8bd8e39b1 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Mar 07 21:58:59 functional-836000 dockerd[7040]: time="2024-03-07T21:58:59.393155551Z" level=info msg="shim disconnected" id=2632c5b685031f404350439dcc1c02fe4247869572b8a8413d7c73b8bd8e39b1 namespace=moby

	Mar 07 21:58:59 functional-836000 dockerd[7040]: time="2024-03-07T21:58:59.393210467Z" level=warning msg="cleaning up after shim disconnected" id=2632c5b685031f404350439dcc1c02fe4247869572b8a8413d7c73b8bd8e39b1 namespace=moby

	Mar 07 21:58:59 functional-836000 dockerd[7040]: time="2024-03-07T21:58:59.393215509Z" level=info msg="cleaning up dead shim" namespace=moby

	Mar 07 21:59:04 functional-836000 dockerd[7040]: time="2024-03-07T21:59:04.689255782Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Mar 07 21:59:04 functional-836000 dockerd[7040]: time="2024-03-07T21:59:04.689462740Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Mar 07 21:59:04 functional-836000 dockerd[7040]: time="2024-03-07T21:59:04.689492239Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Mar 07 21:59:04 functional-836000 dockerd[7040]: time="2024-03-07T21:59:04.689601864Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Mar 07 21:59:04 functional-836000 dockerd[7040]: time="2024-03-07T21:59:04.729275543Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Mar 07 21:59:04 functional-836000 dockerd[7040]: time="2024-03-07T21:59:04.729313918Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Mar 07 21:59:04 functional-836000 dockerd[7040]: time="2024-03-07T21:59:04.729319377Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Mar 07 21:59:04 functional-836000 dockerd[7040]: time="2024-03-07T21:59:04.729362168Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Mar 07 21:59:04 functional-836000 cri-dockerd[7296]: time="2024-03-07T21:59:04Z" level=info msg="Will attempt to re-write config file /var/lib/docker/containers/d9096f302b77258a73f00c68e1adb0bb17e96476f80a1d9d20a43e423f160ea5/resolv.conf as [nameserver 10.96.0.10 search kubernetes-dashboard.svc.cluster.local svc.cluster.local cluster.local options ndots:5]"

	Mar 07 21:59:04 functional-836000 cri-dockerd[7296]: time="2024-03-07T21:59:04Z" level=info msg="Will attempt to re-write config file /var/lib/docker/containers/def2a341c90bcd3a6674f4ee4da06a84d2e17114df104b93a18c2eb1d8a892bd/resolv.conf as [nameserver 10.96.0.10 search kubernetes-dashboard.svc.cluster.local svc.cluster.local cluster.local options ndots:5]"

	Mar 07 21:59:04 functional-836000 dockerd[7034]: time="2024-03-07T21:59:04.996452662Z" level=warning msg="reference for unknown type: " digest="sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c" remote="docker.io/kubernetesui/metrics-scraper@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c"

	Mar 07 21:59:06 functional-836000 cri-dockerd[7296]: time="2024-03-07T21:59:06Z" level=info msg="Stop pulling image docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c: Status: Downloaded newer image for kubernetesui/metrics-scraper@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c"

	Mar 07 21:59:06 functional-836000 dockerd[7040]: time="2024-03-07T21:59:06.511225743Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Mar 07 21:59:06 functional-836000 dockerd[7040]: time="2024-03-07T21:59:06.511271534Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Mar 07 21:59:06 functional-836000 dockerd[7040]: time="2024-03-07T21:59:06.511277451Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Mar 07 21:59:06 functional-836000 dockerd[7040]: time="2024-03-07T21:59:06.511304034Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Mar 07 21:59:06 functional-836000 dockerd[7034]: time="2024-03-07T21:59:06.658930023Z" level=warning msg="reference for unknown type: " digest="sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93" remote="docker.io/kubernetesui/dashboard@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93"

	==> container status <==

	CONTAINER           IMAGE                                                                                                  CREATED              STATE               NAME                        ATTEMPT             POD ID              POD

	3cb8089e490c5       kubernetesui/metrics-scraper@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c   1 second ago         Running             dashboard-metrics-scraper   0                   d9096f302b772       dashboard-metrics-scraper-7fd5cb4ddc-9mwzs

	4b3363a4604a9       72565bf5bbedf                                                                                          10 seconds ago       Exited              echoserver-arm              2                   4cce57943a740       hello-node-759d89bdcc-2htjx

	5cb93fe3154ca       gcr.io/k8s-minikube/busybox@sha256:2d03e6ceeb99250061dd110530b0ece7998cd84121f952adef120ea7c5a6f00e    10 seconds ago       Exited              mount-munger                0                   2632c5b685031       busybox-mount

	27ae7bcacdb2e       72565bf5bbedf                                                                                          15 seconds ago       Exited              echoserver-arm              2                   f5b57944596cc       hello-node-connect-7799dfb7c6-gfnrd

	3f6323894de01       nginx@sha256:c26ae7472d624ba1fafd296e73cecc4f93f853088e6a9c13c0d52f6ca5865107                          30 seconds ago       Running             myfrontend                  0                   7fcbee428c5ca       sp-pod

	8b2b7e7e8a201       nginx@sha256:6a2f8b28e45c4adea04ec207a251fd4a2df03ddc930f782af51e315ebc76e9a9                          47 seconds ago       Running             nginx                       0                   7aa42a6b645df       nginx-svc

	a8555009ff8d7       97e04611ad434                                                                                          About a minute ago   Running             coredns                     2                   d52171e46bed0       coredns-5dd5756b68-zrdj8

	6bf2cc1b4b869       ba04bb24b9575                                                                                          About a minute ago   Running             storage-provisioner         2                   6744a7ad8ca81       storage-provisioner

	659988e2b5e61       3ca3ca488cf13                                                                                          About a minute ago   Running             kube-proxy                  2                   e1ae96c6ede5a       kube-proxy-dx544

	7c54af5852907       9cdd6470f48c8                                                                                          About a minute ago   Running             etcd                        2                   335db5b520297       etcd-functional-836000

	305f47c4b3380       9961cbceaf234                                                                                          About a minute ago   Running             kube-controller-manager     2                   05d5f93cfd4a0       kube-controller-manager-functional-836000

	400735fe39787       05c284c929889                                                                                          About a minute ago   Running             kube-scheduler              2                   9f8fc1989491b       kube-scheduler-functional-836000

	65563d3fd4830       04b4c447bb9d4                                                                                          About a minute ago   Running             kube-apiserver              0                   59a031a7de65e       kube-apiserver-functional-836000

	45aa3aaee8a98       97e04611ad434                                                                                          2 minutes ago        Exited              coredns                     1                   a024b177c0c3d       coredns-5dd5756b68-zrdj8

	c3bf730870421       ba04bb24b9575                                                                                          2 minutes ago        Exited              storage-provisioner         1                   afae5b10ba0e2       storage-provisioner

	9f88217f5abc2       3ca3ca488cf13                                                                                          2 minutes ago        Exited              kube-proxy                  1                   3c000a3fdcb24       kube-proxy-dx544

	7890324dcf522       9cdd6470f48c8                                                                                          2 minutes ago        Exited              etcd                        1                   d4f05b36e8ed4       etcd-functional-836000

	7665b38b1ce37       05c284c929889                                                                                          2 minutes ago        Exited              kube-scheduler              1                   3016947ea2795       kube-scheduler-functional-836000

	50ef766f33a4a       9961cbceaf234                                                                                          2 minutes ago        Exited              kube-controller-manager     1                   ae46771d3bca6       kube-controller-manager-functional-836000

	==> coredns [45aa3aaee8a9] <==

	.:53

	[INFO] plugin/reload: Running configuration SHA512 = ea7a0d73d9d208f758b1f67640ef03c58089b9d9366cf3478df3bb369b210e39f213811b46224f8a04380814b6e0890ccd358f5b5e8c80bc22ac19c8601ee35b

	CoreDNS-1.10.1

	linux/arm64, go1.20, 055b2c3

	[INFO] 127.0.0.1:44139 - 36160 "HINFO IN 3452098514840508433.544491595962185437. udp 56 false 512" NXDOMAIN qr,rd,ra 131 0.009178901s

	[INFO] SIGTERM: Shutting down servers then terminating

	[INFO] plugin/health: Going into lameduck mode for 5s

	==> coredns [a8555009ff8d] <==

	.:53

	[INFO] plugin/reload: Running configuration SHA512 = ea7a0d73d9d208f758b1f67640ef03c58089b9d9366cf3478df3bb369b210e39f213811b46224f8a04380814b6e0890ccd358f5b5e8c80bc22ac19c8601ee35b

	CoreDNS-1.10.1

	linux/arm64, go1.20, 055b2c3

	[INFO] 127.0.0.1:33094 - 12343 "HINFO IN 3951182610231781618.7903230198375525754. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.009296936s

	[INFO] 10.244.0.1:25634 - 53950 "A IN nginx-svc.default.svc.cluster.local. udp 64 false 4096" NOERROR qr,aa,rd 104 0.000094875s

	[INFO] 10.244.0.1:17804 - 14647 "AAAA IN nginx-svc.default.svc.cluster.local. udp 53 false 512" NOERROR qr,aa,rd 146 0.000088875s

	[INFO] 10.244.0.1:26340 - 26852 "A IN nginx-svc.default.svc.cluster.local. udp 53 false 512" NOERROR qr,aa,rd 104 0.000022083s

	[INFO] 10.244.0.1:18252 - 3741 "SVCB IN _dns.resolver.arpa. udp 36 false 512" NXDOMAIN qr,rd,ra 116 0.000938082s

	[INFO] 10.244.0.1:59867 - 12308 "A IN nginx-svc.default.svc.cluster.local. udp 64 false 1232" NOERROR qr,aa,rd 104 0.00005075s

	[INFO] 10.244.0.1:30037 - 44446 "AAAA IN nginx-svc.default.svc.cluster.local. udp 64 false 1232" NOERROR qr,aa,rd 146 0.000054333s

	==> describe nodes <==

	Name:               functional-836000

	Roles:              control-plane

	Labels:             beta.kubernetes.io/arch=arm64

	                    beta.kubernetes.io/os=linux

	                    kubernetes.io/arch=arm64

	                    kubernetes.io/hostname=functional-836000

	                    kubernetes.io/os=linux

	                    minikube.k8s.io/commit=3e3656b8cff33aafa60dd2a07a4b34bce666a6a6

	                    minikube.k8s.io/name=functional-836000

	                    minikube.k8s.io/primary=true

	                    minikube.k8s.io/updated_at=2024_03_07T13_56_31_0700

	                    minikube.k8s.io/version=v1.32.0

	                    node-role.kubernetes.io/control-plane=

	                    node.kubernetes.io/exclude-from-external-load-balancers=

	Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock

	                    node.alpha.kubernetes.io/ttl: 0

	                    volumes.kubernetes.io/controller-managed-attach-detach: true

	CreationTimestamp:  Thu, 07 Mar 2024 21:56:28 +0000

	Taints:             <none>

	Unschedulable:      false

	Lease:

	  HolderIdentity:  functional-836000

	  AcquireTime:     <unset>

	  RenewTime:       Thu, 07 Mar 2024 21:59:06 +0000

	Conditions:

	  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message

	  ----             ------  -----------------                 ------------------                ------                       -------

	  MemoryPressure   False   Thu, 07 Mar 2024 21:58:54 +0000   Thu, 07 Mar 2024 21:56:28 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available

	  DiskPressure     False   Thu, 07 Mar 2024 21:58:54 +0000   Thu, 07 Mar 2024 21:56:28 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure

	  PIDPressure      False   Thu, 07 Mar 2024 21:58:54 +0000   Thu, 07 Mar 2024 21:56:28 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available

	  Ready            True    Thu, 07 Mar 2024 21:58:54 +0000   Thu, 07 Mar 2024 21:56:35 +0000   KubeletReady                 kubelet is posting ready status

	Addresses:

	  InternalIP:  192.168.105.4

	  Hostname:    functional-836000

	Capacity:

	  cpu:                2

	  ephemeral-storage:  17734596Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3904752Ki

	  pods:               110

	Allocatable:

	  cpu:                2

	  ephemeral-storage:  17734596Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3904752Ki

	  pods:               110

	System Info:

	  Machine ID:                 d2e65d77d49641e4a4a956ece2fea5ab

	  System UUID:                d2e65d77d49641e4a4a956ece2fea5ab

	  Boot ID:                    0042e42e-411e-40ab-a99b-9d4b0dd9908f

	  Kernel Version:             5.10.207

	  OS Image:                   Buildroot 2023.02.9

	  Operating System:           linux

	  Architecture:               arm64

	  Container Runtime Version:  docker://24.0.7

	  Kubelet Version:            v1.28.4

	  Kube-Proxy Version:         v1.28.4

	PodCIDR:                      10.244.0.0/24

	PodCIDRs:                     10.244.0.0/24

	Non-terminated Pods:          (13 in total)

	  Namespace                   Name                                          CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age

	  ---------                   ----                                          ------------  ----------  ---------------  -------------  ---

	  default                     hello-node-759d89bdcc-2htjx                   0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         24s

	  default                     hello-node-connect-7799dfb7c6-gfnrd           0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         39s

	  default                     nginx-svc                                     0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         50s

	  default                     sp-pod                                        0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         31s

	  kube-system                 coredns-5dd5756b68-zrdj8                      100m (5%!)(MISSING)     0 (0%!)(MISSING)      70Mi (1%!)(MISSING)        170Mi (4%!)(MISSING)     2m23s

	  kube-system                 etcd-functional-836000                        100m (5%!)(MISSING)     0 (0%!)(MISSING)      100Mi (2%!)(MISSING)       0 (0%!)(MISSING)         2m36s

	  kube-system                 kube-apiserver-functional-836000              250m (12%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         74s

	  kube-system                 kube-controller-manager-functional-836000     200m (10%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         2m36s

	  kube-system                 kube-proxy-dx544                              0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         2m24s

	  kube-system                 kube-scheduler-functional-836000              100m (5%!)(MISSING)     0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         2m36s

	  kube-system                 storage-provisioner                           0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         2m23s

	  kubernetes-dashboard        dashboard-metrics-scraper-7fd5cb4ddc-9mwzs    0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         3s

	  kubernetes-dashboard        kubernetes-dashboard-8694d4445c-scpwb         0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         3s

	Allocated resources:

	  (Total limits may be over 100 percent, i.e., overcommitted.)

	  Resource           Requests    Limits

	  --------           --------    ------

	  cpu                750m (37%!)(MISSING)  0 (0%!)(MISSING)

	  memory             170Mi (4%!)(MISSING)  170Mi (4%!)(MISSING)

	  ephemeral-storage  0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-1Gi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-2Mi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-32Mi     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-64Ki     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	Events:

	  Type    Reason                   Age                    From             Message

	  ----    ------                   ----                   ----             -------

	  Normal  Starting                 2m22s                  kube-proxy       

	  Normal  Starting                 72s                    kube-proxy       

	  Normal  Starting                 2m                     kube-proxy       

	  Normal  Starting                 2m41s                  kubelet          Starting kubelet.

	  Normal  NodeHasSufficientMemory  2m41s (x8 over 2m41s)  kubelet          Node functional-836000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    2m41s (x8 over 2m41s)  kubelet          Node functional-836000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     2m41s (x7 over 2m41s)  kubelet          Node functional-836000 status is now: NodeHasSufficientPID

	  Normal  NodeAllocatableEnforced  2m41s                  kubelet          Updated Node Allocatable limit across pods

	  Normal  NodeHasSufficientPID     2m36s                  kubelet          Node functional-836000 status is now: NodeHasSufficientPID

	  Normal  NodeAllocatableEnforced  2m36s                  kubelet          Updated Node Allocatable limit across pods

	  Normal  NodeHasSufficientMemory  2m36s                  kubelet          Node functional-836000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    2m36s                  kubelet          Node functional-836000 status is now: NodeHasNoDiskPressure

	  Normal  Starting                 2m36s                  kubelet          Starting kubelet.

	  Normal  NodeReady                2m32s                  kubelet          Node functional-836000 status is now: NodeReady

	  Normal  RegisteredNode           2m24s                  node-controller  Node functional-836000 event: Registered Node functional-836000 in Controller

	  Normal  NodeHasNoDiskPressure    2m4s (x8 over 2m4s)    kubelet          Node functional-836000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientMemory  2m4s (x8 over 2m4s)    kubelet          Node functional-836000 status is now: NodeHasSufficientMemory

	  Normal  Starting                 2m4s                   kubelet          Starting kubelet.

	  Normal  NodeHasSufficientPID     2m4s (x7 over 2m4s)    kubelet          Node functional-836000 status is now: NodeHasSufficientPID

	  Normal  NodeAllocatableEnforced  2m4s                   kubelet          Updated Node Allocatable limit across pods

	  Normal  RegisteredNode           109s                   node-controller  Node functional-836000 event: Registered Node functional-836000 in Controller

	  Normal  Starting                 77s                    kubelet          Starting kubelet.

	  Normal  NodeHasSufficientMemory  77s (x8 over 77s)      kubelet          Node functional-836000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    77s (x8 over 77s)      kubelet          Node functional-836000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     77s (x7 over 77s)      kubelet          Node functional-836000 status is now: NodeHasSufficientPID

	  Normal  NodeAllocatableEnforced  77s                    kubelet          Updated Node Allocatable limit across pods

	  Normal  RegisteredNode           62s                    node-controller  Node functional-836000 event: Registered Node functional-836000 in Controller

	==> dmesg <==

	[ +11.252878] kauditd_printk_skb: 27 callbacks suppressed

	[  +1.571281] systemd-fstab-generator[5932]: Ignoring "noauto" option for root device

	[ +17.997753] systemd-fstab-generator[6568]: Ignoring "noauto" option for root device

	[  +0.054983] kauditd_printk_skb: 14 callbacks suppressed

	[  +0.102650] systemd-fstab-generator[6602]: Ignoring "noauto" option for root device

	[  +0.102019] systemd-fstab-generator[6614]: Ignoring "noauto" option for root device

	[  +0.094819] systemd-fstab-generator[6628]: Ignoring "noauto" option for root device

	[  +5.101476] kauditd_printk_skb: 89 callbacks suppressed

	[  +6.354883] systemd-fstab-generator[7188]: Ignoring "noauto" option for root device

	[  +0.083288] systemd-fstab-generator[7200]: Ignoring "noauto" option for root device

	[  +0.082311] systemd-fstab-generator[7212]: Ignoring "noauto" option for root device

	[  +0.081199] systemd-fstab-generator[7232]: Ignoring "noauto" option for root device

	[  +0.216838] systemd-fstab-generator[7437]: Ignoring "noauto" option for root device

	[  +0.786101] systemd-fstab-generator[7555]: Ignoring "noauto" option for root device

	[  +4.359204] kauditd_printk_skb: 202 callbacks suppressed

	[Mar 7 21:58] kauditd_printk_skb: 27 callbacks suppressed

	[  +2.338172] systemd-fstab-generator[8741]: Ignoring "noauto" option for root device

	[  +4.421352] kauditd_printk_skb: 14 callbacks suppressed

	[  +5.232005] kauditd_printk_skb: 17 callbacks suppressed

	[  +5.942255] kauditd_printk_skb: 11 callbacks suppressed

	[ +11.393365] kauditd_printk_skb: 19 callbacks suppressed

	[  +7.933871] kauditd_printk_skb: 25 callbacks suppressed

	[  +8.458681] kauditd_printk_skb: 18 callbacks suppressed

	[  +5.762736] kauditd_printk_skb: 6 callbacks suppressed

	[Mar 7 21:59] kauditd_printk_skb: 7 callbacks suppressed

	==> etcd [7890324dcf52] <==

	{"level":"info","ts":"2024-03-07T21:57:04.15721Z","caller":"embed/etcd.go:569","msg":"cmux::serve","address":"192.168.105.4:2380"}

	{"level":"info","ts":"2024-03-07T21:57:05.343705Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 is starting a new election at term 2"}

	{"level":"info","ts":"2024-03-07T21:57:05.343852Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became pre-candidate at term 2"}

	{"level":"info","ts":"2024-03-07T21:57:05.343905Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 received MsgPreVoteResp from 7520ddf439b1d16 at term 2"}

	{"level":"info","ts":"2024-03-07T21:57:05.343942Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became candidate at term 3"}

	{"level":"info","ts":"2024-03-07T21:57:05.343959Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 received MsgVoteResp from 7520ddf439b1d16 at term 3"}

	{"level":"info","ts":"2024-03-07T21:57:05.343985Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became leader at term 3"}

	{"level":"info","ts":"2024-03-07T21:57:05.344015Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: 7520ddf439b1d16 elected leader 7520ddf439b1d16 at term 3"}

	{"level":"info","ts":"2024-03-07T21:57:05.348638Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2024-03-07T21:57:05.348655Z","caller":"etcdserver/server.go:2062","msg":"published local member to cluster through raft","local-member-id":"7520ddf439b1d16","local-member-attributes":"{Name:functional-836000 ClientURLs:[https://192.168.105.4:2379]}","request-path":"/0/members/7520ddf439b1d16/attributes","cluster-id":"80e92d98c466b02f","publish-timeout":"7s"}

	{"level":"info","ts":"2024-03-07T21:57:05.349432Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2024-03-07T21:57:05.351471Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.105.4:2379"}

	{"level":"info","ts":"2024-03-07T21:57:05.351481Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"127.0.0.1:2379"}

	{"level":"info","ts":"2024-03-07T21:57:05.351619Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}

	{"level":"info","ts":"2024-03-07T21:57:05.353302Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}

	{"level":"info","ts":"2024-03-07T21:57:38.054828Z","caller":"osutil/interrupt_unix.go:64","msg":"received signal; shutting down","signal":"terminated"}

	{"level":"info","ts":"2024-03-07T21:57:38.054854Z","caller":"embed/etcd.go:376","msg":"closing etcd server","name":"functional-836000","data-dir":"/var/lib/minikube/etcd","advertise-peer-urls":["https://192.168.105.4:2380"],"advertise-client-urls":["https://192.168.105.4:2379"]}

	{"level":"warn","ts":"2024-03-07T21:57:38.054925Z","caller":"embed/serve.go:212","msg":"stopping secure grpc server due to error","error":"accept tcp 127.0.0.1:2379: use of closed network connection"}

	{"level":"warn","ts":"2024-03-07T21:57:38.054966Z","caller":"embed/serve.go:214","msg":"stopped secure grpc server due to error","error":"accept tcp 127.0.0.1:2379: use of closed network connection"}

	{"level":"warn","ts":"2024-03-07T21:57:38.069595Z","caller":"embed/serve.go:212","msg":"stopping secure grpc server due to error","error":"accept tcp 192.168.105.4:2379: use of closed network connection"}

	{"level":"warn","ts":"2024-03-07T21:57:38.069686Z","caller":"embed/serve.go:214","msg":"stopped secure grpc server due to error","error":"accept tcp 192.168.105.4:2379: use of closed network connection"}

	{"level":"info","ts":"2024-03-07T21:57:38.069766Z","caller":"etcdserver/server.go:1465","msg":"skipped leadership transfer for single voting member cluster","local-member-id":"7520ddf439b1d16","current-leader-member-id":"7520ddf439b1d16"}

	{"level":"info","ts":"2024-03-07T21:57:38.071429Z","caller":"embed/etcd.go:579","msg":"stopping serving peer traffic","address":"192.168.105.4:2380"}

	{"level":"info","ts":"2024-03-07T21:57:38.071506Z","caller":"embed/etcd.go:584","msg":"stopped serving peer traffic","address":"192.168.105.4:2380"}

	{"level":"info","ts":"2024-03-07T21:57:38.071522Z","caller":"embed/etcd.go:378","msg":"closed etcd server","name":"functional-836000","data-dir":"/var/lib/minikube/etcd","advertise-peer-urls":["https://192.168.105.4:2380"],"advertise-client-urls":["https://192.168.105.4:2379"]}

	==> etcd [7c54af585290] <==

	{"level":"info","ts":"2024-03-07T21:57:52.197762Z","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"/var/lib/minikube/etcd/member/snap","suffix":"snap","max":5,"interval":"30s"}

	{"level":"info","ts":"2024-03-07T21:57:52.197784Z","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"/var/lib/minikube/etcd/member/wal","suffix":"wal","max":5,"interval":"30s"}

	{"level":"info","ts":"2024-03-07T21:57:52.19789Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 switched to configuration voters=(527499358918876438)"}

	{"level":"info","ts":"2024-03-07T21:57:52.197926Z","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"80e92d98c466b02f","local-member-id":"7520ddf439b1d16","added-peer-id":"7520ddf439b1d16","added-peer-peer-urls":["https://192.168.105.4:2380"]}

	{"level":"info","ts":"2024-03-07T21:57:52.197986Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"80e92d98c466b02f","local-member-id":"7520ddf439b1d16","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-07T21:57:52.198012Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-07T21:57:52.199394Z","caller":"embed/etcd.go:726","msg":"starting with client TLS","tls-info":"cert = /var/lib/minikube/certs/etcd/server.crt, key = /var/lib/minikube/certs/etcd/server.key, client-cert=, client-key=, trusted-ca = /var/lib/minikube/certs/etcd/ca.crt, client-cert-auth = true, crl-file = ","cipher-suites":[]}

	{"level":"info","ts":"2024-03-07T21:57:52.201231Z","caller":"embed/etcd.go:597","msg":"serving peer traffic","address":"192.168.105.4:2380"}

	{"level":"info","ts":"2024-03-07T21:57:52.202485Z","caller":"embed/etcd.go:569","msg":"cmux::serve","address":"192.168.105.4:2380"}

	{"level":"info","ts":"2024-03-07T21:57:52.202576Z","caller":"embed/etcd.go:855","msg":"serving metrics","address":"http://127.0.0.1:2381"}

	{"level":"info","ts":"2024-03-07T21:57:52.202606Z","caller":"embed/etcd.go:278","msg":"now serving peer/client/metrics","local-member-id":"7520ddf439b1d16","initial-advertise-peer-urls":["https://192.168.105.4:2380"],"listen-peer-urls":["https://192.168.105.4:2380"],"advertise-client-urls":["https://192.168.105.4:2379"],"listen-client-urls":["https://127.0.0.1:2379","https://192.168.105.4:2379"],"listen-metrics-urls":["http://127.0.0.1:2381"]}

	{"level":"info","ts":"2024-03-07T21:57:53.170555Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 is starting a new election at term 3"}

	{"level":"info","ts":"2024-03-07T21:57:53.170692Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became pre-candidate at term 3"}

	{"level":"info","ts":"2024-03-07T21:57:53.170737Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 received MsgPreVoteResp from 7520ddf439b1d16 at term 3"}

	{"level":"info","ts":"2024-03-07T21:57:53.170769Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became candidate at term 4"}

	{"level":"info","ts":"2024-03-07T21:57:53.170793Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 received MsgVoteResp from 7520ddf439b1d16 at term 4"}

	{"level":"info","ts":"2024-03-07T21:57:53.170835Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became leader at term 4"}

	{"level":"info","ts":"2024-03-07T21:57:53.170855Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: 7520ddf439b1d16 elected leader 7520ddf439b1d16 at term 4"}

	{"level":"info","ts":"2024-03-07T21:57:53.17562Z","caller":"etcdserver/server.go:2062","msg":"published local member to cluster through raft","local-member-id":"7520ddf439b1d16","local-member-attributes":"{Name:functional-836000 ClientURLs:[https://192.168.105.4:2379]}","request-path":"/0/members/7520ddf439b1d16/attributes","cluster-id":"80e92d98c466b02f","publish-timeout":"7s"}

	{"level":"info","ts":"2024-03-07T21:57:53.175696Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2024-03-07T21:57:53.175819Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}

	{"level":"info","ts":"2024-03-07T21:57:53.175845Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}

	{"level":"info","ts":"2024-03-07T21:57:53.175874Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2024-03-07T21:57:53.178434Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.105.4:2379"}

	{"level":"info","ts":"2024-03-07T21:57:53.179143Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"127.0.0.1:2379"}

	==> kernel <==

	 21:59:07 up 2 min,  0 users,  load average: 0.40, 0.27, 0.11

	Linux functional-836000 5.10.207 #1 SMP PREEMPT Thu Feb 22 23:40:42 UTC 2024 aarch64 GNU/Linux

	PRETTY_NAME="Buildroot 2023.02.9"

	==> kube-apiserver [65563d3fd483] <==

	I0307 21:57:53.841382       1 aggregator.go:166] initial CRD sync complete...

	I0307 21:57:53.841397       1 autoregister_controller.go:141] Starting autoregister controller

	I0307 21:57:53.841416       1 cache.go:32] Waiting for caches to sync for autoregister controller

	I0307 21:57:53.841429       1 cache.go:39] Caches are synced for autoregister controller

	I0307 21:57:53.841889       1 cache.go:39] Caches are synced for AvailableConditionController controller

	I0307 21:57:53.842385       1 apf_controller.go:377] Running API Priority and Fairness config worker

	I0307 21:57:53.842408       1 apf_controller.go:380] Running API Priority and Fairness periodic rebalancing process

	E0307 21:57:53.844455       1 controller.go:97] Error removing old endpoints from kubernetes service: no API server IP addresses were listed in storage, refusing to erase all endpoints for the kubernetes Service

	I0307 21:57:53.892682       1 shared_informer.go:318] Caches are synced for node_authorizer

	I0307 21:57:54.742100       1 storage_scheduling.go:111] all system priority classes are created successfully or already exist.

	I0307 21:57:54.870124       1 controller.go:624] quota admission added evaluator for: serviceaccounts

	I0307 21:57:54.873380       1 controller.go:624] quota admission added evaluator for: deployments.apps

	I0307 21:57:54.886711       1 controller.go:624] quota admission added evaluator for: daemonsets.apps

	I0307 21:57:54.898449       1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io

	I0307 21:57:54.900678       1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io

	I0307 21:58:06.029687       1 controller.go:624] quota admission added evaluator for: endpoints

	I0307 21:58:06.130547       1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io

	I0307 21:58:12.889282       1 alloc.go:330] "allocated clusterIPs" service="default/invalid-svc" clusterIPs={"IPv4":"10.102.92.133"}

	I0307 21:58:17.777684       1 alloc.go:330] "allocated clusterIPs" service="default/nginx-svc" clusterIPs={"IPv4":"10.103.241.157"}

	I0307 21:58:28.174316       1 controller.go:624] quota admission added evaluator for: replicasets.apps

	I0307 21:58:28.220046       1 alloc.go:330] "allocated clusterIPs" service="default/hello-node-connect" clusterIPs={"IPv4":"10.110.89.210"}

	I0307 21:58:43.391337       1 alloc.go:330] "allocated clusterIPs" service="default/hello-node" clusterIPs={"IPv4":"10.110.245.48"}

	I0307 21:59:04.268174       1 controller.go:624] quota admission added evaluator for: namespaces

	I0307 21:59:04.332743       1 alloc.go:330] "allocated clusterIPs" service="kubernetes-dashboard/kubernetes-dashboard" clusterIPs={"IPv4":"10.105.191.234"}

	I0307 21:59:04.367185       1 alloc.go:330] "allocated clusterIPs" service="kubernetes-dashboard/dashboard-metrics-scraper" clusterIPs={"IPv4":"10.109.175.24"}

	==> kube-controller-manager [305f47c4b338] <==

	E0307 21:59:04.303948       1 replica_set.go:557] sync "kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" failed with pods "dashboard-metrics-scraper-7fd5cb4ddc-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I0307 21:59:04.304051       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="8.776725ms"

	E0307 21:59:04.304063       1 replica_set.go:557] sync "kubernetes-dashboard/kubernetes-dashboard-8694d4445c" failed with pods "kubernetes-dashboard-8694d4445c-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I0307 21:59:04.306539       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="2.571451ms"

	E0307 21:59:04.306866       1 replica_set.go:557] sync "kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" failed with pods "dashboard-metrics-scraper-7fd5cb4ddc-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I0307 21:59:04.306854       1 event.go:307] "Event occurred" object="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"dashboard-metrics-scraper-7fd5cb4ddc-\" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount \"kubernetes-dashboard\" not found"

	I0307 21:59:04.307820       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="2.34991ms"

	E0307 21:59:04.307938       1 replica_set.go:557] sync "kubernetes-dashboard/kubernetes-dashboard-8694d4445c" failed with pods "kubernetes-dashboard-8694d4445c-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I0307 21:59:04.307930       1 event.go:307] "Event occurred" object="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"kubernetes-dashboard-8694d4445c-\" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount \"kubernetes-dashboard\" not found"

	I0307 21:59:04.310870       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="1.812162ms"

	E0307 21:59:04.310881       1 replica_set.go:557] sync "kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" failed with pods "dashboard-metrics-scraper-7fd5cb4ddc-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I0307 21:59:04.310893       1 event.go:307] "Event occurred" object="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"dashboard-metrics-scraper-7fd5cb4ddc-\" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount \"kubernetes-dashboard\" not found"

	I0307 21:59:04.312399       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="1.780204ms"

	E0307 21:59:04.312458       1 replica_set.go:557] sync "kubernetes-dashboard/kubernetes-dashboard-8694d4445c" failed with pods "kubernetes-dashboard-8694d4445c-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I0307 21:59:04.312450       1 event.go:307] "Event occurred" object="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"kubernetes-dashboard-8694d4445c-\" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount \"kubernetes-dashboard\" not found"

	I0307 21:59:04.333776       1 event.go:307] "Event occurred" object="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: dashboard-metrics-scraper-7fd5cb4ddc-9mwzs"

	I0307 21:59:04.337320       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="5.576359ms"

	I0307 21:59:04.349696       1 event.go:307] "Event occurred" object="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kubernetes-dashboard-8694d4445c-scpwb"

	I0307 21:59:04.363889       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="30.01804ms"

	I0307 21:59:04.365726       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="28.221837ms"

	I0307 21:59:04.365816       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="27.792µs"

	I0307 21:59:04.377378       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="13.383462ms"

	I0307 21:59:04.377402       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="12.125µs"

	I0307 21:59:07.381630       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="4.319245ms"

	I0307 21:59:07.381655       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="13µs"

	==> kube-controller-manager [50ef766f33a4] <==

	I0307 21:57:18.084497       1 shared_informer.go:318] Caches are synced for ReplicationController

	I0307 21:57:18.086665       1 shared_informer.go:318] Caches are synced for HPA

	I0307 21:57:18.088882       1 shared_informer.go:318] Caches are synced for taint

	I0307 21:57:18.088931       1 node_lifecycle_controller.go:1225] "Initializing eviction metric for zone" zone=""

	I0307 21:57:18.088934       1 taint_manager.go:205] "Starting NoExecuteTaintManager"

	I0307 21:57:18.088976       1 taint_manager.go:210] "Sending events to api server"

	I0307 21:57:18.088962       1 node_lifecycle_controller.go:877] "Missing timestamp for Node. Assuming now as a timestamp" node="functional-836000"

	I0307 21:57:18.089066       1 node_lifecycle_controller.go:1071] "Controller detected that zone is now in new state" zone="" newState="Normal"

	I0307 21:57:18.089116       1 shared_informer.go:318] Caches are synced for persistent volume

	I0307 21:57:18.089127       1 event.go:307] "Event occurred" object="functional-836000" fieldPath="" kind="Node" apiVersion="v1" type="Normal" reason="RegisteredNode" message="Node functional-836000 event: Registered Node functional-836000 in Controller"

	I0307 21:57:18.097647       1 shared_informer.go:318] Caches are synced for bootstrap_signer

	I0307 21:57:18.098810       1 shared_informer.go:318] Caches are synced for TTL

	I0307 21:57:18.099892       1 shared_informer.go:318] Caches are synced for expand

	I0307 21:57:18.100963       1 shared_informer.go:318] Caches are synced for service account

	I0307 21:57:18.103129       1 shared_informer.go:318] Caches are synced for stateful set

	I0307 21:57:18.150535       1 shared_informer.go:318] Caches are synced for attach detach

	I0307 21:57:18.187848       1 shared_informer.go:318] Caches are synced for disruption

	I0307 21:57:18.196054       1 shared_informer.go:318] Caches are synced for deployment

	I0307 21:57:18.198156       1 shared_informer.go:318] Caches are synced for ReplicaSet

	I0307 21:57:18.198226       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kube-system/coredns-5dd5756b68" duration="24.333µs"

	I0307 21:57:18.204205       1 shared_informer.go:318] Caches are synced for resource quota

	I0307 21:57:18.295783       1 shared_informer.go:318] Caches are synced for resource quota

	I0307 21:57:18.612951       1 shared_informer.go:318] Caches are synced for garbage collector

	I0307 21:57:18.650140       1 shared_informer.go:318] Caches are synced for garbage collector

	I0307 21:57:18.650152       1 garbagecollector.go:166] "All resource monitors have synced. Proceeding to collect garbage"

	==> kube-proxy [659988e2b5e6] <==

	I0307 21:57:55.149827       1 server_others.go:69] "Using iptables proxy"

	I0307 21:57:55.154254       1 node.go:141] Successfully retrieved node IP: 192.168.105.4

	I0307 21:57:55.162443       1 server_others.go:121] "No iptables support for family" ipFamily="IPv6"

	I0307 21:57:55.162455       1 server.go:634] "kube-proxy running in single-stack mode" ipFamily="IPv4"

	I0307 21:57:55.163075       1 server_others.go:152] "Using iptables Proxier"

	I0307 21:57:55.163095       1 proxier.go:251] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses"

	I0307 21:57:55.163163       1 server.go:846] "Version info" version="v1.28.4"

	I0307 21:57:55.163171       1 server.go:848] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0307 21:57:55.163616       1 config.go:188] "Starting service config controller"

	I0307 21:57:55.163652       1 shared_informer.go:311] Waiting for caches to sync for service config

	I0307 21:57:55.163676       1 config.go:97] "Starting endpoint slice config controller"

	I0307 21:57:55.163690       1 shared_informer.go:311] Waiting for caches to sync for endpoint slice config

	I0307 21:57:55.163855       1 config.go:315] "Starting node config controller"

	I0307 21:57:55.163875       1 shared_informer.go:311] Waiting for caches to sync for node config

	I0307 21:57:55.265093       1 shared_informer.go:318] Caches are synced for node config

	I0307 21:57:55.265093       1 shared_informer.go:318] Caches are synced for service config

	I0307 21:57:55.265101       1 shared_informer.go:318] Caches are synced for endpoint slice config

	==> kube-proxy [9f88217f5abc] <==

	I0307 21:57:06.977267       1 server_others.go:69] "Using iptables proxy"

	I0307 21:57:06.985717       1 node.go:141] Successfully retrieved node IP: 192.168.105.4

	I0307 21:57:07.007475       1 server_others.go:121] "No iptables support for family" ipFamily="IPv6"

	I0307 21:57:07.007486       1 server.go:634] "kube-proxy running in single-stack mode" ipFamily="IPv4"

	I0307 21:57:07.008509       1 server_others.go:152] "Using iptables Proxier"

	I0307 21:57:07.008608       1 proxier.go:251] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses"

	I0307 21:57:07.008762       1 server.go:846] "Version info" version="v1.28.4"

	I0307 21:57:07.008770       1 server.go:848] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0307 21:57:07.009131       1 config.go:188] "Starting service config controller"

	I0307 21:57:07.009143       1 shared_informer.go:311] Waiting for caches to sync for service config

	I0307 21:57:07.009150       1 config.go:97] "Starting endpoint slice config controller"

	I0307 21:57:07.009152       1 shared_informer.go:311] Waiting for caches to sync for endpoint slice config

	I0307 21:57:07.009516       1 config.go:315] "Starting node config controller"

	I0307 21:57:07.009736       1 shared_informer.go:311] Waiting for caches to sync for node config

	I0307 21:57:07.109917       1 shared_informer.go:318] Caches are synced for node config

	I0307 21:57:07.109939       1 shared_informer.go:318] Caches are synced for service config

	I0307 21:57:07.110030       1 shared_informer.go:318] Caches are synced for endpoint slice config

	==> kube-scheduler [400735fe3978] <==

	I0307 21:57:51.733259       1 serving.go:348] Generated self-signed cert in-memory

	W0307 21:57:53.764974       1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system.  Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'

	W0307 21:57:53.764989       1 authentication.go:368] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"

	W0307 21:57:53.764993       1 authentication.go:369] Continuing without authentication configuration. This may treat all requests as anonymous.

	W0307 21:57:53.765006       1 authentication.go:370] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false

	I0307 21:57:53.817849       1 server.go:154] "Starting Kubernetes Scheduler" version="v1.28.4"

	I0307 21:57:53.817891       1 server.go:156] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0307 21:57:53.818734       1 secure_serving.go:213] Serving securely on 127.0.0.1:10259

	I0307 21:57:53.818783       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"

	I0307 21:57:53.818797       1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	I0307 21:57:53.818805       1 tlsconfig.go:240] "Starting DynamicServingCertificateController"

	I0307 21:57:53.919895       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	==> kube-scheduler [7665b38b1ce3] <==

	I0307 21:57:04.442718       1 serving.go:348] Generated self-signed cert in-memory

	W0307 21:57:05.965506       1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system.  Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'

	W0307 21:57:05.965601       1 authentication.go:368] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"

	W0307 21:57:05.965636       1 authentication.go:369] Continuing without authentication configuration. This may treat all requests as anonymous.

	W0307 21:57:05.965652       1 authentication.go:370] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false

	I0307 21:57:05.983145       1 server.go:154] "Starting Kubernetes Scheduler" version="v1.28.4"

	I0307 21:57:05.983259       1 server.go:156] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0307 21:57:05.984293       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"

	I0307 21:57:05.984320       1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	I0307 21:57:05.984585       1 secure_serving.go:213] Serving securely on 127.0.0.1:10259

	I0307 21:57:05.984645       1 tlsconfig.go:240] "Starting DynamicServingCertificateController"

	I0307 21:57:06.084829       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	I0307 21:57:38.069173       1 secure_serving.go:258] Stopped listening on 127.0.0.1:10259

	I0307 21:57:38.069220       1 tlsconfig.go:255] "Shutting down DynamicServingCertificateController"

	E0307 21:57:38.069283       1 run.go:74] "command failed" err="finished without leader elect"

	==> kubelet <==

	Mar 07 21:58:53 functional-836000 kubelet[7562]: I0307 21:58:53.280059    7562 scope.go:117] "RemoveContainer" containerID="5eed9a8372ade809c5b1a95a618675e4bec60daec1a587a3ae756099d255338a"

	Mar 07 21:58:53 functional-836000 kubelet[7562]: I0307 21:58:53.280225    7562 scope.go:117] "RemoveContainer" containerID="27ae7bcacdb2e20876c2ff4fca003a402e7f8657ab05a11d21238d1f11eabf2b"

	Mar 07 21:58:53 functional-836000 kubelet[7562]: E0307 21:58:53.280321    7562 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"echoserver-arm\" with CrashLoopBackOff: \"back-off 20s restarting failed container=echoserver-arm pod=hello-node-connect-7799dfb7c6-gfnrd_default(ec5c5794-652a-4971-b709-4333f321052d)\"" pod="default/hello-node-connect-7799dfb7c6-gfnrd" podUID="ec5c5794-652a-4971-b709-4333f321052d"

	Mar 07 21:58:57 functional-836000 kubelet[7562]: I0307 21:58:57.818034    7562 scope.go:117] "RemoveContainer" containerID="b1bcb645e4d7b4ad6c3bb75e1172967faf7713e7ddf3b10a095d3feb04720236"

	Mar 07 21:58:58 functional-836000 kubelet[7562]: I0307 21:58:58.314499    7562 scope.go:117] "RemoveContainer" containerID="b1bcb645e4d7b4ad6c3bb75e1172967faf7713e7ddf3b10a095d3feb04720236"

	Mar 07 21:58:58 functional-836000 kubelet[7562]: I0307 21:58:58.314601    7562 scope.go:117] "RemoveContainer" containerID="4b3363a4604a9e6cbdd8259e99fa7fe9d91fa75e22d356e896b8b531f4b5da78"

	Mar 07 21:58:58 functional-836000 kubelet[7562]: E0307 21:58:58.315994    7562 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"echoserver-arm\" with CrashLoopBackOff: \"back-off 20s restarting failed container=echoserver-arm pod=hello-node-759d89bdcc-2htjx_default(18ac08df-803f-4ddd-94c1-c13993ed3531)\"" pod="default/hello-node-759d89bdcc-2htjx" podUID="18ac08df-803f-4ddd-94c1-c13993ed3531"

	Mar 07 21:58:59 functional-836000 kubelet[7562]: I0307 21:58:59.560966    7562 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79hsf\" (UniqueName: \"kubernetes.io/projected/e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2-kube-api-access-79hsf\") pod \"e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2\" (UID: \"e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2\") "

	Mar 07 21:58:59 functional-836000 kubelet[7562]: I0307 21:58:59.561003    7562 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"test-volume\" (UniqueName: \"kubernetes.io/host-path/e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2-test-volume\") pod \"e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2\" (UID: \"e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2\") "

	Mar 07 21:58:59 functional-836000 kubelet[7562]: I0307 21:58:59.561054    7562 operation_generator.go:882] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2-test-volume" (OuterVolumeSpecName: "test-volume") pod "e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2" (UID: "e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2"). InnerVolumeSpecName "test-volume". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Mar 07 21:58:59 functional-836000 kubelet[7562]: I0307 21:58:59.564166    7562 operation_generator.go:882] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2-kube-api-access-79hsf" (OuterVolumeSpecName: "kube-api-access-79hsf") pod "e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2" (UID: "e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2"). InnerVolumeSpecName "kube-api-access-79hsf". PluginName "kubernetes.io/projected", VolumeGidValue ""

	Mar 07 21:58:59 functional-836000 kubelet[7562]: I0307 21:58:59.661644    7562 reconciler_common.go:300] "Volume detached for volume \"test-volume\" (UniqueName: \"kubernetes.io/host-path/e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2-test-volume\") on node \"functional-836000\" DevicePath \"\""

	Mar 07 21:58:59 functional-836000 kubelet[7562]: I0307 21:58:59.661663    7562 reconciler_common.go:300] "Volume detached for volume \"kube-api-access-79hsf\" (UniqueName: \"kubernetes.io/projected/e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2-kube-api-access-79hsf\") on node \"functional-836000\" DevicePath \"\""

	Mar 07 21:59:00 functional-836000 kubelet[7562]: I0307 21:59:00.332854    7562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2632c5b685031f404350439dcc1c02fe4247869572b8a8413d7c73b8bd8e39b1"

	Mar 07 21:59:03 functional-836000 kubelet[7562]: I0307 21:59:03.818092    7562 scope.go:117] "RemoveContainer" containerID="27ae7bcacdb2e20876c2ff4fca003a402e7f8657ab05a11d21238d1f11eabf2b"

	Mar 07 21:59:03 functional-836000 kubelet[7562]: E0307 21:59:03.818191    7562 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"echoserver-arm\" with CrashLoopBackOff: \"back-off 20s restarting failed container=echoserver-arm pod=hello-node-connect-7799dfb7c6-gfnrd_default(ec5c5794-652a-4971-b709-4333f321052d)\"" pod="default/hello-node-connect-7799dfb7c6-gfnrd" podUID="ec5c5794-652a-4971-b709-4333f321052d"

	Mar 07 21:59:04 functional-836000 kubelet[7562]: I0307 21:59:04.341272    7562 topology_manager.go:215] "Topology Admit Handler" podUID="ad9e6364-f6cb-4b86-9b2c-9b02771baa9e" podNamespace="kubernetes-dashboard" podName="dashboard-metrics-scraper-7fd5cb4ddc-9mwzs"

	Mar 07 21:59:04 functional-836000 kubelet[7562]: E0307 21:59:04.341311    7562 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2" containerName="mount-munger"

	Mar 07 21:59:04 functional-836000 kubelet[7562]: I0307 21:59:04.341328    7562 memory_manager.go:346] "RemoveStaleState removing state" podUID="e2fa99f7-89f2-4ed9-b7d2-d7f938efa6e2" containerName="mount-munger"

	Mar 07 21:59:04 functional-836000 kubelet[7562]: I0307 21:59:04.362678    7562 topology_manager.go:215] "Topology Admit Handler" podUID="29d360b2-85e0-45ce-93f4-93b0e9c1e75d" podNamespace="kubernetes-dashboard" podName="kubernetes-dashboard-8694d4445c-scpwb"

	Mar 07 21:59:04 functional-836000 kubelet[7562]: I0307 21:59:04.483923    7562 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnqqn\" (UniqueName: \"kubernetes.io/projected/29d360b2-85e0-45ce-93f4-93b0e9c1e75d-kube-api-access-fnqqn\") pod \"kubernetes-dashboard-8694d4445c-scpwb\" (UID: \"29d360b2-85e0-45ce-93f4-93b0e9c1e75d\") " pod="kubernetes-dashboard/kubernetes-dashboard-8694d4445c-scpwb"

	Mar 07 21:59:04 functional-836000 kubelet[7562]: I0307 21:59:04.483972    7562 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad9e6364-f6cb-4b86-9b2c-9b02771baa9e-tmp-volume\") pod \"dashboard-metrics-scraper-7fd5cb4ddc-9mwzs\" (UID: \"ad9e6364-f6cb-4b86-9b2c-9b02771baa9e\") " pod="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc-9mwzs"

	Mar 07 21:59:04 functional-836000 kubelet[7562]: I0307 21:59:04.483986    7562 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr9qr\" (UniqueName: \"kubernetes.io/projected/ad9e6364-f6cb-4b86-9b2c-9b02771baa9e-kube-api-access-pr9qr\") pod \"dashboard-metrics-scraper-7fd5cb4ddc-9mwzs\" (UID: \"ad9e6364-f6cb-4b86-9b2c-9b02771baa9e\") " pod="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc-9mwzs"

	Mar 07 21:59:04 functional-836000 kubelet[7562]: I0307 21:59:04.483996    7562 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-volume\" (UniqueName: \"kubernetes.io/empty-dir/29d360b2-85e0-45ce-93f4-93b0e9c1e75d-tmp-volume\") pod \"kubernetes-dashboard-8694d4445c-scpwb\" (UID: \"29d360b2-85e0-45ce-93f4-93b0e9c1e75d\") " pod="kubernetes-dashboard/kubernetes-dashboard-8694d4445c-scpwb"

	Mar 07 21:59:07 functional-836000 kubelet[7562]: I0307 21:59:07.379439    7562 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc-9mwzs" podStartSLOduration=1.71393632 podCreationTimestamp="2024-03-07 21:59:04 +0000 UTC" firstStartedPulling="2024-03-07 21:59:04.7768897 +0000 UTC m=+74.038608373" lastFinishedPulling="2024-03-07 21:59:06.442366034 +0000 UTC m=+75.704084665" observedRunningTime="2024-03-07 21:59:07.377531198 +0000 UTC m=+76.639249829" watchObservedRunningTime="2024-03-07 21:59:07.379412612 +0000 UTC m=+76.641131243"

	==> storage-provisioner [6bf2cc1b4b86] <==

	I0307 21:57:55.113693       1 storage_provisioner.go:116] Initializing the minikube storage provisioner...

	I0307 21:57:55.120640       1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!

	I0307 21:57:55.120744       1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...

	I0307 21:58:12.512025       1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath

	I0307 21:58:12.512088       1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_functional-836000_b3a679e2-da4c-4c89-8ac9-f831011422cc!

	I0307 21:58:12.512272       1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"fc56af94-8a32-473a-a767-c4f3ed94a534", APIVersion:"v1", ResourceVersion:"616", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' functional-836000_b3a679e2-da4c-4c89-8ac9-f831011422cc became leader

	I0307 21:58:12.613237       1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_functional-836000_b3a679e2-da4c-4c89-8ac9-f831011422cc!

	I0307 21:58:23.631655       1 controller.go:1332] provision "default/myclaim" class "standard": started

	I0307 21:58:23.631726       1 storage_provisioner.go:61] Provisioning volume {&StorageClass{ObjectMeta:{standard    49e6fe59-9d53-43a2-8056-9ef788e6ff43 370 0 2024-03-07 21:56:44 +0000 UTC <nil> <nil> map[addonmanager.kubernetes.io/mode:EnsureExists] map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"},"labels":{"addonmanager.kubernetes.io/mode":"EnsureExists"},"name":"standard"},"provisioner":"k8s.io/minikube-hostpath"}

	 storageclass.kubernetes.io/is-default-class:true] [] []  [{kubectl-client-side-apply Update storage.k8s.io/v1 2024-03-07 21:56:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{},"f:storageclass.kubernetes.io/is-default-class":{}},"f:labels":{".":{},"f:addonmanager.kubernetes.io/mode":{}}},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}}}]},Provisioner:k8s.io/minikube-hostpath,Parameters:map[string]string{},ReclaimPolicy:*Delete,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*Immediate,AllowedTopologies:[]TopologySelectorTerm{},} pvc-c08b2c59-b984-489e-8024-a2b04a4ca4cb &PersistentVolumeClaim{ObjectMeta:{myclaim  default  c08b2c59-b984-489e-8024-a2b04a4ca4cb 675 0 2024-03-07 21:58:23 +0000 UTC <nil> <nil> map[] map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"name":"myclaim","namespace":"default"},"spec":{"accessModes":["Rea

dWriteOnce"],"resources":{"requests":{"storage":"500Mi"}},"volumeMode":"Filesystem"}}

	 volume.beta.kubernetes.io/storage-provisioner:k8s.io/minikube-hostpath volume.kubernetes.io/storage-provisioner:k8s.io/minikube-hostpath] [] [kubernetes.io/pvc-protection]  [{kube-controller-manager Update v1 2024-03-07 21:58:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}}}} {kubectl-client-side-apply Update v1 2024-03-07 21:58:23 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}}}}]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{524288000 0} {<nil>} 500Mi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*standard,VolumeMode:*Filesystem,DataSource:nil,},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:

ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} nil} to /tmp/hostpath-provisioner/default/myclaim

	I0307 21:58:23.632141       1 controller.go:1439] provision "default/myclaim" class "standard": volume "pvc-c08b2c59-b984-489e-8024-a2b04a4ca4cb" provisioned

	I0307 21:58:23.632160       1 controller.go:1456] provision "default/myclaim" class "standard": succeeded

	I0307 21:58:23.632176       1 volume_store.go:212] Trying to save persistentvolume "pvc-c08b2c59-b984-489e-8024-a2b04a4ca4cb"

	I0307 21:58:23.632714       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"myclaim", UID:"c08b2c59-b984-489e-8024-a2b04a4ca4cb", APIVersion:"v1", ResourceVersion:"675", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "default/myclaim"

	I0307 21:58:23.636575       1 volume_store.go:219] persistentvolume "pvc-c08b2c59-b984-489e-8024-a2b04a4ca4cb" saved

	I0307 21:58:23.636784       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"myclaim", UID:"c08b2c59-b984-489e-8024-a2b04a4ca4cb", APIVersion:"v1", ResourceVersion:"675", FieldPath:""}): type: 'Normal' reason: 'ProvisioningSucceeded' Successfully provisioned volume pvc-c08b2c59-b984-489e-8024-a2b04a4ca4cb

	==> storage-provisioner [c3bf73087042] <==

	I0307 21:57:06.945775       1 storage_provisioner.go:116] Initializing the minikube storage provisioner...

	I0307 21:57:06.954186       1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!

	I0307 21:57:06.954208       1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...

	I0307 21:57:24.345145       1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath

	I0307 21:57:24.345242       1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_functional-836000_fc216c22-9cbd-44d2-b629-e97431740c63!

	I0307 21:57:24.345430       1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"fc56af94-8a32-473a-a767-c4f3ed94a534", APIVersion:"v1", ResourceVersion:"509", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' functional-836000_fc216c22-9cbd-44d2-b629-e97431740c63 became leader

	I0307 21:57:24.446045       1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_functional-836000_fc216c22-9cbd-44d2-b629-e97431740c63!

-- /stdout --

helpers_test.go:254: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.APIServer}} -p functional-836000 -n functional-836000

helpers_test.go:261: (dbg) Run:  kubectl --context functional-836000 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running

helpers_test.go:272: non-running pods: busybox-mount kubernetes-dashboard-8694d4445c-scpwb

helpers_test.go:274: ======> post-mortem[TestFunctional/parallel/ServiceCmdConnect]: describe non-running pods <======

helpers_test.go:277: (dbg) Run:  kubectl --context functional-836000 describe pod busybox-mount kubernetes-dashboard-8694d4445c-scpwb

helpers_test.go:277: (dbg) Non-zero exit: kubectl --context functional-836000 describe pod busybox-mount kubernetes-dashboard-8694d4445c-scpwb: exit status 1 (47.854125ms)

-- stdout --

	Name:             busybox-mount

	Namespace:        default

	Priority:         0

	Service Account:  default

	Node:             functional-836000/192.168.105.4

	Start Time:       Thu, 07 Mar 2024 13:58:51 -0800

	Labels:           integration-test=busybox-mount

	Annotations:      <none>

	Status:           Succeeded

	IP:               10.244.0.11

	IPs:

	  IP:  10.244.0.11

	Containers:

	  mount-munger:

	    Container ID:  docker://5cb93fe3154ca9beab350980f6f79fe64d1ce074dbc5f93778afb96bfa7520c0

	    Image:         gcr.io/k8s-minikube/busybox:1.28.4-glibc

	    Image ID:      docker-pullable://gcr.io/k8s-minikube/busybox@sha256:2d03e6ceeb99250061dd110530b0ece7998cd84121f952adef120ea7c5a6f00e

	    Port:          <none>

	    Host Port:     <none>

	    Command:

	      /bin/sh

	      -c

	      --

	    Args:

	      cat /mount-9p/created-by-test; echo test > /mount-9p/created-by-pod; rm /mount-9p/created-by-test-removed-by-pod; echo test > /mount-9p/created-by-pod-removed-by-test date >> /mount-9p/pod-dates

	    State:          Terminated

	      Reason:       Completed

	      Exit Code:    0

	      Started:      Thu, 07 Mar 2024 13:58:57 -0800

	      Finished:     Thu, 07 Mar 2024 13:58:57 -0800

	    Ready:          False

	    Restart Count:  0

	    Environment:    <none>

	    Mounts:

	      /mount-9p from test-volume (rw)

	      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-79hsf (ro)

	Conditions:

	  Type              Status

	  Initialized       True 

	  Ready             False 

	  ContainersReady   False 

	  PodScheduled      True 

	Volumes:

	  test-volume:

	    Type:          HostPath (bare host directory volume)

	    Path:          /mount-9p

	    HostPathType:  

	  kube-api-access-79hsf:

	    Type:                    Projected (a volume that contains injected data from multiple sources)

	    TokenExpirationSeconds:  3607

	    ConfigMapName:           kube-root-ca.crt

	    ConfigMapOptional:       <nil>

	    DownwardAPI:             true

	QoS Class:                   BestEffort

	Node-Selectors:              <none>

	Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s

	                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s

	Events:

	  Type    Reason     Age   From               Message

	  ----    ------     ----  ----               -------

	  Normal  Scheduled  16s   default-scheduler  Successfully assigned default/busybox-mount to functional-836000

	  Normal  Pulling    16s   kubelet            Pulling image "gcr.io/k8s-minikube/busybox:1.28.4-glibc"

	  Normal  Pulled     10s   kubelet            Successfully pulled image "gcr.io/k8s-minikube/busybox:1.28.4-glibc" in 5.671s (5.671s including waiting)

	  Normal  Created    10s   kubelet            Created container mount-munger

	  Normal  Started    10s   kubelet            Started container mount-munger

-- /stdout --

** stderr ** 

	Error from server (NotFound): pods "kubernetes-dashboard-8694d4445c-scpwb" not found

** /stderr **

helpers_test.go:279: kubectl --context functional-836000 describe pod busybox-mount kubernetes-dashboard-8694d4445c-scpwb: exit status 1

--- FAIL: TestFunctional/parallel/ServiceCmdConnect (39.67s)

=== RUN   TestMutliControlPlane/serial/StopSecondaryNode

ha_test.go:363: (dbg) Run:  out/minikube-darwin-arm64 -p ha-827000 node stop m02 -v=7 --alsologtostderr

ha_test.go:363: (dbg) Done: out/minikube-darwin-arm64 -p ha-827000 node stop m02 -v=7 --alsologtostderr: (12.195585834s)

ha_test.go:369: (dbg) Run:  out/minikube-darwin-arm64 -p ha-827000 status -v=7 --alsologtostderr

E0307 14:12:13.852628    1596 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.crt: no such file or directory

ha_test.go:369: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p ha-827000 status -v=7 --alsologtostderr: exit status 7 (3m45.047441042s)

-- stdout --

	ha-827000

	type: Control Plane

	host: Error

	kubelet: Nonexistent

	apiserver: Nonexistent

	kubeconfig: Configured

	ha-827000-m02

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

	ha-827000-m03

	type: Control Plane

	host: Error

	kubelet: Nonexistent

	apiserver: Nonexistent

	kubeconfig: Configured

	ha-827000-m04

	type: Worker

	host: Error

	kubelet: Nonexistent

-- /stdout --

** stderr ** 

	I0307 14:09:13.587780    2967 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:09:13.588339    2967 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:09:13.588345    2967 out.go:304] Setting ErrFile to fd 2...

	I0307 14:09:13.588347    2967 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:09:13.588528    2967 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:09:13.588683    2967 out.go:298] Setting JSON to false

	I0307 14:09:13.588700    2967 mustload.go:65] Loading cluster: ha-827000

	I0307 14:09:13.588939    2967 notify.go:220] Checking for updates...

	I0307 14:09:13.589535    2967 config.go:182] Loaded profile config "ha-827000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:09:13.589544    2967 status.go:255] checking status of ha-827000 ...

	I0307 14:09:13.590343    2967 status.go:330] ha-827000 host status = "Running" (err=<nil>)

	I0307 14:09:13.590354    2967 host.go:66] Checking if "ha-827000" exists ...

	I0307 14:09:13.590461    2967 host.go:66] Checking if "ha-827000" exists ...

	I0307 14:09:13.590574    2967 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0307 14:09:13.590583    2967 sshutil.go:53] new ssh client: &{IP:192.168.105.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000/id_rsa Username:docker}

	W0307 14:10:28.590714    2967 sshutil.go:64] dial failure (will retry): dial tcp 192.168.105.5:22: connect: operation timed out

	W0307 14:10:28.591007    2967 start.go:268] error running df -h /var: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

	E0307 14:10:28.591019    2967 status.go:376] failed to get storage capacity of /var: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

	I0307 14:10:28.591024    2967 status.go:257] ha-827000 status: &{Name:ha-827000 Host:Error Kubelet:Nonexistent APIServer:Nonexistent Kubeconfig:Configured Worker:false TimeToStop: DockerEnv: PodManEnv:}

	E0307 14:10:28.591040    2967 status.go:260] status error: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

	I0307 14:10:28.591044    2967 status.go:255] checking status of ha-827000-m02 ...

	I0307 14:10:28.591258    2967 status.go:330] ha-827000-m02 host status = "Stopped" (err=<nil>)

	I0307 14:10:28.591264    2967 status.go:343] host is not running, skipping remaining checks

	I0307 14:10:28.591266    2967 status.go:257] ha-827000-m02 status: &{Name:ha-827000-m02 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

	I0307 14:10:28.591270    2967 status.go:255] checking status of ha-827000-m03 ...

	I0307 14:10:28.592058    2967 status.go:330] ha-827000-m03 host status = "Running" (err=<nil>)

	I0307 14:10:28.592066    2967 host.go:66] Checking if "ha-827000-m03" exists ...

	I0307 14:10:28.592177    2967 host.go:66] Checking if "ha-827000-m03" exists ...

	I0307 14:10:28.592292    2967 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0307 14:10:28.592298    2967 sshutil.go:53] new ssh client: &{IP:192.168.105.7 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m03/id_rsa Username:docker}

	W0307 14:11:43.591117    2967 sshutil.go:64] dial failure (will retry): dial tcp 192.168.105.7:22: connect: operation timed out

	W0307 14:11:43.591161    2967 start.go:268] error running df -h /var: NewSession: new client: new client: dial tcp 192.168.105.7:22: connect: operation timed out

	E0307 14:11:43.591169    2967 status.go:376] failed to get storage capacity of /var: NewSession: new client: new client: dial tcp 192.168.105.7:22: connect: operation timed out

	I0307 14:11:43.591173    2967 status.go:257] ha-827000-m03 status: &{Name:ha-827000-m03 Host:Error Kubelet:Nonexistent APIServer:Nonexistent Kubeconfig:Configured Worker:false TimeToStop: DockerEnv: PodManEnv:}

	E0307 14:11:43.591182    2967 status.go:260] status error: NewSession: new client: new client: dial tcp 192.168.105.7:22: connect: operation timed out

	I0307 14:11:43.591185    2967 status.go:255] checking status of ha-827000-m04 ...

	I0307 14:11:43.591944    2967 status.go:330] ha-827000-m04 host status = "Running" (err=<nil>)

	I0307 14:11:43.591951    2967 host.go:66] Checking if "ha-827000-m04" exists ...

	I0307 14:11:43.592039    2967 host.go:66] Checking if "ha-827000-m04" exists ...

	I0307 14:11:43.592149    2967 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0307 14:11:43.592155    2967 sshutil.go:53] new ssh client: &{IP:192.168.105.8 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m04/id_rsa Username:docker}

	W0307 14:12:58.592159    2967 sshutil.go:64] dial failure (will retry): dial tcp 192.168.105.8:22: connect: operation timed out

	W0307 14:12:58.592297    2967 start.go:268] error running df -h /var: NewSession: new client: new client: dial tcp 192.168.105.8:22: connect: operation timed out

	E0307 14:12:58.592323    2967 status.go:376] failed to get storage capacity of /var: NewSession: new client: new client: dial tcp 192.168.105.8:22: connect: operation timed out

	I0307 14:12:58.592340    2967 status.go:257] ha-827000-m04 status: &{Name:ha-827000-m04 Host:Error Kubelet:Nonexistent APIServer:Irrelevant Kubeconfig:Irrelevant Worker:true TimeToStop: DockerEnv: PodManEnv:}

	E0307 14:12:58.592380    2967 status.go:260] status error: NewSession: new client: new client: dial tcp 192.168.105.8:22: connect: operation timed out

** /stderr **

ha_test.go:378: status says not three hosts are running: args "out/minikube-darwin-arm64 -p ha-827000 status -v=7 --alsologtostderr": ha-827000

type: Control Plane

host: Error

kubelet: Nonexistent

apiserver: Nonexistent

kubeconfig: Configured

ha-827000-m02

type: Control Plane

host: Stopped

kubelet: Stopped

apiserver: Stopped

kubeconfig: Stopped

ha-827000-m03

type: Control Plane

host: Error

kubelet: Nonexistent

apiserver: Nonexistent

kubeconfig: Configured

ha-827000-m04

type: Worker

host: Error

kubelet: Nonexistent

ha_test.go:381: status says not three kubelets are running: args "out/minikube-darwin-arm64 -p ha-827000 status -v=7 --alsologtostderr": ha-827000

type: Control Plane

host: Error

kubelet: Nonexistent

apiserver: Nonexistent

kubeconfig: Configured

ha-827000-m02

type: Control Plane

host: Stopped

kubelet: Stopped

apiserver: Stopped

kubeconfig: Stopped

ha-827000-m03

type: Control Plane

host: Error

kubelet: Nonexistent

apiserver: Nonexistent

kubeconfig: Configured

ha-827000-m04

type: Worker

host: Error

kubelet: Nonexistent

ha_test.go:384: status says not two apiservers are running: args "out/minikube-darwin-arm64 -p ha-827000 status -v=7 --alsologtostderr": ha-827000

type: Control Plane

host: Error

kubelet: Nonexistent

apiserver: Nonexistent

kubeconfig: Configured

ha-827000-m02

type: Control Plane

host: Stopped

kubelet: Stopped

apiserver: Stopped

kubeconfig: Stopped

ha-827000-m03

type: Control Plane

host: Error

kubelet: Nonexistent

apiserver: Nonexistent

kubeconfig: Configured

ha-827000-m04

type: Worker

host: Error

kubelet: Nonexistent

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p ha-827000 -n ha-827000

E0307 14:13:17.361637    1596 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/functional-836000/client.crt: no such file or directory

E0307 14:13:36.918472    1596 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.crt: no such file or directory

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p ha-827000 -n ha-827000: exit status 3 (1m15.069638334s)

-- stdout --

	Error

-- /stdout --

** stderr ** 

	E0307 14:14:13.662344    3001 status.go:376] failed to get storage capacity of /var: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

	E0307 14:14:13.662370    3001 status.go:249] status error: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

** /stderr **

helpers_test.go:239: status error: exit status 3 (may be ok)

helpers_test.go:241: "ha-827000" host is not running, skipping log retrieval (state="Error")

--- FAIL: TestMutliControlPlane/serial/StopSecondaryNode (312.31s)

=== RUN   TestMutliControlPlane/serial/DegradedAfterControlPlaneNodeStop

ha_test.go:390: (dbg) Run:  out/minikube-darwin-arm64 profile list --output json

ha_test.go:390: (dbg) Done: out/minikube-darwin-arm64 profile list --output json: (2m32.251982708s)

ha_test.go:413: expected profile "ha-827000" in json of 'profile list' to have "Degraded" status but have "Stopped" status. got *"{\"invalid\":[],\"valid\":[{\"Name\":\"ha-827000\",\"Status\":\"Stopped\",\"Config\":{\"Name\":\"ha-827000\",\"KeepContext\":false,\"EmbedCerts\":false,\"MinikubeISO\":\"https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso\",\"KicBaseImage\":\"gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08\",\"Memory\":2200,\"CPUs\":2,\"DiskSize\":20000,\"Driver\":\"qemu2\",\"HyperkitVpnKitSock\":\"\",\"HyperkitVSockPorts\":[],\"DockerEnv\":null,\"ContainerVolumeMounts\":null,\"InsecureRegistry\":null,\"RegistryMirror\":[],\"HostOnlyCIDR\":\"192.168.59.1/24\",\"HypervVirtualSwitch\":\"\",\"HypervUseExternalSwitch\":false,\"HypervExternalAdapter\":\"\",\"KVMNetwork\":\"default\",\"KVMQemuURI\":\"qemu:///system\",\"KVMGPU\":false,\"KVMHidden\":false,\"KVMNUMACount\"

:1,\"APIServerPort\":8443,\"DockerOpt\":null,\"DisableDriverMounts\":false,\"NFSShare\":[],\"NFSSharesRoot\":\"/nfsshares\",\"UUID\":\"\",\"NoVTXCheck\":false,\"DNSProxy\":false,\"HostDNSResolver\":true,\"HostOnlyNicType\":\"virtio\",\"NatNicType\":\"virtio\",\"SSHIPAddress\":\"\",\"SSHUser\":\"root\",\"SSHKey\":\"\",\"SSHPort\":22,\"KubernetesConfig\":{\"KubernetesVersion\":\"v1.28.4\",\"ClusterName\":\"ha-827000\",\"Namespace\":\"default\",\"APIServerHAVIP\":\"192.168.105.254\",\"APIServerName\":\"minikubeCA\",\"APIServerNames\":null,\"APIServerIPs\":null,\"DNSDomain\":\"cluster.local\",\"ContainerRuntime\":\"docker\",\"CRISocket\":\"\",\"NetworkPlugin\":\"cni\",\"FeatureGates\":\"\",\"ServiceCIDR\":\"10.96.0.0/12\",\"ImageRepository\":\"\",\"LoadBalancerStartIP\":\"\",\"LoadBalancerEndIP\":\"\",\"CustomIngressCert\":\"\",\"RegistryAliases\":\"\",\"ExtraOptions\":null,\"ShouldLoadCachedImages\":true,\"EnableDefaultCNI\":false,\"CNI\":\"\"},\"Nodes\":[{\"Name\":\"\",\"IP\":\"192.168.105.5\",\"Port\":8443,\"K

ubernetesVersion\":\"v1.28.4\",\"ContainerRuntime\":\"docker\",\"ControlPlane\":true,\"Worker\":true},{\"Name\":\"m02\",\"IP\":\"192.168.105.6\",\"Port\":8443,\"KubernetesVersion\":\"v1.28.4\",\"ContainerRuntime\":\"docker\",\"ControlPlane\":true,\"Worker\":true},{\"Name\":\"m03\",\"IP\":\"192.168.105.7\",\"Port\":8443,\"KubernetesVersion\":\"v1.28.4\",\"ContainerRuntime\":\"docker\",\"ControlPlane\":true,\"Worker\":true},{\"Name\":\"m04\",\"IP\":\"192.168.105.8\",\"Port\":0,\"KubernetesVersion\":\"v1.28.4\",\"ContainerRuntime\":\"\",\"ControlPlane\":false,\"Worker\":true}],\"Addons\":{\"ambassador\":false,\"auto-pause\":false,\"cloud-spanner\":false,\"csi-hostpath-driver\":false,\"dashboard\":false,\"default-storageclass\":false,\"efk\":false,\"freshpod\":false,\"gcp-auth\":false,\"gvisor\":false,\"headlamp\":false,\"helm-tiller\":false,\"inaccel\":false,\"ingress\":false,\"ingress-dns\":false,\"inspektor-gadget\":false,\"istio\":false,\"istio-provisioner\":false,\"kong\":false,\"kubeflow\":false,\"kubevirt\

":false,\"logviewer\":false,\"metallb\":false,\"metrics-server\":false,\"nvidia-device-plugin\":false,\"nvidia-driver-installer\":false,\"nvidia-gpu-device-plugin\":false,\"olm\":false,\"pod-security-policy\":false,\"portainer\":false,\"registry\":false,\"registry-aliases\":false,\"registry-creds\":false,\"storage-provisioner\":false,\"storage-provisioner-gluster\":false,\"storage-provisioner-rancher\":false,\"volumesnapshots\":false,\"yakd\":false},\"CustomAddonImages\":null,\"CustomAddonRegistries\":null,\"VerifyComponents\":{\"apiserver\":true,\"apps_running\":true,\"default_sa\":true,\"extra\":true,\"kubelet\":true,\"node_ready\":true,\"system_pods\":true},\"StartHostTimeout\":360000000000,\"ScheduledStop\":null,\"ExposedPorts\":[],\"ListenAddress\":\"\",\"Network\":\"socket_vmnet\",\"Subnet\":\"\",\"MultiNodeRequested\":true,\"ExtraDisks\":0,\"CertExpiration\":94608000000000000,\"Mount\":false,\"MountString\":\"/Users:/minikube-host\",\"Mount9PVersion\":\"9p2000.L\",\"MountGID\":\"docker\",\"MountIP\":\"

\",\"MountMSize\":262144,\"MountOptions\":[],\"MountPort\":0,\"MountType\":\"9p\",\"MountUID\":\"docker\",\"BinaryMirror\":\"\",\"DisableOptimizations\":false,\"DisableMetrics\":false,\"CustomQemuFirmwarePath\":\"\",\"SocketVMnetClientPath\":\"/opt/socket_vmnet/bin/socket_vmnet_client\",\"SocketVMnetPath\":\"/var/run/socket_vmnet\",\"StaticIP\":\"\",\"SSHAuthSock\":\"\",\"SSHAgentPID\":0,\"GPUs\":\"\",\"AutoPauseInterval\":60000000000},\"Active\":false}]}"*. args: "out/minikube-darwin-arm64 profile list --output json"

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p ha-827000 -n ha-827000

E0307 14:17:13.845781    1596 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.crt: no such file or directory

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p ha-827000 -n ha-827000: exit status 3 (1m15.043258834s)

-- stdout --

	Error

-- /stdout --

** stderr ** 

	E0307 14:18:00.953868    3069 status.go:376] failed to get storage capacity of /var: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

	E0307 14:18:00.953898    3069 status.go:249] status error: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

** /stderr **

helpers_test.go:239: status error: exit status 3 (may be ok)

helpers_test.go:241: "ha-827000" host is not running, skipping log retrieval (state="Error")

--- FAIL: TestMutliControlPlane/serial/DegradedAfterControlPlaneNodeStop (227.30s)

=== RUN   TestMutliControlPlane/serial/RestartSecondaryNode

ha_test.go:420: (dbg) Run:  out/minikube-darwin-arm64 -p ha-827000 node start m02 -v=7 --alsologtostderr

ha_test.go:420: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p ha-827000 node start m02 -v=7 --alsologtostderr: exit status 80 (5.131140084s)

-- stdout --

	* Starting "ha-827000-m02" control-plane node in "ha-827000" cluster

	* Restarting existing qemu2 VM for "ha-827000-m02" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Restarting existing qemu2 VM for "ha-827000-m02" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0307 14:18:01.009930    3074 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:18:01.010208    3074 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:18:01.010212    3074 out.go:304] Setting ErrFile to fd 2...

	I0307 14:18:01.010215    3074 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:18:01.010371    3074 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:18:01.010693    3074 mustload.go:65] Loading cluster: ha-827000

	I0307 14:18:01.010993    3074 config.go:182] Loaded profile config "ha-827000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	W0307 14:18:01.011285    3074 host.go:58] "ha-827000-m02" host status: Stopped

	I0307 14:18:01.016049    3074 out.go:177] * Starting "ha-827000-m02" control-plane node in "ha-827000" cluster

	I0307 14:18:01.019083    3074 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0307 14:18:01.019100    3074 preload.go:147] Found local preload: /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4

	I0307 14:18:01.019110    3074 cache.go:56] Caching tarball of preloaded images

	I0307 14:18:01.019228    3074 preload.go:173] Found /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0307 14:18:01.019245    3074 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on docker

	I0307 14:18:01.019332    3074 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/ha-827000/config.json ...

	I0307 14:18:01.019790    3074 start.go:360] acquireMachinesLock for ha-827000-m02: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:18:01.019834    3074 start.go:364] duration metric: took 29.708µs to acquireMachinesLock for "ha-827000-m02"

	I0307 14:18:01.019843    3074 start.go:96] Skipping create...Using existing machine configuration

	I0307 14:18:01.019848    3074 fix.go:54] fixHost starting: m02

	I0307 14:18:01.019992    3074 fix.go:112] recreateIfNeeded on ha-827000-m02: state=Stopped err=<nil>

	W0307 14:18:01.019998    3074 fix.go:138] unexpected machine state, will restart: <nil>

	I0307 14:18:01.024036    3074 out.go:177] * Restarting existing qemu2 VM for "ha-827000-m02" ...

	I0307 14:18:01.027052    3074 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2200 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/qemu.pid -device virtio-net-pci,netdev=net0,mac=9e:26:47:45:25:e6 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/disk.qcow2

	I0307 14:18:01.029753    3074 main.go:141] libmachine: STDOUT: 

	I0307 14:18:01.029774    3074 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:18:01.029801    3074 fix.go:56] duration metric: took 9.952125ms for fixHost

	I0307 14:18:01.029805    3074 start.go:83] releasing machines lock for "ha-827000-m02", held for 9.967083ms

	W0307 14:18:01.029811    3074 start.go:713] error starting host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0307 14:18:01.029843    3074 out.go:239] ! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:18:01.029848    3074 start.go:728] Will try again in 5 seconds ...

	I0307 14:18:06.031979    3074 start.go:360] acquireMachinesLock for ha-827000-m02: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:18:06.032462    3074 start.go:364] duration metric: took 404.292µs to acquireMachinesLock for "ha-827000-m02"

	I0307 14:18:06.032627    3074 start.go:96] Skipping create...Using existing machine configuration

	I0307 14:18:06.032649    3074 fix.go:54] fixHost starting: m02

	I0307 14:18:06.033353    3074 fix.go:112] recreateIfNeeded on ha-827000-m02: state=Stopped err=<nil>

	W0307 14:18:06.033378    3074 fix.go:138] unexpected machine state, will restart: <nil>

	I0307 14:18:06.037628    3074 out.go:177] * Restarting existing qemu2 VM for "ha-827000-m02" ...

	I0307 14:18:06.041722    3074 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2200 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/qemu.pid -device virtio-net-pci,netdev=net0,mac=9e:26:47:45:25:e6 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/disk.qcow2

	I0307 14:18:06.051277    3074 main.go:141] libmachine: STDOUT: 

	I0307 14:18:06.051353    3074 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:18:06.051470    3074 fix.go:56] duration metric: took 18.801333ms for fixHost

	I0307 14:18:06.051497    3074 start.go:83] releasing machines lock for "ha-827000-m02", held for 19.01225ms

	W0307 14:18:06.051694    3074 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p ha-827000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p ha-827000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:18:06.056733    3074 out.go:177] 

	W0307 14:18:06.060691    3074 out.go:239] X Exiting due to GUEST_NODE_PROVISION: provisioning host for node: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_NODE_PROVISION: provisioning host for node: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0307 14:18:06.060715    3074 out.go:239] * 

	* 

	W0307 14:18:06.074602    3074 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/minikube_node_6a758bccf1d363a5d0799efcdea444172a621e97_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/minikube_node_6a758bccf1d363a5d0799efcdea444172a621e97_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	I0307 14:18:06.079516    3074 out.go:177] 

** /stderr **

ha_test.go:422: I0307 14:18:01.009930    3074 out.go:291] Setting OutFile to fd 1 ...

I0307 14:18:01.010208    3074 out.go:338] TERM=,COLORTERM=, which probably does not support color

I0307 14:18:01.010212    3074 out.go:304] Setting ErrFile to fd 2...

I0307 14:18:01.010215    3074 out.go:338] TERM=,COLORTERM=, which probably does not support color

I0307 14:18:01.010371    3074 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

I0307 14:18:01.010693    3074 mustload.go:65] Loading cluster: ha-827000

I0307 14:18:01.010993    3074 config.go:182] Loaded profile config "ha-827000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

W0307 14:18:01.011285    3074 host.go:58] "ha-827000-m02" host status: Stopped

I0307 14:18:01.016049    3074 out.go:177] * Starting "ha-827000-m02" control-plane node in "ha-827000" cluster

I0307 14:18:01.019083    3074 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

I0307 14:18:01.019100    3074 preload.go:147] Found local preload: /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4

I0307 14:18:01.019110    3074 cache.go:56] Caching tarball of preloaded images

I0307 14:18:01.019228    3074 preload.go:173] Found /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4 in cache, skipping download

I0307 14:18:01.019245    3074 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on docker

I0307 14:18:01.019332    3074 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/ha-827000/config.json ...

I0307 14:18:01.019790    3074 start.go:360] acquireMachinesLock for ha-827000-m02: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

I0307 14:18:01.019834    3074 start.go:364] duration metric: took 29.708µs to acquireMachinesLock for "ha-827000-m02"

I0307 14:18:01.019843    3074 start.go:96] Skipping create...Using existing machine configuration

I0307 14:18:01.019848    3074 fix.go:54] fixHost starting: m02

I0307 14:18:01.019992    3074 fix.go:112] recreateIfNeeded on ha-827000-m02: state=Stopped err=<nil>

W0307 14:18:01.019998    3074 fix.go:138] unexpected machine state, will restart: <nil>

I0307 14:18:01.024036    3074 out.go:177] * Restarting existing qemu2 VM for "ha-827000-m02" ...

I0307 14:18:01.027052    3074 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2200 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/qemu.pid -device virtio-net-pci,netdev=net0,mac=9e:26:47:45:25:e6 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/disk.qcow2

I0307 14:18:01.029753    3074 main.go:141] libmachine: STDOUT: 

I0307 14:18:01.029774    3074 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

I0307 14:18:01.029801    3074 fix.go:56] duration metric: took 9.952125ms for fixHost

I0307 14:18:01.029805    3074 start.go:83] releasing machines lock for "ha-827000-m02", held for 9.967083ms

W0307 14:18:01.029811    3074 start.go:713] error starting host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

W0307 14:18:01.029843    3074 out.go:239] ! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

I0307 14:18:01.029848    3074 start.go:728] Will try again in 5 seconds ...

I0307 14:18:06.031979    3074 start.go:360] acquireMachinesLock for ha-827000-m02: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

I0307 14:18:06.032462    3074 start.go:364] duration metric: took 404.292µs to acquireMachinesLock for "ha-827000-m02"

I0307 14:18:06.032627    3074 start.go:96] Skipping create...Using existing machine configuration

I0307 14:18:06.032649    3074 fix.go:54] fixHost starting: m02

I0307 14:18:06.033353    3074 fix.go:112] recreateIfNeeded on ha-827000-m02: state=Stopped err=<nil>

W0307 14:18:06.033378    3074 fix.go:138] unexpected machine state, will restart: <nil>

I0307 14:18:06.037628    3074 out.go:177] * Restarting existing qemu2 VM for "ha-827000-m02" ...

I0307 14:18:06.041722    3074 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2200 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/qemu.pid -device virtio-net-pci,netdev=net0,mac=9e:26:47:45:25:e6 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/disk.qcow2

I0307 14:18:06.051277    3074 main.go:141] libmachine: STDOUT: 

I0307 14:18:06.051353    3074 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

I0307 14:18:06.051470    3074 fix.go:56] duration metric: took 18.801333ms for fixHost

I0307 14:18:06.051497    3074 start.go:83] releasing machines lock for "ha-827000-m02", held for 19.01225ms

W0307 14:18:06.051694    3074 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p ha-827000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

* Failed to start qemu2 VM. Running "minikube delete -p ha-827000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

I0307 14:18:06.056733    3074 out.go:177] 

W0307 14:18:06.060691    3074 out.go:239] X Exiting due to GUEST_NODE_PROVISION: provisioning host for node: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

X Exiting due to GUEST_NODE_PROVISION: provisioning host for node: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

W0307 14:18:06.060715    3074 out.go:239] * 

* 

W0307 14:18:06.074602    3074 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

│                                                                                                                         │

│    * If the above advice does not help, please let us know:                                                             │

│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

│                                                                                                                         │

│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

│    * Please also attach the following file to the GitHub issue:                                                         │

│    * - /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/minikube_node_6a758bccf1d363a5d0799efcdea444172a621e97_0.log    │

│                                                                                                                         │

╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

│                                                                                                                         │

│    * If the above advice does not help, please let us know:                                                             │

│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

│                                                                                                                         │

│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

│    * Please also attach the following file to the GitHub issue:                                                         │

│    * - /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/minikube_node_6a758bccf1d363a5d0799efcdea444172a621e97_0.log    │

│                                                                                                                         │

╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

I0307 14:18:06.079516    3074 out.go:177] 

ha_test.go:423: secondary control-plane node start returned an error. args "out/minikube-darwin-arm64 -p ha-827000 node start m02 -v=7 --alsologtostderr": exit status 80

ha_test.go:428: (dbg) Run:  out/minikube-darwin-arm64 -p ha-827000 status -v=7 --alsologtostderr

E0307 14:18:17.354939    1596 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/functional-836000/client.crt: no such file or directory

E0307 14:19:40.419621    1596 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/functional-836000/client.crt: no such file or directory

ha_test.go:428: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p ha-827000 status -v=7 --alsologtostderr: exit status 7 (3m45.081891833s)

-- stdout --

	ha-827000

	type: Control Plane

	host: Error

	kubelet: Nonexistent

	apiserver: Nonexistent

	kubeconfig: Configured

	ha-827000-m02

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

	ha-827000-m03

	type: Control Plane

	host: Error

	kubelet: Nonexistent

	apiserver: Nonexistent

	kubeconfig: Configured

	ha-827000-m04

	type: Worker

	host: Error

	kubelet: Nonexistent

-- /stdout --

** stderr ** 

	I0307 14:18:06.150660    3078 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:18:06.150852    3078 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:18:06.150855    3078 out.go:304] Setting ErrFile to fd 2...

	I0307 14:18:06.150858    3078 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:18:06.151056    3078 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:18:06.151211    3078 out.go:298] Setting JSON to false

	I0307 14:18:06.151226    3078 mustload.go:65] Loading cluster: ha-827000

	I0307 14:18:06.151266    3078 notify.go:220] Checking for updates...

	I0307 14:18:06.151508    3078 config.go:182] Loaded profile config "ha-827000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:18:06.151515    3078 status.go:255] checking status of ha-827000 ...

	I0307 14:18:06.152526    3078 status.go:330] ha-827000 host status = "Running" (err=<nil>)

	I0307 14:18:06.152542    3078 host.go:66] Checking if "ha-827000" exists ...

	I0307 14:18:06.152695    3078 host.go:66] Checking if "ha-827000" exists ...

	I0307 14:18:06.152832    3078 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0307 14:18:06.152842    3078 sshutil.go:53] new ssh client: &{IP:192.168.105.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000/id_rsa Username:docker}

	W0307 14:19:21.153309    3078 sshutil.go:64] dial failure (will retry): dial tcp 192.168.105.5:22: connect: operation timed out

	W0307 14:19:21.156134    3078 start.go:268] error running df -h /var: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

	E0307 14:19:21.156199    3078 status.go:376] failed to get storage capacity of /var: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

	I0307 14:19:21.156274    3078 status.go:257] ha-827000 status: &{Name:ha-827000 Host:Error Kubelet:Nonexistent APIServer:Nonexistent Kubeconfig:Configured Worker:false TimeToStop: DockerEnv: PodManEnv:}

	E0307 14:19:21.156356    3078 status.go:260] status error: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

	I0307 14:19:21.156383    3078 status.go:255] checking status of ha-827000-m02 ...

	I0307 14:19:21.157381    3078 status.go:330] ha-827000-m02 host status = "Stopped" (err=<nil>)

	I0307 14:19:21.157402    3078 status.go:343] host is not running, skipping remaining checks

	I0307 14:19:21.157413    3078 status.go:257] ha-827000-m02 status: &{Name:ha-827000-m02 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

	I0307 14:19:21.157434    3078 status.go:255] checking status of ha-827000-m03 ...

	I0307 14:19:21.160297    3078 status.go:330] ha-827000-m03 host status = "Running" (err=<nil>)

	I0307 14:19:21.160321    3078 host.go:66] Checking if "ha-827000-m03" exists ...

	I0307 14:19:21.160820    3078 host.go:66] Checking if "ha-827000-m03" exists ...

	I0307 14:19:21.161465    3078 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0307 14:19:21.161500    3078 sshutil.go:53] new ssh client: &{IP:192.168.105.7 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m03/id_rsa Username:docker}

	W0307 14:20:36.162268    3078 sshutil.go:64] dial failure (will retry): dial tcp 192.168.105.7:22: connect: operation timed out

	W0307 14:20:36.162313    3078 start.go:268] error running df -h /var: NewSession: new client: new client: dial tcp 192.168.105.7:22: connect: operation timed out

	E0307 14:20:36.162320    3078 status.go:376] failed to get storage capacity of /var: NewSession: new client: new client: dial tcp 192.168.105.7:22: connect: operation timed out

	I0307 14:20:36.162324    3078 status.go:257] ha-827000-m03 status: &{Name:ha-827000-m03 Host:Error Kubelet:Nonexistent APIServer:Nonexistent Kubeconfig:Configured Worker:false TimeToStop: DockerEnv: PodManEnv:}

	E0307 14:20:36.162332    3078 status.go:260] status error: NewSession: new client: new client: dial tcp 192.168.105.7:22: connect: operation timed out

	I0307 14:20:36.162338    3078 status.go:255] checking status of ha-827000-m04 ...

	I0307 14:20:36.163093    3078 status.go:330] ha-827000-m04 host status = "Running" (err=<nil>)

	I0307 14:20:36.163101    3078 host.go:66] Checking if "ha-827000-m04" exists ...

	I0307 14:20:36.163216    3078 host.go:66] Checking if "ha-827000-m04" exists ...

	I0307 14:20:36.163326    3078 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0307 14:20:36.163333    3078 sshutil.go:53] new ssh client: &{IP:192.168.105.8 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m04/id_rsa Username:docker}

	W0307 14:21:51.164187    3078 sshutil.go:64] dial failure (will retry): dial tcp 192.168.105.8:22: connect: operation timed out

	W0307 14:21:51.164247    3078 start.go:268] error running df -h /var: NewSession: new client: new client: dial tcp 192.168.105.8:22: connect: operation timed out

	E0307 14:21:51.164256    3078 status.go:376] failed to get storage capacity of /var: NewSession: new client: new client: dial tcp 192.168.105.8:22: connect: operation timed out

	I0307 14:21:51.164259    3078 status.go:257] ha-827000-m04 status: &{Name:ha-827000-m04 Host:Error Kubelet:Nonexistent APIServer:Irrelevant Kubeconfig:Irrelevant Worker:true TimeToStop: DockerEnv: PodManEnv:}

	E0307 14:21:51.164269    3078 status.go:260] status error: NewSession: new client: new client: dial tcp 192.168.105.8:22: connect: operation timed out

** /stderr **

ha_test.go:432: failed to run minikube status. args "out/minikube-darwin-arm64 -p ha-827000 status -v=7 --alsologtostderr" : exit status 7

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p ha-827000 -n ha-827000

E0307 14:22:13.838776    1596 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.crt: no such file or directory

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p ha-827000 -n ha-827000: exit status 3 (1m15.048008584s)

-- stdout --

	Error

-- /stdout --

** stderr ** 

	E0307 14:23:06.206785    3124 status.go:376] failed to get storage capacity of /var: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

	E0307 14:23:06.206835    3124 status.go:249] status error: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

** /stderr **

helpers_test.go:239: status error: exit status 3 (may be ok)

helpers_test.go:241: "ha-827000" host is not running, skipping log retrieval (state="Error")

--- FAIL: TestMutliControlPlane/serial/RestartSecondaryNode (305.26s)

=== RUN   TestMutliControlPlane/serial/RestartClusterKeepsNodes

ha_test.go:456: (dbg) Run:  out/minikube-darwin-arm64 node list -p ha-827000 -v=7 --alsologtostderr

ha_test.go:462: (dbg) Run:  out/minikube-darwin-arm64 stop -p ha-827000 -v=7 --alsologtostderr

E0307 14:27:13.935412    1596 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.crt: no such file or directory

E0307 14:28:17.445879    1596 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/functional-836000/client.crt: no such file or directory

ha_test.go:462: (dbg) Non-zero exit: out/minikube-darwin-arm64 stop -p ha-827000 -v=7 --alsologtostderr: signal: killed (4m26.322872083s)

-- stdout --

	* Stopping node "ha-827000-m04"  ...

	* Stopping node "ha-827000-m03"  ...

	* Stopping node "ha-827000-m02"  ...

	* Stopping node "ha-827000"  ...

-- /stdout --

** stderr ** 

	I0307 14:25:38.462107    3184 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:25:38.462479    3184 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:25:38.462485    3184 out.go:304] Setting ErrFile to fd 2...

	I0307 14:25:38.462488    3184 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:25:38.462642    3184 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:25:38.462863    3184 out.go:298] Setting JSON to false

	I0307 14:25:38.463545    3184 mustload.go:65] Loading cluster: ha-827000

	I0307 14:25:38.463787    3184 config.go:182] Loaded profile config "ha-827000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:25:38.463862    3184 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/ha-827000/config.json ...

	I0307 14:25:38.464133    3184 mustload.go:65] Loading cluster: ha-827000

	I0307 14:25:38.464217    3184 config.go:182] Loaded profile config "ha-827000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:25:38.464235    3184 stop.go:39] StopHost: ha-827000-m04

	I0307 14:25:38.468413    3184 out.go:177] * Stopping node "ha-827000-m04"  ...

	I0307 14:25:38.476341    3184 machine.go:157] backing up vm config to /var/lib/minikube/backup: [/etc/cni /etc/kubernetes]

	I0307 14:25:38.476372    3184 ssh_runner.go:195] Run: sudo mkdir -p /var/lib/minikube/backup

	I0307 14:25:38.476380    3184 sshutil.go:53] new ssh client: &{IP:192.168.105.8 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m04/id_rsa Username:docker}

	W0307 14:26:53.478455    3184 sshutil.go:64] dial failure (will retry): dial tcp 192.168.105.8:22: connect: operation timed out

	W0307 14:26:53.478542    3184 stop.go:55] failed to complete vm config backup (will continue): create dir: NewSession: new client: new client: dial tcp 192.168.105.8:22: connect: operation timed out

	I0307 14:26:53.478586    3184 main.go:141] libmachine: Stopping "ha-827000-m04"...

	I0307 14:26:56.482489    3184 main.go:141] libmachine: Machine "ha-827000-m04" was stopped.

	I0307 14:26:56.482506    3184 stop.go:75] duration metric: took 1m18.006018125s to stop

	I0307 14:26:56.482522    3184 stop.go:39] StopHost: ha-827000-m03

	I0307 14:26:56.489851    3184 out.go:177] * Stopping node "ha-827000-m03"  ...

	I0307 14:26:56.493804    3184 machine.go:157] backing up vm config to /var/lib/minikube/backup: [/etc/cni /etc/kubernetes]

	I0307 14:26:56.493835    3184 ssh_runner.go:195] Run: sudo mkdir -p /var/lib/minikube/backup

	I0307 14:26:56.493842    3184 sshutil.go:53] new ssh client: &{IP:192.168.105.7 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m03/id_rsa Username:docker}

	W0307 14:28:11.496244    3184 sshutil.go:64] dial failure (will retry): dial tcp 192.168.105.7:22: connect: operation timed out

	W0307 14:28:11.496303    3184 stop.go:55] failed to complete vm config backup (will continue): create dir: NewSession: new client: new client: dial tcp 192.168.105.7:22: connect: operation timed out

	I0307 14:28:11.496321    3184 main.go:141] libmachine: Stopping "ha-827000-m03"...

	I0307 14:28:23.507406    3184 main.go:141] libmachine: Machine "ha-827000-m03" was stopped.

	I0307 14:28:23.507418    3184 stop.go:75] duration metric: took 1m27.013446458s to stop

	I0307 14:28:23.507427    3184 stop.go:39] StopHost: ha-827000-m02

	I0307 14:28:23.510612    3184 out.go:177] * Stopping node "ha-827000-m02"  ...

	I0307 14:28:23.518672    3184 machine.go:157] backing up vm config to /var/lib/minikube/backup: [/etc/cni /etc/kubernetes]

	I0307 14:28:23.518716    3184 ssh_runner.go:195] Run: sudo mkdir -p /var/lib/minikube/backup

	I0307 14:28:23.518725    3184 sshutil.go:53] new ssh client: &{IP:192.168.105.6 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000-m02/id_rsa Username:docker}

	W0307 14:29:38.520891    3184 sshutil.go:64] dial failure (will retry): dial tcp 192.168.105.6:22: connect: operation timed out

	W0307 14:29:38.521155    3184 stop.go:55] failed to complete vm config backup (will continue): create dir: NewSession: new client: new client: dial tcp 192.168.105.6:22: connect: operation timed out

	I0307 14:29:38.521291    3184 main.go:141] libmachine: Stopping "ha-827000-m02"...

	I0307 14:29:38.521470    3184 stop.go:66] stop err: Machine "ha-827000-m02" is already stopped.

	I0307 14:29:38.521501    3184 stop.go:69] host is already stopped

	I0307 14:29:38.521523    3184 stop.go:39] StopHost: ha-827000

	I0307 14:29:38.526808    3184 out.go:177] * Stopping node "ha-827000"  ...

	I0307 14:29:38.535672    3184 machine.go:157] backing up vm config to /var/lib/minikube/backup: [/etc/cni /etc/kubernetes]

	I0307 14:29:38.536083    3184 ssh_runner.go:195] Run: sudo mkdir -p /var/lib/minikube/backup

	I0307 14:29:38.536117    3184 sshutil.go:53] new ssh client: &{IP:192.168.105.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/ha-827000/id_rsa Username:docker}

** /stderr **

ha_test.go:464: failed to run minikube stop. args "out/minikube-darwin-arm64 node list -p ha-827000 -v=7 --alsologtostderr" : signal: killed

ha_test.go:467: (dbg) Run:  out/minikube-darwin-arm64 start -p ha-827000 --wait=true -v=7 --alsologtostderr

ha_test.go:467: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p ha-827000 --wait=true -v=7 --alsologtostderr: context deadline exceeded (2.5µs)

ha_test.go:469: failed to run minikube start. args "out/minikube-darwin-arm64 node list -p ha-827000 -v=7 --alsologtostderr" : context deadline exceeded

ha_test.go:472: (dbg) Run:  out/minikube-darwin-arm64 node list -p ha-827000

ha_test.go:472: (dbg) Non-zero exit: out/minikube-darwin-arm64 node list -p ha-827000: context deadline exceeded (292ns)

ha_test.go:474: failed to run node list. args "out/minikube-darwin-arm64 node list -p ha-827000" : context deadline exceeded

ha_test.go:479: reported node list is not the same after restart. Before restart: ha-827000	192.168.105.5

ha-827000-m02	192.168.105.6

ha-827000-m03	192.168.105.7

ha-827000-m04	192.168.105.8

After restart: 

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p ha-827000 -n ha-827000

E0307 14:30:17.006968    1596 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/addons-263000/client.crt: no such file or directory

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p ha-827000 -n ha-827000: exit status 3 (1m15.080819208s)

-- stdout --

	Error

-- /stdout --

** stderr ** 

	E0307 14:31:19.829529    3243 status.go:376] failed to get storage capacity of /var: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

	E0307 14:31:19.829572    3243 status.go:249] status error: NewSession: new client: new client: dial tcp 192.168.105.5:22: connect: operation timed out

** /stderr **

helpers_test.go:239: status error: exit status 3 (may be ok)

helpers_test.go:241: "ha-827000" host is not running, skipping log retrieval (state="Error")

--- FAIL: TestMutliControlPlane/serial/RestartClusterKeepsNodes (341.44s)

=== RUN   TestImageBuild/serial/Setup

image_test.go:69: (dbg) Run:  out/minikube-darwin-arm64 start -p image-134000 --driver=qemu2 

image_test.go:69: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p image-134000 --driver=qemu2 : exit status 80 (9.838120958s)

-- stdout --

	* [image-134000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting "image-134000" primary control-plane node in "image-134000" cluster

	* Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "image-134000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p image-134000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

image_test.go:70: failed to start minikube with args: "out/minikube-darwin-arm64 start -p image-134000 --driver=qemu2 " : exit status 80

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p image-134000 -n image-134000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p image-134000 -n image-134000: exit status 7 (69.290125ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "image-134000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestImageBuild/serial/Setup (9.91s)

=== RUN   TestJSONOutput/start/Command

json_output_test.go:63: (dbg) Run:  out/minikube-darwin-arm64 start -p json-output-328000 --output=json --user=testUser --memory=2200 --wait=true --driver=qemu2 

json_output_test.go:63: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p json-output-328000 --output=json --user=testUser --memory=2200 --wait=true --driver=qemu2 : exit status 80 (9.764632875s)

-- stdout --

	{"specversion":"1.0","id":"e8180699-b183-4505-8cf5-1c521a60e7f7","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"0","message":"[json-output-328000] minikube v1.32.0 on Darwin 14.3.1 (arm64)","name":"Initial Minikube Setup","totalsteps":"19"}}

	{"specversion":"1.0","id":"8f6fc39f-a07e-4dfe-bfa9-7cb5f06be936","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_LOCATION=18320"}}

	{"specversion":"1.0","id":"874dcd84-b62a-4907-bb8e-4b23197cc716","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig"}}

	{"specversion":"1.0","id":"17c1acd4-7a53-4a2a-bf24-0a84b483ca8c","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_BIN=out/minikube-darwin-arm64"}}

	{"specversion":"1.0","id":"0904a0c8-38cb-4bc4-823a-1aca61eb8d20","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true"}}

	{"specversion":"1.0","id":"213c2868-29dc-43fb-8a52-c968adfca14d","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube"}}

	{"specversion":"1.0","id":"aa08d069-1261-419e-969f-6d848f7cf415","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_FORCE_SYSTEMD="}}

	{"specversion":"1.0","id":"4297ac6e-b5a7-4f25-abd9-acaa4eab06ac","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"1","message":"Using the qemu2 driver based on user configuration","name":"Selecting Driver","totalsteps":"19"}}

	{"specversion":"1.0","id":"f6651696-dc0b-47c8-9997-a87d80c700aa","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"Automatically selected the socket_vmnet network"}}

	{"specversion":"1.0","id":"42998af1-329f-469d-8dea-11c8e2976a54","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"3","message":"Starting \"json-output-328000\" primary control-plane node in \"json-output-328000\" cluster","name":"Starting Node","totalsteps":"19"}}

	{"specversion":"1.0","id":"a9be05bd-636d-4323-907d-fda91f3764b5","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"9","message":"Creating qemu2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...","name":"Creating VM","totalsteps":"19"}}

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	{"specversion":"1.0","id":"8cc20751-4b58-44cb-acf6-cb3163d9f160","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"9","message":"Deleting \"json-output-328000\" in qemu2 ...","name":"Creating VM","totalsteps":"19"}}

	{"specversion":"1.0","id":"8cb1ac9d-624b-476c-868d-4473abfbb5cc","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.error","datacontenttype":"application/json","data":{"message":"StartHost failed, but will try again: creating host: create: creating: Failed to connect to \"/var/run/socket_vmnet\": Connection refused: exit status 1"}}

	{"specversion":"1.0","id":"10a8affe-8d8d-484c-ac34-b37d6f699fe2","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"9","message":"Creating qemu2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...","name":"Creating VM","totalsteps":"19"}}

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	{"specversion":"1.0","id":"195beb62-a9c7-4ca6-8392-6b9d07306e37","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.error","datacontenttype":"application/json","data":{"message":"Failed to start qemu2 VM. Running \"minikube delete -p json-output-328000\" may fix it: creating host: create: creating: Failed to connect to \"/var/run/socket_vmnet\": Connection refused: exit status 1"}}

	{"specversion":"1.0","id":"c9c3eccd-00c0-446c-8469-d5eb59bb76cd","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.error","datacontenttype":"application/json","data":{"advice":"","exitcode":"80","issues":"","message":"error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to \"/var/run/socket_vmnet\": Connection refused: exit status 1","name":"GUEST_PROVISION","url":""}}

	{"specversion":"1.0","id":"ca2e83a2-c904-4cb6-91af-1566aa8077e9","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.error","datacontenttype":"application/json","data":{"message":"╭───────────────────────────────────────────────────────────────────────────────────────────╮\n│                                                                                           │\n│    If the above advice does not help, please let us know:                                 │\n│    https://github.com/kubernetes/minikube/issues/new/choose                               │\n│                                                                                           │\n│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │\n│

│\n╰───────────────────────────────────────────────────────────────────────────────────────────╯"}}

-- /stdout --

json_output_test.go:65: failed to clean up: args "out/minikube-darwin-arm64 start -p json-output-328000 --output=json --user=testUser --memory=2200 --wait=true --driver=qemu2 ": exit status 80

json_output_test.go:213: unable to marshal output: OUTPUT: 

json_output_test.go:70: converting to cloud events: invalid character 'O' looking for beginning of value

--- FAIL: TestJSONOutput/start/Command (9.77s)

=== RUN   TestJSONOutput/pause/Command

json_output_test.go:63: (dbg) Run:  out/minikube-darwin-arm64 pause -p json-output-328000 --output=json --user=testUser

json_output_test.go:63: (dbg) Non-zero exit: out/minikube-darwin-arm64 pause -p json-output-328000 --output=json --user=testUser: exit status 83 (80.504625ms)

-- stdout --

	{"specversion":"1.0","id":"687b268a-0936-4efb-aaa5-021bbe285267","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"The control-plane node json-output-328000 host is not running: state=Stopped"}}

	{"specversion":"1.0","id":"2b2766f6-c7b5-4c6b-9e8a-0156cc451132","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"To start a cluster, run: \"minikube start -p json-output-328000\""}}

-- /stdout --

json_output_test.go:65: failed to clean up: args "out/minikube-darwin-arm64 pause -p json-output-328000 --output=json --user=testUser": exit status 83

--- FAIL: TestJSONOutput/pause/Command (0.08s)

=== RUN   TestJSONOutput/unpause/Command

json_output_test.go:63: (dbg) Run:  out/minikube-darwin-arm64 unpause -p json-output-328000 --output=json --user=testUser

json_output_test.go:63: (dbg) Non-zero exit: out/minikube-darwin-arm64 unpause -p json-output-328000 --output=json --user=testUser: exit status 83 (44.273084ms)

-- stdout --

	* The control-plane node json-output-328000 host is not running: state=Stopped

	  To start a cluster, run: "minikube start -p json-output-328000"

-- /stdout --

json_output_test.go:65: failed to clean up: args "out/minikube-darwin-arm64 unpause -p json-output-328000 --output=json --user=testUser": exit status 83

json_output_test.go:213: unable to marshal output: * The control-plane node json-output-328000 host is not running: state=Stopped

json_output_test.go:70: converting to cloud events: invalid character '*' looking for beginning of value

--- FAIL: TestJSONOutput/unpause/Command (0.04s)

=== RUN   TestMinikubeProfile

minikube_profile_test.go:44: (dbg) Run:  out/minikube-darwin-arm64 start -p first-106000 --driver=qemu2 

minikube_profile_test.go:44: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p first-106000 --driver=qemu2 : exit status 80 (9.827080292s)

-- stdout --

	* [first-106000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting "first-106000" primary control-plane node in "first-106000" cluster

	* Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "first-106000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p first-106000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

minikube_profile_test.go:46: test pre-condition failed. args "out/minikube-darwin-arm64 start -p first-106000 --driver=qemu2 ": exit status 80

panic.go:626: *** TestMinikubeProfile FAILED at 2024-03-07 14:33:14.523843 -0800 PST m=+2807.019823001

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p second-108000 -n second-108000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p second-108000 -n second-108000: exit status 85 (85.666875ms)

-- stdout --

	* Profile "second-108000" not found. Run "minikube profile list" to view all profiles.

	  To start a cluster, run: "minikube start -p second-108000"

-- /stdout --

helpers_test.go:239: status error: exit status 85 (may be ok)

helpers_test.go:241: "second-108000" host is not running, skipping log retrieval (state="* Profile \"second-108000\" not found. Run \"minikube profile list\" to view all profiles.\n  To start a cluster, run: \"minikube start -p second-108000\"")

helpers_test.go:175: Cleaning up "second-108000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p second-108000

panic.go:626: *** TestMinikubeProfile FAILED at 2024-03-07 14:33:14.842018 -0800 PST m=+2807.337997418

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p first-106000 -n first-106000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p first-106000 -n first-106000: exit status 7 (32.044666ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "first-106000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "first-106000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p first-106000

--- FAIL: TestMinikubeProfile (10.28s)

=== RUN   TestMountStart/serial/StartWithMountFirst

mount_start_test.go:98: (dbg) Run:  out/minikube-darwin-arm64 start -p mount-start-1-043000 --memory=2048 --mount --mount-gid 0 --mount-msize 6543 --mount-port 46464 --mount-uid 0 --no-kubernetes --driver=qemu2 

E0307 14:33:17.447626    1596 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/functional-836000/client.crt: no such file or directory

mount_start_test.go:98: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p mount-start-1-043000 --memory=2048 --mount --mount-gid 0 --mount-msize 6543 --mount-port 46464 --mount-uid 0 --no-kubernetes --driver=qemu2 : exit status 80 (10.49571275s)

-- stdout --

	* [mount-start-1-043000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting minikube without Kubernetes in cluster mount-start-1-043000

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "mount-start-1-043000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p mount-start-1-043000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

mount_start_test.go:100: failed to start minikube with args: "out/minikube-darwin-arm64 start -p mount-start-1-043000 --memory=2048 --mount --mount-gid 0 --mount-msize 6543 --mount-port 46464 --mount-uid 0 --no-kubernetes --driver=qemu2 " : exit status 80

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p mount-start-1-043000 -n mount-start-1-043000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p mount-start-1-043000 -n mount-start-1-043000: exit status 7 (68.200625ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "mount-start-1-043000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMountStart/serial/StartWithMountFirst (10.56s)

=== RUN   TestMultiNode/serial/FreshStart2Nodes

multinode_test.go:96: (dbg) Run:  out/minikube-darwin-arm64 start -p multinode-172000 --wait=true --memory=2200 --nodes=2 -v=8 --alsologtostderr --driver=qemu2 

multinode_test.go:96: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p multinode-172000 --wait=true --memory=2200 --nodes=2 -v=8 --alsologtostderr --driver=qemu2 : exit status 80 (9.808816209s)

-- stdout --

	* [multinode-172000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting "multinode-172000" primary control-plane node in "multinode-172000" cluster

	* Creating qemu2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "multinode-172000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0307 14:33:25.898400    3434 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:33:25.898570    3434 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:33:25.898576    3434 out.go:304] Setting ErrFile to fd 2...

	I0307 14:33:25.898578    3434 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:33:25.898875    3434 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:33:25.900179    3434 out.go:298] Setting JSON to false

	I0307 14:33:25.916598    3434 start.go:129] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":3762,"bootTime":1709847043,"procs":467,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.3.1","kernelVersion":"23.3.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0307 14:33:25.916664    3434 start.go:137] gopshost.Virtualization returned error: not implemented yet

	I0307 14:33:25.922372    3434 out.go:177] * [multinode-172000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	I0307 14:33:25.929318    3434 out.go:177]   - MINIKUBE_LOCATION=18320

	I0307 14:33:25.933328    3434 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	I0307 14:33:25.929353    3434 notify.go:220] Checking for updates...

	I0307 14:33:25.939244    3434 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0307 14:33:25.942292    3434 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0307 14:33:25.945213    3434 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	I0307 14:33:25.948256    3434 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0307 14:33:25.951440    3434 driver.go:392] Setting default libvirt URI to qemu:///system

	I0307 14:33:25.954238    3434 out.go:177] * Using the qemu2 driver based on user configuration

	I0307 14:33:25.961263    3434 start.go:297] selected driver: qemu2

	I0307 14:33:25.961268    3434 start.go:901] validating driver "qemu2" against <nil>

	I0307 14:33:25.961273    3434 start.go:912] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0307 14:33:25.963472    3434 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0307 14:33:25.964985    3434 out.go:177] * Automatically selected the socket_vmnet network

	I0307 14:33:25.968354    3434 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0307 14:33:25.968405    3434 cni.go:84] Creating CNI manager for ""

	I0307 14:33:25.968411    3434 cni.go:136] multinode detected (0 nodes found), recommending kindnet

	I0307 14:33:25.968415    3434 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni

	I0307 14:33:25.968446    3434 start.go:340] cluster config:

	{Name:multinode-172000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:multinode-172000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRu

ntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vm

net_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 14:33:25.972856    3434 iso.go:125] acquiring lock: {Name:mk9f11fe8095f5aa8805d47835226b5c64c27e19 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0307 14:33:25.980291    3434 out.go:177] * Starting "multinode-172000" primary control-plane node in "multinode-172000" cluster

	I0307 14:33:25.984256    3434 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0307 14:33:25.984283    3434 preload.go:147] Found local preload: /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4

	I0307 14:33:25.984293    3434 cache.go:56] Caching tarball of preloaded images

	I0307 14:33:25.984348    3434 preload.go:173] Found /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0307 14:33:25.984354    3434 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on docker

	I0307 14:33:25.984579    3434 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/multinode-172000/config.json ...

	I0307 14:33:25.984591    3434 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/multinode-172000/config.json: {Name:mk8b7d9a7660e6e07e56e363562f3bf4e25a4ac6 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0307 14:33:25.984819    3434 start.go:360] acquireMachinesLock for multinode-172000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:33:25.984853    3434 start.go:364] duration metric: took 27.458µs to acquireMachinesLock for "multinode-172000"

	I0307 14:33:25.984864    3434 start.go:93] Provisioning new machine with config: &{Name:multinode-172000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{K

ubernetesVersion:v1.28.4 ClusterName:multinode-172000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[

] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 14:33:25.984896    3434 start.go:125] createHost starting for "" (driver="qemu2")

	I0307 14:33:25.993272    3434 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...

	I0307 14:33:26.011775    3434 start.go:159] libmachine.API.Create for "multinode-172000" (driver="qemu2")

	I0307 14:33:26.011808    3434 client.go:168] LocalClient.Create starting

	I0307 14:33:26.011886    3434 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem

	I0307 14:33:26.011914    3434 main.go:141] libmachine: Decoding PEM data...

	I0307 14:33:26.011925    3434 main.go:141] libmachine: Parsing certificate...

	I0307 14:33:26.011970    3434 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem

	I0307 14:33:26.011992    3434 main.go:141] libmachine: Decoding PEM data...

	I0307 14:33:26.011998    3434 main.go:141] libmachine: Parsing certificate...

	I0307 14:33:26.012367    3434 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso...

	I0307 14:33:26.149548    3434 main.go:141] libmachine: Creating SSH key...

	I0307 14:33:26.269630    3434 main.go:141] libmachine: Creating Disk image...

	I0307 14:33:26.269639    3434 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0307 14:33:26.269832    3434 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2.raw /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2

	I0307 14:33:26.282493    3434 main.go:141] libmachine: STDOUT: 

	I0307 14:33:26.282514    3434 main.go:141] libmachine: STDERR: 

	I0307 14:33:26.282576    3434 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2 +20000M

	I0307 14:33:26.293331    3434 main.go:141] libmachine: STDOUT: Image resized.

	I0307 14:33:26.293351    3434 main.go:141] libmachine: STDERR: 

	I0307 14:33:26.293364    3434 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2.raw and /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2

	I0307 14:33:26.293371    3434 main.go:141] libmachine: Starting QEMU VM...

	I0307 14:33:26.293407    3434 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2200 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/qemu.pid -device virtio-net-pci,netdev=net0,mac=da:8a:6e:f9:0e:d7 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2

	I0307 14:33:26.295115    3434 main.go:141] libmachine: STDOUT: 

	I0307 14:33:26.295130    3434 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:33:26.295148    3434 client.go:171] duration metric: took 283.333ms to LocalClient.Create

	I0307 14:33:28.297346    3434 start.go:128] duration metric: took 2.312425333s to createHost

	I0307 14:33:28.297439    3434 start.go:83] releasing machines lock for "multinode-172000", held for 2.312552125s

	W0307 14:33:28.297509    3434 start.go:713] error starting host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:33:28.305620    3434 out.go:177] * Deleting "multinode-172000" in qemu2 ...

	W0307 14:33:28.331262    3434 out.go:239] ! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:33:28.331303    3434 start.go:728] Will try again in 5 seconds ...

	I0307 14:33:33.333477    3434 start.go:360] acquireMachinesLock for multinode-172000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:33:33.333906    3434 start.go:364] duration metric: took 328.25µs to acquireMachinesLock for "multinode-172000"

	I0307 14:33:33.334025    3434 start.go:93] Provisioning new machine with config: &{Name:multinode-172000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{K

ubernetesVersion:v1.28.4 ClusterName:multinode-172000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[

] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0307 14:33:33.334316    3434 start.go:125] createHost starting for "" (driver="qemu2")

	I0307 14:33:33.343977    3434 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2200MB, Disk=20000MB) ...

	I0307 14:33:33.393784    3434 start.go:159] libmachine.API.Create for "multinode-172000" (driver="qemu2")

	I0307 14:33:33.393831    3434 client.go:168] LocalClient.Create starting

	I0307 14:33:33.393942    3434 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/ca.pem

	I0307 14:33:33.394009    3434 main.go:141] libmachine: Decoding PEM data...

	I0307 14:33:33.394027    3434 main.go:141] libmachine: Parsing certificate...

	I0307 14:33:33.394089    3434 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/18320-1153/.minikube/certs/cert.pem

	I0307 14:33:33.394130    3434 main.go:141] libmachine: Decoding PEM data...

	I0307 14:33:33.394140    3434 main.go:141] libmachine: Parsing certificate...

	I0307 14:33:33.394635    3434 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/18320-1153/.minikube/cache/iso/arm64/minikube-v1.32.1-1708638130-18020-arm64.iso...

	I0307 14:33:33.543650    3434 main.go:141] libmachine: Creating SSH key...

	I0307 14:33:33.610779    3434 main.go:141] libmachine: Creating Disk image...

	I0307 14:33:33.610784    3434 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0307 14:33:33.610958    3434 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2.raw /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2

	I0307 14:33:33.623071    3434 main.go:141] libmachine: STDOUT: 

	I0307 14:33:33.623090    3434 main.go:141] libmachine: STDERR: 

	I0307 14:33:33.623155    3434 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2 +20000M

	I0307 14:33:33.633989    3434 main.go:141] libmachine: STDOUT: Image resized.

	I0307 14:33:33.634005    3434 main.go:141] libmachine: STDERR: 

	I0307 14:33:33.634016    3434 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2.raw and /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2

	I0307 14:33:33.634021    3434 main.go:141] libmachine: Starting QEMU VM...

	I0307 14:33:33.634066    3434 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2200 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/qemu.pid -device virtio-net-pci,netdev=net0,mac=ea:c0:e3:0f:13:78 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2

	I0307 14:33:33.635830    3434 main.go:141] libmachine: STDOUT: 

	I0307 14:33:33.635848    3434 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:33:33.635862    3434 client.go:171] duration metric: took 242.026708ms to LocalClient.Create

	I0307 14:33:35.638138    3434 start.go:128] duration metric: took 2.3037555s to createHost

	I0307 14:33:35.638225    3434 start.go:83] releasing machines lock for "multinode-172000", held for 2.304290625s

	W0307 14:33:35.638674    3434 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p multinode-172000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p multinode-172000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:33:35.646334    3434 out.go:177] 

	W0307 14:33:35.652301    3434 out.go:239] X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0307 14:33:35.652349    3434 out.go:239] * 

	* 

	W0307 14:33:35.654465    3434 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0307 14:33:35.663153    3434 out.go:177] 

** /stderr **

multinode_test.go:98: failed to start cluster. args "out/minikube-darwin-arm64 start -p multinode-172000 --wait=true --memory=2200 --nodes=2 -v=8 --alsologtostderr --driver=qemu2 " : exit status 80

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (68.344333ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/FreshStart2Nodes (9.88s)

=== RUN   TestMultiNode/serial/DeployApp2Nodes

multinode_test.go:493: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- apply -f ./testdata/multinodes/multinode-pod-dns-test.yaml

multinode_test.go:493: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- apply -f ./testdata/multinodes/multinode-pod-dns-test.yaml: exit status 1 (131.670375ms)

** stderr ** 

	error: cluster "multinode-172000" does not exist

** /stderr **

multinode_test.go:495: failed to create busybox deployment to multinode cluster

multinode_test.go:498: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- rollout status deployment/busybox

multinode_test.go:498: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- rollout status deployment/busybox: exit status 1 (58.96ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:500: failed to deploy busybox to multinode cluster

multinode_test.go:505: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:505: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (58.225416ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:508: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:505: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:505: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (105.048458ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:508: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:505: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:505: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (104.357041ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:508: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:505: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:505: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (100.622125ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:508: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:505: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:505: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (104.506584ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:508: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:505: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:505: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (105.197458ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:508: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:505: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:505: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (104.784375ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:508: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:505: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:505: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (103.548417ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:508: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:505: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:505: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (104.881084ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:508: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:505: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}'

multinode_test.go:505: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].status.podIP}': exit status 1 (104.14175ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:508: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:524: failed to resolve pod IPs: failed to retrieve Pod IPs (may be temporary): exit status 1

multinode_test.go:528: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].metadata.name}'

multinode_test.go:528: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].metadata.name}': exit status 1 (58.1765ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:530: failed get Pod names

multinode_test.go:536: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- exec  -- nslookup kubernetes.io

multinode_test.go:536: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- exec  -- nslookup kubernetes.io: exit status 1 (56.990125ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:538: Pod  could not resolve 'kubernetes.io': exit status 1

multinode_test.go:546: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- exec  -- nslookup kubernetes.default

multinode_test.go:546: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- exec  -- nslookup kubernetes.default: exit status 1 (58.185166ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:548: Pod  could not resolve 'kubernetes.default': exit status 1

multinode_test.go:554: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- exec  -- nslookup kubernetes.default.svc.cluster.local

multinode_test.go:554: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- exec  -- nslookup kubernetes.default.svc.cluster.local: exit status 1 (57.131209ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:556: Pod  could not resolve local service (kubernetes.default.svc.cluster.local): exit status 1

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (31.543833ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/DeployApp2Nodes (97.30s)

=== RUN   TestMultiNode/serial/PingHostFrom2Pods

multinode_test.go:564: (dbg) Run:  out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].metadata.name}'

multinode_test.go:564: (dbg) Non-zero exit: out/minikube-darwin-arm64 kubectl -p multinode-172000 -- get pods -o jsonpath='{.items[*].metadata.name}': exit status 1 (58.339458ms)

** stderr ** 

	error: no server found for cluster "multinode-172000"

** /stderr **

multinode_test.go:566: failed to get Pod names: exit status 1

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (32.202416ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/PingHostFrom2Pods (0.09s)

=== RUN   TestMultiNode/serial/AddNode

multinode_test.go:121: (dbg) Run:  out/minikube-darwin-arm64 node add -p multinode-172000 -v 3 --alsologtostderr

multinode_test.go:121: (dbg) Non-zero exit: out/minikube-darwin-arm64 node add -p multinode-172000 -v 3 --alsologtostderr: exit status 83 (44.364625ms)

-- stdout --

	* The control-plane node multinode-172000 host is not running: state=Stopped

	  To start a cluster, run: "minikube start -p multinode-172000"

-- /stdout --

** stderr ** 

	I0307 14:35:13.171150    3521 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:13.171510    3521 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:13.171513    3521 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:13.171515    3521 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:13.171671    3521 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:13.171915    3521 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:13.172101    3521 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:13.177534    3521 out.go:177] * The control-plane node multinode-172000 host is not running: state=Stopped

	I0307 14:35:13.180564    3521 out.go:177]   To start a cluster, run: "minikube start -p multinode-172000"

** /stderr **

multinode_test.go:123: failed to add node to current cluster. args "out/minikube-darwin-arm64 node add -p multinode-172000 -v 3 --alsologtostderr" : exit status 83

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (31.83675ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/AddNode (0.08s)

=== RUN   TestMultiNode/serial/MultiNodeLabels

multinode_test.go:221: (dbg) Run:  kubectl --context multinode-172000 get nodes -o "jsonpath=[{range .items[*]}{.metadata.labels},{end}]"

multinode_test.go:221: (dbg) Non-zero exit: kubectl --context multinode-172000 get nodes -o "jsonpath=[{range .items[*]}{.metadata.labels},{end}]": exit status 1 (29.438084ms)

** stderr ** 

	Error in configuration: context was not found for specified context: multinode-172000

** /stderr **

multinode_test.go:223: failed to 'kubectl get nodes' with args "kubectl --context multinode-172000 get nodes -o \"jsonpath=[{range .items[*]}{.metadata.labels},{end}]\"": exit status 1

multinode_test.go:230: failed to decode json from label list: args "kubectl --context multinode-172000 get nodes -o \"jsonpath=[{range .items[*]}{.metadata.labels},{end}]\"": unexpected end of JSON input

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (31.943125ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/MultiNodeLabels (0.06s)

=== RUN   TestMultiNode/serial/ProfileList

multinode_test.go:143: (dbg) Run:  out/minikube-darwin-arm64 profile list --output json

multinode_test.go:166: expected profile "multinode-172000" in json of 'profile list' include 3 nodes but have 1 nodes. got *"{\"invalid\":[],\"valid\":[{\"Name\":\"multinode-172000\",\"Status\":\"Stopped\",\"Config\":{\"Name\":\"multinode-172000\",\"KeepContext\":false,\"EmbedCerts\":false,\"MinikubeISO\":\"https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso\",\"KicBaseImage\":\"gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08\",\"Memory\":2200,\"CPUs\":2,\"DiskSize\":20000,\"Driver\":\"qemu2\",\"HyperkitVpnKitSock\":\"\",\"HyperkitVSockPorts\":[],\"DockerEnv\":null,\"ContainerVolumeMounts\":null,\"InsecureRegistry\":null,\"RegistryMirror\":[],\"HostOnlyCIDR\":\"192.168.59.1/24\",\"HypervVirtualSwitch\":\"\",\"HypervUseExternalSwitch\":false,\"HypervExternalAdapter\":\"\",\"KVMNetwork\":\"default\",\"KVMQemuURI\":\"qemu:///system\",\"KVMGPU\":false,\"KVMHidden\":false,\"KVMNU

MACount\":1,\"APIServerPort\":8443,\"DockerOpt\":null,\"DisableDriverMounts\":false,\"NFSShare\":[],\"NFSSharesRoot\":\"/nfsshares\",\"UUID\":\"\",\"NoVTXCheck\":false,\"DNSProxy\":false,\"HostDNSResolver\":true,\"HostOnlyNicType\":\"virtio\",\"NatNicType\":\"virtio\",\"SSHIPAddress\":\"\",\"SSHUser\":\"root\",\"SSHKey\":\"\",\"SSHPort\":22,\"KubernetesConfig\":{\"KubernetesVersion\":\"v1.28.4\",\"ClusterName\":\"multinode-172000\",\"Namespace\":\"default\",\"APIServerHAVIP\":\"\",\"APIServerName\":\"minikubeCA\",\"APIServerNames\":null,\"APIServerIPs\":null,\"DNSDomain\":\"cluster.local\",\"ContainerRuntime\":\"docker\",\"CRISocket\":\"\",\"NetworkPlugin\":\"cni\",\"FeatureGates\":\"\",\"ServiceCIDR\":\"10.96.0.0/12\",\"ImageRepository\":\"\",\"LoadBalancerStartIP\":\"\",\"LoadBalancerEndIP\":\"\",\"CustomIngressCert\":\"\",\"RegistryAliases\":\"\",\"ExtraOptions\":null,\"ShouldLoadCachedImages\":true,\"EnableDefaultCNI\":false,\"CNI\":\"\"},\"Nodes\":[{\"Name\":\"\",\"IP\":\"\",\"Port\":8443,\"KubernetesVer

sion\":\"v1.28.4\",\"ContainerRuntime\":\"docker\",\"ControlPlane\":true,\"Worker\":true}],\"Addons\":null,\"CustomAddonImages\":null,\"CustomAddonRegistries\":null,\"VerifyComponents\":{\"apiserver\":true,\"apps_running\":true,\"default_sa\":true,\"extra\":true,\"kubelet\":true,\"node_ready\":true,\"system_pods\":true},\"StartHostTimeout\":360000000000,\"ScheduledStop\":null,\"ExposedPorts\":[],\"ListenAddress\":\"\",\"Network\":\"socket_vmnet\",\"Subnet\":\"\",\"MultiNodeRequested\":true,\"ExtraDisks\":0,\"CertExpiration\":94608000000000000,\"Mount\":false,\"MountString\":\"/Users:/minikube-host\",\"Mount9PVersion\":\"9p2000.L\",\"MountGID\":\"docker\",\"MountIP\":\"\",\"MountMSize\":262144,\"MountOptions\":[],\"MountPort\":0,\"MountType\":\"9p\",\"MountUID\":\"docker\",\"BinaryMirror\":\"\",\"DisableOptimizations\":false,\"DisableMetrics\":false,\"CustomQemuFirmwarePath\":\"\",\"SocketVMnetClientPath\":\"/opt/socket_vmnet/bin/socket_vmnet_client\",\"SocketVMnetPath\":\"/var/run/socket_vmnet\",\"StaticIP\":

\"\",\"SSHAuthSock\":\"\",\"SSHAgentPID\":0,\"GPUs\":\"\",\"AutoPauseInterval\":60000000000},\"Active\":false}]}"*. args: "out/minikube-darwin-arm64 profile list --output json"

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (31.261125ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/ProfileList (0.10s)

=== RUN   TestMultiNode/serial/CopyFile

multinode_test.go:184: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status --output json --alsologtostderr

multinode_test.go:184: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status --output json --alsologtostderr: exit status 7 (31.6245ms)

-- stdout --

	{"Name":"multinode-172000","Host":"Stopped","Kubelet":"Stopped","APIServer":"Stopped","Kubeconfig":"Stopped","Worker":false}

-- /stdout --

** stderr ** 

	I0307 14:35:13.412592    3534 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:13.412778    3534 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:13.412781    3534 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:13.412783    3534 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:13.412907    3534 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:13.413033    3534 out.go:298] Setting JSON to true

	I0307 14:35:13.413046    3534 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:13.413110    3534 notify.go:220] Checking for updates...

	I0307 14:35:13.413243    3534 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:13.413249    3534 status.go:255] checking status of multinode-172000 ...

	I0307 14:35:13.413474    3534 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:35:13.413477    3534 status.go:343] host is not running, skipping remaining checks

	I0307 14:35:13.413480    3534 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:191: failed to decode json from status: args "out/minikube-darwin-arm64 -p multinode-172000 status --output json --alsologtostderr": json: cannot unmarshal object into Go value of type []cmd.Status

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (31.234583ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/CopyFile (0.06s)

=== RUN   TestMultiNode/serial/StopNode

multinode_test.go:248: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 node stop m03

multinode_test.go:248: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 node stop m03: exit status 85 (46.698584ms)

-- stdout --

-- /stdout --

** stderr ** 

	X Exiting due to GUEST_NODE_RETRIEVE: retrieving node: Could not find node m03

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/minikube_node_295f67d8757edd996fe5c1e7ccde72c355ccf4dc_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

multinode_test.go:250: node stop returned an error. args "out/minikube-darwin-arm64 -p multinode-172000 node stop m03": exit status 85

multinode_test.go:254: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status

multinode_test.go:254: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status: exit status 7 (31.585458ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

multinode_test.go:261: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status --alsologtostderr

multinode_test.go:261: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status --alsologtostderr: exit status 7 (31.685041ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:35:13.554714    3542 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:13.554877    3542 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:13.554880    3542 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:13.554882    3542 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:13.555019    3542 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:13.555137    3542 out.go:298] Setting JSON to false

	I0307 14:35:13.555155    3542 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:13.555204    3542 notify.go:220] Checking for updates...

	I0307 14:35:13.555368    3542 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:13.555376    3542 status.go:255] checking status of multinode-172000 ...

	I0307 14:35:13.555589    3542 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:35:13.555592    3542 status.go:343] host is not running, skipping remaining checks

	I0307 14:35:13.555595    3542 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:267: incorrect number of running kubelets: args "out/minikube-darwin-arm64 -p multinode-172000 status --alsologtostderr": multinode-172000

type: Control Plane

host: Stopped

kubelet: Stopped

apiserver: Stopped

kubeconfig: Stopped

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (31.585708ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/StopNode (0.14s)

=== RUN   TestMultiNode/serial/StartAfterStop

multinode_test.go:282: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 node start m03 -v=7 --alsologtostderr

multinode_test.go:282: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 node start m03 -v=7 --alsologtostderr: exit status 85 (45.754208ms)

-- stdout --

-- /stdout --

** stderr ** 

	I0307 14:35:13.618330    3546 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:13.618535    3546 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:13.618538    3546 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:13.618541    3546 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:13.618666    3546 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:13.618931    3546 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:13.619120    3546 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:13.622291    3546 out.go:177] 

	W0307 14:35:13.625201    3546 out.go:239] X Exiting due to GUEST_NODE_RETRIEVE: retrieving node: Could not find node m03

	X Exiting due to GUEST_NODE_RETRIEVE: retrieving node: Could not find node m03

	W0307 14:35:13.625213    3546 out.go:239] * 

	* 

	W0307 14:35:13.626802    3546 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/minikube_node_1c3a1297795327375b61f3ff5a4ef34c9b2fc69b_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                         │

	│    * If the above advice does not help, please let us know:                                                             │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

	│                                                                                                                         │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

	│    * Please also attach the following file to the GitHub issue:                                                         │

	│    * - /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/minikube_node_1c3a1297795327375b61f3ff5a4ef34c9b2fc69b_0.log    │

	│                                                                                                                         │

	╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

	I0307 14:35:13.630224    3546 out.go:177] 

** /stderr **

multinode_test.go:284: I0307 14:35:13.618330    3546 out.go:291] Setting OutFile to fd 1 ...

I0307 14:35:13.618535    3546 out.go:338] TERM=,COLORTERM=, which probably does not support color

I0307 14:35:13.618538    3546 out.go:304] Setting ErrFile to fd 2...

I0307 14:35:13.618541    3546 out.go:338] TERM=,COLORTERM=, which probably does not support color

I0307 14:35:13.618666    3546 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

I0307 14:35:13.618931    3546 mustload.go:65] Loading cluster: multinode-172000

I0307 14:35:13.619120    3546 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

I0307 14:35:13.622291    3546 out.go:177] 

W0307 14:35:13.625201    3546 out.go:239] X Exiting due to GUEST_NODE_RETRIEVE: retrieving node: Could not find node m03

X Exiting due to GUEST_NODE_RETRIEVE: retrieving node: Could not find node m03

W0307 14:35:13.625213    3546 out.go:239] * 

* 

W0307 14:35:13.626802    3546 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

│                                                                                                                         │

│    * If the above advice does not help, please let us know:                                                             │

│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

│                                                                                                                         │

│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

│    * Please also attach the following file to the GitHub issue:                                                         │

│    * - /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/minikube_node_1c3a1297795327375b61f3ff5a4ef34c9b2fc69b_0.log    │

│                                                                                                                         │

╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

│                                                                                                                         │

│    * If the above advice does not help, please let us know:                                                             │

│      https://github.com/kubernetes/minikube/issues/new/choose                                                           │

│                                                                                                                         │

│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                │

│    * Please also attach the following file to the GitHub issue:                                                         │

│    * - /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/minikube_node_1c3a1297795327375b61f3ff5a4ef34c9b2fc69b_0.log    │

│                                                                                                                         │

╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

I0307 14:35:13.630224    3546 out.go:177] 

multinode_test.go:285: node start returned an error. args "out/minikube-darwin-arm64 -p multinode-172000 node start m03 -v=7 --alsologtostderr": exit status 85

multinode_test.go:290: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr

multinode_test.go:290: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr: exit status 7 (31.400416ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:35:13.663928    3548 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:13.664089    3548 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:13.664092    3548 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:13.664094    3548 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:13.664234    3548 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:13.664358    3548 out.go:298] Setting JSON to false

	I0307 14:35:13.664370    3548 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:13.664427    3548 notify.go:220] Checking for updates...

	I0307 14:35:13.664569    3548 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:13.664576    3548 status.go:255] checking status of multinode-172000 ...

	I0307 14:35:13.664773    3548 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:35:13.664776    3548 status.go:343] host is not running, skipping remaining checks

	I0307 14:35:13.664778    3548 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:290: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr

multinode_test.go:290: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr: exit status 7 (77.971666ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:35:14.958976    3550 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:14.959150    3550 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:14.959155    3550 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:14.959158    3550 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:14.959345    3550 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:14.959517    3550 out.go:298] Setting JSON to false

	I0307 14:35:14.959534    3550 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:14.959576    3550 notify.go:220] Checking for updates...

	I0307 14:35:14.959806    3550 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:14.959814    3550 status.go:255] checking status of multinode-172000 ...

	I0307 14:35:14.960084    3550 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:35:14.960089    3550 status.go:343] host is not running, skipping remaining checks

	I0307 14:35:14.960092    3550 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:290: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr

multinode_test.go:290: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr: exit status 7 (73.662041ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:35:16.816054    3552 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:16.816285    3552 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:16.816290    3552 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:16.816293    3552 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:16.816454    3552 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:16.816627    3552 out.go:298] Setting JSON to false

	I0307 14:35:16.816643    3552 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:16.816681    3552 notify.go:220] Checking for updates...

	I0307 14:35:16.816892    3552 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:16.816900    3552 status.go:255] checking status of multinode-172000 ...

	I0307 14:35:16.817181    3552 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:35:16.817186    3552 status.go:343] host is not running, skipping remaining checks

	I0307 14:35:16.817189    3552 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:290: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr

multinode_test.go:290: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr: exit status 7 (76.503209ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:35:18.925601    3554 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:18.925757    3554 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:18.925761    3554 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:18.925764    3554 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:18.925954    3554 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:18.926109    3554 out.go:298] Setting JSON to false

	I0307 14:35:18.926125    3554 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:18.926162    3554 notify.go:220] Checking for updates...

	I0307 14:35:18.926373    3554 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:18.926380    3554 status.go:255] checking status of multinode-172000 ...

	I0307 14:35:18.926640    3554 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:35:18.926645    3554 status.go:343] host is not running, skipping remaining checks

	I0307 14:35:18.926648    3554 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:290: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr

multinode_test.go:290: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr: exit status 7 (78.047625ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:35:21.638069    3556 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:21.638246    3556 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:21.638251    3556 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:21.638254    3556 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:21.638419    3556 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:21.638585    3556 out.go:298] Setting JSON to false

	I0307 14:35:21.638600    3556 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:21.638627    3556 notify.go:220] Checking for updates...

	I0307 14:35:21.638867    3556 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:21.638874    3556 status.go:255] checking status of multinode-172000 ...

	I0307 14:35:21.639169    3556 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:35:21.639174    3556 status.go:343] host is not running, skipping remaining checks

	I0307 14:35:21.639177    3556 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:290: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr

multinode_test.go:290: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr: exit status 7 (76.687834ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:35:26.286376    3558 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:26.286557    3558 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:26.286561    3558 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:26.286564    3558 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:26.286711    3558 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:26.286885    3558 out.go:298] Setting JSON to false

	I0307 14:35:26.286900    3558 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:26.286943    3558 notify.go:220] Checking for updates...

	I0307 14:35:26.287200    3558 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:26.287210    3558 status.go:255] checking status of multinode-172000 ...

	I0307 14:35:26.287481    3558 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:35:26.287487    3558 status.go:343] host is not running, skipping remaining checks

	I0307 14:35:26.287490    3558 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:290: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr

multinode_test.go:290: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr: exit status 7 (75.899125ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:35:33.493125    3560 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:33.493294    3560 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:33.493299    3560 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:33.493301    3560 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:33.493464    3560 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:33.493620    3560 out.go:298] Setting JSON to false

	I0307 14:35:33.493634    3560 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:33.493670    3560 notify.go:220] Checking for updates...

	I0307 14:35:33.493889    3560 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:33.493897    3560 status.go:255] checking status of multinode-172000 ...

	I0307 14:35:33.494171    3560 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:35:33.494176    3560 status.go:343] host is not running, skipping remaining checks

	I0307 14:35:33.494179    3560 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:290: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr

multinode_test.go:290: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr: exit status 7 (74.998791ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:35:46.914311    3562 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:35:46.914493    3562 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:46.914498    3562 out.go:304] Setting ErrFile to fd 2...

	I0307 14:35:46.914501    3562 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:35:46.914662    3562 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:35:46.914822    3562 out.go:298] Setting JSON to false

	I0307 14:35:46.914838    3562 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:35:46.914875    3562 notify.go:220] Checking for updates...

	I0307 14:35:46.915072    3562 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:35:46.915079    3562 status.go:255] checking status of multinode-172000 ...

	I0307 14:35:46.915345    3562 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:35:46.915350    3562 status.go:343] host is not running, skipping remaining checks

	I0307 14:35:46.915353    3562 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:290: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr

multinode_test.go:290: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr: exit status 7 (77.081833ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:36:02.804343    3568 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:36:02.804554    3568 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:36:02.804558    3568 out.go:304] Setting ErrFile to fd 2...

	I0307 14:36:02.804561    3568 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:36:02.804727    3568 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:36:02.804906    3568 out.go:298] Setting JSON to false

	I0307 14:36:02.804923    3568 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:36:02.804947    3568 notify.go:220] Checking for updates...

	I0307 14:36:02.805201    3568 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:36:02.805209    3568 status.go:255] checking status of multinode-172000 ...

	I0307 14:36:02.805502    3568 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:36:02.805507    3568 status.go:343] host is not running, skipping remaining checks

	I0307 14:36:02.805510    3568 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:294: failed to run minikube status. args "out/minikube-darwin-arm64 -p multinode-172000 status -v=7 --alsologtostderr" : exit status 7

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (34.964541ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/StartAfterStop (49.25s)

=== RUN   TestMultiNode/serial/RestartKeepsNodes

multinode_test.go:314: (dbg) Run:  out/minikube-darwin-arm64 node list -p multinode-172000

multinode_test.go:321: (dbg) Run:  out/minikube-darwin-arm64 stop -p multinode-172000

multinode_test.go:321: (dbg) Done: out/minikube-darwin-arm64 stop -p multinode-172000: (3.717384333s)

multinode_test.go:326: (dbg) Run:  out/minikube-darwin-arm64 start -p multinode-172000 --wait=true -v=8 --alsologtostderr

multinode_test.go:326: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p multinode-172000 --wait=true -v=8 --alsologtostderr: exit status 80 (5.216200042s)

-- stdout --

	* [multinode-172000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on existing profile

	* Starting "multinode-172000" primary control-plane node in "multinode-172000" cluster

	* Restarting existing qemu2 VM for "multinode-172000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Restarting existing qemu2 VM for "multinode-172000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0307 14:36:06.656003    3592 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:36:06.656155    3592 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:36:06.656162    3592 out.go:304] Setting ErrFile to fd 2...

	I0307 14:36:06.656166    3592 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:36:06.656370    3592 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:36:06.657606    3592 out.go:298] Setting JSON to false

	I0307 14:36:06.676514    3592 start.go:129] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":3923,"bootTime":1709847043,"procs":467,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.3.1","kernelVersion":"23.3.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0307 14:36:06.676579    3592 start.go:137] gopshost.Virtualization returned error: not implemented yet

	I0307 14:36:06.681654    3592 out.go:177] * [multinode-172000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	I0307 14:36:06.684624    3592 out.go:177]   - MINIKUBE_LOCATION=18320

	I0307 14:36:06.688538    3592 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	I0307 14:36:06.684669    3592 notify.go:220] Checking for updates...

	I0307 14:36:06.692419    3592 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0307 14:36:06.695536    3592 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0307 14:36:06.698537    3592 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	I0307 14:36:06.701591    3592 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0307 14:36:06.704938    3592 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:36:06.705009    3592 driver.go:392] Setting default libvirt URI to qemu:///system

	I0307 14:36:06.709538    3592 out.go:177] * Using the qemu2 driver based on existing profile

	I0307 14:36:06.716477    3592 start.go:297] selected driver: qemu2

	I0307 14:36:06.716482    3592 start.go:901] validating driver "qemu2" against &{Name:multinode-172000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{Kube

rnetesVersion:v1.28.4 ClusterName:multinode-172000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] M

ountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 14:36:06.716541    3592 start.go:912] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0307 14:36:06.718961    3592 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0307 14:36:06.719011    3592 cni.go:84] Creating CNI manager for ""

	I0307 14:36:06.719016    3592 cni.go:136] multinode detected (1 nodes found), recommending kindnet

	I0307 14:36:06.719071    3592 start.go:340] cluster config:

	{Name:multinode-172000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:multinode-172000 Namespace:default APIServerH

AVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:fals

e DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 14:36:06.723644    3592 iso.go:125] acquiring lock: {Name:mk9f11fe8095f5aa8805d47835226b5c64c27e19 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0307 14:36:06.729495    3592 out.go:177] * Starting "multinode-172000" primary control-plane node in "multinode-172000" cluster

	I0307 14:36:06.733485    3592 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0307 14:36:06.733500    3592 preload.go:147] Found local preload: /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4

	I0307 14:36:06.733508    3592 cache.go:56] Caching tarball of preloaded images

	I0307 14:36:06.733562    3592 preload.go:173] Found /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0307 14:36:06.733567    3592 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on docker

	I0307 14:36:06.733628    3592 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/multinode-172000/config.json ...

	I0307 14:36:06.734136    3592 start.go:360] acquireMachinesLock for multinode-172000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:36:06.734171    3592 start.go:364] duration metric: took 28.166µs to acquireMachinesLock for "multinode-172000"

	I0307 14:36:06.734180    3592 start.go:96] Skipping create...Using existing machine configuration

	I0307 14:36:06.734184    3592 fix.go:54] fixHost starting: 

	I0307 14:36:06.734314    3592 fix.go:112] recreateIfNeeded on multinode-172000: state=Stopped err=<nil>

	W0307 14:36:06.734323    3592 fix.go:138] unexpected machine state, will restart: <nil>

	I0307 14:36:06.741528    3592 out.go:177] * Restarting existing qemu2 VM for "multinode-172000" ...

	I0307 14:36:06.745566    3592 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2200 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/qemu.pid -device virtio-net-pci,netdev=net0,mac=ea:c0:e3:0f:13:78 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2

	I0307 14:36:06.747668    3592 main.go:141] libmachine: STDOUT: 

	I0307 14:36:06.747691    3592 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:36:06.747723    3592 fix.go:56] duration metric: took 13.535666ms for fixHost

	I0307 14:36:06.747727    3592 start.go:83] releasing machines lock for "multinode-172000", held for 13.551167ms

	W0307 14:36:06.747734    3592 start.go:713] error starting host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0307 14:36:06.747777    3592 out.go:239] ! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:36:06.747783    3592 start.go:728] Will try again in 5 seconds ...

	I0307 14:36:11.749806    3592 start.go:360] acquireMachinesLock for multinode-172000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:36:11.750256    3592 start.go:364] duration metric: took 335.125µs to acquireMachinesLock for "multinode-172000"

	I0307 14:36:11.750373    3592 start.go:96] Skipping create...Using existing machine configuration

	I0307 14:36:11.750397    3592 fix.go:54] fixHost starting: 

	I0307 14:36:11.751078    3592 fix.go:112] recreateIfNeeded on multinode-172000: state=Stopped err=<nil>

	W0307 14:36:11.751104    3592 fix.go:138] unexpected machine state, will restart: <nil>

	I0307 14:36:11.756449    3592 out.go:177] * Restarting existing qemu2 VM for "multinode-172000" ...

	I0307 14:36:11.761570    3592 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2200 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/qemu.pid -device virtio-net-pci,netdev=net0,mac=ea:c0:e3:0f:13:78 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2

	I0307 14:36:11.771514    3592 main.go:141] libmachine: STDOUT: 

	I0307 14:36:11.771641    3592 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:36:11.771733    3592 fix.go:56] duration metric: took 21.334125ms for fixHost

	I0307 14:36:11.771750    3592 start.go:83] releasing machines lock for "multinode-172000", held for 21.47125ms

	W0307 14:36:11.772126    3592 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p multinode-172000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p multinode-172000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:36:11.778175    3592 out.go:177] 

	W0307 14:36:11.782448    3592 out.go:239] X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0307 14:36:11.782474    3592 out.go:239] * 

	* 

	W0307 14:36:11.784979    3592 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0307 14:36:11.793415    3592 out.go:177] 

** /stderr **

multinode_test.go:328: failed to run minikube start. args "out/minikube-darwin-arm64 node list -p multinode-172000" : exit status 80

multinode_test.go:331: (dbg) Run:  out/minikube-darwin-arm64 node list -p multinode-172000

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (33.695708ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/RestartKeepsNodes (9.07s)

=== RUN   TestMultiNode/serial/DeleteNode

multinode_test.go:416: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 node delete m03

multinode_test.go:416: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 node delete m03: exit status 83 (41.550708ms)

-- stdout --

	* The control-plane node multinode-172000 host is not running: state=Stopped

	  To start a cluster, run: "minikube start -p multinode-172000"

-- /stdout --

multinode_test.go:418: node delete returned an error. args "out/minikube-darwin-arm64 -p multinode-172000 node delete m03": exit status 83

multinode_test.go:422: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status --alsologtostderr

multinode_test.go:422: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status --alsologtostderr: exit status 7 (31.918667ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:36:11.983960    3606 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:36:11.984109    3606 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:36:11.984113    3606 out.go:304] Setting ErrFile to fd 2...

	I0307 14:36:11.984115    3606 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:36:11.984251    3606 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:36:11.984377    3606 out.go:298] Setting JSON to false

	I0307 14:36:11.984388    3606 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:36:11.984439    3606 notify.go:220] Checking for updates...

	I0307 14:36:11.984589    3606 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:36:11.984595    3606 status.go:255] checking status of multinode-172000 ...

	I0307 14:36:11.984797    3606 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:36:11.984801    3606 status.go:343] host is not running, skipping remaining checks

	I0307 14:36:11.984803    3606 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:424: failed to run minikube status. args "out/minikube-darwin-arm64 -p multinode-172000 status --alsologtostderr" : exit status 7

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (31.673292ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/DeleteNode (0.11s)

=== RUN   TestMultiNode/serial/StopMultiNode

multinode_test.go:345: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 stop

multinode_test.go:345: (dbg) Done: out/minikube-darwin-arm64 -p multinode-172000 stop: (1.858111875s)

multinode_test.go:351: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status

multinode_test.go:351: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status: exit status 7 (65.839583ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

multinode_test.go:358: (dbg) Run:  out/minikube-darwin-arm64 -p multinode-172000 status --alsologtostderr

multinode_test.go:358: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p multinode-172000 status --alsologtostderr: exit status 7 (33.665708ms)

-- stdout --

	multinode-172000

	type: Control Plane

	host: Stopped

	kubelet: Stopped

	apiserver: Stopped

	kubeconfig: Stopped

-- /stdout --

** stderr ** 

	I0307 14:36:13.973897    3626 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:36:13.974047    3626 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:36:13.974050    3626 out.go:304] Setting ErrFile to fd 2...

	I0307 14:36:13.974053    3626 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:36:13.974178    3626 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:36:13.974303    3626 out.go:298] Setting JSON to false

	I0307 14:36:13.974314    3626 mustload.go:65] Loading cluster: multinode-172000

	I0307 14:36:13.974380    3626 notify.go:220] Checking for updates...

	I0307 14:36:13.974499    3626 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:36:13.974504    3626 status.go:255] checking status of multinode-172000 ...

	I0307 14:36:13.974738    3626 status.go:330] multinode-172000 host status = "Stopped" (err=<nil>)

	I0307 14:36:13.974743    3626 status.go:343] host is not running, skipping remaining checks

	I0307 14:36:13.974746    3626 status.go:257] multinode-172000 status: &{Name:multinode-172000 Host:Stopped Kubelet:Stopped APIServer:Stopped Kubeconfig:Stopped Worker:false TimeToStop: DockerEnv: PodManEnv:}

** /stderr **

multinode_test.go:364: incorrect number of stopped hosts: args "out/minikube-darwin-arm64 -p multinode-172000 status --alsologtostderr": multinode-172000

type: Control Plane

host: Stopped

kubelet: Stopped

apiserver: Stopped

kubeconfig: Stopped

multinode_test.go:368: incorrect number of stopped kubelets: args "out/minikube-darwin-arm64 -p multinode-172000 status --alsologtostderr": multinode-172000

type: Control Plane

host: Stopped

kubelet: Stopped

apiserver: Stopped

kubeconfig: Stopped

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (31.371125ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/StopMultiNode (1.99s)

=== RUN   TestMultiNode/serial/RestartMultiNode

multinode_test.go:376: (dbg) Run:  out/minikube-darwin-arm64 start -p multinode-172000 --wait=true -v=8 --alsologtostderr --driver=qemu2 

multinode_test.go:376: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p multinode-172000 --wait=true -v=8 --alsologtostderr --driver=qemu2 : exit status 80 (5.18542s)

-- stdout --

	* [multinode-172000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	  - MINIKUBE_LOCATION=18320

	  - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on existing profile

	* Starting "multinode-172000" primary control-plane node in "multinode-172000" cluster

	* Restarting existing qemu2 VM for "multinode-172000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Restarting existing qemu2 VM for "multinode-172000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0307 14:36:14.036778    3630 out.go:291] Setting OutFile to fd 1 ...

	I0307 14:36:14.036886    3630 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:36:14.036889    3630 out.go:304] Setting ErrFile to fd 2...

	I0307 14:36:14.036892    3630 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0307 14:36:14.037010    3630 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/18320-1153/.minikube/bin

	I0307 14:36:14.038003    3630 out.go:298] Setting JSON to false

	I0307 14:36:14.054083    3630 start.go:129] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":3931,"bootTime":1709847043,"procs":471,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.3.1","kernelVersion":"23.3.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0307 14:36:14.054142    3630 start.go:137] gopshost.Virtualization returned error: not implemented yet

	I0307 14:36:14.058049    3630 out.go:177] * [multinode-172000] minikube v1.32.0 on Darwin 14.3.1 (arm64)

	I0307 14:36:14.065817    3630 out.go:177]   - MINIKUBE_LOCATION=18320

	I0307 14:36:14.065876    3630 notify.go:220] Checking for updates...

	I0307 14:36:14.069817    3630 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/18320-1153/kubeconfig

	I0307 14:36:14.072777    3630 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0307 14:36:14.075809    3630 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0307 14:36:14.078881    3630 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/18320-1153/.minikube

	I0307 14:36:14.081743    3630 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0307 14:36:14.085011    3630 config.go:182] Loaded profile config "multinode-172000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.4

	I0307 14:36:14.085267    3630 driver.go:392] Setting default libvirt URI to qemu:///system

	I0307 14:36:14.089781    3630 out.go:177] * Using the qemu2 driver based on existing profile

	I0307 14:36:14.096790    3630 start.go:297] selected driver: qemu2

	I0307 14:36:14.096796    3630 start.go:901] validating driver "qemu2" against &{Name:multinode-172000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{Kube

rnetesVersion:v1.28.4 ClusterName:multinode-172000 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] M

ountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 14:36:14.096853    3630 start.go:912] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0307 14:36:14.099104    3630 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0307 14:36:14.099152    3630 cni.go:84] Creating CNI manager for ""

	I0307 14:36:14.099158    3630 cni.go:136] multinode detected (1 nodes found), recommending kindnet

	I0307 14:36:14.099203    3630 start.go:340] cluster config:

	{Name:multinode-172000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/18020/minikube-v1.32.1-1708638130-18020-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:multinode-172000 Namespace:default APIServerH

AVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:fals

e DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0307 14:36:14.103605    3630 iso.go:125] acquiring lock: {Name:mk9f11fe8095f5aa8805d47835226b5c64c27e19 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0307 14:36:14.110815    3630 out.go:177] * Starting "multinode-172000" primary control-plane node in "multinode-172000" cluster

	I0307 14:36:14.114807    3630 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime docker

	I0307 14:36:14.114823    3630 preload.go:147] Found local preload: /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4

	I0307 14:36:14.114841    3630 cache.go:56] Caching tarball of preloaded images

	I0307 14:36:14.114903    3630 preload.go:173] Found /Users/jenkins/minikube-integration/18320-1153/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0307 14:36:14.114910    3630 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on docker

	I0307 14:36:14.114973    3630 profile.go:142] Saving config to /Users/jenkins/minikube-integration/18320-1153/.minikube/profiles/multinode-172000/config.json ...

	I0307 14:36:14.115448    3630 start.go:360] acquireMachinesLock for multinode-172000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:36:14.115474    3630 start.go:364] duration metric: took 19.708µs to acquireMachinesLock for "multinode-172000"

	I0307 14:36:14.115481    3630 start.go:96] Skipping create...Using existing machine configuration

	I0307 14:36:14.115488    3630 fix.go:54] fixHost starting: 

	I0307 14:36:14.115609    3630 fix.go:112] recreateIfNeeded on multinode-172000: state=Stopped err=<nil>

	W0307 14:36:14.115618    3630 fix.go:138] unexpected machine state, will restart: <nil>

	I0307 14:36:14.123793    3630 out.go:177] * Restarting existing qemu2 VM for "multinode-172000" ...

	I0307 14:36:14.127825    3630 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2200 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/qemu.pid -device virtio-net-pci,netdev=net0,mac=ea:c0:e3:0f:13:78 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2

	I0307 14:36:14.129853    3630 main.go:141] libmachine: STDOUT: 

	I0307 14:36:14.129869    3630 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:36:14.129898    3630 fix.go:56] duration metric: took 14.408833ms for fixHost

	I0307 14:36:14.129904    3630 start.go:83] releasing machines lock for "multinode-172000", held for 14.42625ms

	W0307 14:36:14.129909    3630 start.go:713] error starting host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0307 14:36:14.129948    3630 out.go:239] ! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:36:14.129953    3630 start.go:728] Will try again in 5 seconds ...

	I0307 14:36:19.132096    3630 start.go:360] acquireMachinesLock for multinode-172000: {Name:mk6de7c633b714084d6e2f7358f5de3d48a293fc Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0307 14:36:19.132380    3630 start.go:364] duration metric: took 221.917µs to acquireMachinesLock for "multinode-172000"

	I0307 14:36:19.132504    3630 start.go:96] Skipping create...Using existing machine configuration

	I0307 14:36:19.132528    3630 fix.go:54] fixHost starting: 

	I0307 14:36:19.133157    3630 fix.go:112] recreateIfNeeded on multinode-172000: state=Stopped err=<nil>

	W0307 14:36:19.133181    3630 fix.go:138] unexpected machine state, will restart: <nil>

	I0307 14:36:19.138657    3630 out.go:177] * Restarting existing qemu2 VM for "multinode-172000" ...

	I0307 14:36:19.146716    3630 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2200 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/qemu.pid -device virtio-net-pci,netdev=net0,mac=ea:c0:e3:0f:13:78 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/18320-1153/.minikube/machines/multinode-172000/disk.qcow2

	I0307 14:36:19.156029    3630 main.go:141] libmachine: STDOUT: 

	I0307 14:36:19.156097    3630 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0307 14:36:19.156171    3630 fix.go:56] duration metric: took 23.649334ms for fixHost

	I0307 14:36:19.156192    3630 start.go:83] releasing machines lock for "multinode-172000", held for 23.787584ms

	W0307 14:36:19.156402    3630 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p multinode-172000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p multinode-172000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0307 14:36:19.163491    3630 out.go:177] 

	W0307 14:36:19.167615    3630 out.go:239] X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0307 14:36:19.167644    3630 out.go:239] * 

	* 

	W0307 14:36:19.170019    3630 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0307 14:36:19.178641    3630 out.go:177] 

** /stderr **

multinode_test.go:378: failed to start cluster. args "out/minikube-darwin-arm64 start -p multinode-172000 --wait=true -v=8 --alsologtostderr --driver=qemu2 " : exit status 80

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p multinode-172000 -n multinode-172000: exit status 7 (69.729375ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "multinode-172000" host is not running, skipping log retrieval (state="Stopped")

--- FAIL: TestMultiNode/serial/RestartMultiNode (5.26s)