=== RUN   TestDownloadOnly/v1.16.0/json-events

aaa_download_only_test.go:69: (dbg) Run:  out/minikube-darwin-arm64 start -o=json --download-only -p download-only-242000 --force --alsologtostderr --kubernetes-version=v1.16.0 --container-runtime=docker --driver=qemu2 

aaa_download_only_test.go:69: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -o=json --download-only -p download-only-242000 --force --alsologtostderr --kubernetes-version=v1.16.0 --container-runtime=docker --driver=qemu2 : exit status 40 (17.18728125s)

-- stdout --

	{"specversion":"1.0","id":"0a4292e4-b52f-443b-842f-5ccf28323b5f","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"0","message":"[download-only-242000] minikube v1.31.2 on Darwin 13.5.2 (arm64)","name":"Initial Minikube Setup","totalsteps":"19"}}

	{"specversion":"1.0","id":"7bcb4809-7d41-4ac9-a264-6a3b9f75fe90","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_LOCATION=17263"}}

	{"specversion":"1.0","id":"187c1539-12ce-4bfc-95b4-f86037e2aa31","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig"}}

	{"specversion":"1.0","id":"af0fe2df-0392-4bd2-97be-82164d28ea3d","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_BIN=out/minikube-darwin-arm64"}}

	{"specversion":"1.0","id":"b5539383-3de7-4f01-80b7-32b20f3a9dc2","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true"}}

	{"specversion":"1.0","id":"1b4e229c-e30d-4e8c-a58f-7c00d22c8a44","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube"}}

	{"specversion":"1.0","id":"6ba644e3-9410-448b-9bc2-e3672dd857ed","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.warning","datacontenttype":"application/json","data":{"message":"minikube skips various validations when --force is supplied; this may lead to unexpected behavior"}}

	{"specversion":"1.0","id":"a8a053f1-cbaa-463a-8d34-75da73bfee5a","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"1","message":"Using the qemu2 driver based on user configuration","name":"Selecting Driver","totalsteps":"19"}}

	{"specversion":"1.0","id":"50ece0ce-e7cd-4644-8ff1-68f87375e67b","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"Automatically selected the socket_vmnet network"}}

	{"specversion":"1.0","id":"92b70260-8e39-4249-9978-b059c5d05e7b","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"1","message":"Downloading VM boot image ...","name":"Selecting Driver","totalsteps":"19"}}

	{"specversion":"1.0","id":"ddb71033-f4fa-415f-8ee6-7fa282313a79","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"3","message":"Starting control plane node download-only-242000 in cluster download-only-242000","name":"Starting Node","totalsteps":"19"}}

	{"specversion":"1.0","id":"36526bff-16af-41e5-9ade-29c255eb4d66","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"3","message":"Downloading Kubernetes v1.16.0 preload ...","name":"Starting Node","totalsteps":"19"}}

	{"specversion":"1.0","id":"81e3b15c-77c9-45ff-8968-31555d1ed0c9","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.error","datacontenttype":"application/json","data":{"advice":"","exitcode":"40","issues":"","message":"Failed to cache kubectl: download failed: https://dl.k8s.io/release/v1.16.0/bin/darwin/arm64/kubectl?checksum=file:https://dl.k8s.io/release/v1.16.0/bin/darwin/arm64/kubectl.sha1: getter: \u0026{Ctx:context.Background Src:https://dl.k8s.io/release/v1.16.0/bin/darwin/arm64/kubectl?checksum=file:https://dl.k8s.io/release/v1.16.0/bin/darwin/arm64/kubectl.sha1 Dst:/Users/jenkins/minikube-integration/17263-1251/.minikube/cache/darwin/arm64/v1.16.0/kubectl.download Pwd: Mode:2 Umask:---------- Detectors:[0x1083497e0 0x1083497e0 0x1083497e0 0x1083497e0 0x1083497e0 0x1083497e0 0x1083497e0] Decompressors:map[bz2:0x1400013cdd0 gz:0x1400013cdd8 tar:0x1400013cd10 tar.bz2:0x1400013cd20 tar.gz:0x1400013cd30 tar.xz:0x1400013cd50 tar.zst:0x1400013cd60 tbz2:0x1400013cd20 tgz:0x140001

3cd30 txz:0x1400013cd50 tzst:0x1400013cd60 xz:0x1400013cde0 zip:0x1400013ce20 zst:0x1400013cde8] Getters:map[file:0x14000cbcdb0 http:0x1400017e910 https:0x1400017e960] Dir:false ProgressListener:\u003cnil\u003e Insecure:false DisableSymlinks:false Options:[]}: invalid checksum: Error downloading checksum file: bad response code: 404","name":"INET_CACHE_KUBECTL","url":""}}

	{"specversion":"1.0","id":"a9e28e35-ed81-47e0-b44c-ec563048bf85","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.error","datacontenttype":"application/json","data":{"message":"╭───────────────────────────────────────────────────────────────────────────────────────────╮\n│                                                                                           │\n│    If the above advice does not help, please let us know:                                 │\n│    https://github.com/kubernetes/minikube/issues/new/choose                               │\n│                                                                                           │\n│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │\n│

│\n╰───────────────────────────────────────────────────────────────────────────────────────────╯"}}

-- /stdout --

** stderr ** 

	I0918 11:51:43.673352    1670 out.go:296] Setting OutFile to fd 1 ...

	I0918 11:51:43.673496    1670 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 11:51:43.673499    1670 out.go:309] Setting ErrFile to fd 2...

	I0918 11:51:43.673502    1670 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 11:51:43.673628    1670 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17263-1251/.minikube/bin

	W0918 11:51:43.673712    1670 root.go:314] Error reading config file at /Users/jenkins/minikube-integration/17263-1251/.minikube/config/config.json: open /Users/jenkins/minikube-integration/17263-1251/.minikube/config/config.json: no such file or directory

	I0918 11:51:43.674814    1670 out.go:303] Setting JSON to true

	I0918 11:51:43.691096    1670 start.go:128] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":1277,"bootTime":1695061826,"procs":415,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.5.2","kernelVersion":"22.6.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0918 11:51:43.691153    1670 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0918 11:51:43.696788    1670 out.go:97] [download-only-242000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	I0918 11:51:43.700713    1670 out.go:169] MINIKUBE_LOCATION=17263

	I0918 11:51:43.696908    1670 notify.go:220] Checking for updates...

	W0918 11:51:43.696937    1670 preload.go:295] Failed to list preload files: open /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball: no such file or directory

	I0918 11:51:43.707561    1670 out.go:169] KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 11:51:43.711781    1670 out.go:169] MINIKUBE_BIN=out/minikube-darwin-arm64

	I0918 11:51:43.714758    1670 out.go:169] MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0918 11:51:43.716116    1670 out.go:169] MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	W0918 11:51:43.721757    1670 out.go:272] minikube skips various validations when --force is supplied; this may lead to unexpected behavior

	I0918 11:51:43.721987    1670 driver.go:373] Setting default libvirt URI to qemu:///system

	I0918 11:51:43.727772    1670 out.go:97] Using the qemu2 driver based on user configuration

	I0918 11:51:43.727778    1670 start.go:298] selected driver: qemu2

	I0918 11:51:43.727792    1670 start.go:902] validating driver "qemu2" against <nil>

	I0918 11:51:43.727853    1670 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0918 11:51:43.731695    1670 out.go:169] Automatically selected the socket_vmnet network

	I0918 11:51:43.737213    1670 start_flags.go:384] Using suggested 4000MB memory alloc based on sys=16384MB, container=0MB

	I0918 11:51:43.737298    1670 start_flags.go:904] Wait components to verify : map[apiserver:true system_pods:true]

	I0918 11:51:43.737361    1670 cni.go:84] Creating CNI manager for ""

	I0918 11:51:43.737378    1670 cni.go:162] CNI unnecessary in this configuration, recommending no CNI

	I0918 11:51:43.737384    1670 start_flags.go:321] config:

	{Name:download-only-242000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:download-only-242000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRun

time:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 11:51:43.742567    1670 iso.go:125] acquiring lock: {Name:mk34e23c181861c65264ec384f06e6cfa001aa08 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0918 11:51:43.745703    1670 out.go:97] Downloading VM boot image ...

	I0918 11:51:43.745736    1670 download.go:107] Downloading: https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso?checksum=file:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso.sha256 -> /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso

	I0918 11:51:50.801506    1670 out.go:97] Starting control plane node download-only-242000 in cluster download-only-242000

	I0918 11:51:50.801529    1670 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker

	I0918 11:51:50.861845    1670 preload.go:119] Found remote preload: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.16.0/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-arm64.tar.lz4

	I0918 11:51:50.861853    1670 cache.go:57] Caching tarball of preloaded images

	I0918 11:51:50.862024    1670 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker

	I0918 11:51:50.865690    1670 out.go:97] Downloading Kubernetes v1.16.0 preload ...

	I0918 11:51:50.865697    1670 preload.go:238] getting checksum for preloaded-images-k8s-v18-v1.16.0-docker-overlay2-arm64.tar.lz4 ...

	I0918 11:51:50.946758    1670 download.go:107] Downloading: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.16.0/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-arm64.tar.lz4?checksum=md5:a000baffb0664b293d602f95ed25caa6 -> /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-arm64.tar.lz4

	I0918 11:51:59.686637    1670 preload.go:249] saving checksum for preloaded-images-k8s-v18-v1.16.0-docker-overlay2-arm64.tar.lz4 ...

	I0918 11:51:59.686787    1670 preload.go:256] verifying checksum of /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-arm64.tar.lz4 ...

	I0918 11:52:00.326560    1670 cache.go:60] Finished verifying existence of preloaded tar for  v1.16.0 on docker

	I0918 11:52:00.326753    1670 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/download-only-242000/config.json ...

	I0918 11:52:00.326771    1670 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/download-only-242000/config.json: {Name:mk2d38f7178624dd8e5685d2e554cb81270be80a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:00.327027    1670 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker

	I0918 11:52:00.327185    1670 download.go:107] Downloading: https://dl.k8s.io/release/v1.16.0/bin/darwin/arm64/kubectl?checksum=file:https://dl.k8s.io/release/v1.16.0/bin/darwin/arm64/kubectl.sha1 -> /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/darwin/arm64/v1.16.0/kubectl

	I0918 11:52:00.800545    1670 out.go:169] 

	W0918 11:52:00.804618    1670 out_reason.go:110] Failed to cache kubectl: download failed: https://dl.k8s.io/release/v1.16.0/bin/darwin/arm64/kubectl?checksum=file:https://dl.k8s.io/release/v1.16.0/bin/darwin/arm64/kubectl.sha1: getter: &{Ctx:context.Background Src:https://dl.k8s.io/release/v1.16.0/bin/darwin/arm64/kubectl?checksum=file:https://dl.k8s.io/release/v1.16.0/bin/darwin/arm64/kubectl.sha1 Dst:/Users/jenkins/minikube-integration/17263-1251/.minikube/cache/darwin/arm64/v1.16.0/kubectl.download Pwd: Mode:2 Umask:---------- Detectors:[0x1083497e0 0x1083497e0 0x1083497e0 0x1083497e0 0x1083497e0 0x1083497e0 0x1083497e0] Decompressors:map[bz2:0x1400013cdd0 gz:0x1400013cdd8 tar:0x1400013cd10 tar.bz2:0x1400013cd20 tar.gz:0x1400013cd30 tar.xz:0x1400013cd50 tar.zst:0x1400013cd60 tbz2:0x1400013cd20 tgz:0x1400013cd30 txz:0x1400013cd50 tzst:0x1400013cd60 xz:0x1400013cde0 zip:0x1400013ce20 zst:0x1400013cde8] Getters:map[file:0x14000cbcdb0 http:0x1400017e910 https:0x1400017e960] Dir:false ProgressListener

:<nil> Insecure:false DisableSymlinks:false Options:[]}: invalid checksum: Error downloading checksum file: bad response code: 404

	W0918 11:52:00.804643    1670 out_reason.go:110] 

	W0918 11:52:00.809566    1670 out.go:229] ╭───────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                           │

	│    If the above advice does not help, please let us know:                                 │

	│    https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                           │

	│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                           │

	╰───────────────────────────────────────────────────────────────────────────────────────────╯

	I0918 11:52:00.813560    1670 out.go:169] 

** /stderr **

aaa_download_only_test.go:71: failed to download only. args: ["start" "-o=json" "--download-only" "-p" "download-only-242000" "--force" "--alsologtostderr" "--kubernetes-version=v1.16.0" "--container-runtime=docker" "--driver=qemu2" ""] exit status 40

--- FAIL: TestDownloadOnly/v1.16.0/json-events (17.19s)

=== RUN   TestDownloadOnly/v1.16.0/kubectl

aaa_download_only_test.go:160: expected the file for binary exist at "/Users/jenkins/minikube-integration/17263-1251/.minikube/cache/darwin/arm64/v1.16.0/kubectl" but got error stat /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/darwin/arm64/v1.16.0/kubectl: no such file or directory

--- FAIL: TestDownloadOnly/v1.16.0/kubectl (0.00s)

=== RUN   TestOffline

=== PAUSE TestOffline

=== CONT  TestOffline

aab_offline_test.go:55: (dbg) Run:  out/minikube-darwin-arm64 start -p offline-docker-612000 --alsologtostderr -v=1 --memory=2048 --wait=true --driver=qemu2 

aab_offline_test.go:55: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p offline-docker-612000 --alsologtostderr -v=1 --memory=2048 --wait=true --driver=qemu2 : exit status 80 (9.768181584s)

-- stdout --

	* [offline-docker-612000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	  - MINIKUBE_LOCATION=17263

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting control plane node offline-docker-612000 in cluster offline-docker-612000

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "offline-docker-612000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0918 12:38:08.292178    3838 out.go:296] Setting OutFile to fd 1 ...

	I0918 12:38:08.292340    3838 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:38:08.292343    3838 out.go:309] Setting ErrFile to fd 2...

	I0918 12:38:08.292346    3838 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:38:08.292481    3838 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17263-1251/.minikube/bin

	I0918 12:38:08.293470    3838 out.go:303] Setting JSON to false

	I0918 12:38:08.309952    3838 start.go:128] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":4062,"bootTime":1695061826,"procs":403,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.5.2","kernelVersion":"22.6.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0918 12:38:08.310038    3838 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0918 12:38:08.314715    3838 out.go:177] * [offline-docker-612000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	I0918 12:38:08.326750    3838 out.go:177]   - MINIKUBE_LOCATION=17263

	I0918 12:38:08.330726    3838 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 12:38:08.326858    3838 notify.go:220] Checking for updates...

	I0918 12:38:08.336590    3838 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0918 12:38:08.339896    3838 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0918 12:38:08.342698    3838 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	I0918 12:38:08.344192    3838 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0918 12:38:08.347870    3838 driver.go:373] Setting default libvirt URI to qemu:///system

	I0918 12:38:08.351769    3838 out.go:177] * Using the qemu2 driver based on user configuration

	I0918 12:38:08.357748    3838 start.go:298] selected driver: qemu2

	I0918 12:38:08.357754    3838 start.go:902] validating driver "qemu2" against <nil>

	I0918 12:38:08.357760    3838 start.go:913] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0918 12:38:08.359815    3838 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0918 12:38:08.362748    3838 out.go:177] * Automatically selected the socket_vmnet network

	I0918 12:38:08.365767    3838 start_flags.go:922] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0918 12:38:08.365784    3838 cni.go:84] Creating CNI manager for ""

	I0918 12:38:08.365790    3838 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 12:38:08.365793    3838 start_flags.go:316] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0918 12:38:08.365797    3838 start_flags.go:321] config:

	{Name:offline-docker-612000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:offline-docker-612000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerR

untime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthS

ock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 12:38:08.369974    3838 iso.go:125] acquiring lock: {Name:mk34e23c181861c65264ec384f06e6cfa001aa08 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0918 12:38:08.372629    3838 out.go:177] * Starting control plane node offline-docker-612000 in cluster offline-docker-612000

	I0918 12:38:08.380687    3838 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 12:38:08.380712    3838 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4

	I0918 12:38:08.380723    3838 cache.go:57] Caching tarball of preloaded images

	I0918 12:38:08.380791    3838 preload.go:174] Found /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0918 12:38:08.380796    3838 cache.go:60] Finished verifying existence of preloaded tar for  v1.28.2 on docker

	I0918 12:38:08.380995    3838 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/offline-docker-612000/config.json ...

	I0918 12:38:08.381008    3838 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/offline-docker-612000/config.json: {Name:mkbaf75ec2851f5165d7018699b7880738d6fc35 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:38:08.381225    3838 start.go:365] acquiring machines lock for offline-docker-612000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 12:38:08.381262    3838 start.go:369] acquired machines lock for "offline-docker-612000" in 28.75µs

	I0918 12:38:08.381275    3838 start.go:93] Provisioning new machine with config: &{Name:offline-docker-612000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kuberne

tesConfig:{KubernetesVersion:v1.28.2 ClusterName:offline-docker-612000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP

: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 12:38:08.381301    3838 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 12:38:08.385610    3838 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0918 12:38:08.399705    3838 start.go:159] libmachine.API.Create for "offline-docker-612000" (driver="qemu2")

	I0918 12:38:08.399729    3838 client.go:168] LocalClient.Create starting

	I0918 12:38:08.399803    3838 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 12:38:08.399828    3838 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:08.399840    3838 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:08.399885    3838 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 12:38:08.399903    3838 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:08.399912    3838 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:08.400245    3838 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 12:38:08.515564    3838 main.go:141] libmachine: Creating SSH key...

	I0918 12:38:08.611751    3838 main.go:141] libmachine: Creating Disk image...

	I0918 12:38:08.611764    3838 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 12:38:08.611919    3838 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2

	I0918 12:38:08.620628    3838 main.go:141] libmachine: STDOUT: 

	I0918 12:38:08.620643    3838 main.go:141] libmachine: STDERR: 

	I0918 12:38:08.620694    3838 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2 +20000M

	I0918 12:38:08.628401    3838 main.go:141] libmachine: STDOUT: Image resized.

	I0918 12:38:08.628424    3838 main.go:141] libmachine: STDERR: 

	I0918 12:38:08.628453    3838 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2

	I0918 12:38:08.628462    3838 main.go:141] libmachine: Starting QEMU VM...

	I0918 12:38:08.628508    3838 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/qemu.pid -device virtio-net-pci,netdev=net0,mac=3e:28:91:25:d0:0b -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2

	I0918 12:38:08.630270    3838 main.go:141] libmachine: STDOUT: 

	I0918 12:38:08.630297    3838 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0918 12:38:08.630321    3838 client.go:171] LocalClient.Create took 230.588541ms

	I0918 12:38:10.632348    3838 start.go:128] duration metric: createHost completed in 2.251081958s

	I0918 12:38:10.632366    3838 start.go:83] releasing machines lock for "offline-docker-612000", held for 2.251141916s

	W0918 12:38:10.632379    3838 start.go:688] error starting host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:10.637847    3838 out.go:177] * Deleting "offline-docker-612000" in qemu2 ...

	W0918 12:38:10.655236    3838 out.go:239] ! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:10.655243    3838 start.go:703] Will try again in 5 seconds ...

	I0918 12:38:15.657308    3838 start.go:365] acquiring machines lock for offline-docker-612000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 12:38:15.657592    3838 start.go:369] acquired machines lock for "offline-docker-612000" in 216.459µs

	I0918 12:38:15.657683    3838 start.go:93] Provisioning new machine with config: &{Name:offline-docker-612000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kuberne

tesConfig:{KubernetesVersion:v1.28.2 ClusterName:offline-docker-612000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP

: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 12:38:15.657851    3838 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 12:38:15.669078    3838 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0918 12:38:15.710262    3838 start.go:159] libmachine.API.Create for "offline-docker-612000" (driver="qemu2")

	I0918 12:38:15.710305    3838 client.go:168] LocalClient.Create starting

	I0918 12:38:15.710440    3838 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 12:38:15.710500    3838 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:15.710534    3838 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:15.710609    3838 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 12:38:15.710657    3838 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:15.710674    3838 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:15.711196    3838 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 12:38:15.898596    3838 main.go:141] libmachine: Creating SSH key...

	I0918 12:38:15.977592    3838 main.go:141] libmachine: Creating Disk image...

	I0918 12:38:15.977601    3838 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 12:38:15.977744    3838 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2

	I0918 12:38:15.986454    3838 main.go:141] libmachine: STDOUT: 

	I0918 12:38:15.986469    3838 main.go:141] libmachine: STDERR: 

	I0918 12:38:15.986523    3838 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2 +20000M

	I0918 12:38:15.993922    3838 main.go:141] libmachine: STDOUT: Image resized.

	I0918 12:38:15.993936    3838 main.go:141] libmachine: STDERR: 

	I0918 12:38:15.993951    3838 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2

	I0918 12:38:15.993959    3838 main.go:141] libmachine: Starting QEMU VM...

	I0918 12:38:15.993992    3838 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/qemu.pid -device virtio-net-pci,netdev=net0,mac=32:d5:01:25:08:fe -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/offline-docker-612000/disk.qcow2

	I0918 12:38:15.995560    3838 main.go:141] libmachine: STDOUT: 

	I0918 12:38:15.995575    3838 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0918 12:38:15.995588    3838 client.go:171] LocalClient.Create took 285.283042ms

	I0918 12:38:17.997761    3838 start.go:128] duration metric: createHost completed in 2.339917625s

	I0918 12:38:17.997887    3838 start.go:83] releasing machines lock for "offline-docker-612000", held for 2.340272291s

	W0918 12:38:17.998382    3838 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p offline-docker-612000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p offline-docker-612000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:18.006048    3838 out.go:177] 

	W0918 12:38:18.010092    3838 out.go:239] X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0918 12:38:18.010118    3838 out.go:239] * 

	* 

	W0918 12:38:18.012749    3838 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0918 12:38:18.021018    3838 out.go:177] 

** /stderr **

aab_offline_test.go:58: out/minikube-darwin-arm64 start -p offline-docker-612000 --alsologtostderr -v=1 --memory=2048 --wait=true --driver=qemu2  failed: exit status 80

panic.go:523: *** TestOffline FAILED at 2023-09-18 12:38:18.035404 -0700 PDT m=+2794.512153792

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p offline-docker-612000 -n offline-docker-612000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p offline-docker-612000 -n offline-docker-612000: exit status 7 (65.498916ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "offline-docker-612000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "offline-docker-612000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p offline-docker-612000

--- FAIL: TestOffline (9.94s)

=== RUN   TestAddons/parallel/Registry

=== PAUSE TestAddons/parallel/Registry

=== CONT  TestAddons/parallel/Registry

addons_test.go:304: failed waiting for registry replicacontroller to stabilize: timed out waiting for the condition

addons_test.go:306: registry stabilized in 6m0.001488375s

addons_test.go:308: (dbg) TestAddons/parallel/Registry: waiting 6m0s for pods matching "actual-registry=true" in namespace "kube-system" ...

addons_test.go:308: ***** TestAddons/parallel/Registry: pod "actual-registry=true" failed to start within 6m0s: context deadline exceeded ****

addons_test.go:308: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.APIServer}} -p addons-221000 -n addons-221000

addons_test.go:308: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.APIServer}} -p addons-221000 -n addons-221000: exit status 7 (35.473917ms)

-- stdout --

	Nonexistent

-- /stdout --

** stderr ** 

	E0918 12:11:00.331738    1990 status.go:249] status error: host: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

** /stderr **

addons_test.go:308: status error: exit status 7 (may be ok)

addons_test.go:308: "addons-221000" apiserver is not running, skipping kubectl commands (state="Nonexistent")

addons_test.go:309: failed waiting for pod actual-registry: actual-registry=true within 6m0s: context deadline exceeded

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p addons-221000 -n addons-221000

helpers_test.go:244: <<< TestAddons/parallel/Registry FAILED: start of post-mortem logs <<<

helpers_test.go:245: ======>  post-mortem[TestAddons/parallel/Registry]: minikube logs <======

helpers_test.go:247: (dbg) Run:  out/minikube-darwin-arm64 -p addons-221000 logs -n 25

helpers_test.go:252: TestAddons/parallel/Registry logs: 

-- stdout --

	* 

	* ==> Audit <==

	* |---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| Command |              Args              |       Profile        |  User   | Version |     Start Time      |      End Time       |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| start   | -o=json --download-only        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:51 PDT |                     |

	|         | -p download-only-242000        |                      |         |         |                     |                     |

	|         | --force --alsologtostderr      |                      |         |         |                     |                     |

	|         | --kubernetes-version=v1.16.0   |                      |         |         |                     |                     |

	|         | --container-runtime=docker     |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	| start   | -o=json --download-only        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT |                     |

	|         | -p download-only-242000        |                      |         |         |                     |                     |

	|         | --force --alsologtostderr      |                      |         |         |                     |                     |

	|         | --kubernetes-version=v1.28.2   |                      |         |         |                     |                     |

	|         | --container-runtime=docker     |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	| delete  | --all                          | minikube             | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| delete  | -p download-only-242000        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| delete  | -p download-only-242000        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| start   | --download-only -p             | binary-mirror-077000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT |                     |

	|         | binary-mirror-077000           |                      |         |         |                     |                     |

	|         | --alsologtostderr              |                      |         |         |                     |                     |

	|         | --binary-mirror                |                      |         |         |                     |                     |

	|         | http://127.0.0.1:49414         |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	| delete  | -p binary-mirror-077000        | binary-mirror-077000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| start   | -p addons-221000               | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:59 PDT |

	|         | --wait=true --memory=4000      |                      |         |         |                     |                     |

	|         | --alsologtostderr              |                      |         |         |                     |                     |

	|         | --addons=registry              |                      |         |         |                     |                     |

	|         | --addons=metrics-server        |                      |         |         |                     |                     |

	|         | --addons=volumesnapshots       |                      |         |         |                     |                     |

	|         | --addons=csi-hostpath-driver   |                      |         |         |                     |                     |

	|         | --addons=gcp-auth              |                      |         |         |                     |                     |

	|         | --addons=cloud-spanner         |                      |         |         |                     |                     |

	|         | --addons=inspektor-gadget      |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	|         | --addons=ingress               |                      |         |         |                     |                     |

	|         | --addons=ingress-dns           |                      |         |         |                     |                     |

	| addons  | disable cloud-spanner -p       | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:11 PDT |                     |

	|         | addons-221000                  |                      |         |         |                     |                     |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	* 

	* ==> Last Start <==

	* Log file created at: 2023/09/18 11:52:16

	Running on machine: MacOS-M1-Agent-1

	Binary: Built with gc go1.21.1 for darwin/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0918 11:52:16.711602    1740 out.go:296] Setting OutFile to fd 1 ...

	I0918 11:52:16.711748    1740 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 11:52:16.711751    1740 out.go:309] Setting ErrFile to fd 2...

	I0918 11:52:16.711753    1740 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 11:52:16.711880    1740 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17263-1251/.minikube/bin

	I0918 11:52:16.712918    1740 out.go:303] Setting JSON to false

	I0918 11:52:16.728001    1740 start.go:128] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":1310,"bootTime":1695061826,"procs":408,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.5.2","kernelVersion":"22.6.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0918 11:52:16.728087    1740 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0918 11:52:16.732378    1740 out.go:177] * [addons-221000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	I0918 11:52:16.739454    1740 out.go:177]   - MINIKUBE_LOCATION=17263

	I0918 11:52:16.743421    1740 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 11:52:16.739507    1740 notify.go:220] Checking for updates...

	I0918 11:52:16.749403    1740 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0918 11:52:16.752377    1740 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0918 11:52:16.755381    1740 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	I0918 11:52:16.758417    1740 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0918 11:52:16.761446    1740 driver.go:373] Setting default libvirt URI to qemu:///system

	I0918 11:52:16.765371    1740 out.go:177] * Using the qemu2 driver based on user configuration

	I0918 11:52:16.777355    1740 start.go:298] selected driver: qemu2

	I0918 11:52:16.777364    1740 start.go:902] validating driver "qemu2" against <nil>

	I0918 11:52:16.777372    1740 start.go:913] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0918 11:52:16.779390    1740 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0918 11:52:16.782385    1740 out.go:177] * Automatically selected the socket_vmnet network

	I0918 11:52:16.785462    1740 start_flags.go:922] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0918 11:52:16.785488    1740 cni.go:84] Creating CNI manager for ""

	I0918 11:52:16.785496    1740 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 11:52:16.785507    1740 start_flags.go:316] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0918 11:52:16.785513    1740 start_flags.go:321] config:

	{Name:addons-221000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CR

ISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID

:0 AutoPauseInterval:1m0s}

	I0918 11:52:16.789634    1740 iso.go:125] acquiring lock: {Name:mk34e23c181861c65264ec384f06e6cfa001aa08 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0918 11:52:16.798394    1740 out.go:177] * Starting control plane node addons-221000 in cluster addons-221000

	I0918 11:52:16.802195    1740 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 11:52:16.802217    1740 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4

	I0918 11:52:16.802234    1740 cache.go:57] Caching tarball of preloaded images

	I0918 11:52:16.802301    1740 preload.go:174] Found /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0918 11:52:16.802315    1740 cache.go:60] Finished verifying existence of preloaded tar for  v1.28.2 on docker

	I0918 11:52:16.802542    1740 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/config.json ...

	I0918 11:52:16.802555    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/config.json: {Name:mk6624c585fbc7911138df2cd59d1f2e10251cf1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:16.802799    1740 start.go:365] acquiring machines lock for addons-221000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 11:52:16.802873    1740 start.go:369] acquired machines lock for "addons-221000" in 68.417µs

	I0918 11:52:16.802886    1740 start.go:93] Provisioning new machine with config: &{Name:addons-221000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi

g:{KubernetesVersion:v1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262

144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 11:52:16.802925    1740 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 11:52:16.810242    1740 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...

	I0918 11:52:17.161676    1740 start.go:159] libmachine.API.Create for "addons-221000" (driver="qemu2")

	I0918 11:52:17.161722    1740 client.go:168] LocalClient.Create starting

	I0918 11:52:17.161932    1740 main.go:141] libmachine: Creating CA: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 11:52:17.253776    1740 main.go:141] libmachine: Creating client certificate: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 11:52:17.312301    1740 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 11:52:17.776256    1740 main.go:141] libmachine: Creating SSH key...

	I0918 11:52:17.897328    1740 main.go:141] libmachine: Creating Disk image...

	I0918 11:52:17.897334    1740 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 11:52:17.897524    1740 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2

	I0918 11:52:17.933044    1740 main.go:141] libmachine: STDOUT: 

	I0918 11:52:17.933072    1740 main.go:141] libmachine: STDERR: 

	I0918 11:52:17.933136    1740 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2 +20000M

	I0918 11:52:17.940597    1740 main.go:141] libmachine: STDOUT: Image resized.

	I0918 11:52:17.940609    1740 main.go:141] libmachine: STDERR: 

	I0918 11:52:17.940623    1740 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2

	I0918 11:52:17.940628    1740 main.go:141] libmachine: Starting QEMU VM...

	I0918 11:52:17.940657    1740 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 4000 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/qemu.pid -device virtio-net-pci,netdev=net0,mac=ce:ae:e8:0a:fd:16 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2

	I0918 11:52:18.008779    1740 main.go:141] libmachine: STDOUT: 

	I0918 11:52:18.008804    1740 main.go:141] libmachine: STDERR: 

	I0918 11:52:18.008808    1740 main.go:141] libmachine: Attempt 0

	I0918 11:52:18.008820    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:20.011046    1740 main.go:141] libmachine: Attempt 1

	I0918 11:52:20.011124    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:22.013479    1740 main.go:141] libmachine: Attempt 2

	I0918 11:52:22.013559    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:24.015632    1740 main.go:141] libmachine: Attempt 3

	I0918 11:52:24.015645    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:26.017675    1740 main.go:141] libmachine: Attempt 4

	I0918 11:52:26.017681    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:28.018761    1740 main.go:141] libmachine: Attempt 5

	I0918 11:52:28.018782    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:30.020886    1740 main.go:141] libmachine: Attempt 6

	I0918 11:52:30.020920    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:30.021070    1740 main.go:141] libmachine: Found 1 entries in /var/db/dhcpd_leases!

	I0918 11:52:30.021123    1740 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:ce:ae:e8:a:fd:16 ID:1,ce:ae:e8:a:fd:16 Lease:0x6509edec}

	I0918 11:52:30.021130    1740 main.go:141] libmachine: Found match: ce:ae:e8:a:fd:16

	I0918 11:52:30.021140    1740 main.go:141] libmachine: IP: 192.168.105.2

	I0918 11:52:30.021147    1740 main.go:141] libmachine: Waiting for VM to start (ssh -p 22 docker@192.168.105.2)...

	I0918 11:52:31.026067    1740 machine.go:88] provisioning docker machine ...

	I0918 11:52:31.026085    1740 buildroot.go:166] provisioning hostname "addons-221000"

	I0918 11:52:31.026964    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.027231    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.027237    1740 main.go:141] libmachine: About to run SSH command:

	sudo hostname addons-221000 && echo "addons-221000" | sudo tee /etc/hostname

	I0918 11:52:31.084404    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: addons-221000

	I0918 11:52:31.084473    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.084732    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.084740    1740 main.go:141] libmachine: About to run SSH command:

			if ! grep -xq '.*\saddons-221000' /etc/hosts; then

				if grep -xq '127.0.1.1\s.*' /etc/hosts; then

					sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 addons-221000/g' /etc/hosts;

				else 

					echo '127.0.1.1 addons-221000' | sudo tee -a /etc/hosts; 

				fi

			fi

	I0918 11:52:31.144009    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	I0918 11:52:31.144022    1740 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/17263-1251/.minikube CaCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/17263-1251/.minikube}

	I0918 11:52:31.144033    1740 buildroot.go:174] setting up certificates

	I0918 11:52:31.144038    1740 provision.go:83] configureAuth start

	I0918 11:52:31.144042    1740 provision.go:138] copyHostCerts

	I0918 11:52:31.144138    1740 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.pem (1082 bytes)

	I0918 11:52:31.144342    1740 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/cert.pem (1123 bytes)

	I0918 11:52:31.144435    1740 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/key.pem (1679 bytes)

	I0918 11:52:31.144503    1740 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem org=jenkins.addons-221000 san=[192.168.105.2 192.168.105.2 localhost 127.0.0.1 minikube addons-221000]

	I0918 11:52:31.225327    1740 provision.go:172] copyRemoteCerts

	I0918 11:52:31.225385    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker

	I0918 11:52:31.225394    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.256352    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)

	I0918 11:52:31.263330    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem --> /etc/docker/server.pem (1216 bytes)

	I0918 11:52:31.270197    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)

	I0918 11:52:31.276715    1740 provision.go:86] duration metric: configureAuth took 132.670667ms

	I0918 11:52:31.276723    1740 buildroot.go:189] setting minikube options for container-runtime

	I0918 11:52:31.276820    1740 config.go:182] Loaded profile config "addons-221000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 11:52:31.276857    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.277075    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.277080    1740 main.go:141] libmachine: About to run SSH command:

	df --output=fstype / | tail -n 1

	I0918 11:52:31.337901    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs

	I0918 11:52:31.337910    1740 buildroot.go:70] root file system type: tmpfs

	I0918 11:52:31.337970    1740 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...

	I0918 11:52:31.338012    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.338275    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.338315    1740 main.go:141] libmachine: About to run SSH command:

	sudo mkdir -p /lib/systemd/system && printf %!s(MISSING) "[Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP \$MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	" | sudo tee /lib/systemd/system/docker.service.new

	I0918 11:52:31.400816    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP $MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	I0918 11:52:31.400863    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.401116    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.401126    1740 main.go:141] libmachine: About to run SSH command:

	sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }

	I0918 11:52:31.746670    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory

	Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

	I0918 11:52:31.746684    1740 machine.go:91] provisioned docker machine in 720.613ms

	I0918 11:52:31.746690    1740 client.go:171] LocalClient.Create took 14.585099291s

	I0918 11:52:31.746703    1740 start.go:167] duration metric: libmachine.API.Create for "addons-221000" took 14.585173417s

	I0918 11:52:31.746707    1740 start.go:300] post-start starting for "addons-221000" (driver="qemu2")

	I0918 11:52:31.746711    1740 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]

	I0918 11:52:31.746780    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs

	I0918 11:52:31.746790    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.775601    1740 ssh_runner.go:195] Run: cat /etc/os-release

	I0918 11:52:31.776975    1740 info.go:137] Remote host: Buildroot 2021.02.12

	I0918 11:52:31.776983    1740 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17263-1251/.minikube/addons for local assets ...

	I0918 11:52:31.777055    1740 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17263-1251/.minikube/files for local assets ...

	I0918 11:52:31.777083    1740 start.go:303] post-start completed in 30.374292ms

	I0918 11:52:31.777437    1740 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/config.json ...

	I0918 11:52:31.777603    1740 start.go:128] duration metric: createHost completed in 14.974815417s

	I0918 11:52:31.777667    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.777884    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.777888    1740 main.go:141] libmachine: About to run SSH command:

	date +%!s(MISSING).%!N(MISSING)

	I0918 11:52:31.833629    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: 1695063151.431021085

	I0918 11:52:31.833635    1740 fix.go:206] guest clock: 1695063151.431021085

	I0918 11:52:31.833638    1740 fix.go:219] Guest: 2023-09-18 11:52:31.431021085 -0700 PDT Remote: 2023-09-18 11:52:31.777608 -0700 PDT m=+15.083726834 (delta=-346.586915ms)

	I0918 11:52:31.833654    1740 fix.go:190] guest clock delta is within tolerance: -346.586915ms

	I0918 11:52:31.833656    1740 start.go:83] releasing machines lock for "addons-221000", held for 15.0309195s

	I0918 11:52:31.833905    1740 ssh_runner.go:195] Run: cat /version.json

	I0918 11:52:31.833915    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.833930    1740 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/

	I0918 11:52:31.833973    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.902596    1740 ssh_runner.go:195] Run: systemctl --version

	I0918 11:52:31.904777    1740 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"

	W0918 11:52:31.906638    1740 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found

	I0918 11:52:31.906668    1740 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;

	I0918 11:52:31.911697    1740 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)

	I0918 11:52:31.911704    1740 start.go:469] detecting cgroup driver to use...

	I0918 11:52:31.911799    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock

	" | sudo tee /etc/crictl.yaml"

	I0918 11:52:31.917320    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"

	I0918 11:52:31.920500    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"

	I0918 11:52:31.923811    1740 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...

	I0918 11:52:31.923843    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"

	I0918 11:52:31.926950    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0918 11:52:31.929666    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"

	I0918 11:52:31.932680    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0918 11:52:31.936002    1740 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"

	I0918 11:52:31.939362    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"

	I0918 11:52:31.942186    1740 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables

	I0918 11:52:31.944734    1740 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

	I0918 11:52:31.947664    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:32.027464    1740 ssh_runner.go:195] Run: sudo systemctl restart containerd

	I0918 11:52:32.036559    1740 start.go:469] detecting cgroup driver to use...

	I0918 11:52:32.036614    1740 ssh_runner.go:195] Run: sudo systemctl cat docker.service

	I0918 11:52:32.042440    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0918 11:52:32.047583    1740 ssh_runner.go:195] Run: sudo systemctl stop -f containerd

	I0918 11:52:32.053840    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0918 11:52:32.058225    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I0918 11:52:32.062305    1740 ssh_runner.go:195] Run: sudo systemctl stop -f crio

	I0918 11:52:32.098720    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I0918 11:52:32.103440    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock

	" | sudo tee /etc/crictl.yaml"

	I0918 11:52:32.108844    1740 ssh_runner.go:195] Run: which cri-dockerd

	I0918 11:52:32.110173    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d

	I0918 11:52:32.112731    1740 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)

	I0918 11:52:32.117532    1740 ssh_runner.go:195] Run: sudo systemctl unmask docker.service

	I0918 11:52:32.194769    1740 ssh_runner.go:195] Run: sudo systemctl enable docker.socket

	I0918 11:52:32.269401    1740 docker.go:535] configuring docker to use "cgroupfs" as cgroup driver...

	I0918 11:52:32.269417    1740 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)

	I0918 11:52:32.274373    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:32.355030    1740 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0918 11:52:33.517984    1740 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.1629475s)

	I0918 11:52:33.518044    1740 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I0918 11:52:33.595160    1740 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket

	I0918 11:52:33.670332    1740 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I0918 11:52:33.746578    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:33.822625    1740 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket

	I0918 11:52:33.829925    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:33.909957    1740 ssh_runner.go:195] Run: sudo systemctl restart cri-docker

	I0918 11:52:33.933398    1740 start.go:516] Will wait 60s for socket path /var/run/cri-dockerd.sock

	I0918 11:52:33.933487    1740 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock

	I0918 11:52:33.935757    1740 start.go:537] Will wait 60s for crictl version

	I0918 11:52:33.935801    1740 ssh_runner.go:195] Run: which crictl

	I0918 11:52:33.937147    1740 ssh_runner.go:195] Run: sudo /usr/bin/crictl version

	I0918 11:52:33.952602    1740 start.go:553] Version:  0.1.0

	RuntimeName:  docker

	RuntimeVersion:  24.0.6

	RuntimeApiVersion:  v1alpha2

	I0918 11:52:33.952673    1740 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0918 11:52:33.962082    1740 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0918 11:52:33.975334    1740 out.go:204] * Preparing Kubernetes v1.28.2 on Docker 24.0.6 ...

	I0918 11:52:33.975416    1740 ssh_runner.go:195] Run: grep 192.168.105.1	host.minikube.internal$ /etc/hosts

	I0918 11:52:33.976970    1740 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.105.1	host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0918 11:52:33.980853    1740 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 11:52:33.980897    1740 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0918 11:52:33.986139    1740 docker.go:636] Got preloaded images: 

	I0918 11:52:33.986147    1740 docker.go:642] registry.k8s.io/kube-apiserver:v1.28.2 wasn't preloaded

	I0918 11:52:33.986189    1740 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0918 11:52:33.988954    1740 ssh_runner.go:195] Run: which lz4

	I0918 11:52:33.990479    1740 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4

	I0918 11:52:33.991766    1740 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4: Process exited with status 1

	stdout:

	stderr:

	stat: cannot statx '/preloaded.tar.lz4': No such file or directory

	I0918 11:52:33.991780    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 --> /preloaded.tar.lz4 (356993689 bytes)

	I0918 11:52:35.310513    1740 docker.go:600] Took 1.320057 seconds to copy over tarball

	I0918 11:52:35.310582    1740 ssh_runner.go:195] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4

	I0918 11:52:36.348518    1740 ssh_runner.go:235] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (1.03793175s)

	I0918 11:52:36.348535    1740 ssh_runner.go:146] rm: /preloaded.tar.lz4

	I0918 11:52:36.364745    1740 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0918 11:52:36.368295    1740 ssh_runner.go:362] scp memory --> /var/lib/docker/image/overlay2/repositories.json (2629 bytes)

	I0918 11:52:36.373429    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:36.450305    1740 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0918 11:52:38.940530    1740 ssh_runner.go:235] Completed: sudo systemctl restart docker: (2.490231167s)

	I0918 11:52:38.940627    1740 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0918 11:52:38.946699    1740 docker.go:636] Got preloaded images: -- stdout --

	registry.k8s.io/kube-apiserver:v1.28.2

	registry.k8s.io/kube-scheduler:v1.28.2

	registry.k8s.io/kube-proxy:v1.28.2

	registry.k8s.io/kube-controller-manager:v1.28.2

	registry.k8s.io/etcd:3.5.9-0

	registry.k8s.io/coredns/coredns:v1.10.1

	registry.k8s.io/pause:3.9

	gcr.io/k8s-minikube/storage-provisioner:v5

	-- /stdout --

	I0918 11:52:38.946709    1740 cache_images.go:84] Images are preloaded, skipping loading

	I0918 11:52:38.946766    1740 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}

	I0918 11:52:38.954428    1740 cni.go:84] Creating CNI manager for ""

	I0918 11:52:38.954439    1740 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 11:52:38.954458    1740 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16

	I0918 11:52:38.954467    1740 kubeadm.go:176] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.105.2 APIServerPort:8443 KubernetesVersion:v1.28.2 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:addons-221000 NodeName:addons-221000 DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.105.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.105.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/ku

bernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}

	I0918 11:52:38.954540    1740 kubeadm.go:181] kubeadm config:

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: InitConfiguration

	localAPIEndpoint:

	  advertiseAddress: 192.168.105.2

	  bindPort: 8443

	bootstrapTokens:

	  - groups:

	      - system:bootstrappers:kubeadm:default-node-token

	    ttl: 24h0m0s

	    usages:

	      - signing

	      - authentication

	nodeRegistration:

	  criSocket: unix:///var/run/cri-dockerd.sock

	  name: "addons-221000"

	  kubeletExtraArgs:

	    node-ip: 192.168.105.2

	  taints: []

	---

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: ClusterConfiguration

	apiServer:

	  certSANs: ["127.0.0.1", "localhost", "192.168.105.2"]

	  extraArgs:

	    enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"

	controllerManager:

	  extraArgs:

	    allocate-node-cidrs: "true"

	    leader-elect: "false"

	scheduler:

	  extraArgs:

	    leader-elect: "false"

	certificatesDir: /var/lib/minikube/certs

	clusterName: mk

	controlPlaneEndpoint: control-plane.minikube.internal:8443

	etcd:

	  local:

	    dataDir: /var/lib/minikube/etcd

	    extraArgs:

	      proxy-refresh-interval: "70000"

	kubernetesVersion: v1.28.2

	networking:

	  dnsDomain: cluster.local

	  podSubnet: "10.244.0.0/16"

	  serviceSubnet: 10.96.0.0/12

	---

	apiVersion: kubelet.config.k8s.io/v1beta1

	kind: KubeletConfiguration

	authentication:

	  x509:

	    clientCAFile: /var/lib/minikube/certs/ca.crt

	cgroupDriver: cgroupfs

	hairpinMode: hairpin-veth

	runtimeRequestTimeout: 15m

	clusterDomain: "cluster.local"

	# disable disk resource management by default

	imageGCHighThresholdPercent: 100

	evictionHard:

	  nodefs.available: "0%!"(MISSING)

	  nodefs.inodesFree: "0%!"(MISSING)

	  imagefs.available: "0%!"(MISSING)

	failSwapOn: false

	staticPodPath: /etc/kubernetes/manifests

	---

	apiVersion: kubeproxy.config.k8s.io/v1alpha1

	kind: KubeProxyConfiguration

	clusterCIDR: "10.244.0.0/16"

	metricsBindAddress: 0.0.0.0:10249

	conntrack:

	  maxPerCore: 0

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"

	  tcpEstablishedTimeout: 0s

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"

	  tcpCloseWaitTimeout: 0s

	I0918 11:52:38.954592    1740 kubeadm.go:976] kubelet [Unit]

	Wants=docker.socket

	[Service]

	ExecStart=

	ExecStart=/var/lib/minikube/binaries/v1.28.2/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --hostname-override=addons-221000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.105.2

	[Install]

	 config:

	{KubernetesVersion:v1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}

	I0918 11:52:38.954661    1740 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.2

	I0918 11:52:38.957531    1740 binaries.go:44] Found k8s binaries, skipping transfer

	I0918 11:52:38.957562    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube

	I0918 11:52:38.960385    1740 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (375 bytes)

	I0918 11:52:38.965592    1740 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)

	I0918 11:52:38.970301    1740 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2099 bytes)

	I0918 11:52:38.975165    1740 ssh_runner.go:195] Run: grep 192.168.105.2	control-plane.minikube.internal$ /etc/hosts

	I0918 11:52:38.976400    1740 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.105.2	control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0918 11:52:38.980255    1740 certs.go:56] Setting up /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000 for IP: 192.168.105.2

	I0918 11:52:38.980276    1740 certs.go:190] acquiring lock for shared ca certs: {Name:mkfac81ee65979b8c4f5ece6243c3a0190531689 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:38.980470    1740 certs.go:204] generating minikubeCA CA: /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key

	I0918 11:52:39.170828    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt ...

	I0918 11:52:39.170844    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt: {Name:mk0f303ee67627c25d1d04e1887861f15cdad763 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.171150    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key ...

	I0918 11:52:39.171155    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key: {Name:mkc5e20e8161cfdcfc3d5dcd8300765ea2c12112 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.171271    1740 certs.go:204] generating proxyClientCA CA: /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key

	I0918 11:52:39.287022    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.crt ...

	I0918 11:52:39.287027    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.crt: {Name:mk54c49c3c44ff09930e6c0f57238b89cff4c5e9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.287171    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key ...

	I0918 11:52:39.287173    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key: {Name:mk05faae5769358f82565f32c1f37a244f2478c1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.287315    1740 certs.go:319] generating minikube-user signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.key

	I0918 11:52:39.287337    1740 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.crt with IP's: []

	I0918 11:52:39.376234    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.crt ...

	I0918 11:52:39.376241    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.crt: {Name:mkc8e654c6f2522197f557cb47d266f15eebaadd Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.376467    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.key ...

	I0918 11:52:39.376471    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.key: {Name:mkf345dd56f86115b31ecd965617f4c21d6a0cd8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.376571    1740 certs.go:319] generating minikube signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969

	I0918 11:52:39.376580    1740 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969 with IP's: [192.168.105.2 10.96.0.1 127.0.0.1 10.0.0.1]

	I0918 11:52:39.429944    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969 ...

	I0918 11:52:39.429952    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969: {Name:mkd69eb587bd0dc6ccdbaa88b78f4f92f2b47b12 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.430095    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969 ...

	I0918 11:52:39.430098    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969: {Name:mk76ea3a7fbbef2305f74e52afdf06cda921c8bb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.430199    1740 certs.go:337] copying /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969 -> /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt

	I0918 11:52:39.430382    1740 certs.go:341] copying /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969 -> /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key

	I0918 11:52:39.430499    1740 certs.go:319] generating aggregator signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key

	I0918 11:52:39.430509    1740 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt with IP's: []

	I0918 11:52:39.698555    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt ...

	I0918 11:52:39.698563    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt: {Name:mk6d7a924ed10f0012b290ec4e0ea6bf1b7bfc02 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.698767    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key ...

	I0918 11:52:39.698773    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key: {Name:mk8d78e9179e4c57e4602e98d4fc6a37885b4d01 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.699037    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem (1679 bytes)

	I0918 11:52:39.699062    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem (1082 bytes)

	I0918 11:52:39.699081    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem (1123 bytes)

	I0918 11:52:39.699100    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem (1679 bytes)

	I0918 11:52:39.699418    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)

	I0918 11:52:39.707305    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)

	I0918 11:52:39.713956    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)

	I0918 11:52:39.720614    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)

	I0918 11:52:39.727643    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)

	I0918 11:52:39.734609    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)

	I0918 11:52:39.741305    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)

	I0918 11:52:39.748141    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)

	I0918 11:52:39.755277    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)

	I0918 11:52:39.762025    1740 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)

	I0918 11:52:39.767701    1740 ssh_runner.go:195] Run: openssl version

	I0918 11:52:39.769871    1740 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"

	I0918 11:52:39.772909    1740 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem

	I0918 11:52:39.774536    1740 certs.go:480] hashing: -rw-r--r-- 1 root root 1111 Sep 18 18:52 /usr/share/ca-certificates/minikubeCA.pem

	I0918 11:52:39.774555    1740 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem

	I0918 11:52:39.776417    1740 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"

	I0918 11:52:39.779698    1740 ssh_runner.go:195] Run: ls /var/lib/minikube/certs/etcd

	I0918 11:52:39.781162    1740 certs.go:353] certs directory doesn't exist, likely first start: ls /var/lib/minikube/certs/etcd: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/var/lib/minikube/certs/etcd': No such file or directory

	I0918 11:52:39.781200    1740 kubeadm.go:404] StartCluster: {Name:addons-221000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v

1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.105.2 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 Mount

Options:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 11:52:39.781263    1740 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}

	I0918 11:52:39.787256    1740 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd

	I0918 11:52:39.790167    1740 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml

	I0918 11:52:39.792879    1740 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf

	I0918 11:52:39.795890    1740 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory

	I0918 11:52:39.795906    1740 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"

	I0918 11:52:39.820130    1740 kubeadm.go:322] [init] Using Kubernetes version: v1.28.2

	I0918 11:52:39.820157    1740 kubeadm.go:322] [preflight] Running pre-flight checks

	I0918 11:52:39.874262    1740 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster

	I0918 11:52:39.874320    1740 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection

	I0918 11:52:39.874401    1740 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	I0918 11:52:39.936649    1740 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"

	I0918 11:52:39.946863    1740 out.go:204]   - Generating certificates and keys ...

	I0918 11:52:39.946906    1740 kubeadm.go:322] [certs] Using existing ca certificate authority

	I0918 11:52:39.946940    1740 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk

	I0918 11:52:40.057135    1740 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key

	I0918 11:52:40.267412    1740 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key

	I0918 11:52:40.415260    1740 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key

	I0918 11:52:40.592293    1740 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key

	I0918 11:52:40.714190    1740 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key

	I0918 11:52:40.714252    1740 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [addons-221000 localhost] and IPs [192.168.105.2 127.0.0.1 ::1]

	I0918 11:52:40.818329    1740 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key

	I0918 11:52:40.818397    1740 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [addons-221000 localhost] and IPs [192.168.105.2 127.0.0.1 ::1]

	I0918 11:52:41.068370    1740 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key

	I0918 11:52:41.110794    1740 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key

	I0918 11:52:41.218301    1740 kubeadm.go:322] [certs] Generating "sa" key and public key

	I0918 11:52:41.218335    1740 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	I0918 11:52:41.282421    1740 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file

	I0918 11:52:41.650315    1740 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file

	I0918 11:52:41.733907    1740 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file

	I0918 11:52:41.925252    1740 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file

	I0918 11:52:41.925561    1740 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	I0918 11:52:41.927413    1740 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"

	I0918 11:52:41.931680    1740 out.go:204]   - Booting up control plane ...

	I0918 11:52:41.931754    1740 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"

	I0918 11:52:41.931794    1740 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"

	I0918 11:52:41.931831    1740 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"

	I0918 11:52:41.935171    1740 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	I0918 11:52:41.935565    1740 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	I0918 11:52:41.935586    1740 kubeadm.go:322] [kubelet-start] Starting the kubelet

	I0918 11:52:42.024365    1740 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	I0918 11:52:45.527756    1740 kubeadm.go:322] [apiclient] All control plane components are healthy after 3.503453 seconds

	I0918 11:52:45.527852    1740 kubeadm.go:322] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace

	I0918 11:52:45.533290    1740 kubeadm.go:322] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster

	I0918 11:52:46.043984    1740 kubeadm.go:322] [upload-certs] Skipping phase. Please see --upload-certs

	I0918 11:52:46.044088    1740 kubeadm.go:322] [mark-control-plane] Marking the node addons-221000 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]

	I0918 11:52:46.548611    1740 kubeadm.go:322] [bootstrap-token] Using token: 0otx18.vbdfa1zgl84pbc1n

	I0918 11:52:46.552403    1740 out.go:204]   - Configuring RBAC rules ...

	I0918 11:52:46.552463    1740 kubeadm.go:322] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles

	I0918 11:52:46.553348    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes

	I0918 11:52:46.557357    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials

	I0918 11:52:46.558552    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token

	I0918 11:52:46.559879    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster

	I0918 11:52:46.560890    1740 kubeadm.go:322] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace

	I0918 11:52:46.567944    1740 kubeadm.go:322] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key

	I0918 11:52:46.739677    1740 kubeadm.go:322] [addons] Applied essential addon: CoreDNS

	I0918 11:52:46.956246    1740 kubeadm.go:322] [addons] Applied essential addon: kube-proxy

	I0918 11:52:46.958050    1740 kubeadm.go:322] 

	I0918 11:52:46.958085    1740 kubeadm.go:322] Your Kubernetes control-plane has initialized successfully!

	I0918 11:52:46.958096    1740 kubeadm.go:322] 

	I0918 11:52:46.958138    1740 kubeadm.go:322] To start using your cluster, you need to run the following as a regular user:

	I0918 11:52:46.958143    1740 kubeadm.go:322] 

	I0918 11:52:46.958157    1740 kubeadm.go:322]   mkdir -p $HOME/.kube

	I0918 11:52:46.958186    1740 kubeadm.go:322]   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

	I0918 11:52:46.958221    1740 kubeadm.go:322]   sudo chown $(id -u):$(id -g) $HOME/.kube/config

	I0918 11:52:46.958226    1740 kubeadm.go:322] 

	I0918 11:52:46.958261    1740 kubeadm.go:322] Alternatively, if you are the root user, you can run:

	I0918 11:52:46.958267    1740 kubeadm.go:322] 

	I0918 11:52:46.958291    1740 kubeadm.go:322]   export KUBECONFIG=/etc/kubernetes/admin.conf

	I0918 11:52:46.958294    1740 kubeadm.go:322] 

	I0918 11:52:46.958317    1740 kubeadm.go:322] You should now deploy a pod network to the cluster.

	I0918 11:52:46.958375    1740 kubeadm.go:322] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

	I0918 11:52:46.958411    1740 kubeadm.go:322]   https://kubernetes.io/docs/concepts/cluster-administration/addons/

	I0918 11:52:46.958416    1740 kubeadm.go:322] 

	I0918 11:52:46.958458    1740 kubeadm.go:322] You can now join any number of control-plane nodes by copying certificate authorities

	I0918 11:52:46.958503    1740 kubeadm.go:322] and service account keys on each node and then running the following as root:

	I0918 11:52:46.958507    1740 kubeadm.go:322] 

	I0918 11:52:46.958562    1740 kubeadm.go:322]   kubeadm join control-plane.minikube.internal:8443 --token 0otx18.vbdfa1zgl84pbc1n \

	I0918 11:52:46.958623    1740 kubeadm.go:322] 	--discovery-token-ca-cert-hash sha256:ba7ba5242719fff40b98ade8a053fc0c6dded3185e28e2742ca7444c7c25a7a5 \

	I0918 11:52:46.958634    1740 kubeadm.go:322] 	--control-plane 

	I0918 11:52:46.958636    1740 kubeadm.go:322] 

	I0918 11:52:46.958676    1740 kubeadm.go:322] Then you can join any number of worker nodes by running the following on each as root:

	I0918 11:52:46.958681    1740 kubeadm.go:322] 

	I0918 11:52:46.958735    1740 kubeadm.go:322] kubeadm join control-plane.minikube.internal:8443 --token 0otx18.vbdfa1zgl84pbc1n \

	I0918 11:52:46.958805    1740 kubeadm.go:322] 	--discovery-token-ca-cert-hash sha256:ba7ba5242719fff40b98ade8a053fc0c6dded3185e28e2742ca7444c7c25a7a5 

	I0918 11:52:46.958862    1740 kubeadm.go:322] 	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	I0918 11:52:46.958868    1740 cni.go:84] Creating CNI manager for ""

	I0918 11:52:46.958880    1740 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 11:52:46.967096    1740 out.go:177] * Configuring bridge CNI (Container Networking Interface) ...

	I0918 11:52:46.970213    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/cni/net.d

	I0918 11:52:46.973479    1740 ssh_runner.go:362] scp memory --> /etc/cni/net.d/1-k8s.conflist (457 bytes)

	I0918 11:52:46.977991    1740 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"

	I0918 11:52:46.978031    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:46.978044    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl label nodes minikube.k8s.io/version=v1.31.2 minikube.k8s.io/commit=ae2adeb20cc3a4b5f1decc7c8f53736ec04c4a36 minikube.k8s.io/name=addons-221000 minikube.k8s.io/updated_at=2023_09_18T11_52_46_0700 minikube.k8s.io/primary=true --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:47.038282    1740 ops.go:34] apiserver oom_adj: -16

	I0918 11:52:47.038334    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:47.073908    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:47.620721    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:48.118781    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:48.620650    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:49.120663    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:49.620741    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:50.120032    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:50.620638    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:51.118777    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:51.618788    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:52.120234    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:52.619043    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:53.120606    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:53.619266    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:54.118979    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:54.618916    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:55.120620    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:55.620599    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:56.120585    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:56.618629    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:57.120587    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:57.618783    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:58.120645    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:58.620559    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:59.120556    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:59.619252    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:53:00.118796    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:53:00.169662    1740 kubeadm.go:1081] duration metric: took 13.191788791s to wait for elevateKubeSystemPrivileges.

	I0918 11:53:00.169677    1740 kubeadm.go:406] StartCluster complete in 20.38867075s

	I0918 11:53:00.169687    1740 settings.go:142] acquiring lock: {Name:mke420f28dda4f7a752738b3e6d571dc4216779e Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:53:00.169849    1740 settings.go:150] Updating kubeconfig:  /Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 11:53:00.170110    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/kubeconfig: {Name:mk07020c5b974cf07ca0cda25f72a521eb245fa1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:53:00.170308    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"

	I0918 11:53:00.170433    1740 config.go:182] Loaded profile config "addons-221000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 11:53:00.170379    1740 addons.go:499] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:true csi-hostpath-driver:true dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:true gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:true ingress-dns:true inspektor-gadget:true istio:false istio-provisioner:false kong:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:true registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false volumesnapshots:true]

	I0918 11:53:00.170512    1740 addons.go:69] Setting ingress=true in profile "addons-221000"

	I0918 11:53:00.170517    1740 addons.go:69] Setting ingress-dns=true in profile "addons-221000"

	I0918 11:53:00.170520    1740 addons.go:231] Setting addon ingress=true in "addons-221000"

	I0918 11:53:00.170523    1740 addons.go:231] Setting addon ingress-dns=true in "addons-221000"

	I0918 11:53:00.170533    1740 addons.go:69] Setting metrics-server=true in profile "addons-221000"

	I0918 11:53:00.170538    1740 addons.go:231] Setting addon metrics-server=true in "addons-221000"

	I0918 11:53:00.170552    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170556    1740 addons.go:69] Setting inspektor-gadget=true in profile "addons-221000"

	I0918 11:53:00.170560    1740 addons.go:69] Setting csi-hostpath-driver=true in profile "addons-221000"

	I0918 11:53:00.170564    1740 addons.go:69] Setting gcp-auth=true in profile "addons-221000"

	I0918 11:53:00.170569    1740 mustload.go:65] Loading cluster: addons-221000

	I0918 11:53:00.170573    1740 addons.go:69] Setting registry=true in profile "addons-221000"

	I0918 11:53:00.170577    1740 addons.go:231] Setting addon registry=true in "addons-221000"

	I0918 11:53:00.170587    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170595    1740 addons.go:69] Setting default-storageclass=true in profile "addons-221000"

	I0918 11:53:00.170618    1740 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "addons-221000"

	I0918 11:53:00.170629    1740 addons.go:69] Setting storage-provisioner=true in profile "addons-221000"

	I0918 11:53:00.170639    1740 config.go:182] Loaded profile config "addons-221000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 11:53:00.170657    1740 addons.go:231] Setting addon storage-provisioner=true in "addons-221000"

	I0918 11:53:00.170705    1740 host.go:66] Checking if "addons-221000" exists ...

	W0918 11:53:00.170820    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	W0918 11:53:00.170825    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	W0918 11:53:00.170829    1740 addons.go:277] "addons-221000" is not running, setting ingress-dns=true and skipping enablement (err=<nil>)

	W0918 11:53:00.170831    1740 addons.go:277] "addons-221000" is not running, setting registry=true and skipping enablement (err=<nil>)

	I0918 11:53:00.170552    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170834    1740 addons.go:467] Verifying addon registry=true in "addons-221000"

	I0918 11:53:00.170556    1740 addons.go:69] Setting cloud-spanner=true in profile "addons-221000"

	I0918 11:53:00.175435    1740 out.go:177] * Verifying registry addon...

	I0918 11:53:00.170560    1740 addons.go:231] Setting addon inspektor-gadget=true in "addons-221000"

	I0918 11:53:00.170569    1740 addons.go:231] Setting addon csi-hostpath-driver=true in "addons-221000"

	I0918 11:53:00.170865    1740 addons.go:231] Setting addon cloud-spanner=true in "addons-221000"

	I0918 11:53:00.170553    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170512    1740 addons.go:69] Setting volumesnapshots=true in profile "addons-221000"

	W0918 11:53:00.171149    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	W0918 11:53:00.171162    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	I0918 11:53:00.171465    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.188489    1740 out.go:177]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5

	I0918 11:53:00.182591    1740 host.go:66] Checking if "addons-221000" exists ...

	W0918 11:53:00.182599    1740 addons.go:277] "addons-221000" is not running, setting ingress=true and skipping enablement (err=<nil>)

	W0918 11:53:00.182603    1740 addons_storage_classes.go:55] "addons-221000" is not running, writing default-storageclass=true to disk and skipping enablement

	I0918 11:53:00.182614    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.182623    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.182641    1740 addons.go:231] Setting addon volumesnapshots=true in "addons-221000"

	I0918 11:53:00.183250    1740 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=registry" in ns "kube-system" ...

	I0918 11:53:00.196542    1740 out.go:177]   - Using image registry.k8s.io/metrics-server/metrics-server:v0.6.4

	I0918 11:53:00.192833    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.192879    1740 addons.go:423] installing /etc/kubernetes/addons/storage-provisioner.yaml

	I0918 11:53:00.192896    1740 addons.go:467] Verifying addon ingress=true in "addons-221000"

	I0918 11:53:00.192903    1740 addons.go:231] Setting addon default-storageclass=true in "addons-221000"

	W0918 11:53:00.193202    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	I0918 11:53:00.196510    1740 kapi.go:248] "coredns" deployment in "kube-system" namespace and "addons-221000" context rescaled to 1 replicas

	I0918 11:53:00.199509    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)

	I0918 11:53:00.199541    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-apiservice.yaml

	I0918 11:53:00.199554    1740 host.go:66] Checking if "addons-221000" exists ...

	W0918 11:53:00.199564    1740 addons.go:277] "addons-221000" is not running, setting cloud-spanner=true and skipping enablement (err=<nil>)

	I0918 11:53:00.206530    1740 out.go:177]   - Using image ghcr.io/inspektor-gadget/inspektor-gadget:v0.20.0

	I0918 11:53:00.210356    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)

	I0918 11:53:00.210367    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.210376    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/livenessprobe:v2.8.0

	I0918 11:53:00.210391    1740 start.go:223] Will wait 6m0s for node &{Name: IP:192.168.105.2 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 11:53:00.211130    1740 addons.go:423] installing /etc/kubernetes/addons/storageclass.yaml

	I0918 11:53:00.213481    1740 out.go:177] * Verifying ingress addon...

	I0918 11:53:00.214506    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/snapshot-controller:v6.1.0

	I0918 11:53:00.214532    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.214630    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)

	I0918 11:53:00.218827    1740 out.go:177] * Verifying Kubernetes components...

	I0918 11:53:00.221560    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^        forward . \/etc\/resolv.conf.*/i \        hosts {\n           192.168.105.1 host.minikube.internal\n           fallthrough\n        }' -e '/^        errors *$/i \        log' | sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"

	I0918 11:53:00.218836    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.229444    1740 addons.go:423] installing /etc/kubernetes/addons/ig-namespace.yaml

	I0918 11:53:00.229456    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-namespace.yaml (55 bytes)

	I0918 11:53:00.217515    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml

	I0918 11:53:00.229466    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml (934 bytes)

	I0918 11:53:00.229467    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.229472    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.220086    1740 kapi.go:86] Found 0 Pods for label selector kubernetes.io/minikube-addons=registry

	I0918 11:53:00.221623    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I0918 11:53:00.236499    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-resizer:v1.6.0

	I0918 11:53:00.234112    1740 kapi.go:75] Waiting for pod with label "app.kubernetes.io/name=ingress-nginx" in ns "ingress-nginx" ...

	I0918 11:53:00.242380    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0

	I0918 11:53:00.251491    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-provisioner:v3.3.0

	I0918 11:53:00.252703    1740 kapi.go:86] Found 0 Pods for label selector app.kubernetes.io/name=ingress-nginx

	I0918 11:53:00.260396    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-attacher:v4.0.0

	I0918 11:53:00.267323    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.7.0

	I0918 11:53:00.274307    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.6.0

	I0918 11:53:00.284458    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/hostpathplugin:v1.9.0

	I0918 11:53:00.287512    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-attacher.yaml

	I0918 11:53:00.287521    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-attacher.yaml (3073 bytes)

	I0918 11:53:00.287531    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.322513    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-server-deployment.yaml

	I0918 11:53:00.322523    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1907 bytes)

	I0918 11:53:00.336060    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml

	I0918 11:53:00.340017    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml

	I0918 11:53:00.346686    1740 addons.go:423] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml

	I0918 11:53:00.346693    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml (6471 bytes)

	I0918 11:53:00.362466    1740 addons.go:423] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml

	I0918 11:53:00.362477    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml (23126 bytes)

	I0918 11:53:00.363923    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-server-rbac.yaml

	I0918 11:53:00.363928    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)

	I0918 11:53:00.368153    1740 addons.go:423] installing /etc/kubernetes/addons/ig-serviceaccount.yaml

	I0918 11:53:00.368161    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-serviceaccount.yaml (80 bytes)

	I0918 11:53:00.376728    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-hostpath.yaml

	I0918 11:53:00.376740    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-hostpath.yaml (4266 bytes)

	I0918 11:53:00.395920    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-server-service.yaml

	I0918 11:53:00.395931    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)

	I0918 11:53:00.403313    1740 addons.go:423] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml

	I0918 11:53:00.403320    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml (19582 bytes)

	I0918 11:53:00.404660    1740 addons.go:423] installing /etc/kubernetes/addons/ig-role.yaml

	I0918 11:53:00.404665    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-role.yaml (210 bytes)

	I0918 11:53:00.430003    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml

	I0918 11:53:00.430014    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml (3038 bytes)

	I0918 11:53:00.492665    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-provisioner.yaml

	I0918 11:53:00.492677    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-provisioner.yaml (4442 bytes)

	I0918 11:53:00.495149    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml

	I0918 11:53:00.495157    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml (3545 bytes)

	I0918 11:53:00.500095    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml

	I0918 11:53:00.522789    1740 addons.go:423] installing /etc/kubernetes/addons/ig-rolebinding.yaml

	I0918 11:53:00.522799    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-rolebinding.yaml (244 bytes)

	I0918 11:53:00.561433    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-resizer.yaml

	I0918 11:53:00.561445    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-resizer.yaml (2943 bytes)

	I0918 11:53:00.561578    1740 addons.go:423] installing /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0918 11:53:00.561584    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml (1475 bytes)

	I0918 11:53:00.579630    1740 addons.go:423] installing /etc/kubernetes/addons/ig-clusterrole.yaml

	I0918 11:53:00.579641    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-clusterrole.yaml (1485 bytes)

	I0918 11:53:00.604202    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0918 11:53:00.607378    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-snapshotter.yaml

	I0918 11:53:00.607389    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-snapshotter.yaml (3149 bytes)

	I0918 11:53:00.624742    1740 addons.go:423] installing /etc/kubernetes/addons/ig-clusterrolebinding.yaml

	I0918 11:53:00.624755    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-clusterrolebinding.yaml (274 bytes)

	I0918 11:53:00.641432    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-attacher.yaml

	I0918 11:53:00.641441    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-attacher.yaml (2143 bytes)

	I0918 11:53:00.675716    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml

	I0918 11:53:00.675728    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml (1274 bytes)

	I0918 11:53:00.684962    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-plugin.yaml

	I0918 11:53:00.684971    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-plugin.yaml (8201 bytes)

	I0918 11:53:00.690643    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-resizer.yaml

	I0918 11:53:00.690652    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-resizer.yaml (2191 bytes)

	I0918 11:53:00.691887    1740 addons.go:423] installing /etc/kubernetes/addons/ig-crd.yaml

	I0918 11:53:00.691893    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-crd.yaml (5216 bytes)

	I0918 11:53:00.701987    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-storageclass.yaml

	I0918 11:53:00.701999    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-storageclass.yaml (846 bytes)

	I0918 11:53:00.719094    1740 addons.go:423] installing /etc/kubernetes/addons/ig-daemonset.yaml

	I0918 11:53:00.719103    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-daemonset.yaml (7741 bytes)

	I0918 11:53:00.821368    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/ig-namespace.yaml -f /etc/kubernetes/addons/ig-serviceaccount.yaml -f /etc/kubernetes/addons/ig-role.yaml -f /etc/kubernetes/addons/ig-rolebinding.yaml -f /etc/kubernetes/addons/ig-clusterrole.yaml -f /etc/kubernetes/addons/ig-clusterrolebinding.yaml -f /etc/kubernetes/addons/ig-crd.yaml -f /etc/kubernetes/addons/ig-daemonset.yaml

	I0918 11:53:00.824330    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml

	I0918 11:53:01.167637    1740 start.go:917] {"host.minikube.internal": 192.168.105.1} host record injected into CoreDNS's ConfigMap

	I0918 11:53:01.168100    1740 node_ready.go:35] waiting up to 6m0s for node "addons-221000" to be "Ready" ...

	I0918 11:53:01.169940    1740 node_ready.go:49] node "addons-221000" has status "Ready":"True"

	I0918 11:53:01.169963    1740 node_ready.go:38] duration metric: took 1.836333ms waiting for node "addons-221000" to be "Ready" ...

	I0918 11:53:01.169968    1740 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I0918 11:53:01.172987    1740 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5dd5756b68-mbgns" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:01.659785    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (1.159683334s)

	I0918 11:53:01.659805    1740 addons.go:467] Verifying addon metrics-server=true in "addons-221000"

	I0918 11:53:01.659835    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: (1.055627667s)

	W0918 11:53:01.659861    1740 addons.go:449] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1

	stdout:

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created

	serviceaccount/snapshot-controller created

	clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created

	clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created

	role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	deployment.apps/snapshot-controller created

	stderr:

	error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"

	ensure CRDs are installed first

	I0918 11:53:01.659882    1740 retry.go:31] will retry after 288.841008ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1

	stdout:

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created

	serviceaccount/snapshot-controller created

	clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created

	clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created

	role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	deployment.apps/snapshot-controller created

	stderr:

	error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"

	ensure CRDs are installed first

	I0918 11:53:01.660144    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (1.320131291s)

	I0918 11:53:01.949280    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply --force -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0918 11:53:02.183526    1740 pod_ready.go:92] pod "coredns-5dd5756b68-mbgns" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.183539    1740 pod_ready.go:81] duration metric: took 1.010553542s waiting for pod "coredns-5dd5756b68-mbgns" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.183545    1740 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.184891    1740 pod_ready.go:97] error getting pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace (skipping!): pods "coredns-5dd5756b68-z4cmf" not found

	I0918 11:53:02.184904    1740 pod_ready.go:81] duration metric: took 1.354709ms waiting for pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace to be "Ready" ...

	E0918 11:53:02.184909    1740 pod_ready.go:66] WaitExtra: waitPodCondition: error getting pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace (skipping!): pods "coredns-5dd5756b68-z4cmf" not found

	I0918 11:53:02.184914    1740 pod_ready.go:78] waiting up to 6m0s for pod "etcd-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.187700    1740 pod_ready.go:92] pod "etcd-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.187709    1740 pod_ready.go:81] duration metric: took 2.791875ms waiting for pod "etcd-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.187714    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.190228    1740 pod_ready.go:92] pod "kube-apiserver-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.190236    1740 pod_ready.go:81] duration metric: took 2.518208ms waiting for pod "kube-apiserver-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.190240    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.571748    1740 pod_ready.go:92] pod "kube-controller-manager-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.571761    1740 pod_ready.go:81] duration metric: took 381.518375ms waiting for pod "kube-controller-manager-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.571765    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-q7gqn" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.823208    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml: (1.998874458s)

	I0918 11:53:02.823230    1740 addons.go:467] Verifying addon csi-hostpath-driver=true in "addons-221000"

	I0918 11:53:02.828329    1740 out.go:177] * Verifying csi-hostpath-driver addon...

	I0918 11:53:02.838784    1740 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=csi-hostpath-driver" in ns "kube-system" ...

	I0918 11:53:02.842760    1740 kapi.go:86] Found 3 Pods for label selector kubernetes.io/minikube-addons=csi-hostpath-driver

	I0918 11:53:02.842767    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:02.847417    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:02.972129    1740 pod_ready.go:92] pod "kube-proxy-q7gqn" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.972138    1740 pod_ready.go:81] duration metric: took 400.3735ms waiting for pod "kube-proxy-q7gqn" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.972143    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:03.351863    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:03.371796    1740 pod_ready.go:92] pod "kube-scheduler-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:03.371806    1740 pod_ready.go:81] duration metric: took 399.662875ms waiting for pod "kube-scheduler-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:03.371810    1740 pod_ready.go:38] duration metric: took 2.201856875s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I0918 11:53:03.371820    1740 api_server.go:52] waiting for apiserver process to appear ...

	I0918 11:53:03.371875    1740 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I0918 11:53:03.851785    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:04.352157    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:04.679993    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply --force -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: (2.73071775s)

	I0918 11:53:04.680002    1740 ssh_runner.go:235] Completed: sudo pgrep -xnf kube-apiserver.*minikube.*: (1.308127625s)

	I0918 11:53:04.680021    1740 api_server.go:72] duration metric: took 4.46545525s to wait for apiserver process to appear ...

	I0918 11:53:04.680025    1740 api_server.go:88] waiting for apiserver healthz status ...

	I0918 11:53:04.680031    1740 api_server.go:253] Checking apiserver healthz at https://192.168.105.2:8443/healthz ...

	I0918 11:53:04.683995    1740 api_server.go:279] https://192.168.105.2:8443/healthz returned 200:

	ok

	I0918 11:53:04.684750    1740 api_server.go:141] control plane version: v1.28.2

	I0918 11:53:04.684756    1740 api_server.go:131] duration metric: took 4.728917ms to wait for apiserver health ...

	I0918 11:53:04.684760    1740 system_pods.go:43] waiting for kube-system pods to appear ...

	I0918 11:53:04.689157    1740 system_pods.go:59] 13 kube-system pods found

	I0918 11:53:04.689166    1740 system_pods.go:61] "coredns-5dd5756b68-mbgns" [376db80e-bef7-49a8-805c-d250bbb5ddc5] Running

	I0918 11:53:04.689171    1740 system_pods.go:61] "csi-hostpath-attacher-0" [b3eb2340-f156-4127-b844-79013849b5d5] Pending / Ready:ContainersNotReady (containers with unready status: [csi-attacher]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-attacher])

	I0918 11:53:04.689175    1740 system_pods.go:61] "csi-hostpath-resizer-0" [a39fc7e1-21e6-43e0-8a71-76fc3122aa67] Pending / Ready:ContainersNotReady (containers with unready status: [csi-resizer]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-resizer])

	I0918 11:53:04.689182    1740 system_pods.go:61] "csi-hostpathplugin-s878j" [f7db805b-46b1-4d4f-b620-4bb9732a0ba0] Pending / Ready:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter])

	I0918 11:53:04.689185    1740 system_pods.go:61] "etcd-addons-221000" [4500c338-1b0c-4d39-b5b6-76d42cf285f5] Running

	I0918 11:53:04.689188    1740 system_pods.go:61] "kube-apiserver-addons-221000" [93a9cc2e-3463-4332-a37c-86437106ed5e] Running

	I0918 11:53:04.689190    1740 system_pods.go:61] "kube-controller-manager-addons-221000" [eaeef50f-2b51-43b2-91c9-a4f97f5460ae] Running

	I0918 11:53:04.689193    1740 system_pods.go:61] "kube-proxy-q7gqn" [e971c33c-7d1b-47b7-9ff5-3a629f12fb57] Running

	I0918 11:53:04.689195    1740 system_pods.go:61] "kube-scheduler-addons-221000" [9d39af63-6f0a-4855-9435-f8e7af26869e] Running

	I0918 11:53:04.689199    1740 system_pods.go:61] "metrics-server-7c66d45ddc-ph4qt" [193ff040-48cb-4429-8aa3-4065ecb5241d] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I0918 11:53:04.689205    1740 system_pods.go:61] "snapshot-controller-58dbcc7b99-89j9m" [d9a96c2a-2231-4dea-abbf-16875dd2b1d7] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.689210    1740 system_pods.go:61] "snapshot-controller-58dbcc7b99-xwwxn" [5a80446c-a3a8-4ce7-8ac4-2894087691fb] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.689214    1740 system_pods.go:61] "storage-provisioner" [88ff8527-97e2-4317-8d5b-a2502e8cb7f7] Running

	I0918 11:53:04.689217    1740 system_pods.go:74] duration metric: took 4.45475ms to wait for pod list to return data ...

	I0918 11:53:04.689220    1740 default_sa.go:34] waiting for default service account to be created ...

	I0918 11:53:04.690951    1740 default_sa.go:45] found service account: "default"

	I0918 11:53:04.690958    1740 default_sa.go:55] duration metric: took 1.736083ms for default service account to be created ...

	I0918 11:53:04.690961    1740 system_pods.go:116] waiting for k8s-apps to be running ...

	I0918 11:53:04.694962    1740 system_pods.go:86] 13 kube-system pods found

	I0918 11:53:04.694971    1740 system_pods.go:89] "coredns-5dd5756b68-mbgns" [376db80e-bef7-49a8-805c-d250bbb5ddc5] Running

	I0918 11:53:04.694976    1740 system_pods.go:89] "csi-hostpath-attacher-0" [b3eb2340-f156-4127-b844-79013849b5d5] Pending / Ready:ContainersNotReady (containers with unready status: [csi-attacher]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-attacher])

	I0918 11:53:04.694979    1740 system_pods.go:89] "csi-hostpath-resizer-0" [a39fc7e1-21e6-43e0-8a71-76fc3122aa67] Pending / Ready:ContainersNotReady (containers with unready status: [csi-resizer]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-resizer])

	I0918 11:53:04.694983    1740 system_pods.go:89] "csi-hostpathplugin-s878j" [f7db805b-46b1-4d4f-b620-4bb9732a0ba0] Pending / Ready:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter])

	I0918 11:53:04.694986    1740 system_pods.go:89] "etcd-addons-221000" [4500c338-1b0c-4d39-b5b6-76d42cf285f5] Running

	I0918 11:53:04.694988    1740 system_pods.go:89] "kube-apiserver-addons-221000" [93a9cc2e-3463-4332-a37c-86437106ed5e] Running

	I0918 11:53:04.694990    1740 system_pods.go:89] "kube-controller-manager-addons-221000" [eaeef50f-2b51-43b2-91c9-a4f97f5460ae] Running

	I0918 11:53:04.694994    1740 system_pods.go:89] "kube-proxy-q7gqn" [e971c33c-7d1b-47b7-9ff5-3a629f12fb57] Running

	I0918 11:53:04.694996    1740 system_pods.go:89] "kube-scheduler-addons-221000" [9d39af63-6f0a-4855-9435-f8e7af26869e] Running

	I0918 11:53:04.694999    1740 system_pods.go:89] "metrics-server-7c66d45ddc-ph4qt" [193ff040-48cb-4429-8aa3-4065ecb5241d] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I0918 11:53:04.695003    1740 system_pods.go:89] "snapshot-controller-58dbcc7b99-89j9m" [d9a96c2a-2231-4dea-abbf-16875dd2b1d7] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.695006    1740 system_pods.go:89] "snapshot-controller-58dbcc7b99-xwwxn" [5a80446c-a3a8-4ce7-8ac4-2894087691fb] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.695009    1740 system_pods.go:89] "storage-provisioner" [88ff8527-97e2-4317-8d5b-a2502e8cb7f7] Running

	I0918 11:53:04.695012    1740 system_pods.go:126] duration metric: took 4.049ms to wait for k8s-apps to be running ...

	I0918 11:53:04.695014    1740 system_svc.go:44] waiting for kubelet service to be running ....

	I0918 11:53:04.695074    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I0918 11:53:04.700754    1740 system_svc.go:56] duration metric: took 5.736541ms WaitForService to wait for kubelet.

	I0918 11:53:04.700761    1740 kubeadm.go:581] duration metric: took 4.486195916s to wait for : map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] ...

	I0918 11:53:04.700771    1740 node_conditions.go:102] verifying NodePressure condition ...

	I0918 11:53:04.702224    1740 node_conditions.go:122] node storage ephemeral capacity is 17784760Ki

	I0918 11:53:04.702234    1740 node_conditions.go:123] node cpu capacity is 2

	I0918 11:53:04.702239    1740 node_conditions.go:105] duration metric: took 1.466083ms to run NodePressure ...

	I0918 11:53:04.702244    1740 start.go:228] waiting for startup goroutines ...

	I0918 11:53:04.851794    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:05.352908    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:05.851843    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:06.351684    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:06.787954    1740 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_application_credentials.json (162 bytes)

	I0918 11:53:06.787972    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:06.819465    1740 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_cloud_project (12 bytes)

	I0918 11:53:06.824867    1740 addons.go:231] Setting addon gcp-auth=true in "addons-221000"

	I0918 11:53:06.824887    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:06.825624    1740 ssh_runner.go:195] Run: cat /var/lib/minikube/google_application_credentials.json

	I0918 11:53:06.825631    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:06.851875    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:06.860327    1740 out.go:177]   - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20230407

	I0918 11:53:06.868318    1740 out.go:177]   - Using image gcr.io/k8s-minikube/gcp-auth-webhook:v0.1.0

	I0918 11:53:06.871302    1740 addons.go:423] installing /etc/kubernetes/addons/gcp-auth-ns.yaml

	I0918 11:53:06.871308    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-ns.yaml (700 bytes)

	I0918 11:53:06.876186    1740 addons.go:423] installing /etc/kubernetes/addons/gcp-auth-service.yaml

	I0918 11:53:06.876191    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-service.yaml (788 bytes)

	I0918 11:53:06.881437    1740 addons.go:423] installing /etc/kubernetes/addons/gcp-auth-webhook.yaml

	I0918 11:53:06.881443    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-webhook.yaml (5412 bytes)

	I0918 11:53:06.887709    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/gcp-auth-ns.yaml -f /etc/kubernetes/addons/gcp-auth-service.yaml -f /etc/kubernetes/addons/gcp-auth-webhook.yaml

	I0918 11:53:07.124491    1740 addons.go:467] Verifying addon gcp-auth=true in "addons-221000"

	I0918 11:53:07.129030    1740 out.go:177] * Verifying gcp-auth addon...

	I0918 11:53:07.137330    1740 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=gcp-auth" in ns "gcp-auth" ...

	I0918 11:53:07.139319    1740 kapi.go:86] Found 1 Pods for label selector kubernetes.io/minikube-addons=gcp-auth

	I0918 11:53:07.139325    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:07.142056    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:07.351786    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:07.647787    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:07.851746    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:08.144923    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:08.351959    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:08.646023    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:08.851939    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:09.146053    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:09.352896    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:09.645776    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:09.851792    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:10.146843    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:10.351428    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:10.645894    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:10.852236    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:11.145951    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:11.352216    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:11.645774    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:11.852232    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:12.145929    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:12.355119    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:12.840629    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:12.851603    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:13.145852    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:13.351962    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:13.646245    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:13.852112    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:14.144894    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:14.351204    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:14.646046    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:15.059939    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:15.145929    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:15.351906    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:15.646033    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:15.852022    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:16.145893    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:16.351976    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:16.645740    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:16.852007    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:17.145547    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:17.352003    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:17.646147    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:17.853011    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:18.145960    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:18.353519    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:18.646257    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:18.851829    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:19.143968    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:19.351778    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:19.645563    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:19.851678    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:20.145637    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:20.351425    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:20.645727    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:20.852080    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:21.145053    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:21.352010    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:21.645631    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:21.851983    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:22.146102    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:22.351559    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:22.645364    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:22.851995    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:23.145511    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:23.351664    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:23.646427    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:23.851813    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:24.144653    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:24.351670    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:24.645732    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:24.851659    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:25.145755    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:25.350286    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:25.645572    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:25.851968    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:26.145692    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:26.352042    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:26.645575    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:26.852307    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:27.145498    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:27.352114    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:27.645719    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:27.851965    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:28.145987    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:28.351772    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:28.645594    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:28.851993    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:29.145747    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:29.351502    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:29.645697    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:29.851953    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:30.145495    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:30.351423    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:30.645330    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:30.852142    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:31.145422    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:31.352025    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:31.645572    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:31.852021    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:32.145742    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:32.351596    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:32.645761    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:32.851846    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:33.145666    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:33.351607    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:33.646262    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:33.852115    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:34.145556    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:34.353136    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:34.644719    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:34.852049    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:35.145343    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:35.351551    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:35.645361    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:35.851476    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:36.143766    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:36.351679    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:36.645788    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:36.851584    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:37.145933    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:37.351445    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:37.646034    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:37.851557    1740 kapi.go:107] duration metric: took 35.013102459s to wait for kubernetes.io/minikube-addons=csi-hostpath-driver ...

	I0918 11:53:38.145661    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:38.645759    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:39.145382    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:39.645858    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:40.145851    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:40.645842    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:41.145511    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:41.646176    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:42.145937    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:42.645521    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:43.145380    1740 kapi.go:107] duration metric: took 36.008387458s to wait for kubernetes.io/minikube-addons=gcp-auth ...

	I0918 11:53:43.148984    1740 out.go:177] * Your GCP credentials will now be mounted into every pod created in the addons-221000 cluster.

	I0918 11:53:43.152958    1740 out.go:177] * If you don't want your credentials mounted into a specific pod, add a label with the `gcp-auth-skip-secret` key to your pod configuration.

	I0918 11:53:43.156871    1740 out.go:177] * If you want existing pods to be mounted with credentials, either recreate them or rerun addons enable with --refresh.

	I0918 11:59:00.190235    1740 kapi.go:107] duration metric: took 6m0.011650291s to wait for kubernetes.io/minikube-addons=registry ...

	W0918 11:59:00.190354    1740 out.go:239] ! Enabling 'registry' returned an error: running callbacks: [waiting for kubernetes.io/minikube-addons=registry pods: context deadline exceeded]

	I0918 11:59:00.236861    1740 kapi.go:107] duration metric: took 6m0.007418791s to wait for app.kubernetes.io/name=ingress-nginx ...

	W0918 11:59:00.236887    1740 out.go:239] ! Enabling 'ingress' returned an error: running callbacks: [waiting for app.kubernetes.io/name=ingress-nginx pods: context deadline exceeded]

	I0918 11:59:00.245622    1740 out.go:177] * Enabled addons: ingress-dns, cloud-spanner, default-storageclass, metrics-server, storage-provisioner, inspektor-gadget, volumesnapshots, csi-hostpath-driver, gcp-auth

	I0918 11:59:00.252559    1740 addons.go:502] enable addons completed in 6m0.086876584s: enabled=[ingress-dns cloud-spanner default-storageclass metrics-server storage-provisioner inspektor-gadget volumesnapshots csi-hostpath-driver gcp-auth]

	I0918 11:59:00.252570    1740 start.go:233] waiting for cluster config update ...

	I0918 11:59:00.252577    1740 start.go:242] writing updated cluster config ...

	I0918 11:59:00.253046    1740 ssh_runner.go:195] Run: rm -f paused

	I0918 11:59:00.282509    1740 start.go:600] kubectl: 1.27.2, cluster: 1.28.2 (minor skew: 1)

	I0918 11:59:00.285533    1740 out.go:177] * Done! kubectl is now configured to use "addons-221000" cluster and "default" namespace by default

	* 

	* ==> Docker <==

	* -- Journal begins at Mon 2023-09-18 18:52:28 UTC, ends at Mon 2023-09-18 19:11:00 UTC. --

	Sep 18 18:53:31 addons-221000 dockerd[1106]: time="2023-09-18T18:53:31.913045758Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 18:53:31 addons-221000 dockerd[1106]: time="2023-09-18T18:53:31.913052008Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 18:53:32 addons-221000 dockerd[1100]: time="2023-09-18T18:53:32.000341717Z" level=warning msg="reference for unknown type: " digest="sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8" remote="registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8"

	Sep 18 18:53:34 addons-221000 cri-dockerd[995]: time="2023-09-18T18:53:34Z" level=info msg="Stop pulling image registry.k8s.io/sig-storage/csi-provisioner:v3.3.0@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8: Status: Downloaded newer image for registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8"

	Sep 18 18:53:34 addons-221000 dockerd[1106]: time="2023-09-18T18:53:34.455925540Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 18:53:34 addons-221000 dockerd[1106]: time="2023-09-18T18:53:34.456111241Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 18:53:34 addons-221000 dockerd[1106]: time="2023-09-18T18:53:34.456126491Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 18:53:34 addons-221000 dockerd[1106]: time="2023-09-18T18:53:34.456165406Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 18:53:34 addons-221000 dockerd[1100]: time="2023-09-18T18:53:34.546723654Z" level=warning msg="reference for unknown type: " digest="sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f" remote="registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f"

	Sep 18 18:53:36 addons-221000 cri-dockerd[995]: time="2023-09-18T18:53:36Z" level=info msg="Stop pulling image registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f: Status: Downloaded newer image for registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f"

	Sep 18 18:53:36 addons-221000 dockerd[1106]: time="2023-09-18T18:53:36.766332627Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 18:53:36 addons-221000 dockerd[1106]: time="2023-09-18T18:53:36.766362584Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 18:53:36 addons-221000 dockerd[1106]: time="2023-09-18T18:53:36.766375125Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 18:53:36 addons-221000 dockerd[1106]: time="2023-09-18T18:53:36.766381709Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 18:53:40 addons-221000 dockerd[1106]: time="2023-09-18T18:53:40.152856514Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 18:53:40 addons-221000 dockerd[1106]: time="2023-09-18T18:53:40.152909804Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 18:53:40 addons-221000 dockerd[1106]: time="2023-09-18T18:53:40.152924220Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 18:53:40 addons-221000 dockerd[1106]: time="2023-09-18T18:53:40.152934887Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 18:53:40 addons-221000 cri-dockerd[995]: time="2023-09-18T18:53:40Z" level=info msg="Will attempt to re-write config file /var/lib/docker/containers/e321131e2d88d58235861ac8364e2e9ea1e94ae5e42fd565fa9e65fa17e39a30/resolv.conf as [nameserver 10.96.0.10 search gcp-auth.svc.cluster.local svc.cluster.local cluster.local options ndots:5]"

	Sep 18 18:53:40 addons-221000 dockerd[1100]: time="2023-09-18T18:53:40.332879779Z" level=warning msg="reference for unknown type: " digest="sha256:d730651bb6584f969d95d8279a754cf9d8d31b5055c43dbdb8d7363a8c6371cf" remote="gcr.io/k8s-minikube/gcp-auth-webhook@sha256:d730651bb6584f969d95d8279a754cf9d8d31b5055c43dbdb8d7363a8c6371cf"

	Sep 18 18:53:41 addons-221000 cri-dockerd[995]: time="2023-09-18T18:53:41Z" level=info msg="Stop pulling image gcr.io/k8s-minikube/gcp-auth-webhook:v0.1.0@sha256:d730651bb6584f969d95d8279a754cf9d8d31b5055c43dbdb8d7363a8c6371cf: Status: Downloaded newer image for gcr.io/k8s-minikube/gcp-auth-webhook@sha256:d730651bb6584f969d95d8279a754cf9d8d31b5055c43dbdb8d7363a8c6371cf"

	Sep 18 18:53:42 addons-221000 dockerd[1106]: time="2023-09-18T18:53:42.114894841Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 18:53:42 addons-221000 dockerd[1106]: time="2023-09-18T18:53:42.115057171Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 18:53:42 addons-221000 dockerd[1106]: time="2023-09-18T18:53:42.115064546Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 18:53:42 addons-221000 dockerd[1106]: time="2023-09-18T18:53:42.115841863Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	* 

	* ==> container status <==

	* CONTAINER           IMAGE                                                                                                                                        CREATED             STATE               NAME                                     ATTEMPT             POD ID

	6009e365438d8       gcr.io/k8s-minikube/gcp-auth-webhook@sha256:d730651bb6584f969d95d8279a754cf9d8d31b5055c43dbdb8d7363a8c6371cf                                 17 minutes ago      Running             gcp-auth                                 0                   e321131e2d88d

	3bb40d34efc29       registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f                          17 minutes ago      Running             csi-snapshotter                          0                   2dc44d50472a8

	503c9a6a65b7d       registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8                          17 minutes ago      Running             csi-provisioner                          0                   2dc44d50472a8

	d722c62039b0b       registry.k8s.io/sig-storage/livenessprobe@sha256:cacee2b5c36dd59d4c7e8469c05c9e4ef53ecb2df9025fa8c10cdaf61bce62f0                            17 minutes ago      Running             liveness-probe                           0                   2dc44d50472a8

	ddc75a67ab827       registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5                           17 minutes ago      Running             hostpath                                 0                   2dc44d50472a8

	6b4de2f3acf09       registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:f1c25991bac2fbb7f5fcf91ed9438df31e30edee6bed5a780464238aa09ad24c                17 minutes ago      Running             node-driver-registrar                    0                   2dc44d50472a8

	70e58783f3748       registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7                              17 minutes ago      Running             csi-resizer                              0                   5ec9976a9ac7e

	2c2e40482e98d       registry.k8s.io/sig-storage/csi-external-health-monitor-controller@sha256:80b9ba94aa2afe24553d69bd165a6a51552d1582d68618ec00d3b804a7d9193c   17 minutes ago      Running             csi-external-health-monitor-controller   0                   2dc44d50472a8

	d0bd02275ae05       registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b                             17 minutes ago      Running             csi-attacher                             0                   b64a9c9d34ad2

	afd4040b11d6e       ghcr.io/inspektor-gadget/inspektor-gadget@sha256:01b7311f9512411ef6530e09dbdd3aeaea0abc4101227dbead4d44c36b255ca7                            17 minutes ago      Running             gadget                                   0                   a7e6c06852560

	ab33a0f8756c5       registry.k8s.io/metrics-server/metrics-server@sha256:ee4304963fb035239bb5c5e8c10f2f38ee80efc16ecbdb9feb7213c17ae2e86e                        17 minutes ago      Running             metrics-server                           0                   b759b3bc87e67

	68cb499e8a4ea       registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280                      17 minutes ago      Running             volume-snapshot-controller               0                   322c06e1ffed3

	db4658d155f25       registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280                      17 minutes ago      Running             volume-snapshot-controller               0                   f31f590bd0703

	96b1e19d34d0f       ba04bb24b9575                                                                                                                                17 minutes ago      Running             storage-provisioner                      0                   31030bebae5dd

	3397ed73112e1       97e04611ad434                                                                                                                                18 minutes ago      Running             coredns                                  0                   ecf214ed85c34

	3b8b236037bf7       7da62c127fc0f                                                                                                                                18 minutes ago      Running             kube-proxy                               0                   efd5b0f304a7c

	17d16f9191cb9       64fc40cee3716                                                                                                                                18 minutes ago      Running             kube-scheduler                           0                   6dcf2ed48fa0d

	cb85fb8fd00cf       89d57b83c1786                                                                                                                                18 minutes ago      Running             kube-controller-manager                  0                   d0674da883f4f

	4a8eb16a561d8       30bb499447fe1                                                                                                                                18 minutes ago      Running             kube-apiserver                           0                   2a8be15cc8448

	2a7ec1fe69df2       9cdd6470f48c8                                                                                                                                18 minutes ago      Running             etcd                                     0                   0f317e031c491

	* 

	* ==> coredns [3397ed73112e] <==

	* .:53

	[INFO] plugin/reload: Running configuration SHA512 = 591cf328cccc12bc490481273e738df59329c62c0b729d94e8b61db9961c2fa5f046dd37f1cf888b953814040d180f52594972691cd6ff41be96639138a43908

	CoreDNS-1.10.1

	linux/arm64, go1.20, 055b2c3

	[INFO] Reloading

	[INFO] plugin/reload: Running configuration SHA512 = ea7a0d73d9d208f758b1f67640ef03c58089b9d9366cf3478df3bb369b210e39f213811b46224f8a04380814b6e0890ccd358f5b5e8c80bc22ac19c8601ee35b

	[INFO] Reloading complete

	[INFO] 127.0.0.1:51523 - 34394 "HINFO IN 2890994018648264973.7586751697303130781. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.046989834s

	[INFO] 10.244.0.11:51746 - 24273 "AAAA IN storage.googleapis.com.gcp-auth.svc.cluster.local. udp 78 false 1232" NXDOMAIN qr,aa,rd 160 0.000125498s

	[INFO] 10.244.0.11:38492 - 22517 "A IN storage.googleapis.com.gcp-auth.svc.cluster.local. udp 78 false 1232" NXDOMAIN qr,aa,rd 160 0.000252786s

	[INFO] 10.244.0.11:44956 - 48338 "A IN storage.googleapis.com.svc.cluster.local. udp 69 false 1232" NXDOMAIN qr,aa,rd 151 0.000035291s

	[INFO] 10.244.0.11:34979 - 63104 "AAAA IN storage.googleapis.com.svc.cluster.local. udp 69 false 1232" NXDOMAIN qr,aa,rd 151 0.000059957s

	[INFO] 10.244.0.11:38551 - 60811 "A IN storage.googleapis.com.cluster.local. udp 65 false 1232" NXDOMAIN qr,aa,rd 147 0.000037916s

	[INFO] 10.244.0.11:44454 - 13651 "AAAA IN storage.googleapis.com.cluster.local. udp 65 false 1232" NXDOMAIN qr,aa,rd 147 0.000042082s

	[INFO] 10.244.0.11:41401 - 32727 "AAAA IN storage.googleapis.com. udp 51 false 1232" NOERROR qr,rd,ra 240 0.001047395s

	[INFO] 10.244.0.11:40924 - 8451 "A IN storage.googleapis.com. udp 51 false 1232" NOERROR qr,rd,ra 496 0.001040478s

	* 

	* ==> describe nodes <==

	* Name:               addons-221000

	Roles:              control-plane

	Labels:             beta.kubernetes.io/arch=arm64

	                    beta.kubernetes.io/os=linux

	                    kubernetes.io/arch=arm64

	                    kubernetes.io/hostname=addons-221000

	                    kubernetes.io/os=linux

	                    minikube.k8s.io/commit=ae2adeb20cc3a4b5f1decc7c8f53736ec04c4a36

	                    minikube.k8s.io/name=addons-221000

	                    minikube.k8s.io/primary=true

	                    minikube.k8s.io/updated_at=2023_09_18T11_52_46_0700

	                    minikube.k8s.io/version=v1.31.2

	                    node-role.kubernetes.io/control-plane=

	                    node.kubernetes.io/exclude-from-external-load-balancers=

	                    topology.hostpath.csi/node=addons-221000

	Annotations:        csi.volume.kubernetes.io/nodeid: {"hostpath.csi.k8s.io":"addons-221000"}

	                    kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock

	                    node.alpha.kubernetes.io/ttl: 0

	                    volumes.kubernetes.io/controller-managed-attach-detach: true

	CreationTimestamp:  Mon, 18 Sep 2023 18:52:43 +0000

	Taints:             <none>

	Unschedulable:      false

	Lease:

	  HolderIdentity:  addons-221000

	  AcquireTime:     <unset>

	  RenewTime:       Mon, 18 Sep 2023 19:10:58 +0000

	Conditions:

	  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message

	  ----             ------  -----------------                 ------------------                ------                       -------

	  MemoryPressure   False   Mon, 18 Sep 2023 19:09:09 +0000   Mon, 18 Sep 2023 18:52:43 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available

	  DiskPressure     False   Mon, 18 Sep 2023 19:09:09 +0000   Mon, 18 Sep 2023 18:52:43 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure

	  PIDPressure      False   Mon, 18 Sep 2023 19:09:09 +0000   Mon, 18 Sep 2023 18:52:43 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available

	  Ready            True    Mon, 18 Sep 2023 19:09:09 +0000   Mon, 18 Sep 2023 18:52:48 +0000   KubeletReady                 kubelet is posting ready status

	Addresses:

	  InternalIP:  192.168.105.2

	  Hostname:    addons-221000

	Capacity:

	  cpu:                2

	  ephemeral-storage:  17784760Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3905012Ki

	  pods:               110

	Allocatable:

	  cpu:                2

	  ephemeral-storage:  17784760Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3905012Ki

	  pods:               110

	System Info:

	  Machine ID:                 605bc2fc72a045ae88e907db961da3d3

	  System UUID:                605bc2fc72a045ae88e907db961da3d3

	  Boot ID:                    6a9990c2-fe5e-48d8-97ca-ea50d8c8e3b8

	  Kernel Version:             5.10.57

	  OS Image:                   Buildroot 2021.02.12

	  Operating System:           linux

	  Architecture:               arm64

	  Container Runtime Version:  docker://24.0.6

	  Kubelet Version:            v1.28.2

	  Kube-Proxy Version:         v1.28.2

	PodCIDR:                      10.244.0.0/24

	PodCIDRs:                     10.244.0.0/24

	Non-terminated Pods:          (15 in total)

	  Namespace                   Name                                     CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age

	  ---------                   ----                                     ------------  ----------  ---------------  -------------  ---

	  gadget                      gadget-2qmph                             0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         17m

	  gcp-auth                    gcp-auth-d4c87556c-2vm8d                 0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         17m

	  kube-system                 coredns-5dd5756b68-mbgns                 100m (5%!)(MISSING)     0 (0%!)(MISSING)      70Mi (1%!)(MISSING)        170Mi (4%!)(MISSING)     18m

	  kube-system                 csi-hostpath-attacher-0                  0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         17m

	  kube-system                 csi-hostpath-resizer-0                   0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         17m

	  kube-system                 csi-hostpathplugin-s878j                 0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         17m

	  kube-system                 etcd-addons-221000                       100m (5%!)(MISSING)     0 (0%!)(MISSING)      100Mi (2%!)(MISSING)       0 (0%!)(MISSING)         18m

	  kube-system                 kube-apiserver-addons-221000             250m (12%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         18m

	  kube-system                 kube-controller-manager-addons-221000    200m (10%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         18m

	  kube-system                 kube-proxy-q7gqn                         0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         18m

	  kube-system                 kube-scheduler-addons-221000             100m (5%!)(MISSING)     0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         18m

	  kube-system                 metrics-server-7c66d45ddc-ph4qt          100m (5%!)(MISSING)     0 (0%!)(MISSING)      200Mi (5%!)(MISSING)       0 (0%!)(MISSING)         17m

	  kube-system                 snapshot-controller-58dbcc7b99-89j9m     0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         17m

	  kube-system                 snapshot-controller-58dbcc7b99-xwwxn     0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         17m

	  kube-system                 storage-provisioner                      0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         17m

	Allocated resources:

	  (Total limits may be over 100 percent, i.e., overcommitted.)

	  Resource           Requests    Limits

	  --------           --------    ------

	  cpu                850m (42%!)(MISSING)  0 (0%!)(MISSING)

	  memory             370Mi (9%!)(MISSING)  170Mi (4%!)(MISSING)

	  ephemeral-storage  0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-1Gi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-2Mi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-32Mi     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-64Ki     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	Events:

	  Type    Reason                   Age   From             Message

	  ----    ------                   ----  ----             -------

	  Normal  Starting                 18m   kube-proxy       

	  Normal  Starting                 18m   kubelet          Starting kubelet.

	  Normal  NodeAllocatableEnforced  18m   kubelet          Updated Node Allocatable limit across pods

	  Normal  NodeHasSufficientMemory  18m   kubelet          Node addons-221000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    18m   kubelet          Node addons-221000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     18m   kubelet          Node addons-221000 status is now: NodeHasSufficientPID

	  Normal  NodeReady                18m   kubelet          Node addons-221000 status is now: NodeReady

	  Normal  RegisteredNode           18m   node-controller  Node addons-221000 event: Registered Node addons-221000 in Controller

	* 

	* ==> dmesg <==

	* [  +0.000000] KASLR disabled due to lack of seed

	[  +0.641705] EINJ: EINJ table not found.

	[  +0.512113] systemd-fstab-generator[117]: Ignoring "noauto" for root device

	[  +0.044199] systemd[1]: systemd-journald.service: unit configures an IP firewall, but the local system does not support BPF/cgroup firewalling.

	[  +0.000795] systemd[1]: (This warning is only shown for the first unit using IP firewalling.)

	[  +3.076142] systemd-fstab-generator[480]: Ignoring "noauto" for root device

	[  +0.068934] systemd-fstab-generator[491]: Ignoring "noauto" for root device

	[  +0.415221] systemd-fstab-generator[666]: Ignoring "noauto" for root device

	[  +0.169313] systemd-fstab-generator[702]: Ignoring "noauto" for root device

	[  +0.075007] systemd-fstab-generator[713]: Ignoring "noauto" for root device

	[  +0.084392] systemd-fstab-generator[726]: Ignoring "noauto" for root device

	[  +1.144805] kauditd_printk_skb: 17 callbacks suppressed

	[  +0.094980] systemd-fstab-generator[914]: Ignoring "noauto" for root device

	[  +0.076494] systemd-fstab-generator[925]: Ignoring "noauto" for root device

	[  +0.074272] systemd-fstab-generator[936]: Ignoring "noauto" for root device

	[  +0.076208] systemd-fstab-generator[947]: Ignoring "noauto" for root device

	[  +0.088287] systemd-fstab-generator[988]: Ignoring "noauto" for root device

	[  +2.538937] systemd-fstab-generator[1093]: Ignoring "noauto" for root device

	[  +2.473246] kauditd_printk_skb: 29 callbacks suppressed

	[  +3.092849] systemd-fstab-generator[1411]: Ignoring "noauto" for root device

	[  +4.633357] systemd-fstab-generator[2291]: Ignoring "noauto" for root device

	[Sep18 18:53] kauditd_printk_skb: 41 callbacks suppressed

	[  +1.646092] TCP: eth0: Driver has suspect GRO implementation, TCP performance may be compromised.

	[  +4.523270] kauditd_printk_skb: 55 callbacks suppressed

	[ +11.548658] kauditd_printk_skb: 8 callbacks suppressed

	* 

	* ==> etcd [2a7ec1fe69df] <==

	* {"level":"info","ts":"2023-09-18T18:52:43.020227Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2023-09-18T18:52:43.020734Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"127.0.0.1:2379"}

	{"level":"info","ts":"2023-09-18T18:52:43.023404Z","caller":"etcdserver/server.go:2571","msg":"setting up initial cluster version using v2 API","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:52:43.023573Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2023-09-18T18:52:43.02399Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.105.2:2379"}

	{"level":"info","ts":"2023-09-18T18:52:43.024241Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}

	{"level":"info","ts":"2023-09-18T18:52:43.02431Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}

	{"level":"info","ts":"2023-09-18T18:52:43.024433Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"6e03e7863b4f9c54","local-member-id":"c46d288d2fcb0590","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:52:43.024491Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:52:43.025343Z","caller":"etcdserver/server.go:2595","msg":"cluster version is updated","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:53:12.839558Z","caller":"traceutil/trace.go:171","msg":"trace[499305584] linearizableReadLoop","detail":"{readStateIndex:693; appliedIndex:692; }","duration":"193.895541ms","start":"2023-09-18T18:53:12.645653Z","end":"2023-09-18T18:53:12.839549Z","steps":["trace[499305584] 'read index received'  (duration: 193.791791ms)","trace[499305584] 'applied index is now lower than readState.Index'  (duration: 103.417µs)"],"step_count":2}

	{"level":"info","ts":"2023-09-18T18:53:12.839607Z","caller":"traceutil/trace.go:171","msg":"trace[593448589] transaction","detail":"{read_only:false; response_revision:671; number_of_response:1; }","duration":"309.762625ms","start":"2023-09-18T18:53:12.529838Z","end":"2023-09-18T18:53:12.839601Z","steps":["trace[593448589] 'process raft request'  (duration: 309.63075ms)"],"step_count":1}

	{"level":"warn","ts":"2023-09-18T18:53:12.839655Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"193.989167ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/gcp-auth/\" range_end:\"/registry/pods/gcp-auth0\" ","response":"range_response_count:3 size:10532"}

	{"level":"info","ts":"2023-09-18T18:53:12.839671Z","caller":"traceutil/trace.go:171","msg":"trace[1312627899] range","detail":"{range_begin:/registry/pods/gcp-auth/; range_end:/registry/pods/gcp-auth0; response_count:3; response_revision:671; }","duration":"194.031ms","start":"2023-09-18T18:53:12.645637Z","end":"2023-09-18T18:53:12.839668Z","steps":["trace[1312627899] 'agreement among raft nodes before linearized reading'  (duration: 193.963833ms)"],"step_count":1}

	{"level":"warn","ts":"2023-09-18T18:53:12.8398Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2023-09-18T18:53:12.529832Z","time spent":"309.7915ms","remote":"127.0.0.1:54556","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1098,"response count":0,"response size":39,"request content":"compare:<target:MOD key:\"/registry/services/endpoints/kube-system/k8s.io-minikube-hostpath\" mod_revision:669 > success:<request_put:<key:\"/registry/services/endpoints/kube-system/k8s.io-minikube-hostpath\" value_size:1025 >> failure:<request_range:<key:\"/registry/services/endpoints/kube-system/k8s.io-minikube-hostpath\" > >"}

	{"level":"info","ts":"2023-09-18T18:53:15.058529Z","caller":"traceutil/trace.go:171","msg":"trace[42488278] linearizableReadLoop","detail":"{readStateIndex:694; appliedIndex:693; }","duration":"207.837416ms","start":"2023-09-18T18:53:14.850683Z","end":"2023-09-18T18:53:15.05852Z","steps":["trace[42488278] 'read index received'  (duration: 207.76525ms)","trace[42488278] 'applied index is now lower than readState.Index'  (duration: 71.708µs)"],"step_count":2}

	{"level":"warn","ts":"2023-09-18T18:53:15.058651Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"207.966125ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/\" range_end:\"/registry/pods/kube-system0\" ","response":"range_response_count:13 size:63849"}

	{"level":"info","ts":"2023-09-18T18:53:15.058696Z","caller":"traceutil/trace.go:171","msg":"trace[2008509018] range","detail":"{range_begin:/registry/pods/kube-system/; range_end:/registry/pods/kube-system0; response_count:13; response_revision:672; }","duration":"208.018666ms","start":"2023-09-18T18:53:14.850673Z","end":"2023-09-18T18:53:15.058691Z","steps":["trace[2008509018] 'agreement among raft nodes before linearized reading'  (duration: 207.886666ms)"],"step_count":1}

	{"level":"info","ts":"2023-09-18T18:53:15.058824Z","caller":"traceutil/trace.go:171","msg":"trace[1590636669] transaction","detail":"{read_only:false; response_revision:672; number_of_response:1; }","duration":"214.775084ms","start":"2023-09-18T18:53:14.844046Z","end":"2023-09-18T18:53:15.058821Z","steps":["trace[1590636669] 'process raft request'  (duration: 214.424584ms)"],"step_count":1}

	{"level":"info","ts":"2023-09-18T19:02:43.443637Z","caller":"mvcc/index.go:214","msg":"compact tree index","revision":1094}

	{"level":"info","ts":"2023-09-18T19:02:43.458272Z","caller":"mvcc/kvstore_compaction.go:66","msg":"finished scheduled compaction","compact-revision":1094,"took":"14.26046ms","hash":1108450172}

	{"level":"info","ts":"2023-09-18T19:02:43.458292Z","caller":"mvcc/hash.go:137","msg":"storing new hash","hash":1108450172,"revision":1094,"compact-revision":-1}

	{"level":"info","ts":"2023-09-18T19:07:43.446092Z","caller":"mvcc/index.go:214","msg":"compact tree index","revision":1454}

	{"level":"info","ts":"2023-09-18T19:07:43.44678Z","caller":"mvcc/kvstore_compaction.go:66","msg":"finished scheduled compaction","compact-revision":1454,"took":"494.332µs","hash":2695871163}

	{"level":"info","ts":"2023-09-18T19:07:43.446796Z","caller":"mvcc/hash.go:137","msg":"storing new hash","hash":2695871163,"revision":1454,"compact-revision":1094}

	* 

	* ==> gcp-auth [6009e365438d] <==

	* 2023/09/18 18:53:42 GCP Auth Webhook started!

	* 

	* ==> kernel <==

	*  19:11:00 up 18 min,  0 users,  load average: 0.06, 0.14, 0.16

	Linux addons-221000 5.10.57 #1 SMP PREEMPT Fri Sep 15 19:03:18 UTC 2023 aarch64 GNU/Linux

	PRETTY_NAME="Buildroot 2021.02.12"

	* 

	* ==> kube-apiserver [4a8eb16a561d] <==

	* I0918 19:01:44.199428       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:02:44.199298       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:02:44.256077       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:02:44.256094       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:02:44.256307       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:02:44.256319       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:02:44.256347       1 handler.go:232] Adding GroupVersion gadget.kinvolk.io v1alpha1 to ResourceManager

	I0918 19:02:44.267692       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:03:44.200100       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:04:44.199686       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:05:44.199355       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:06:44.199779       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:07:44.199253       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:07:44.256724       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:07:44.256790       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:07:44.256927       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:07:44.256967       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:07:44.257034       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:07:44.257066       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:07:44.257262       1 handler.go:232] Adding GroupVersion gadget.kinvolk.io v1alpha1 to ResourceManager

	I0918 19:07:44.288772       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:08:44.199786       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	W0918 19:09:04.182369       1 watcher.go:245] watch chan error: etcdserver: mvcc: required revision has been compacted

	I0918 19:09:44.199278       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:10:44.200176       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	* 

	* ==> kube-controller-manager [cb85fb8fd00c] <==

	* I0918 18:53:29.248178       1 shared_informer.go:311] Waiting for caches to sync for resource quota

	I0918 18:53:29.348997       1 shared_informer.go:318] Caches are synced for resource quota

	I0918 18:53:29.586694       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-create"

	I0918 18:53:29.602966       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-patch"

	I0918 18:53:29.662572       1 shared_informer.go:311] Waiting for caches to sync for garbage collector

	I0918 18:53:29.707847       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-create"

	I0918 18:53:29.763726       1 shared_informer.go:318] Caches are synced for garbage collector

	I0918 18:53:30.588673       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-create"

	I0918 18:53:30.591170       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-create"

	I0918 18:53:30.592513       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-create"

	I0918 18:53:30.592695       1 event.go:307] "Event occurred" object="gcp-auth/gcp-auth-certs-create" fieldPath="" kind="Job" apiVersion="batch/v1" type="Normal" reason="Completed" message="Job completed"

	I0918 18:53:30.619905       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-patch"

	I0918 18:53:30.641992       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-create"

	I0918 18:53:30.699064       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-patch"

	I0918 18:53:31.622753       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-patch"

	I0918 18:53:31.625237       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-patch"

	I0918 18:53:31.627760       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-patch"

	I0918 18:53:31.627944       1 event.go:307] "Event occurred" object="gcp-auth/gcp-auth-certs-patch" fieldPath="" kind="Job" apiVersion="batch/v1" type="Normal" reason="Completed" message="Job completed"

	I0918 18:53:31.655550       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-patch"

	I0918 18:53:42.736705       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="gcp-auth/gcp-auth-d4c87556c" duration="4.14133ms"

	I0918 18:53:42.736793       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="gcp-auth/gcp-auth-d4c87556c" duration="11.667µs"

	I0918 18:54:00.007554       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-create"

	I0918 18:54:00.025814       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-create"

	I0918 18:54:01.003894       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-patch"

	I0918 18:54:01.011510       1 job_controller.go:562] "enqueueing job" key="gcp-auth/gcp-auth-certs-patch"

	* 

	* ==> kube-proxy [3b8b236037bf] <==

	* I0918 18:53:00.480730       1 server_others.go:69] "Using iptables proxy"

	I0918 18:53:00.488175       1 node.go:141] Successfully retrieved node IP: 192.168.105.2

	I0918 18:53:00.508601       1 server_others.go:121] "No iptables support for family" ipFamily="IPv6"

	I0918 18:53:00.508621       1 server.go:634] "kube-proxy running in single-stack mode" ipFamily="IPv4"

	I0918 18:53:00.509405       1 server_others.go:152] "Using iptables Proxier"

	I0918 18:53:00.509431       1 proxier.go:251] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses"

	I0918 18:53:00.509518       1 server.go:846] "Version info" version="v1.28.2"

	I0918 18:53:00.509524       1 server.go:848] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0918 18:53:00.512081       1 config.go:188] "Starting service config controller"

	I0918 18:53:00.512089       1 shared_informer.go:311] Waiting for caches to sync for service config

	I0918 18:53:00.512105       1 config.go:97] "Starting endpoint slice config controller"

	I0918 18:53:00.512107       1 shared_informer.go:311] Waiting for caches to sync for endpoint slice config

	I0918 18:53:00.512284       1 config.go:315] "Starting node config controller"

	I0918 18:53:00.512287       1 shared_informer.go:311] Waiting for caches to sync for node config

	I0918 18:53:01.015326       1 shared_informer.go:318] Caches are synced for node config

	I0918 18:53:01.015354       1 shared_informer.go:318] Caches are synced for service config

	I0918 18:53:01.015384       1 shared_informer.go:318] Caches are synced for endpoint slice config

	* 

	* ==> kube-scheduler [17d16f9191cb] <==

	* W0918 18:52:43.862760       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	E0918 18:52:43.863344       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	W0918 18:52:43.862808       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	E0918 18:52:43.863392       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	W0918 18:52:43.862823       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	E0918 18:52:43.863442       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	W0918 18:52:43.862835       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	E0918 18:52:43.863451       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	W0918 18:52:43.862850       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope

	E0918 18:52:43.863466       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope

	W0918 18:52:43.862867       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	E0918 18:52:43.863530       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	W0918 18:52:43.862878       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	E0918 18:52:43.863565       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	W0918 18:52:43.862692       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	E0918 18:52:43.863575       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	W0918 18:52:43.863300       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	E0918 18:52:43.863661       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	W0918 18:52:44.751836       1 reflector.go:535] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	E0918 18:52:44.751859       1 reflector.go:147] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	W0918 18:52:44.752492       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope

	E0918 18:52:44.752505       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope

	W0918 18:52:44.760405       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	E0918 18:52:44.760417       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	I0918 18:52:46.760908       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	* 

	* ==> kubelet <==

	* -- Journal begins at Mon 2023-09-18 18:52:28 UTC, ends at Mon 2023-09-18 19:11:00 UTC. --

	Sep 18 19:05:46 addons-221000 kubelet[2297]: E0918 19:05:46.790235    2297 iptables.go:575] "Could not set up iptables canary" err=<

	Sep 18 19:05:46 addons-221000 kubelet[2297]:         error creating chain "KUBE-KUBELET-CANARY": exit status 3: ip6tables v1.8.6 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)

	Sep 18 19:05:46 addons-221000 kubelet[2297]:         Perhaps ip6tables or your kernel needs to be upgraded.

	Sep 18 19:05:46 addons-221000 kubelet[2297]:  > table="nat" chain="KUBE-KUBELET-CANARY"

	Sep 18 19:06:46 addons-221000 kubelet[2297]: E0918 19:06:46.789994    2297 iptables.go:575] "Could not set up iptables canary" err=<

	Sep 18 19:06:46 addons-221000 kubelet[2297]:         error creating chain "KUBE-KUBELET-CANARY": exit status 3: ip6tables v1.8.6 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)

	Sep 18 19:06:46 addons-221000 kubelet[2297]:         Perhaps ip6tables or your kernel needs to be upgraded.

	Sep 18 19:06:46 addons-221000 kubelet[2297]:  > table="nat" chain="KUBE-KUBELET-CANARY"

	Sep 18 19:07:46 addons-221000 kubelet[2297]: E0918 19:07:46.790185    2297 iptables.go:575] "Could not set up iptables canary" err=<

	Sep 18 19:07:46 addons-221000 kubelet[2297]:         error creating chain "KUBE-KUBELET-CANARY": exit status 3: ip6tables v1.8.6 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)

	Sep 18 19:07:46 addons-221000 kubelet[2297]:         Perhaps ip6tables or your kernel needs to be upgraded.

	Sep 18 19:07:46 addons-221000 kubelet[2297]:  > table="nat" chain="KUBE-KUBELET-CANARY"

	Sep 18 19:07:46 addons-221000 kubelet[2297]: W0918 19:07:46.810475    2297 machine.go:65] Cannot read vendor id correctly, set empty.

	Sep 18 19:08:46 addons-221000 kubelet[2297]: E0918 19:08:46.790690    2297 iptables.go:575] "Could not set up iptables canary" err=<

	Sep 18 19:08:46 addons-221000 kubelet[2297]:         error creating chain "KUBE-KUBELET-CANARY": exit status 3: ip6tables v1.8.6 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)

	Sep 18 19:08:46 addons-221000 kubelet[2297]:         Perhaps ip6tables or your kernel needs to be upgraded.

	Sep 18 19:08:46 addons-221000 kubelet[2297]:  > table="nat" chain="KUBE-KUBELET-CANARY"

	Sep 18 19:09:46 addons-221000 kubelet[2297]: E0918 19:09:46.790177    2297 iptables.go:575] "Could not set up iptables canary" err=<

	Sep 18 19:09:46 addons-221000 kubelet[2297]:         error creating chain "KUBE-KUBELET-CANARY": exit status 3: ip6tables v1.8.6 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)

	Sep 18 19:09:46 addons-221000 kubelet[2297]:         Perhaps ip6tables or your kernel needs to be upgraded.

	Sep 18 19:09:46 addons-221000 kubelet[2297]:  > table="nat" chain="KUBE-KUBELET-CANARY"

	Sep 18 19:10:46 addons-221000 kubelet[2297]: E0918 19:10:46.790244    2297 iptables.go:575] "Could not set up iptables canary" err=<

	Sep 18 19:10:46 addons-221000 kubelet[2297]:         error creating chain "KUBE-KUBELET-CANARY": exit status 3: ip6tables v1.8.6 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)

	Sep 18 19:10:46 addons-221000 kubelet[2297]:         Perhaps ip6tables or your kernel needs to be upgraded.

	Sep 18 19:10:46 addons-221000 kubelet[2297]:  > table="nat" chain="KUBE-KUBELET-CANARY"

	* 

	* ==> storage-provisioner [96b1e19d34d0] <==

	* I0918 18:53:02.494940       1 storage_provisioner.go:116] Initializing the minikube storage provisioner...

	I0918 18:53:02.503242       1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!

	I0918 18:53:02.503674       1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...

	I0918 18:53:02.507046       1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath

	I0918 18:53:02.507107       1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_addons-221000_d01c2a4d-820c-47b2-b527-e47d47f0ebf9!

	I0918 18:53:02.507893       1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"993ff549-fac0-4a25-b8bc-6e13c7f3eb70", APIVersion:"v1", ResourceVersion:"490", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' addons-221000_d01c2a4d-820c-47b2-b527-e47d47f0ebf9 became leader

	I0918 18:53:02.607443       1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_addons-221000_d01c2a4d-820c-47b2-b527-e47d47f0ebf9!

-- /stdout --

helpers_test.go:254: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.APIServer}} -p addons-221000 -n addons-221000

helpers_test.go:261: (dbg) Run:  kubectl --context addons-221000 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running

helpers_test.go:285: <<< TestAddons/parallel/Registry FAILED: end of post-mortem logs <<<

helpers_test.go:286: ---------------------/post-mortem---------------------------------

--- FAIL: TestAddons/parallel/Registry (720.82s)

=== RUN   TestAddons/parallel/Ingress

=== PAUSE TestAddons/parallel/Ingress

=== CONT  TestAddons/parallel/Ingress

addons_test.go:183: (dbg) Run:  kubectl --context addons-221000 wait --for=condition=ready --namespace=ingress-nginx pod --selector=app.kubernetes.io/component=controller --timeout=90s

addons_test.go:183: (dbg) Non-zero exit: kubectl --context addons-221000 wait --for=condition=ready --namespace=ingress-nginx pod --selector=app.kubernetes.io/component=controller --timeout=90s: exit status 1 (35.61225ms)

** stderr ** 

	error: no matching resources found

** /stderr **

addons_test.go:184: failed waiting for ingress-nginx-controller : exit status 1

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p addons-221000 -n addons-221000

helpers_test.go:244: <<< TestAddons/parallel/Ingress FAILED: start of post-mortem logs <<<

helpers_test.go:245: ======>  post-mortem[TestAddons/parallel/Ingress]: minikube logs <======

helpers_test.go:247: (dbg) Run:  out/minikube-darwin-arm64 -p addons-221000 logs -n 25

helpers_test.go:252: TestAddons/parallel/Ingress logs: 

-- stdout --

	* 

	* ==> Audit <==

	* |---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| Command |              Args              |       Profile        |  User   | Version |     Start Time      |      End Time       |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| start   | -o=json --download-only        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:51 PDT |                     |

	|         | -p download-only-242000        |                      |         |         |                     |                     |

	|         | --force --alsologtostderr      |                      |         |         |                     |                     |

	|         | --kubernetes-version=v1.16.0   |                      |         |         |                     |                     |

	|         | --container-runtime=docker     |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	| start   | -o=json --download-only        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT |                     |

	|         | -p download-only-242000        |                      |         |         |                     |                     |

	|         | --force --alsologtostderr      |                      |         |         |                     |                     |

	|         | --kubernetes-version=v1.28.2   |                      |         |         |                     |                     |

	|         | --container-runtime=docker     |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	| delete  | --all                          | minikube             | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| delete  | -p download-only-242000        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| delete  | -p download-only-242000        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| start   | --download-only -p             | binary-mirror-077000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT |                     |

	|         | binary-mirror-077000           |                      |         |         |                     |                     |

	|         | --alsologtostderr              |                      |         |         |                     |                     |

	|         | --binary-mirror                |                      |         |         |                     |                     |

	|         | http://127.0.0.1:49414         |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	| delete  | -p binary-mirror-077000        | binary-mirror-077000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| start   | -p addons-221000               | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:59 PDT |

	|         | --wait=true --memory=4000      |                      |         |         |                     |                     |

	|         | --alsologtostderr              |                      |         |         |                     |                     |

	|         | --addons=registry              |                      |         |         |                     |                     |

	|         | --addons=metrics-server        |                      |         |         |                     |                     |

	|         | --addons=volumesnapshots       |                      |         |         |                     |                     |

	|         | --addons=csi-hostpath-driver   |                      |         |         |                     |                     |

	|         | --addons=gcp-auth              |                      |         |         |                     |                     |

	|         | --addons=cloud-spanner         |                      |         |         |                     |                     |

	|         | --addons=inspektor-gadget      |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	|         | --addons=ingress               |                      |         |         |                     |                     |

	|         | --addons=ingress-dns           |                      |         |         |                     |                     |

	| addons  | disable cloud-spanner -p       | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:11 PDT |                     |

	|         | addons-221000                  |                      |         |         |                     |                     |

	| addons  | enable headlamp                | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:11 PDT | 18 Sep 23 12:11 PDT |

	|         | -p addons-221000               |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1         |                      |         |         |                     |                     |

	| addons  | addons-221000 addons           | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:11 PDT | 18 Sep 23 12:12 PDT |

	|         | disable csi-hostpath-driver    |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1         |                      |         |         |                     |                     |

	| addons  | addons-221000 addons           | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:12 PDT | 18 Sep 23 12:12 PDT |

	|         | disable volumesnapshots        |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1         |                      |         |         |                     |                     |

	| addons  | addons-221000 addons           | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:12 PDT | 18 Sep 23 12:12 PDT |

	|         | disable metrics-server         |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1         |                      |         |         |                     |                     |

	| addons  | disable inspektor-gadget -p    | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:12 PDT | 18 Sep 23 12:12 PDT |

	|         | addons-221000                  |                      |         |         |                     |                     |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	* 

	* ==> Last Start <==

	* Log file created at: 2023/09/18 11:52:16

	Running on machine: MacOS-M1-Agent-1

	Binary: Built with gc go1.21.1 for darwin/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0918 11:52:16.711602    1740 out.go:296] Setting OutFile to fd 1 ...

	I0918 11:52:16.711748    1740 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 11:52:16.711751    1740 out.go:309] Setting ErrFile to fd 2...

	I0918 11:52:16.711753    1740 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 11:52:16.711880    1740 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17263-1251/.minikube/bin

	I0918 11:52:16.712918    1740 out.go:303] Setting JSON to false

	I0918 11:52:16.728001    1740 start.go:128] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":1310,"bootTime":1695061826,"procs":408,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.5.2","kernelVersion":"22.6.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0918 11:52:16.728087    1740 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0918 11:52:16.732378    1740 out.go:177] * [addons-221000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	I0918 11:52:16.739454    1740 out.go:177]   - MINIKUBE_LOCATION=17263

	I0918 11:52:16.743421    1740 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 11:52:16.739507    1740 notify.go:220] Checking for updates...

	I0918 11:52:16.749403    1740 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0918 11:52:16.752377    1740 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0918 11:52:16.755381    1740 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	I0918 11:52:16.758417    1740 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0918 11:52:16.761446    1740 driver.go:373] Setting default libvirt URI to qemu:///system

	I0918 11:52:16.765371    1740 out.go:177] * Using the qemu2 driver based on user configuration

	I0918 11:52:16.777355    1740 start.go:298] selected driver: qemu2

	I0918 11:52:16.777364    1740 start.go:902] validating driver "qemu2" against <nil>

	I0918 11:52:16.777372    1740 start.go:913] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0918 11:52:16.779390    1740 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0918 11:52:16.782385    1740 out.go:177] * Automatically selected the socket_vmnet network

	I0918 11:52:16.785462    1740 start_flags.go:922] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0918 11:52:16.785488    1740 cni.go:84] Creating CNI manager for ""

	I0918 11:52:16.785496    1740 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 11:52:16.785507    1740 start_flags.go:316] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0918 11:52:16.785513    1740 start_flags.go:321] config:

	{Name:addons-221000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CR

ISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID

:0 AutoPauseInterval:1m0s}

	I0918 11:52:16.789634    1740 iso.go:125] acquiring lock: {Name:mk34e23c181861c65264ec384f06e6cfa001aa08 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0918 11:52:16.798394    1740 out.go:177] * Starting control plane node addons-221000 in cluster addons-221000

	I0918 11:52:16.802195    1740 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 11:52:16.802217    1740 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4

	I0918 11:52:16.802234    1740 cache.go:57] Caching tarball of preloaded images

	I0918 11:52:16.802301    1740 preload.go:174] Found /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0918 11:52:16.802315    1740 cache.go:60] Finished verifying existence of preloaded tar for  v1.28.2 on docker

	I0918 11:52:16.802542    1740 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/config.json ...

	I0918 11:52:16.802555    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/config.json: {Name:mk6624c585fbc7911138df2cd59d1f2e10251cf1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:16.802799    1740 start.go:365] acquiring machines lock for addons-221000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 11:52:16.802873    1740 start.go:369] acquired machines lock for "addons-221000" in 68.417µs

	I0918 11:52:16.802886    1740 start.go:93] Provisioning new machine with config: &{Name:addons-221000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi

g:{KubernetesVersion:v1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262

144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 11:52:16.802925    1740 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 11:52:16.810242    1740 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...

	I0918 11:52:17.161676    1740 start.go:159] libmachine.API.Create for "addons-221000" (driver="qemu2")

	I0918 11:52:17.161722    1740 client.go:168] LocalClient.Create starting

	I0918 11:52:17.161932    1740 main.go:141] libmachine: Creating CA: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 11:52:17.253776    1740 main.go:141] libmachine: Creating client certificate: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 11:52:17.312301    1740 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 11:52:17.776256    1740 main.go:141] libmachine: Creating SSH key...

	I0918 11:52:17.897328    1740 main.go:141] libmachine: Creating Disk image...

	I0918 11:52:17.897334    1740 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 11:52:17.897524    1740 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2

	I0918 11:52:17.933044    1740 main.go:141] libmachine: STDOUT: 

	I0918 11:52:17.933072    1740 main.go:141] libmachine: STDERR: 

	I0918 11:52:17.933136    1740 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2 +20000M

	I0918 11:52:17.940597    1740 main.go:141] libmachine: STDOUT: Image resized.

	I0918 11:52:17.940609    1740 main.go:141] libmachine: STDERR: 

	I0918 11:52:17.940623    1740 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2

	I0918 11:52:17.940628    1740 main.go:141] libmachine: Starting QEMU VM...

	I0918 11:52:17.940657    1740 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 4000 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/qemu.pid -device virtio-net-pci,netdev=net0,mac=ce:ae:e8:0a:fd:16 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2

	I0918 11:52:18.008779    1740 main.go:141] libmachine: STDOUT: 

	I0918 11:52:18.008804    1740 main.go:141] libmachine: STDERR: 

	I0918 11:52:18.008808    1740 main.go:141] libmachine: Attempt 0

	I0918 11:52:18.008820    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:20.011046    1740 main.go:141] libmachine: Attempt 1

	I0918 11:52:20.011124    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:22.013479    1740 main.go:141] libmachine: Attempt 2

	I0918 11:52:22.013559    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:24.015632    1740 main.go:141] libmachine: Attempt 3

	I0918 11:52:24.015645    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:26.017675    1740 main.go:141] libmachine: Attempt 4

	I0918 11:52:26.017681    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:28.018761    1740 main.go:141] libmachine: Attempt 5

	I0918 11:52:28.018782    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:30.020886    1740 main.go:141] libmachine: Attempt 6

	I0918 11:52:30.020920    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:30.021070    1740 main.go:141] libmachine: Found 1 entries in /var/db/dhcpd_leases!

	I0918 11:52:30.021123    1740 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:ce:ae:e8:a:fd:16 ID:1,ce:ae:e8:a:fd:16 Lease:0x6509edec}

	I0918 11:52:30.021130    1740 main.go:141] libmachine: Found match: ce:ae:e8:a:fd:16

	I0918 11:52:30.021140    1740 main.go:141] libmachine: IP: 192.168.105.2

	I0918 11:52:30.021147    1740 main.go:141] libmachine: Waiting for VM to start (ssh -p 22 docker@192.168.105.2)...

	I0918 11:52:31.026067    1740 machine.go:88] provisioning docker machine ...

	I0918 11:52:31.026085    1740 buildroot.go:166] provisioning hostname "addons-221000"

	I0918 11:52:31.026964    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.027231    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.027237    1740 main.go:141] libmachine: About to run SSH command:

	sudo hostname addons-221000 && echo "addons-221000" | sudo tee /etc/hostname

	I0918 11:52:31.084404    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: addons-221000

	I0918 11:52:31.084473    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.084732    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.084740    1740 main.go:141] libmachine: About to run SSH command:

			if ! grep -xq '.*\saddons-221000' /etc/hosts; then

				if grep -xq '127.0.1.1\s.*' /etc/hosts; then

					sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 addons-221000/g' /etc/hosts;

				else 

					echo '127.0.1.1 addons-221000' | sudo tee -a /etc/hosts; 

				fi

			fi

	I0918 11:52:31.144009    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	I0918 11:52:31.144022    1740 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/17263-1251/.minikube CaCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/17263-1251/.minikube}

	I0918 11:52:31.144033    1740 buildroot.go:174] setting up certificates

	I0918 11:52:31.144038    1740 provision.go:83] configureAuth start

	I0918 11:52:31.144042    1740 provision.go:138] copyHostCerts

	I0918 11:52:31.144138    1740 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.pem (1082 bytes)

	I0918 11:52:31.144342    1740 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/cert.pem (1123 bytes)

	I0918 11:52:31.144435    1740 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/key.pem (1679 bytes)

	I0918 11:52:31.144503    1740 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem org=jenkins.addons-221000 san=[192.168.105.2 192.168.105.2 localhost 127.0.0.1 minikube addons-221000]

	I0918 11:52:31.225327    1740 provision.go:172] copyRemoteCerts

	I0918 11:52:31.225385    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker

	I0918 11:52:31.225394    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.256352    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)

	I0918 11:52:31.263330    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem --> /etc/docker/server.pem (1216 bytes)

	I0918 11:52:31.270197    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)

	I0918 11:52:31.276715    1740 provision.go:86] duration metric: configureAuth took 132.670667ms

	I0918 11:52:31.276723    1740 buildroot.go:189] setting minikube options for container-runtime

	I0918 11:52:31.276820    1740 config.go:182] Loaded profile config "addons-221000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 11:52:31.276857    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.277075    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.277080    1740 main.go:141] libmachine: About to run SSH command:

	df --output=fstype / | tail -n 1

	I0918 11:52:31.337901    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs

	I0918 11:52:31.337910    1740 buildroot.go:70] root file system type: tmpfs

	I0918 11:52:31.337970    1740 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...

	I0918 11:52:31.338012    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.338275    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.338315    1740 main.go:141] libmachine: About to run SSH command:

	sudo mkdir -p /lib/systemd/system && printf %!s(MISSING) "[Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP \$MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	" | sudo tee /lib/systemd/system/docker.service.new

	I0918 11:52:31.400816    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP $MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	I0918 11:52:31.400863    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.401116    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.401126    1740 main.go:141] libmachine: About to run SSH command:

	sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }

	I0918 11:52:31.746670    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory

	Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

	I0918 11:52:31.746684    1740 machine.go:91] provisioned docker machine in 720.613ms

	I0918 11:52:31.746690    1740 client.go:171] LocalClient.Create took 14.585099291s

	I0918 11:52:31.746703    1740 start.go:167] duration metric: libmachine.API.Create for "addons-221000" took 14.585173417s

	I0918 11:52:31.746707    1740 start.go:300] post-start starting for "addons-221000" (driver="qemu2")

	I0918 11:52:31.746711    1740 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]

	I0918 11:52:31.746780    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs

	I0918 11:52:31.746790    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.775601    1740 ssh_runner.go:195] Run: cat /etc/os-release

	I0918 11:52:31.776975    1740 info.go:137] Remote host: Buildroot 2021.02.12

	I0918 11:52:31.776983    1740 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17263-1251/.minikube/addons for local assets ...

	I0918 11:52:31.777055    1740 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17263-1251/.minikube/files for local assets ...

	I0918 11:52:31.777083    1740 start.go:303] post-start completed in 30.374292ms

	I0918 11:52:31.777437    1740 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/config.json ...

	I0918 11:52:31.777603    1740 start.go:128] duration metric: createHost completed in 14.974815417s

	I0918 11:52:31.777667    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.777884    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.777888    1740 main.go:141] libmachine: About to run SSH command:

	date +%!s(MISSING).%!N(MISSING)

	I0918 11:52:31.833629    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: 1695063151.431021085

	I0918 11:52:31.833635    1740 fix.go:206] guest clock: 1695063151.431021085

	I0918 11:52:31.833638    1740 fix.go:219] Guest: 2023-09-18 11:52:31.431021085 -0700 PDT Remote: 2023-09-18 11:52:31.777608 -0700 PDT m=+15.083726834 (delta=-346.586915ms)

	I0918 11:52:31.833654    1740 fix.go:190] guest clock delta is within tolerance: -346.586915ms

	I0918 11:52:31.833656    1740 start.go:83] releasing machines lock for "addons-221000", held for 15.0309195s

	I0918 11:52:31.833905    1740 ssh_runner.go:195] Run: cat /version.json

	I0918 11:52:31.833915    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.833930    1740 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/

	I0918 11:52:31.833973    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.902596    1740 ssh_runner.go:195] Run: systemctl --version

	I0918 11:52:31.904777    1740 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"

	W0918 11:52:31.906638    1740 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found

	I0918 11:52:31.906668    1740 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;

	I0918 11:52:31.911697    1740 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)

	I0918 11:52:31.911704    1740 start.go:469] detecting cgroup driver to use...

	I0918 11:52:31.911799    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock

	" | sudo tee /etc/crictl.yaml"

	I0918 11:52:31.917320    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"

	I0918 11:52:31.920500    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"

	I0918 11:52:31.923811    1740 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...

	I0918 11:52:31.923843    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"

	I0918 11:52:31.926950    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0918 11:52:31.929666    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"

	I0918 11:52:31.932680    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0918 11:52:31.936002    1740 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"

	I0918 11:52:31.939362    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"

	I0918 11:52:31.942186    1740 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables

	I0918 11:52:31.944734    1740 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

	I0918 11:52:31.947664    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:32.027464    1740 ssh_runner.go:195] Run: sudo systemctl restart containerd

	I0918 11:52:32.036559    1740 start.go:469] detecting cgroup driver to use...

	I0918 11:52:32.036614    1740 ssh_runner.go:195] Run: sudo systemctl cat docker.service

	I0918 11:52:32.042440    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0918 11:52:32.047583    1740 ssh_runner.go:195] Run: sudo systemctl stop -f containerd

	I0918 11:52:32.053840    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0918 11:52:32.058225    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I0918 11:52:32.062305    1740 ssh_runner.go:195] Run: sudo systemctl stop -f crio

	I0918 11:52:32.098720    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I0918 11:52:32.103440    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock

	" | sudo tee /etc/crictl.yaml"

	I0918 11:52:32.108844    1740 ssh_runner.go:195] Run: which cri-dockerd

	I0918 11:52:32.110173    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d

	I0918 11:52:32.112731    1740 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)

	I0918 11:52:32.117532    1740 ssh_runner.go:195] Run: sudo systemctl unmask docker.service

	I0918 11:52:32.194769    1740 ssh_runner.go:195] Run: sudo systemctl enable docker.socket

	I0918 11:52:32.269401    1740 docker.go:535] configuring docker to use "cgroupfs" as cgroup driver...

	I0918 11:52:32.269417    1740 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)

	I0918 11:52:32.274373    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:32.355030    1740 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0918 11:52:33.517984    1740 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.1629475s)

	I0918 11:52:33.518044    1740 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I0918 11:52:33.595160    1740 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket

	I0918 11:52:33.670332    1740 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I0918 11:52:33.746578    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:33.822625    1740 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket

	I0918 11:52:33.829925    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:33.909957    1740 ssh_runner.go:195] Run: sudo systemctl restart cri-docker

	I0918 11:52:33.933398    1740 start.go:516] Will wait 60s for socket path /var/run/cri-dockerd.sock

	I0918 11:52:33.933487    1740 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock

	I0918 11:52:33.935757    1740 start.go:537] Will wait 60s for crictl version

	I0918 11:52:33.935801    1740 ssh_runner.go:195] Run: which crictl

	I0918 11:52:33.937147    1740 ssh_runner.go:195] Run: sudo /usr/bin/crictl version

	I0918 11:52:33.952602    1740 start.go:553] Version:  0.1.0

	RuntimeName:  docker

	RuntimeVersion:  24.0.6

	RuntimeApiVersion:  v1alpha2

	I0918 11:52:33.952673    1740 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0918 11:52:33.962082    1740 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0918 11:52:33.975334    1740 out.go:204] * Preparing Kubernetes v1.28.2 on Docker 24.0.6 ...

	I0918 11:52:33.975416    1740 ssh_runner.go:195] Run: grep 192.168.105.1	host.minikube.internal$ /etc/hosts

	I0918 11:52:33.976970    1740 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.105.1	host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0918 11:52:33.980853    1740 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 11:52:33.980897    1740 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0918 11:52:33.986139    1740 docker.go:636] Got preloaded images: 

	I0918 11:52:33.986147    1740 docker.go:642] registry.k8s.io/kube-apiserver:v1.28.2 wasn't preloaded

	I0918 11:52:33.986189    1740 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0918 11:52:33.988954    1740 ssh_runner.go:195] Run: which lz4

	I0918 11:52:33.990479    1740 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4

	I0918 11:52:33.991766    1740 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4: Process exited with status 1

	stdout:

	stderr:

	stat: cannot statx '/preloaded.tar.lz4': No such file or directory

	I0918 11:52:33.991780    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 --> /preloaded.tar.lz4 (356993689 bytes)

	I0918 11:52:35.310513    1740 docker.go:600] Took 1.320057 seconds to copy over tarball

	I0918 11:52:35.310582    1740 ssh_runner.go:195] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4

	I0918 11:52:36.348518    1740 ssh_runner.go:235] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (1.03793175s)

	I0918 11:52:36.348535    1740 ssh_runner.go:146] rm: /preloaded.tar.lz4

	I0918 11:52:36.364745    1740 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0918 11:52:36.368295    1740 ssh_runner.go:362] scp memory --> /var/lib/docker/image/overlay2/repositories.json (2629 bytes)

	I0918 11:52:36.373429    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:36.450305    1740 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0918 11:52:38.940530    1740 ssh_runner.go:235] Completed: sudo systemctl restart docker: (2.490231167s)

	I0918 11:52:38.940627    1740 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0918 11:52:38.946699    1740 docker.go:636] Got preloaded images: -- stdout --

	registry.k8s.io/kube-apiserver:v1.28.2

	registry.k8s.io/kube-scheduler:v1.28.2

	registry.k8s.io/kube-proxy:v1.28.2

	registry.k8s.io/kube-controller-manager:v1.28.2

	registry.k8s.io/etcd:3.5.9-0

	registry.k8s.io/coredns/coredns:v1.10.1

	registry.k8s.io/pause:3.9

	gcr.io/k8s-minikube/storage-provisioner:v5

	-- /stdout --

	I0918 11:52:38.946709    1740 cache_images.go:84] Images are preloaded, skipping loading

	I0918 11:52:38.946766    1740 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}

	I0918 11:52:38.954428    1740 cni.go:84] Creating CNI manager for ""

	I0918 11:52:38.954439    1740 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 11:52:38.954458    1740 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16

	I0918 11:52:38.954467    1740 kubeadm.go:176] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.105.2 APIServerPort:8443 KubernetesVersion:v1.28.2 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:addons-221000 NodeName:addons-221000 DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.105.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.105.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/ku

bernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}

	I0918 11:52:38.954540    1740 kubeadm.go:181] kubeadm config:

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: InitConfiguration

	localAPIEndpoint:

	  advertiseAddress: 192.168.105.2

	  bindPort: 8443

	bootstrapTokens:

	  - groups:

	      - system:bootstrappers:kubeadm:default-node-token

	    ttl: 24h0m0s

	    usages:

	      - signing

	      - authentication

	nodeRegistration:

	  criSocket: unix:///var/run/cri-dockerd.sock

	  name: "addons-221000"

	  kubeletExtraArgs:

	    node-ip: 192.168.105.2

	  taints: []

	---

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: ClusterConfiguration

	apiServer:

	  certSANs: ["127.0.0.1", "localhost", "192.168.105.2"]

	  extraArgs:

	    enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"

	controllerManager:

	  extraArgs:

	    allocate-node-cidrs: "true"

	    leader-elect: "false"

	scheduler:

	  extraArgs:

	    leader-elect: "false"

	certificatesDir: /var/lib/minikube/certs

	clusterName: mk

	controlPlaneEndpoint: control-plane.minikube.internal:8443

	etcd:

	  local:

	    dataDir: /var/lib/minikube/etcd

	    extraArgs:

	      proxy-refresh-interval: "70000"

	kubernetesVersion: v1.28.2

	networking:

	  dnsDomain: cluster.local

	  podSubnet: "10.244.0.0/16"

	  serviceSubnet: 10.96.0.0/12

	---

	apiVersion: kubelet.config.k8s.io/v1beta1

	kind: KubeletConfiguration

	authentication:

	  x509:

	    clientCAFile: /var/lib/minikube/certs/ca.crt

	cgroupDriver: cgroupfs

	hairpinMode: hairpin-veth

	runtimeRequestTimeout: 15m

	clusterDomain: "cluster.local"

	# disable disk resource management by default

	imageGCHighThresholdPercent: 100

	evictionHard:

	  nodefs.available: "0%!"(MISSING)

	  nodefs.inodesFree: "0%!"(MISSING)

	  imagefs.available: "0%!"(MISSING)

	failSwapOn: false

	staticPodPath: /etc/kubernetes/manifests

	---

	apiVersion: kubeproxy.config.k8s.io/v1alpha1

	kind: KubeProxyConfiguration

	clusterCIDR: "10.244.0.0/16"

	metricsBindAddress: 0.0.0.0:10249

	conntrack:

	  maxPerCore: 0

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"

	  tcpEstablishedTimeout: 0s

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"

	  tcpCloseWaitTimeout: 0s

	I0918 11:52:38.954592    1740 kubeadm.go:976] kubelet [Unit]

	Wants=docker.socket

	[Service]

	ExecStart=

	ExecStart=/var/lib/minikube/binaries/v1.28.2/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --hostname-override=addons-221000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.105.2

	[Install]

	 config:

	{KubernetesVersion:v1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}

	I0918 11:52:38.954661    1740 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.2

	I0918 11:52:38.957531    1740 binaries.go:44] Found k8s binaries, skipping transfer

	I0918 11:52:38.957562    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube

	I0918 11:52:38.960385    1740 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (375 bytes)

	I0918 11:52:38.965592    1740 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)

	I0918 11:52:38.970301    1740 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2099 bytes)

	I0918 11:52:38.975165    1740 ssh_runner.go:195] Run: grep 192.168.105.2	control-plane.minikube.internal$ /etc/hosts

	I0918 11:52:38.976400    1740 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.105.2	control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0918 11:52:38.980255    1740 certs.go:56] Setting up /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000 for IP: 192.168.105.2

	I0918 11:52:38.980276    1740 certs.go:190] acquiring lock for shared ca certs: {Name:mkfac81ee65979b8c4f5ece6243c3a0190531689 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:38.980470    1740 certs.go:204] generating minikubeCA CA: /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key

	I0918 11:52:39.170828    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt ...

	I0918 11:52:39.170844    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt: {Name:mk0f303ee67627c25d1d04e1887861f15cdad763 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.171150    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key ...

	I0918 11:52:39.171155    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key: {Name:mkc5e20e8161cfdcfc3d5dcd8300765ea2c12112 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.171271    1740 certs.go:204] generating proxyClientCA CA: /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key

	I0918 11:52:39.287022    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.crt ...

	I0918 11:52:39.287027    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.crt: {Name:mk54c49c3c44ff09930e6c0f57238b89cff4c5e9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.287171    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key ...

	I0918 11:52:39.287173    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key: {Name:mk05faae5769358f82565f32c1f37a244f2478c1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.287315    1740 certs.go:319] generating minikube-user signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.key

	I0918 11:52:39.287337    1740 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.crt with IP's: []

	I0918 11:52:39.376234    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.crt ...

	I0918 11:52:39.376241    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.crt: {Name:mkc8e654c6f2522197f557cb47d266f15eebaadd Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.376467    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.key ...

	I0918 11:52:39.376471    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.key: {Name:mkf345dd56f86115b31ecd965617f4c21d6a0cd8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.376571    1740 certs.go:319] generating minikube signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969

	I0918 11:52:39.376580    1740 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969 with IP's: [192.168.105.2 10.96.0.1 127.0.0.1 10.0.0.1]

	I0918 11:52:39.429944    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969 ...

	I0918 11:52:39.429952    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969: {Name:mkd69eb587bd0dc6ccdbaa88b78f4f92f2b47b12 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.430095    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969 ...

	I0918 11:52:39.430098    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969: {Name:mk76ea3a7fbbef2305f74e52afdf06cda921c8bb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.430199    1740 certs.go:337] copying /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969 -> /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt

	I0918 11:52:39.430382    1740 certs.go:341] copying /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969 -> /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key

	I0918 11:52:39.430499    1740 certs.go:319] generating aggregator signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key

	I0918 11:52:39.430509    1740 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt with IP's: []

	I0918 11:52:39.698555    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt ...

	I0918 11:52:39.698563    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt: {Name:mk6d7a924ed10f0012b290ec4e0ea6bf1b7bfc02 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.698767    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key ...

	I0918 11:52:39.698773    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key: {Name:mk8d78e9179e4c57e4602e98d4fc6a37885b4d01 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.699037    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem (1679 bytes)

	I0918 11:52:39.699062    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem (1082 bytes)

	I0918 11:52:39.699081    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem (1123 bytes)

	I0918 11:52:39.699100    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem (1679 bytes)

	I0918 11:52:39.699418    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)

	I0918 11:52:39.707305    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)

	I0918 11:52:39.713956    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)

	I0918 11:52:39.720614    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)

	I0918 11:52:39.727643    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)

	I0918 11:52:39.734609    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)

	I0918 11:52:39.741305    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)

	I0918 11:52:39.748141    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)

	I0918 11:52:39.755277    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)

	I0918 11:52:39.762025    1740 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)

	I0918 11:52:39.767701    1740 ssh_runner.go:195] Run: openssl version

	I0918 11:52:39.769871    1740 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"

	I0918 11:52:39.772909    1740 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem

	I0918 11:52:39.774536    1740 certs.go:480] hashing: -rw-r--r-- 1 root root 1111 Sep 18 18:52 /usr/share/ca-certificates/minikubeCA.pem

	I0918 11:52:39.774555    1740 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem

	I0918 11:52:39.776417    1740 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"

	I0918 11:52:39.779698    1740 ssh_runner.go:195] Run: ls /var/lib/minikube/certs/etcd

	I0918 11:52:39.781162    1740 certs.go:353] certs directory doesn't exist, likely first start: ls /var/lib/minikube/certs/etcd: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/var/lib/minikube/certs/etcd': No such file or directory

	I0918 11:52:39.781200    1740 kubeadm.go:404] StartCluster: {Name:addons-221000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v

1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.105.2 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 Mount

Options:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 11:52:39.781263    1740 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}

	I0918 11:52:39.787256    1740 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd

	I0918 11:52:39.790167    1740 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml

	I0918 11:52:39.792879    1740 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf

	I0918 11:52:39.795890    1740 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory

	I0918 11:52:39.795906    1740 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"

	I0918 11:52:39.820130    1740 kubeadm.go:322] [init] Using Kubernetes version: v1.28.2

	I0918 11:52:39.820157    1740 kubeadm.go:322] [preflight] Running pre-flight checks

	I0918 11:52:39.874262    1740 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster

	I0918 11:52:39.874320    1740 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection

	I0918 11:52:39.874401    1740 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	I0918 11:52:39.936649    1740 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"

	I0918 11:52:39.946863    1740 out.go:204]   - Generating certificates and keys ...

	I0918 11:52:39.946906    1740 kubeadm.go:322] [certs] Using existing ca certificate authority

	I0918 11:52:39.946940    1740 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk

	I0918 11:52:40.057135    1740 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key

	I0918 11:52:40.267412    1740 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key

	I0918 11:52:40.415260    1740 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key

	I0918 11:52:40.592293    1740 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key

	I0918 11:52:40.714190    1740 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key

	I0918 11:52:40.714252    1740 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [addons-221000 localhost] and IPs [192.168.105.2 127.0.0.1 ::1]

	I0918 11:52:40.818329    1740 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key

	I0918 11:52:40.818397    1740 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [addons-221000 localhost] and IPs [192.168.105.2 127.0.0.1 ::1]

	I0918 11:52:41.068370    1740 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key

	I0918 11:52:41.110794    1740 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key

	I0918 11:52:41.218301    1740 kubeadm.go:322] [certs] Generating "sa" key and public key

	I0918 11:52:41.218335    1740 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	I0918 11:52:41.282421    1740 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file

	I0918 11:52:41.650315    1740 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file

	I0918 11:52:41.733907    1740 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file

	I0918 11:52:41.925252    1740 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file

	I0918 11:52:41.925561    1740 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	I0918 11:52:41.927413    1740 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"

	I0918 11:52:41.931680    1740 out.go:204]   - Booting up control plane ...

	I0918 11:52:41.931754    1740 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"

	I0918 11:52:41.931794    1740 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"

	I0918 11:52:41.931831    1740 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"

	I0918 11:52:41.935171    1740 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	I0918 11:52:41.935565    1740 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	I0918 11:52:41.935586    1740 kubeadm.go:322] [kubelet-start] Starting the kubelet

	I0918 11:52:42.024365    1740 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	I0918 11:52:45.527756    1740 kubeadm.go:322] [apiclient] All control plane components are healthy after 3.503453 seconds

	I0918 11:52:45.527852    1740 kubeadm.go:322] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace

	I0918 11:52:45.533290    1740 kubeadm.go:322] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster

	I0918 11:52:46.043984    1740 kubeadm.go:322] [upload-certs] Skipping phase. Please see --upload-certs

	I0918 11:52:46.044088    1740 kubeadm.go:322] [mark-control-plane] Marking the node addons-221000 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]

	I0918 11:52:46.548611    1740 kubeadm.go:322] [bootstrap-token] Using token: 0otx18.vbdfa1zgl84pbc1n

	I0918 11:52:46.552403    1740 out.go:204]   - Configuring RBAC rules ...

	I0918 11:52:46.552463    1740 kubeadm.go:322] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles

	I0918 11:52:46.553348    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes

	I0918 11:52:46.557357    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials

	I0918 11:52:46.558552    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token

	I0918 11:52:46.559879    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster

	I0918 11:52:46.560890    1740 kubeadm.go:322] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace

	I0918 11:52:46.567944    1740 kubeadm.go:322] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key

	I0918 11:52:46.739677    1740 kubeadm.go:322] [addons] Applied essential addon: CoreDNS

	I0918 11:52:46.956246    1740 kubeadm.go:322] [addons] Applied essential addon: kube-proxy

	I0918 11:52:46.958050    1740 kubeadm.go:322] 

	I0918 11:52:46.958085    1740 kubeadm.go:322] Your Kubernetes control-plane has initialized successfully!

	I0918 11:52:46.958096    1740 kubeadm.go:322] 

	I0918 11:52:46.958138    1740 kubeadm.go:322] To start using your cluster, you need to run the following as a regular user:

	I0918 11:52:46.958143    1740 kubeadm.go:322] 

	I0918 11:52:46.958157    1740 kubeadm.go:322]   mkdir -p $HOME/.kube

	I0918 11:52:46.958186    1740 kubeadm.go:322]   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

	I0918 11:52:46.958221    1740 kubeadm.go:322]   sudo chown $(id -u):$(id -g) $HOME/.kube/config

	I0918 11:52:46.958226    1740 kubeadm.go:322] 

	I0918 11:52:46.958261    1740 kubeadm.go:322] Alternatively, if you are the root user, you can run:

	I0918 11:52:46.958267    1740 kubeadm.go:322] 

	I0918 11:52:46.958291    1740 kubeadm.go:322]   export KUBECONFIG=/etc/kubernetes/admin.conf

	I0918 11:52:46.958294    1740 kubeadm.go:322] 

	I0918 11:52:46.958317    1740 kubeadm.go:322] You should now deploy a pod network to the cluster.

	I0918 11:52:46.958375    1740 kubeadm.go:322] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

	I0918 11:52:46.958411    1740 kubeadm.go:322]   https://kubernetes.io/docs/concepts/cluster-administration/addons/

	I0918 11:52:46.958416    1740 kubeadm.go:322] 

	I0918 11:52:46.958458    1740 kubeadm.go:322] You can now join any number of control-plane nodes by copying certificate authorities

	I0918 11:52:46.958503    1740 kubeadm.go:322] and service account keys on each node and then running the following as root:

	I0918 11:52:46.958507    1740 kubeadm.go:322] 

	I0918 11:52:46.958562    1740 kubeadm.go:322]   kubeadm join control-plane.minikube.internal:8443 --token 0otx18.vbdfa1zgl84pbc1n \

	I0918 11:52:46.958623    1740 kubeadm.go:322] 	--discovery-token-ca-cert-hash sha256:ba7ba5242719fff40b98ade8a053fc0c6dded3185e28e2742ca7444c7c25a7a5 \

	I0918 11:52:46.958634    1740 kubeadm.go:322] 	--control-plane 

	I0918 11:52:46.958636    1740 kubeadm.go:322] 

	I0918 11:52:46.958676    1740 kubeadm.go:322] Then you can join any number of worker nodes by running the following on each as root:

	I0918 11:52:46.958681    1740 kubeadm.go:322] 

	I0918 11:52:46.958735    1740 kubeadm.go:322] kubeadm join control-plane.minikube.internal:8443 --token 0otx18.vbdfa1zgl84pbc1n \

	I0918 11:52:46.958805    1740 kubeadm.go:322] 	--discovery-token-ca-cert-hash sha256:ba7ba5242719fff40b98ade8a053fc0c6dded3185e28e2742ca7444c7c25a7a5 

	I0918 11:52:46.958862    1740 kubeadm.go:322] 	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	I0918 11:52:46.958868    1740 cni.go:84] Creating CNI manager for ""

	I0918 11:52:46.958880    1740 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 11:52:46.967096    1740 out.go:177] * Configuring bridge CNI (Container Networking Interface) ...

	I0918 11:52:46.970213    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/cni/net.d

	I0918 11:52:46.973479    1740 ssh_runner.go:362] scp memory --> /etc/cni/net.d/1-k8s.conflist (457 bytes)

	I0918 11:52:46.977991    1740 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"

	I0918 11:52:46.978031    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:46.978044    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl label nodes minikube.k8s.io/version=v1.31.2 minikube.k8s.io/commit=ae2adeb20cc3a4b5f1decc7c8f53736ec04c4a36 minikube.k8s.io/name=addons-221000 minikube.k8s.io/updated_at=2023_09_18T11_52_46_0700 minikube.k8s.io/primary=true --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:47.038282    1740 ops.go:34] apiserver oom_adj: -16

	I0918 11:52:47.038334    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:47.073908    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:47.620721    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:48.118781    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:48.620650    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:49.120663    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:49.620741    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:50.120032    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:50.620638    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:51.118777    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:51.618788    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:52.120234    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:52.619043    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:53.120606    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:53.619266    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:54.118979    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:54.618916    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:55.120620    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:55.620599    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:56.120585    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:56.618629    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:57.120587    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:57.618783    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:58.120645    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:58.620559    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:59.120556    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:59.619252    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:53:00.118796    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:53:00.169662    1740 kubeadm.go:1081] duration metric: took 13.191788791s to wait for elevateKubeSystemPrivileges.

	I0918 11:53:00.169677    1740 kubeadm.go:406] StartCluster complete in 20.38867075s

	I0918 11:53:00.169687    1740 settings.go:142] acquiring lock: {Name:mke420f28dda4f7a752738b3e6d571dc4216779e Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:53:00.169849    1740 settings.go:150] Updating kubeconfig:  /Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 11:53:00.170110    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/kubeconfig: {Name:mk07020c5b974cf07ca0cda25f72a521eb245fa1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:53:00.170308    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"

	I0918 11:53:00.170433    1740 config.go:182] Loaded profile config "addons-221000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 11:53:00.170379    1740 addons.go:499] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:true csi-hostpath-driver:true dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:true gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:true ingress-dns:true inspektor-gadget:true istio:false istio-provisioner:false kong:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:true registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false volumesnapshots:true]

	I0918 11:53:00.170512    1740 addons.go:69] Setting ingress=true in profile "addons-221000"

	I0918 11:53:00.170517    1740 addons.go:69] Setting ingress-dns=true in profile "addons-221000"

	I0918 11:53:00.170520    1740 addons.go:231] Setting addon ingress=true in "addons-221000"

	I0918 11:53:00.170523    1740 addons.go:231] Setting addon ingress-dns=true in "addons-221000"

	I0918 11:53:00.170533    1740 addons.go:69] Setting metrics-server=true in profile "addons-221000"

	I0918 11:53:00.170538    1740 addons.go:231] Setting addon metrics-server=true in "addons-221000"

	I0918 11:53:00.170552    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170556    1740 addons.go:69] Setting inspektor-gadget=true in profile "addons-221000"

	I0918 11:53:00.170560    1740 addons.go:69] Setting csi-hostpath-driver=true in profile "addons-221000"

	I0918 11:53:00.170564    1740 addons.go:69] Setting gcp-auth=true in profile "addons-221000"

	I0918 11:53:00.170569    1740 mustload.go:65] Loading cluster: addons-221000

	I0918 11:53:00.170573    1740 addons.go:69] Setting registry=true in profile "addons-221000"

	I0918 11:53:00.170577    1740 addons.go:231] Setting addon registry=true in "addons-221000"

	I0918 11:53:00.170587    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170595    1740 addons.go:69] Setting default-storageclass=true in profile "addons-221000"

	I0918 11:53:00.170618    1740 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "addons-221000"

	I0918 11:53:00.170629    1740 addons.go:69] Setting storage-provisioner=true in profile "addons-221000"

	I0918 11:53:00.170639    1740 config.go:182] Loaded profile config "addons-221000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 11:53:00.170657    1740 addons.go:231] Setting addon storage-provisioner=true in "addons-221000"

	I0918 11:53:00.170705    1740 host.go:66] Checking if "addons-221000" exists ...

	W0918 11:53:00.170820    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	W0918 11:53:00.170825    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	W0918 11:53:00.170829    1740 addons.go:277] "addons-221000" is not running, setting ingress-dns=true and skipping enablement (err=<nil>)

	W0918 11:53:00.170831    1740 addons.go:277] "addons-221000" is not running, setting registry=true and skipping enablement (err=<nil>)

	I0918 11:53:00.170552    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170834    1740 addons.go:467] Verifying addon registry=true in "addons-221000"

	I0918 11:53:00.170556    1740 addons.go:69] Setting cloud-spanner=true in profile "addons-221000"

	I0918 11:53:00.175435    1740 out.go:177] * Verifying registry addon...

	I0918 11:53:00.170560    1740 addons.go:231] Setting addon inspektor-gadget=true in "addons-221000"

	I0918 11:53:00.170569    1740 addons.go:231] Setting addon csi-hostpath-driver=true in "addons-221000"

	I0918 11:53:00.170865    1740 addons.go:231] Setting addon cloud-spanner=true in "addons-221000"

	I0918 11:53:00.170553    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170512    1740 addons.go:69] Setting volumesnapshots=true in profile "addons-221000"

	W0918 11:53:00.171149    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	W0918 11:53:00.171162    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	I0918 11:53:00.171465    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.188489    1740 out.go:177]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5

	I0918 11:53:00.182591    1740 host.go:66] Checking if "addons-221000" exists ...

	W0918 11:53:00.182599    1740 addons.go:277] "addons-221000" is not running, setting ingress=true and skipping enablement (err=<nil>)

	W0918 11:53:00.182603    1740 addons_storage_classes.go:55] "addons-221000" is not running, writing default-storageclass=true to disk and skipping enablement

	I0918 11:53:00.182614    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.182623    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.182641    1740 addons.go:231] Setting addon volumesnapshots=true in "addons-221000"

	I0918 11:53:00.183250    1740 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=registry" in ns "kube-system" ...

	I0918 11:53:00.196542    1740 out.go:177]   - Using image registry.k8s.io/metrics-server/metrics-server:v0.6.4

	I0918 11:53:00.192833    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.192879    1740 addons.go:423] installing /etc/kubernetes/addons/storage-provisioner.yaml

	I0918 11:53:00.192896    1740 addons.go:467] Verifying addon ingress=true in "addons-221000"

	I0918 11:53:00.192903    1740 addons.go:231] Setting addon default-storageclass=true in "addons-221000"

	W0918 11:53:00.193202    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	I0918 11:53:00.196510    1740 kapi.go:248] "coredns" deployment in "kube-system" namespace and "addons-221000" context rescaled to 1 replicas

	I0918 11:53:00.199509    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)

	I0918 11:53:00.199541    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-apiservice.yaml

	I0918 11:53:00.199554    1740 host.go:66] Checking if "addons-221000" exists ...

	W0918 11:53:00.199564    1740 addons.go:277] "addons-221000" is not running, setting cloud-spanner=true and skipping enablement (err=<nil>)

	I0918 11:53:00.206530    1740 out.go:177]   - Using image ghcr.io/inspektor-gadget/inspektor-gadget:v0.20.0

	I0918 11:53:00.210356    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)

	I0918 11:53:00.210367    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.210376    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/livenessprobe:v2.8.0

	I0918 11:53:00.210391    1740 start.go:223] Will wait 6m0s for node &{Name: IP:192.168.105.2 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 11:53:00.211130    1740 addons.go:423] installing /etc/kubernetes/addons/storageclass.yaml

	I0918 11:53:00.213481    1740 out.go:177] * Verifying ingress addon...

	I0918 11:53:00.214506    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/snapshot-controller:v6.1.0

	I0918 11:53:00.214532    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.214630    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)

	I0918 11:53:00.218827    1740 out.go:177] * Verifying Kubernetes components...

	I0918 11:53:00.221560    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^        forward . \/etc\/resolv.conf.*/i \        hosts {\n           192.168.105.1 host.minikube.internal\n           fallthrough\n        }' -e '/^        errors *$/i \        log' | sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"

	I0918 11:53:00.218836    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.229444    1740 addons.go:423] installing /etc/kubernetes/addons/ig-namespace.yaml

	I0918 11:53:00.229456    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-namespace.yaml (55 bytes)

	I0918 11:53:00.217515    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml

	I0918 11:53:00.229466    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml (934 bytes)

	I0918 11:53:00.229467    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.229472    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.220086    1740 kapi.go:86] Found 0 Pods for label selector kubernetes.io/minikube-addons=registry

	I0918 11:53:00.221623    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I0918 11:53:00.236499    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-resizer:v1.6.0

	I0918 11:53:00.234112    1740 kapi.go:75] Waiting for pod with label "app.kubernetes.io/name=ingress-nginx" in ns "ingress-nginx" ...

	I0918 11:53:00.242380    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0

	I0918 11:53:00.251491    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-provisioner:v3.3.0

	I0918 11:53:00.252703    1740 kapi.go:86] Found 0 Pods for label selector app.kubernetes.io/name=ingress-nginx

	I0918 11:53:00.260396    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-attacher:v4.0.0

	I0918 11:53:00.267323    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.7.0

	I0918 11:53:00.274307    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.6.0

	I0918 11:53:00.284458    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/hostpathplugin:v1.9.0

	I0918 11:53:00.287512    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-attacher.yaml

	I0918 11:53:00.287521    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-attacher.yaml (3073 bytes)

	I0918 11:53:00.287531    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.322513    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-server-deployment.yaml

	I0918 11:53:00.322523    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1907 bytes)

	I0918 11:53:00.336060    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml

	I0918 11:53:00.340017    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml

	I0918 11:53:00.346686    1740 addons.go:423] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml

	I0918 11:53:00.346693    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml (6471 bytes)

	I0918 11:53:00.362466    1740 addons.go:423] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml

	I0918 11:53:00.362477    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml (23126 bytes)

	I0918 11:53:00.363923    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-server-rbac.yaml

	I0918 11:53:00.363928    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)

	I0918 11:53:00.368153    1740 addons.go:423] installing /etc/kubernetes/addons/ig-serviceaccount.yaml

	I0918 11:53:00.368161    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-serviceaccount.yaml (80 bytes)

	I0918 11:53:00.376728    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-hostpath.yaml

	I0918 11:53:00.376740    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-hostpath.yaml (4266 bytes)

	I0918 11:53:00.395920    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-server-service.yaml

	I0918 11:53:00.395931    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)

	I0918 11:53:00.403313    1740 addons.go:423] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml

	I0918 11:53:00.403320    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml (19582 bytes)

	I0918 11:53:00.404660    1740 addons.go:423] installing /etc/kubernetes/addons/ig-role.yaml

	I0918 11:53:00.404665    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-role.yaml (210 bytes)

	I0918 11:53:00.430003    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml

	I0918 11:53:00.430014    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml (3038 bytes)

	I0918 11:53:00.492665    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-provisioner.yaml

	I0918 11:53:00.492677    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-provisioner.yaml (4442 bytes)

	I0918 11:53:00.495149    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml

	I0918 11:53:00.495157    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml (3545 bytes)

	I0918 11:53:00.500095    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml

	I0918 11:53:00.522789    1740 addons.go:423] installing /etc/kubernetes/addons/ig-rolebinding.yaml

	I0918 11:53:00.522799    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-rolebinding.yaml (244 bytes)

	I0918 11:53:00.561433    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-resizer.yaml

	I0918 11:53:00.561445    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-resizer.yaml (2943 bytes)

	I0918 11:53:00.561578    1740 addons.go:423] installing /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0918 11:53:00.561584    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml (1475 bytes)

	I0918 11:53:00.579630    1740 addons.go:423] installing /etc/kubernetes/addons/ig-clusterrole.yaml

	I0918 11:53:00.579641    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-clusterrole.yaml (1485 bytes)

	I0918 11:53:00.604202    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0918 11:53:00.607378    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-snapshotter.yaml

	I0918 11:53:00.607389    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-snapshotter.yaml (3149 bytes)

	I0918 11:53:00.624742    1740 addons.go:423] installing /etc/kubernetes/addons/ig-clusterrolebinding.yaml

	I0918 11:53:00.624755    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-clusterrolebinding.yaml (274 bytes)

	I0918 11:53:00.641432    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-attacher.yaml

	I0918 11:53:00.641441    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-attacher.yaml (2143 bytes)

	I0918 11:53:00.675716    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml

	I0918 11:53:00.675728    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml (1274 bytes)

	I0918 11:53:00.684962    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-plugin.yaml

	I0918 11:53:00.684971    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-plugin.yaml (8201 bytes)

	I0918 11:53:00.690643    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-resizer.yaml

	I0918 11:53:00.690652    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-resizer.yaml (2191 bytes)

	I0918 11:53:00.691887    1740 addons.go:423] installing /etc/kubernetes/addons/ig-crd.yaml

	I0918 11:53:00.691893    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-crd.yaml (5216 bytes)

	I0918 11:53:00.701987    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-storageclass.yaml

	I0918 11:53:00.701999    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-storageclass.yaml (846 bytes)

	I0918 11:53:00.719094    1740 addons.go:423] installing /etc/kubernetes/addons/ig-daemonset.yaml

	I0918 11:53:00.719103    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-daemonset.yaml (7741 bytes)

	I0918 11:53:00.821368    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/ig-namespace.yaml -f /etc/kubernetes/addons/ig-serviceaccount.yaml -f /etc/kubernetes/addons/ig-role.yaml -f /etc/kubernetes/addons/ig-rolebinding.yaml -f /etc/kubernetes/addons/ig-clusterrole.yaml -f /etc/kubernetes/addons/ig-clusterrolebinding.yaml -f /etc/kubernetes/addons/ig-crd.yaml -f /etc/kubernetes/addons/ig-daemonset.yaml

	I0918 11:53:00.824330    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml

	I0918 11:53:01.167637    1740 start.go:917] {"host.minikube.internal": 192.168.105.1} host record injected into CoreDNS's ConfigMap

	I0918 11:53:01.168100    1740 node_ready.go:35] waiting up to 6m0s for node "addons-221000" to be "Ready" ...

	I0918 11:53:01.169940    1740 node_ready.go:49] node "addons-221000" has status "Ready":"True"

	I0918 11:53:01.169963    1740 node_ready.go:38] duration metric: took 1.836333ms waiting for node "addons-221000" to be "Ready" ...

	I0918 11:53:01.169968    1740 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I0918 11:53:01.172987    1740 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5dd5756b68-mbgns" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:01.659785    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (1.159683334s)

	I0918 11:53:01.659805    1740 addons.go:467] Verifying addon metrics-server=true in "addons-221000"

	I0918 11:53:01.659835    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: (1.055627667s)

	W0918 11:53:01.659861    1740 addons.go:449] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1

	stdout:

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created

	serviceaccount/snapshot-controller created

	clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created

	clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created

	role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	deployment.apps/snapshot-controller created

	stderr:

	error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"

	ensure CRDs are installed first

	I0918 11:53:01.659882    1740 retry.go:31] will retry after 288.841008ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1

	stdout:

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created

	serviceaccount/snapshot-controller created

	clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created

	clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created

	role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	deployment.apps/snapshot-controller created

	stderr:

	error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"

	ensure CRDs are installed first

	I0918 11:53:01.660144    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (1.320131291s)

	I0918 11:53:01.949280    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply --force -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0918 11:53:02.183526    1740 pod_ready.go:92] pod "coredns-5dd5756b68-mbgns" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.183539    1740 pod_ready.go:81] duration metric: took 1.010553542s waiting for pod "coredns-5dd5756b68-mbgns" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.183545    1740 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.184891    1740 pod_ready.go:97] error getting pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace (skipping!): pods "coredns-5dd5756b68-z4cmf" not found

	I0918 11:53:02.184904    1740 pod_ready.go:81] duration metric: took 1.354709ms waiting for pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace to be "Ready" ...

	E0918 11:53:02.184909    1740 pod_ready.go:66] WaitExtra: waitPodCondition: error getting pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace (skipping!): pods "coredns-5dd5756b68-z4cmf" not found

	I0918 11:53:02.184914    1740 pod_ready.go:78] waiting up to 6m0s for pod "etcd-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.187700    1740 pod_ready.go:92] pod "etcd-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.187709    1740 pod_ready.go:81] duration metric: took 2.791875ms waiting for pod "etcd-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.187714    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.190228    1740 pod_ready.go:92] pod "kube-apiserver-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.190236    1740 pod_ready.go:81] duration metric: took 2.518208ms waiting for pod "kube-apiserver-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.190240    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.571748    1740 pod_ready.go:92] pod "kube-controller-manager-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.571761    1740 pod_ready.go:81] duration metric: took 381.518375ms waiting for pod "kube-controller-manager-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.571765    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-q7gqn" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.823208    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml: (1.998874458s)

	I0918 11:53:02.823230    1740 addons.go:467] Verifying addon csi-hostpath-driver=true in "addons-221000"

	I0918 11:53:02.828329    1740 out.go:177] * Verifying csi-hostpath-driver addon...

	I0918 11:53:02.838784    1740 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=csi-hostpath-driver" in ns "kube-system" ...

	I0918 11:53:02.842760    1740 kapi.go:86] Found 3 Pods for label selector kubernetes.io/minikube-addons=csi-hostpath-driver

	I0918 11:53:02.842767    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:02.847417    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:02.972129    1740 pod_ready.go:92] pod "kube-proxy-q7gqn" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.972138    1740 pod_ready.go:81] duration metric: took 400.3735ms waiting for pod "kube-proxy-q7gqn" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.972143    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:03.351863    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:03.371796    1740 pod_ready.go:92] pod "kube-scheduler-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:03.371806    1740 pod_ready.go:81] duration metric: took 399.662875ms waiting for pod "kube-scheduler-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:03.371810    1740 pod_ready.go:38] duration metric: took 2.201856875s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I0918 11:53:03.371820    1740 api_server.go:52] waiting for apiserver process to appear ...

	I0918 11:53:03.371875    1740 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I0918 11:53:03.851785    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:04.352157    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:04.679993    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply --force -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: (2.73071775s)

	I0918 11:53:04.680002    1740 ssh_runner.go:235] Completed: sudo pgrep -xnf kube-apiserver.*minikube.*: (1.308127625s)

	I0918 11:53:04.680021    1740 api_server.go:72] duration metric: took 4.46545525s to wait for apiserver process to appear ...

	I0918 11:53:04.680025    1740 api_server.go:88] waiting for apiserver healthz status ...

	I0918 11:53:04.680031    1740 api_server.go:253] Checking apiserver healthz at https://192.168.105.2:8443/healthz ...

	I0918 11:53:04.683995    1740 api_server.go:279] https://192.168.105.2:8443/healthz returned 200:

	ok

	I0918 11:53:04.684750    1740 api_server.go:141] control plane version: v1.28.2

	I0918 11:53:04.684756    1740 api_server.go:131] duration metric: took 4.728917ms to wait for apiserver health ...

	I0918 11:53:04.684760    1740 system_pods.go:43] waiting for kube-system pods to appear ...

	I0918 11:53:04.689157    1740 system_pods.go:59] 13 kube-system pods found

	I0918 11:53:04.689166    1740 system_pods.go:61] "coredns-5dd5756b68-mbgns" [376db80e-bef7-49a8-805c-d250bbb5ddc5] Running

	I0918 11:53:04.689171    1740 system_pods.go:61] "csi-hostpath-attacher-0" [b3eb2340-f156-4127-b844-79013849b5d5] Pending / Ready:ContainersNotReady (containers with unready status: [csi-attacher]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-attacher])

	I0918 11:53:04.689175    1740 system_pods.go:61] "csi-hostpath-resizer-0" [a39fc7e1-21e6-43e0-8a71-76fc3122aa67] Pending / Ready:ContainersNotReady (containers with unready status: [csi-resizer]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-resizer])

	I0918 11:53:04.689182    1740 system_pods.go:61] "csi-hostpathplugin-s878j" [f7db805b-46b1-4d4f-b620-4bb9732a0ba0] Pending / Ready:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter])

	I0918 11:53:04.689185    1740 system_pods.go:61] "etcd-addons-221000" [4500c338-1b0c-4d39-b5b6-76d42cf285f5] Running

	I0918 11:53:04.689188    1740 system_pods.go:61] "kube-apiserver-addons-221000" [93a9cc2e-3463-4332-a37c-86437106ed5e] Running

	I0918 11:53:04.689190    1740 system_pods.go:61] "kube-controller-manager-addons-221000" [eaeef50f-2b51-43b2-91c9-a4f97f5460ae] Running

	I0918 11:53:04.689193    1740 system_pods.go:61] "kube-proxy-q7gqn" [e971c33c-7d1b-47b7-9ff5-3a629f12fb57] Running

	I0918 11:53:04.689195    1740 system_pods.go:61] "kube-scheduler-addons-221000" [9d39af63-6f0a-4855-9435-f8e7af26869e] Running

	I0918 11:53:04.689199    1740 system_pods.go:61] "metrics-server-7c66d45ddc-ph4qt" [193ff040-48cb-4429-8aa3-4065ecb5241d] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I0918 11:53:04.689205    1740 system_pods.go:61] "snapshot-controller-58dbcc7b99-89j9m" [d9a96c2a-2231-4dea-abbf-16875dd2b1d7] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.689210    1740 system_pods.go:61] "snapshot-controller-58dbcc7b99-xwwxn" [5a80446c-a3a8-4ce7-8ac4-2894087691fb] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.689214    1740 system_pods.go:61] "storage-provisioner" [88ff8527-97e2-4317-8d5b-a2502e8cb7f7] Running

	I0918 11:53:04.689217    1740 system_pods.go:74] duration metric: took 4.45475ms to wait for pod list to return data ...

	I0918 11:53:04.689220    1740 default_sa.go:34] waiting for default service account to be created ...

	I0918 11:53:04.690951    1740 default_sa.go:45] found service account: "default"

	I0918 11:53:04.690958    1740 default_sa.go:55] duration metric: took 1.736083ms for default service account to be created ...

	I0918 11:53:04.690961    1740 system_pods.go:116] waiting for k8s-apps to be running ...

	I0918 11:53:04.694962    1740 system_pods.go:86] 13 kube-system pods found

	I0918 11:53:04.694971    1740 system_pods.go:89] "coredns-5dd5756b68-mbgns" [376db80e-bef7-49a8-805c-d250bbb5ddc5] Running

	I0918 11:53:04.694976    1740 system_pods.go:89] "csi-hostpath-attacher-0" [b3eb2340-f156-4127-b844-79013849b5d5] Pending / Ready:ContainersNotReady (containers with unready status: [csi-attacher]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-attacher])

	I0918 11:53:04.694979    1740 system_pods.go:89] "csi-hostpath-resizer-0" [a39fc7e1-21e6-43e0-8a71-76fc3122aa67] Pending / Ready:ContainersNotReady (containers with unready status: [csi-resizer]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-resizer])

	I0918 11:53:04.694983    1740 system_pods.go:89] "csi-hostpathplugin-s878j" [f7db805b-46b1-4d4f-b620-4bb9732a0ba0] Pending / Ready:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter])

	I0918 11:53:04.694986    1740 system_pods.go:89] "etcd-addons-221000" [4500c338-1b0c-4d39-b5b6-76d42cf285f5] Running

	I0918 11:53:04.694988    1740 system_pods.go:89] "kube-apiserver-addons-221000" [93a9cc2e-3463-4332-a37c-86437106ed5e] Running

	I0918 11:53:04.694990    1740 system_pods.go:89] "kube-controller-manager-addons-221000" [eaeef50f-2b51-43b2-91c9-a4f97f5460ae] Running

	I0918 11:53:04.694994    1740 system_pods.go:89] "kube-proxy-q7gqn" [e971c33c-7d1b-47b7-9ff5-3a629f12fb57] Running

	I0918 11:53:04.694996    1740 system_pods.go:89] "kube-scheduler-addons-221000" [9d39af63-6f0a-4855-9435-f8e7af26869e] Running

	I0918 11:53:04.694999    1740 system_pods.go:89] "metrics-server-7c66d45ddc-ph4qt" [193ff040-48cb-4429-8aa3-4065ecb5241d] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I0918 11:53:04.695003    1740 system_pods.go:89] "snapshot-controller-58dbcc7b99-89j9m" [d9a96c2a-2231-4dea-abbf-16875dd2b1d7] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.695006    1740 system_pods.go:89] "snapshot-controller-58dbcc7b99-xwwxn" [5a80446c-a3a8-4ce7-8ac4-2894087691fb] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.695009    1740 system_pods.go:89] "storage-provisioner" [88ff8527-97e2-4317-8d5b-a2502e8cb7f7] Running

	I0918 11:53:04.695012    1740 system_pods.go:126] duration metric: took 4.049ms to wait for k8s-apps to be running ...

	I0918 11:53:04.695014    1740 system_svc.go:44] waiting for kubelet service to be running ....

	I0918 11:53:04.695074    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I0918 11:53:04.700754    1740 system_svc.go:56] duration metric: took 5.736541ms WaitForService to wait for kubelet.

	I0918 11:53:04.700761    1740 kubeadm.go:581] duration metric: took 4.486195916s to wait for : map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] ...

	I0918 11:53:04.700771    1740 node_conditions.go:102] verifying NodePressure condition ...

	I0918 11:53:04.702224    1740 node_conditions.go:122] node storage ephemeral capacity is 17784760Ki

	I0918 11:53:04.702234    1740 node_conditions.go:123] node cpu capacity is 2

	I0918 11:53:04.702239    1740 node_conditions.go:105] duration metric: took 1.466083ms to run NodePressure ...

	I0918 11:53:04.702244    1740 start.go:228] waiting for startup goroutines ...

	I0918 11:53:04.851794    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:05.352908    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:05.851843    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:06.351684    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:06.787954    1740 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_application_credentials.json (162 bytes)

	I0918 11:53:06.787972    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:06.819465    1740 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_cloud_project (12 bytes)

	I0918 11:53:06.824867    1740 addons.go:231] Setting addon gcp-auth=true in "addons-221000"

	I0918 11:53:06.824887    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:06.825624    1740 ssh_runner.go:195] Run: cat /var/lib/minikube/google_application_credentials.json

	I0918 11:53:06.825631    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:06.851875    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:06.860327    1740 out.go:177]   - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20230407

	I0918 11:53:06.868318    1740 out.go:177]   - Using image gcr.io/k8s-minikube/gcp-auth-webhook:v0.1.0

	I0918 11:53:06.871302    1740 addons.go:423] installing /etc/kubernetes/addons/gcp-auth-ns.yaml

	I0918 11:53:06.871308    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-ns.yaml (700 bytes)

	I0918 11:53:06.876186    1740 addons.go:423] installing /etc/kubernetes/addons/gcp-auth-service.yaml

	I0918 11:53:06.876191    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-service.yaml (788 bytes)

	I0918 11:53:06.881437    1740 addons.go:423] installing /etc/kubernetes/addons/gcp-auth-webhook.yaml

	I0918 11:53:06.881443    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-webhook.yaml (5412 bytes)

	I0918 11:53:06.887709    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/gcp-auth-ns.yaml -f /etc/kubernetes/addons/gcp-auth-service.yaml -f /etc/kubernetes/addons/gcp-auth-webhook.yaml

	I0918 11:53:07.124491    1740 addons.go:467] Verifying addon gcp-auth=true in "addons-221000"

	I0918 11:53:07.129030    1740 out.go:177] * Verifying gcp-auth addon...

	I0918 11:53:07.137330    1740 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=gcp-auth" in ns "gcp-auth" ...

	I0918 11:53:07.139319    1740 kapi.go:86] Found 1 Pods for label selector kubernetes.io/minikube-addons=gcp-auth

	I0918 11:53:07.139325    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:07.142056    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:07.351786    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:07.647787    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:07.851746    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:08.144923    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:08.351959    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:08.646023    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:08.851939    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:09.146053    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:09.352896    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:09.645776    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:09.851792    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:10.146843    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:10.351428    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:10.645894    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:10.852236    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:11.145951    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:11.352216    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:11.645774    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:11.852232    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:12.145929    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:12.355119    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:12.840629    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:12.851603    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:13.145852    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:13.351962    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:13.646245    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:13.852112    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:14.144894    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:14.351204    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:14.646046    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:15.059939    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:15.145929    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:15.351906    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:15.646033    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:15.852022    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:16.145893    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:16.351976    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:16.645740    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:16.852007    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:17.145547    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:17.352003    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:17.646147    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:17.853011    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:18.145960    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:18.353519    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:18.646257    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:18.851829    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:19.143968    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:19.351778    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:19.645563    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:19.851678    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:20.145637    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:20.351425    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:20.645727    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:20.852080    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:21.145053    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:21.352010    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:21.645631    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:21.851983    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:22.146102    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:22.351559    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:22.645364    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:22.851995    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:23.145511    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:23.351664    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:23.646427    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:23.851813    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:24.144653    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:24.351670    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:24.645732    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:24.851659    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:25.145755    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:25.350286    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:25.645572    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:25.851968    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:26.145692    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:26.352042    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:26.645575    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:26.852307    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:27.145498    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:27.352114    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:27.645719    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:27.851965    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:28.145987    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:28.351772    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:28.645594    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:28.851993    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:29.145747    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:29.351502    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:29.645697    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:29.851953    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:30.145495    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:30.351423    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:30.645330    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:30.852142    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:31.145422    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:31.352025    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:31.645572    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:31.852021    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:32.145742    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:32.351596    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:32.645761    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:32.851846    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:33.145666    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:33.351607    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:33.646262    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:33.852115    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:34.145556    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:34.353136    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:34.644719    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:34.852049    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:35.145343    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:35.351551    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:35.645361    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:35.851476    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:36.143766    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:36.351679    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:36.645788    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:36.851584    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:37.145933    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:37.351445    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:37.646034    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:37.851557    1740 kapi.go:107] duration metric: took 35.013102459s to wait for kubernetes.io/minikube-addons=csi-hostpath-driver ...

	I0918 11:53:38.145661    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:38.645759    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:39.145382    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:39.645858    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:40.145851    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:40.645842    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:41.145511    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:41.646176    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:42.145937    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:42.645521    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:43.145380    1740 kapi.go:107] duration metric: took 36.008387458s to wait for kubernetes.io/minikube-addons=gcp-auth ...

	I0918 11:53:43.148984    1740 out.go:177] * Your GCP credentials will now be mounted into every pod created in the addons-221000 cluster.

	I0918 11:53:43.152958    1740 out.go:177] * If you don't want your credentials mounted into a specific pod, add a label with the `gcp-auth-skip-secret` key to your pod configuration.

	I0918 11:53:43.156871    1740 out.go:177] * If you want existing pods to be mounted with credentials, either recreate them or rerun addons enable with --refresh.

	I0918 11:59:00.190235    1740 kapi.go:107] duration metric: took 6m0.011650291s to wait for kubernetes.io/minikube-addons=registry ...

	W0918 11:59:00.190354    1740 out.go:239] ! Enabling 'registry' returned an error: running callbacks: [waiting for kubernetes.io/minikube-addons=registry pods: context deadline exceeded]

	I0918 11:59:00.236861    1740 kapi.go:107] duration metric: took 6m0.007418791s to wait for app.kubernetes.io/name=ingress-nginx ...

	W0918 11:59:00.236887    1740 out.go:239] ! Enabling 'ingress' returned an error: running callbacks: [waiting for app.kubernetes.io/name=ingress-nginx pods: context deadline exceeded]

	I0918 11:59:00.245622    1740 out.go:177] * Enabled addons: ingress-dns, cloud-spanner, default-storageclass, metrics-server, storage-provisioner, inspektor-gadget, volumesnapshots, csi-hostpath-driver, gcp-auth

	I0918 11:59:00.252559    1740 addons.go:502] enable addons completed in 6m0.086876584s: enabled=[ingress-dns cloud-spanner default-storageclass metrics-server storage-provisioner inspektor-gadget volumesnapshots csi-hostpath-driver gcp-auth]

	I0918 11:59:00.252570    1740 start.go:233] waiting for cluster config update ...

	I0918 11:59:00.252577    1740 start.go:242] writing updated cluster config ...

	I0918 11:59:00.253046    1740 ssh_runner.go:195] Run: rm -f paused

	I0918 11:59:00.282509    1740 start.go:600] kubectl: 1.27.2, cluster: 1.28.2 (minor skew: 1)

	I0918 11:59:00.285533    1740 out.go:177] * Done! kubectl is now configured to use "addons-221000" cluster and "default" namespace by default

	* 

	* ==> Docker <==

	* -- Journal begins at Mon 2023-09-18 18:52:28 UTC, ends at Mon 2023-09-18 19:12:19 UTC. --

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.378956054Z" level=info msg="shim disconnected" id=db4658d155f2578fc668ba94399fd4b64164a06274f25bb873a3470c521d2b3b namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.378988221Z" level=warning msg="cleaning up after shim disconnected" id=db4658d155f2578fc668ba94399fd4b64164a06274f25bb873a3470c521d2b3b namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.378992929Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1100]: time="2023-09-18T19:12:04.436636476Z" level=info msg="ignoring event" container=f31f590bd070326784aa894bb3b130e04a21f13ff6b7835f98e9c9c994be5c89 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.437103975Z" level=info msg="shim disconnected" id=f31f590bd070326784aa894bb3b130e04a21f13ff6b7835f98e9c9c994be5c89 namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.437132058Z" level=warning msg="cleaning up after shim disconnected" id=f31f590bd070326784aa894bb3b130e04a21f13ff6b7835f98e9c9c994be5c89 namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.437137642Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.441266547Z" level=warning msg="cleanup warnings time=\"2023-09-18T19:12:04Z\" level=warning msg=\"failed to remove runc container\" error=\"runc did not terminate successfully: exit status 255: \" runtime=io.containerd.runc.v2\n" namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1100]: time="2023-09-18T19:12:10.222057782Z" level=info msg="ignoring event" container=ab33a0f8756c5abdbe52f99802252c9d88620a35e7026b8d48ddc961d9fd608a module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.222121782Z" level=info msg="shim disconnected" id=ab33a0f8756c5abdbe52f99802252c9d88620a35e7026b8d48ddc961d9fd608a namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.222148615Z" level=warning msg="cleaning up after shim disconnected" id=ab33a0f8756c5abdbe52f99802252c9d88620a35e7026b8d48ddc961d9fd608a namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.222152574Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1100]: time="2023-09-18T19:12:10.284872898Z" level=info msg="ignoring event" container=b759b3bc87e672ab6709e17bb94b6e35eee5335882a2a4303af5bb7802b4d324 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.284977606Z" level=info msg="shim disconnected" id=b759b3bc87e672ab6709e17bb94b6e35eee5335882a2a4303af5bb7802b4d324 namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.285020689Z" level=warning msg="cleaning up after shim disconnected" id=b759b3bc87e672ab6709e17bb94b6e35eee5335882a2a4303af5bb7802b4d324 namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.285025106Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:10 addons-221000 cri-dockerd[995]: time="2023-09-18T19:12:10Z" level=error msg="EOF Failed to get stats from container ab33a0f8756c5abdbe52f99802252c9d88620a35e7026b8d48ddc961d9fd608a"

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.398430134Z" level=info msg="shim disconnected" id=afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1100]: time="2023-09-18T19:12:14.398477592Z" level=info msg="ignoring event" container=afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.398687758Z" level=warning msg="cleaning up after shim disconnected" id=afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.398698092Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.419717366Z" level=info msg="shim disconnected" id=a7e6c06852560a1d0bdec45aab8087733475e5299415ad38f796c38431947c7a namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.419822949Z" level=warning msg="cleaning up after shim disconnected" id=a7e6c06852560a1d0bdec45aab8087733475e5299415ad38f796c38431947c7a namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.419832116Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1100]: time="2023-09-18T19:12:14.420080948Z" level=info msg="ignoring event" container=a7e6c06852560a1d0bdec45aab8087733475e5299415ad38f796c38431947c7a module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	* 

	* ==> container status <==

	* CONTAINER           IMAGE                                                                                                          CREATED              STATE               NAME                      ATTEMPT             POD ID

	e55be7fdce01f       ghcr.io/headlamp-k8s/headlamp@sha256:498ea22dc5acadaa4015e7a50335d21fdce45d9e8f1f8adf29c2777da4182f98          About a minute ago   Running             headlamp                  0                   6eea9635aa263

	6009e365438d8       gcr.io/k8s-minikube/gcp-auth-webhook@sha256:d730651bb6584f969d95d8279a754cf9d8d31b5055c43dbdb8d7363a8c6371cf   18 minutes ago       Running             gcp-auth                  0                   e321131e2d88d

	96b1e19d34d0f       ba04bb24b9575                                                                                                  19 minutes ago       Running             storage-provisioner       0                   31030bebae5dd

	3397ed73112e1       97e04611ad434                                                                                                  19 minutes ago       Running             coredns                   0                   ecf214ed85c34

	3b8b236037bf7       7da62c127fc0f                                                                                                  19 minutes ago       Running             kube-proxy                0                   efd5b0f304a7c

	17d16f9191cb9       64fc40cee3716                                                                                                  19 minutes ago       Running             kube-scheduler            0                   6dcf2ed48fa0d

	cb85fb8fd00cf       89d57b83c1786                                                                                                  19 minutes ago       Running             kube-controller-manager   0                   d0674da883f4f

	4a8eb16a561d8       30bb499447fe1                                                                                                  19 minutes ago       Running             kube-apiserver            0                   2a8be15cc8448

	2a7ec1fe69df2       9cdd6470f48c8                                                                                                  19 minutes ago       Running             etcd                      0                   0f317e031c491

	* 

	* ==> coredns [3397ed73112e] <==

	* .:53

	[INFO] plugin/reload: Running configuration SHA512 = 591cf328cccc12bc490481273e738df59329c62c0b729d94e8b61db9961c2fa5f046dd37f1cf888b953814040d180f52594972691cd6ff41be96639138a43908

	CoreDNS-1.10.1

	linux/arm64, go1.20, 055b2c3

	[INFO] Reloading

	[INFO] plugin/reload: Running configuration SHA512 = ea7a0d73d9d208f758b1f67640ef03c58089b9d9366cf3478df3bb369b210e39f213811b46224f8a04380814b6e0890ccd358f5b5e8c80bc22ac19c8601ee35b

	[INFO] Reloading complete

	[INFO] 127.0.0.1:51523 - 34394 "HINFO IN 2890994018648264973.7586751697303130781. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.046989834s

	[INFO] 10.244.0.11:51746 - 24273 "AAAA IN storage.googleapis.com.gcp-auth.svc.cluster.local. udp 78 false 1232" NXDOMAIN qr,aa,rd 160 0.000125498s

	[INFO] 10.244.0.11:38492 - 22517 "A IN storage.googleapis.com.gcp-auth.svc.cluster.local. udp 78 false 1232" NXDOMAIN qr,aa,rd 160 0.000252786s

	[INFO] 10.244.0.11:44956 - 48338 "A IN storage.googleapis.com.svc.cluster.local. udp 69 false 1232" NXDOMAIN qr,aa,rd 151 0.000035291s

	[INFO] 10.244.0.11:34979 - 63104 "AAAA IN storage.googleapis.com.svc.cluster.local. udp 69 false 1232" NXDOMAIN qr,aa,rd 151 0.000059957s

	[INFO] 10.244.0.11:38551 - 60811 "A IN storage.googleapis.com.cluster.local. udp 65 false 1232" NXDOMAIN qr,aa,rd 147 0.000037916s

	[INFO] 10.244.0.11:44454 - 13651 "AAAA IN storage.googleapis.com.cluster.local. udp 65 false 1232" NXDOMAIN qr,aa,rd 147 0.000042082s

	[INFO] 10.244.0.11:41401 - 32727 "AAAA IN storage.googleapis.com. udp 51 false 1232" NOERROR qr,rd,ra 240 0.001047395s

	[INFO] 10.244.0.11:40924 - 8451 "A IN storage.googleapis.com. udp 51 false 1232" NOERROR qr,rd,ra 496 0.001040478s

	* 

	* ==> describe nodes <==

	* Name:               addons-221000

	Roles:              control-plane

	Labels:             beta.kubernetes.io/arch=arm64

	                    beta.kubernetes.io/os=linux

	                    kubernetes.io/arch=arm64

	                    kubernetes.io/hostname=addons-221000

	                    kubernetes.io/os=linux

	                    minikube.k8s.io/commit=ae2adeb20cc3a4b5f1decc7c8f53736ec04c4a36

	                    minikube.k8s.io/name=addons-221000

	                    minikube.k8s.io/primary=true

	                    minikube.k8s.io/updated_at=2023_09_18T11_52_46_0700

	                    minikube.k8s.io/version=v1.31.2

	                    node-role.kubernetes.io/control-plane=

	                    node.kubernetes.io/exclude-from-external-load-balancers=

	                    topology.hostpath.csi/node=addons-221000

	Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock

	                    node.alpha.kubernetes.io/ttl: 0

	                    volumes.kubernetes.io/controller-managed-attach-detach: true

	CreationTimestamp:  Mon, 18 Sep 2023 18:52:43 +0000

	Taints:             <none>

	Unschedulable:      false

	Lease:

	  HolderIdentity:  addons-221000

	  AcquireTime:     <unset>

	  RenewTime:       Mon, 18 Sep 2023 19:12:19 +0000

	Conditions:

	  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message

	  ----             ------  -----------------                 ------------------                ------                       -------

	  MemoryPressure   False   Mon, 18 Sep 2023 19:11:52 +0000   Mon, 18 Sep 2023 18:52:43 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available

	  DiskPressure     False   Mon, 18 Sep 2023 19:11:52 +0000   Mon, 18 Sep 2023 18:52:43 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure

	  PIDPressure      False   Mon, 18 Sep 2023 19:11:52 +0000   Mon, 18 Sep 2023 18:52:43 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available

	  Ready            True    Mon, 18 Sep 2023 19:11:52 +0000   Mon, 18 Sep 2023 18:52:48 +0000   KubeletReady                 kubelet is posting ready status

	Addresses:

	  InternalIP:  192.168.105.2

	  Hostname:    addons-221000

	Capacity:

	  cpu:                2

	  ephemeral-storage:  17784760Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3905012Ki

	  pods:               110

	Allocatable:

	  cpu:                2

	  ephemeral-storage:  17784760Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3905012Ki

	  pods:               110

	System Info:

	  Machine ID:                 605bc2fc72a045ae88e907db961da3d3

	  System UUID:                605bc2fc72a045ae88e907db961da3d3

	  Boot ID:                    6a9990c2-fe5e-48d8-97ca-ea50d8c8e3b8

	  Kernel Version:             5.10.57

	  OS Image:                   Buildroot 2021.02.12

	  Operating System:           linux

	  Architecture:               arm64

	  Container Runtime Version:  docker://24.0.6

	  Kubelet Version:            v1.28.2

	  Kube-Proxy Version:         v1.28.2

	PodCIDR:                      10.244.0.0/24

	PodCIDRs:                     10.244.0.0/24

	Non-terminated Pods:          (9 in total)

	  Namespace                   Name                                     CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age

	  ---------                   ----                                     ------------  ----------  ---------------  -------------  ---

	  gcp-auth                    gcp-auth-d4c87556c-2vm8d                 0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	  headlamp                    headlamp-699c48fb74-6fg5z                0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         78s

	  kube-system                 coredns-5dd5756b68-mbgns                 100m (5%!)(MISSING)     0 (0%!)(MISSING)      70Mi (1%!)(MISSING)        170Mi (4%!)(MISSING)     19m

	  kube-system                 etcd-addons-221000                       100m (5%!)(MISSING)     0 (0%!)(MISSING)      100Mi (2%!)(MISSING)       0 (0%!)(MISSING)         19m

	  kube-system                 kube-apiserver-addons-221000             250m (12%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	  kube-system                 kube-controller-manager-addons-221000    200m (10%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	  kube-system                 kube-proxy-q7gqn                         0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	  kube-system                 kube-scheduler-addons-221000             100m (5%!)(MISSING)     0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	  kube-system                 storage-provisioner                      0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	Allocated resources:

	  (Total limits may be over 100 percent, i.e., overcommitted.)

	  Resource           Requests    Limits

	  --------           --------    ------

	  cpu                750m (37%!)(MISSING)  0 (0%!)(MISSING)

	  memory             170Mi (4%!)(MISSING)  170Mi (4%!)(MISSING)

	  ephemeral-storage  0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-1Gi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-2Mi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-32Mi     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-64Ki     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	Events:

	  Type    Reason                   Age   From             Message

	  ----    ------                   ----  ----             -------

	  Normal  Starting                 19m   kube-proxy       

	  Normal  Starting                 19m   kubelet          Starting kubelet.

	  Normal  NodeAllocatableEnforced  19m   kubelet          Updated Node Allocatable limit across pods

	  Normal  NodeHasSufficientMemory  19m   kubelet          Node addons-221000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    19m   kubelet          Node addons-221000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     19m   kubelet          Node addons-221000 status is now: NodeHasSufficientPID

	  Normal  NodeReady                19m   kubelet          Node addons-221000 status is now: NodeReady

	  Normal  RegisteredNode           19m   node-controller  Node addons-221000 event: Registered Node addons-221000 in Controller

	* 

	* ==> dmesg <==

	* [  +0.641705] EINJ: EINJ table not found.

	[  +0.512113] systemd-fstab-generator[117]: Ignoring "noauto" for root device

	[  +0.044199] systemd[1]: systemd-journald.service: unit configures an IP firewall, but the local system does not support BPF/cgroup firewalling.

	[  +0.000795] systemd[1]: (This warning is only shown for the first unit using IP firewalling.)

	[  +3.076142] systemd-fstab-generator[480]: Ignoring "noauto" for root device

	[  +0.068934] systemd-fstab-generator[491]: Ignoring "noauto" for root device

	[  +0.415221] systemd-fstab-generator[666]: Ignoring "noauto" for root device

	[  +0.169313] systemd-fstab-generator[702]: Ignoring "noauto" for root device

	[  +0.075007] systemd-fstab-generator[713]: Ignoring "noauto" for root device

	[  +0.084392] systemd-fstab-generator[726]: Ignoring "noauto" for root device

	[  +1.144805] kauditd_printk_skb: 17 callbacks suppressed

	[  +0.094980] systemd-fstab-generator[914]: Ignoring "noauto" for root device

	[  +0.076494] systemd-fstab-generator[925]: Ignoring "noauto" for root device

	[  +0.074272] systemd-fstab-generator[936]: Ignoring "noauto" for root device

	[  +0.076208] systemd-fstab-generator[947]: Ignoring "noauto" for root device

	[  +0.088287] systemd-fstab-generator[988]: Ignoring "noauto" for root device

	[  +2.538937] systemd-fstab-generator[1093]: Ignoring "noauto" for root device

	[  +2.473246] kauditd_printk_skb: 29 callbacks suppressed

	[  +3.092849] systemd-fstab-generator[1411]: Ignoring "noauto" for root device

	[  +4.633357] systemd-fstab-generator[2291]: Ignoring "noauto" for root device

	[Sep18 18:53] kauditd_printk_skb: 41 callbacks suppressed

	[  +1.646092] TCP: eth0: Driver has suspect GRO implementation, TCP performance may be compromised.

	[  +4.523270] kauditd_printk_skb: 55 callbacks suppressed

	[ +11.548658] kauditd_printk_skb: 8 callbacks suppressed

	[Sep18 19:12] kauditd_printk_skb: 12 callbacks suppressed

	* 

	* ==> etcd [2a7ec1fe69df] <==

	* {"level":"info","ts":"2023-09-18T18:52:43.023404Z","caller":"etcdserver/server.go:2571","msg":"setting up initial cluster version using v2 API","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:52:43.023573Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2023-09-18T18:52:43.02399Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.105.2:2379"}

	{"level":"info","ts":"2023-09-18T18:52:43.024241Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}

	{"level":"info","ts":"2023-09-18T18:52:43.02431Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}

	{"level":"info","ts":"2023-09-18T18:52:43.024433Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"6e03e7863b4f9c54","local-member-id":"c46d288d2fcb0590","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:52:43.024491Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:52:43.025343Z","caller":"etcdserver/server.go:2595","msg":"cluster version is updated","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:53:12.839558Z","caller":"traceutil/trace.go:171","msg":"trace[499305584] linearizableReadLoop","detail":"{readStateIndex:693; appliedIndex:692; }","duration":"193.895541ms","start":"2023-09-18T18:53:12.645653Z","end":"2023-09-18T18:53:12.839549Z","steps":["trace[499305584] 'read index received'  (duration: 193.791791ms)","trace[499305584] 'applied index is now lower than readState.Index'  (duration: 103.417µs)"],"step_count":2}

	{"level":"info","ts":"2023-09-18T18:53:12.839607Z","caller":"traceutil/trace.go:171","msg":"trace[593448589] transaction","detail":"{read_only:false; response_revision:671; number_of_response:1; }","duration":"309.762625ms","start":"2023-09-18T18:53:12.529838Z","end":"2023-09-18T18:53:12.839601Z","steps":["trace[593448589] 'process raft request'  (duration: 309.63075ms)"],"step_count":1}

	{"level":"warn","ts":"2023-09-18T18:53:12.839655Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"193.989167ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/gcp-auth/\" range_end:\"/registry/pods/gcp-auth0\" ","response":"range_response_count:3 size:10532"}

	{"level":"info","ts":"2023-09-18T18:53:12.839671Z","caller":"traceutil/trace.go:171","msg":"trace[1312627899] range","detail":"{range_begin:/registry/pods/gcp-auth/; range_end:/registry/pods/gcp-auth0; response_count:3; response_revision:671; }","duration":"194.031ms","start":"2023-09-18T18:53:12.645637Z","end":"2023-09-18T18:53:12.839668Z","steps":["trace[1312627899] 'agreement among raft nodes before linearized reading'  (duration: 193.963833ms)"],"step_count":1}

	{"level":"warn","ts":"2023-09-18T18:53:12.8398Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2023-09-18T18:53:12.529832Z","time spent":"309.7915ms","remote":"127.0.0.1:54556","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1098,"response count":0,"response size":39,"request content":"compare:<target:MOD key:\"/registry/services/endpoints/kube-system/k8s.io-minikube-hostpath\" mod_revision:669 > success:<request_put:<key:\"/registry/services/endpoints/kube-system/k8s.io-minikube-hostpath\" value_size:1025 >> failure:<request_range:<key:\"/registry/services/endpoints/kube-system/k8s.io-minikube-hostpath\" > >"}

	{"level":"info","ts":"2023-09-18T18:53:15.058529Z","caller":"traceutil/trace.go:171","msg":"trace[42488278] linearizableReadLoop","detail":"{readStateIndex:694; appliedIndex:693; }","duration":"207.837416ms","start":"2023-09-18T18:53:14.850683Z","end":"2023-09-18T18:53:15.05852Z","steps":["trace[42488278] 'read index received'  (duration: 207.76525ms)","trace[42488278] 'applied index is now lower than readState.Index'  (duration: 71.708µs)"],"step_count":2}

	{"level":"warn","ts":"2023-09-18T18:53:15.058651Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"207.966125ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/\" range_end:\"/registry/pods/kube-system0\" ","response":"range_response_count:13 size:63849"}

	{"level":"info","ts":"2023-09-18T18:53:15.058696Z","caller":"traceutil/trace.go:171","msg":"trace[2008509018] range","detail":"{range_begin:/registry/pods/kube-system/; range_end:/registry/pods/kube-system0; response_count:13; response_revision:672; }","duration":"208.018666ms","start":"2023-09-18T18:53:14.850673Z","end":"2023-09-18T18:53:15.058691Z","steps":["trace[2008509018] 'agreement among raft nodes before linearized reading'  (duration: 207.886666ms)"],"step_count":1}

	{"level":"info","ts":"2023-09-18T18:53:15.058824Z","caller":"traceutil/trace.go:171","msg":"trace[1590636669] transaction","detail":"{read_only:false; response_revision:672; number_of_response:1; }","duration":"214.775084ms","start":"2023-09-18T18:53:14.844046Z","end":"2023-09-18T18:53:15.058821Z","steps":["trace[1590636669] 'process raft request'  (duration: 214.424584ms)"],"step_count":1}

	{"level":"info","ts":"2023-09-18T19:02:43.443637Z","caller":"mvcc/index.go:214","msg":"compact tree index","revision":1094}

	{"level":"info","ts":"2023-09-18T19:02:43.458272Z","caller":"mvcc/kvstore_compaction.go:66","msg":"finished scheduled compaction","compact-revision":1094,"took":"14.26046ms","hash":1108450172}

	{"level":"info","ts":"2023-09-18T19:02:43.458292Z","caller":"mvcc/hash.go:137","msg":"storing new hash","hash":1108450172,"revision":1094,"compact-revision":-1}

	{"level":"info","ts":"2023-09-18T19:07:43.446092Z","caller":"mvcc/index.go:214","msg":"compact tree index","revision":1454}

	{"level":"info","ts":"2023-09-18T19:07:43.44678Z","caller":"mvcc/kvstore_compaction.go:66","msg":"finished scheduled compaction","compact-revision":1454,"took":"494.332µs","hash":2695871163}

	{"level":"info","ts":"2023-09-18T19:07:43.446796Z","caller":"mvcc/hash.go:137","msg":"storing new hash","hash":2695871163,"revision":1454,"compact-revision":1094}

	{"level":"info","ts":"2023-09-18T19:11:06.589478Z","caller":"traceutil/trace.go:171","msg":"trace[847243809] transaction","detail":"{read_only:false; response_revision:2088; number_of_response:1; }","duration":"105.795246ms","start":"2023-09-18T19:11:06.483671Z","end":"2023-09-18T19:11:06.589466Z","steps":["trace[847243809] 'process raft request'  (duration: 76.680577ms)","trace[847243809] 'compare'  (duration: 28.988461ms)"],"step_count":2}

	{"level":"info","ts":"2023-09-18T19:11:19.569525Z","caller":"traceutil/trace.go:171","msg":"trace[199883351] transaction","detail":"{read_only:false; response_revision:2131; number_of_response:1; }","duration":"161.881881ms","start":"2023-09-18T19:11:19.407634Z","end":"2023-09-18T19:11:19.569516Z","steps":["trace[199883351] 'process raft request'  (duration: 161.812756ms)"],"step_count":1}

	* 

	* ==> gcp-auth [6009e365438d] <==

	* 2023/09/18 18:53:42 GCP Auth Webhook started!

	2023/09/18 19:11:01 Ready to marshal response ...

	2023/09/18 19:11:01 Ready to write response ...

	2023/09/18 19:11:01 Ready to marshal response ...

	2023/09/18 19:11:01 Ready to write response ...

	2023/09/18 19:11:01 Ready to marshal response ...

	2023/09/18 19:11:01 Ready to write response ...

	2023/09/18 19:11:15 Ready to marshal response ...

	2023/09/18 19:11:15 Ready to write response ...

	2023/09/18 19:11:48 Ready to marshal response ...

	2023/09/18 19:11:48 Ready to write response ...

	* 

	* ==> kernel <==

	*  19:12:19 up 19 min,  0 users,  load average: 0.23, 0.19, 0.18

	Linux addons-221000 5.10.57 #1 SMP PREEMPT Fri Sep 15 19:03:18 UTC 2023 aarch64 GNU/Linux

	PRETTY_NAME="Buildroot 2021.02.12"

	* 

	* ==> kube-apiserver [4a8eb16a561d] <==

	* I0918 19:10:44.200176       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:11:01.618081       1 alloc.go:330] "allocated clusterIPs" service="headlamp/headlamp" clusterIPs={"IPv4":"10.105.100.41"}

	I0918 19:11:27.087628       1 controller.go:624] quota admission added evaluator for: volumesnapshots.snapshot.storage.k8s.io

	I0918 19:11:44.199807       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.823931       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.823948       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.825604       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.825722       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.833256       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.833284       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.846841       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.846862       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.847044       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.847095       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.856162       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.856179       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.857357       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.857367       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	W0918 19:12:04.848160       1 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io

	W0918 19:12:04.857030       1 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io

	W0918 19:12:04.865591       1 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io

	I0918 19:12:11.428741       1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Nothing (removed from the queue).

	I0918 19:12:14.330149       1 handler.go:232] Adding GroupVersion gadget.kinvolk.io v1alpha1 to ResourceManager

	I0918 19:12:14.335152       1 handler.go:232] Adding GroupVersion gadget.kinvolk.io v1alpha1 to ResourceManager

	W0918 19:12:15.339237       1 cacher.go:171] Terminating all watchers from cacher traces.gadget.kinvolk.io

	* 

	* ==> kube-controller-manager [cb85fb8fd00c] <==

	* E0918 19:12:04.866260       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:05.815227       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:05.815247       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:06.314592       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:06.314612       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:06.404971       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:06.404987       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:07.765663       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:07.765681       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:09.044305       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:09.044327       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	I0918 19:12:09.124215       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kube-system/metrics-server-7c66d45ddc" duration="1.875µs"

	W0918 19:12:09.340711       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:09.340733       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:12.778356       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:12.778375       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:13.814310       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:13.814362       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:15.053536       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:15.053562       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:15.339981       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:16.531417       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:16.531440       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:18.180272       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:18.180294       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	* 

	* ==> kube-proxy [3b8b236037bf] <==

	* I0918 18:53:00.480730       1 server_others.go:69] "Using iptables proxy"

	I0918 18:53:00.488175       1 node.go:141] Successfully retrieved node IP: 192.168.105.2

	I0918 18:53:00.508601       1 server_others.go:121] "No iptables support for family" ipFamily="IPv6"

	I0918 18:53:00.508621       1 server.go:634] "kube-proxy running in single-stack mode" ipFamily="IPv4"

	I0918 18:53:00.509405       1 server_others.go:152] "Using iptables Proxier"

	I0918 18:53:00.509431       1 proxier.go:251] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses"

	I0918 18:53:00.509518       1 server.go:846] "Version info" version="v1.28.2"

	I0918 18:53:00.509524       1 server.go:848] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0918 18:53:00.512081       1 config.go:188] "Starting service config controller"

	I0918 18:53:00.512089       1 shared_informer.go:311] Waiting for caches to sync for service config

	I0918 18:53:00.512105       1 config.go:97] "Starting endpoint slice config controller"

	I0918 18:53:00.512107       1 shared_informer.go:311] Waiting for caches to sync for endpoint slice config

	I0918 18:53:00.512284       1 config.go:315] "Starting node config controller"

	I0918 18:53:00.512287       1 shared_informer.go:311] Waiting for caches to sync for node config

	I0918 18:53:01.015326       1 shared_informer.go:318] Caches are synced for node config

	I0918 18:53:01.015354       1 shared_informer.go:318] Caches are synced for service config

	I0918 18:53:01.015384       1 shared_informer.go:318] Caches are synced for endpoint slice config

	* 

	* ==> kube-scheduler [17d16f9191cb] <==

	* W0918 18:52:43.862760       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	E0918 18:52:43.863344       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	W0918 18:52:43.862808       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	E0918 18:52:43.863392       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	W0918 18:52:43.862823       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	E0918 18:52:43.863442       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	W0918 18:52:43.862835       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	E0918 18:52:43.863451       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	W0918 18:52:43.862850       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope

	E0918 18:52:43.863466       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope

	W0918 18:52:43.862867       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	E0918 18:52:43.863530       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	W0918 18:52:43.862878       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	E0918 18:52:43.863565       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	W0918 18:52:43.862692       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	E0918 18:52:43.863575       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	W0918 18:52:43.863300       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	E0918 18:52:43.863661       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	W0918 18:52:44.751836       1 reflector.go:535] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	E0918 18:52:44.751859       1 reflector.go:147] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	W0918 18:52:44.752492       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope

	E0918 18:52:44.752505       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope

	W0918 18:52:44.760405       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	E0918 18:52:44.760417       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	I0918 18:52:46.760908       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	* 

	* ==> kubelet <==

	* -- Journal begins at Mon 2023-09-18 18:52:28 UTC, ends at Mon 2023-09-18 19:12:20 UTC. --

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524125    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-run\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524134    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-host\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524142    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"bpffs\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-bpffs\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524150    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"modules\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-modules\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524157    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"debugfs\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-debugfs\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524165    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"cgroup\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-cgroup\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524209    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-cgroup" (OuterVolumeSpecName: "cgroup") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "cgroup". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524422    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-bpffs" (OuterVolumeSpecName: "bpffs") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "bpffs". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524436    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-run" (OuterVolumeSpecName: "run") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "run". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524443    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-host" (OuterVolumeSpecName: "host") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "host". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524450    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-modules" (OuterVolumeSpecName: "modules") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "modules". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524522    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-debugfs" (OuterVolumeSpecName: "debugfs") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "debugfs". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.525925    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eea1399-50e6-40e6-8424-bacd2f982bff-kube-api-access-pfdln" (OuterVolumeSpecName: "kube-api-access-pfdln") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "kube-api-access-pfdln". PluginName "kubernetes.io/projected", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625197    2297 reconciler_common.go:300] "Volume detached for volume \"bpffs\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-bpffs\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625210    2297 reconciler_common.go:300] "Volume detached for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-host\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625217    2297 reconciler_common.go:300] "Volume detached for volume \"cgroup\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-cgroup\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625222    2297 reconciler_common.go:300] "Volume detached for volume \"modules\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-modules\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625227    2297 reconciler_common.go:300] "Volume detached for volume \"debugfs\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-debugfs\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625233    2297 reconciler_common.go:300] "Volume detached for volume \"kube-api-access-pfdln\" (UniqueName: \"kubernetes.io/projected/7eea1399-50e6-40e6-8424-bacd2f982bff-kube-api-access-pfdln\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625238    2297 reconciler_common.go:300] "Volume detached for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-run\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:15 addons-221000 kubelet[2297]: I0918 19:12:15.398217    2297 scope.go:117] "RemoveContainer" containerID="afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a"

	Sep 18 19:12:15 addons-221000 kubelet[2297]: I0918 19:12:15.407228    2297 scope.go:117] "RemoveContainer" containerID="afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a"

	Sep 18 19:12:15 addons-221000 kubelet[2297]: E0918 19:12:15.407605    2297 remote_runtime.go:432] "ContainerStatus from runtime service failed" err="rpc error: code = Unknown desc = Error response from daemon: No such container: afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a" containerID="afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a"

	Sep 18 19:12:15 addons-221000 kubelet[2297]: I0918 19:12:15.407627    2297 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"docker","ID":"afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a"} err="failed to get container status \"afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a\": rpc error: code = Unknown desc = Error response from daemon: No such container: afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a"

	Sep 18 19:12:16 addons-221000 kubelet[2297]: I0918 19:12:16.788684    2297 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7eea1399-50e6-40e6-8424-bacd2f982bff" path="/var/lib/kubelet/pods/7eea1399-50e6-40e6-8424-bacd2f982bff/volumes"

	* 

	* ==> storage-provisioner [96b1e19d34d0] <==

	* I0918 18:53:02.494940       1 storage_provisioner.go:116] Initializing the minikube storage provisioner...

	I0918 18:53:02.503242       1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!

	I0918 18:53:02.503674       1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...

	I0918 18:53:02.507046       1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath

	I0918 18:53:02.507107       1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_addons-221000_d01c2a4d-820c-47b2-b527-e47d47f0ebf9!

	I0918 18:53:02.507893       1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"993ff549-fac0-4a25-b8bc-6e13c7f3eb70", APIVersion:"v1", ResourceVersion:"490", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' addons-221000_d01c2a4d-820c-47b2-b527-e47d47f0ebf9 became leader

	I0918 18:53:02.607443       1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_addons-221000_d01c2a4d-820c-47b2-b527-e47d47f0ebf9!

-- /stdout --

helpers_test.go:254: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.APIServer}} -p addons-221000 -n addons-221000

helpers_test.go:261: (dbg) Run:  kubectl --context addons-221000 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running

helpers_test.go:285: <<< TestAddons/parallel/Ingress FAILED: end of post-mortem logs <<<

helpers_test.go:286: ---------------------/post-mortem---------------------------------

--- FAIL: TestAddons/parallel/Ingress (0.78s)

=== RUN   TestAddons/parallel/CloudSpanner

=== PAUSE TestAddons/parallel/CloudSpanner

=== CONT  TestAddons/parallel/CloudSpanner

addons_test.go:831: failed waiting for cloud-spanner-emulator deployment to stabilize: timed out waiting for the condition

addons_test.go:833: (dbg) TestAddons/parallel/CloudSpanner: waiting 6m0s for pods matching "app=cloud-spanner-emulator" in namespace "default" ...

addons_test.go:833: ***** TestAddons/parallel/CloudSpanner: pod "app=cloud-spanner-emulator" failed to start within 6m0s: context deadline exceeded ****

addons_test.go:833: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.APIServer}} -p addons-221000 -n addons-221000

addons_test.go:833: TestAddons/parallel/CloudSpanner: showing logs for failed pods as of 2023-09-18 12:11:00.386318 -0700 PDT m=+1156.802890167

addons_test.go:834: failed waiting for app=cloud-spanner-emulator pod: app=cloud-spanner-emulator within 6m0s: context deadline exceeded

addons_test.go:836: (dbg) Run:  out/minikube-darwin-arm64 addons disable cloud-spanner -p addons-221000

addons_test.go:836: (dbg) Non-zero exit: out/minikube-darwin-arm64 addons disable cloud-spanner -p addons-221000: exit status 10 (1m24.272363125s)

-- stdout --

-- /stdout --

** stderr ** 

	X Exiting due to MK_ADDON_DISABLE: disable failed: run callbacks: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl delete --force --ignore-not-found -f /etc/kubernetes/addons/deployment.yaml: Process exited with status 1

	stdout:

	stderr:

	error: the path "/etc/kubernetes/addons/deployment.yaml" does not exist

	]

	* 

	╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                           │

	│    * If the above advice does not help, please let us know:                                                               │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                             │

	│                                                                                                                           │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                  │

	│    * Please also attach the following file to the GitHub issue:                                                           │

	│    * - /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/minikube_addons_e93ff976b7e98e1dc466aded9385c0856b6d1b41_0.log    │

	│                                                                                                                           │

	╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

addons_test.go:837: failed to disable cloud-spanner addon: args "out/minikube-darwin-arm64 addons disable cloud-spanner -p addons-221000" : exit status 10

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p addons-221000 -n addons-221000

helpers_test.go:244: <<< TestAddons/parallel/CloudSpanner FAILED: start of post-mortem logs <<<

helpers_test.go:245: ======>  post-mortem[TestAddons/parallel/CloudSpanner]: minikube logs <======

helpers_test.go:247: (dbg) Run:  out/minikube-darwin-arm64 -p addons-221000 logs -n 25

helpers_test.go:252: TestAddons/parallel/CloudSpanner logs: 

-- stdout --

	* 

	* ==> Audit <==

	* |---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| Command |              Args              |       Profile        |  User   | Version |     Start Time      |      End Time       |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| start   | -o=json --download-only        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:51 PDT |                     |

	|         | -p download-only-242000        |                      |         |         |                     |                     |

	|         | --force --alsologtostderr      |                      |         |         |                     |                     |

	|         | --kubernetes-version=v1.16.0   |                      |         |         |                     |                     |

	|         | --container-runtime=docker     |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	| start   | -o=json --download-only        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT |                     |

	|         | -p download-only-242000        |                      |         |         |                     |                     |

	|         | --force --alsologtostderr      |                      |         |         |                     |                     |

	|         | --kubernetes-version=v1.28.2   |                      |         |         |                     |                     |

	|         | --container-runtime=docker     |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	| delete  | --all                          | minikube             | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| delete  | -p download-only-242000        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| delete  | -p download-only-242000        | download-only-242000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| start   | --download-only -p             | binary-mirror-077000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT |                     |

	|         | binary-mirror-077000           |                      |         |         |                     |                     |

	|         | --alsologtostderr              |                      |         |         |                     |                     |

	|         | --binary-mirror                |                      |         |         |                     |                     |

	|         | http://127.0.0.1:49414         |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	| delete  | -p binary-mirror-077000        | binary-mirror-077000 | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:52 PDT |

	| start   | -p addons-221000               | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 11:52 PDT | 18 Sep 23 11:59 PDT |

	|         | --wait=true --memory=4000      |                      |         |         |                     |                     |

	|         | --alsologtostderr              |                      |         |         |                     |                     |

	|         | --addons=registry              |                      |         |         |                     |                     |

	|         | --addons=metrics-server        |                      |         |         |                     |                     |

	|         | --addons=volumesnapshots       |                      |         |         |                     |                     |

	|         | --addons=csi-hostpath-driver   |                      |         |         |                     |                     |

	|         | --addons=gcp-auth              |                      |         |         |                     |                     |

	|         | --addons=cloud-spanner         |                      |         |         |                     |                     |

	|         | --addons=inspektor-gadget      |                      |         |         |                     |                     |

	|         | --driver=qemu2                 |                      |         |         |                     |                     |

	|         | --addons=ingress               |                      |         |         |                     |                     |

	|         | --addons=ingress-dns           |                      |         |         |                     |                     |

	| addons  | disable cloud-spanner -p       | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:11 PDT |                     |

	|         | addons-221000                  |                      |         |         |                     |                     |

	| addons  | enable headlamp                | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:11 PDT | 18 Sep 23 12:11 PDT |

	|         | -p addons-221000               |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1         |                      |         |         |                     |                     |

	| addons  | addons-221000 addons           | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:11 PDT | 18 Sep 23 12:12 PDT |

	|         | disable csi-hostpath-driver    |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1         |                      |         |         |                     |                     |

	| addons  | addons-221000 addons           | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:12 PDT | 18 Sep 23 12:12 PDT |

	|         | disable volumesnapshots        |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1         |                      |         |         |                     |                     |

	| addons  | addons-221000 addons           | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:12 PDT | 18 Sep 23 12:12 PDT |

	|         | disable metrics-server         |                      |         |         |                     |                     |

	|         | --alsologtostderr -v=1         |                      |         |         |                     |                     |

	| addons  | disable inspektor-gadget -p    | addons-221000        | jenkins | v1.31.2 | 18 Sep 23 12:12 PDT | 18 Sep 23 12:12 PDT |

	|         | addons-221000                  |                      |         |         |                     |                     |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	* 

	* ==> Last Start <==

	* Log file created at: 2023/09/18 11:52:16

	Running on machine: MacOS-M1-Agent-1

	Binary: Built with gc go1.21.1 for darwin/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0918 11:52:16.711602    1740 out.go:296] Setting OutFile to fd 1 ...

	I0918 11:52:16.711748    1740 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 11:52:16.711751    1740 out.go:309] Setting ErrFile to fd 2...

	I0918 11:52:16.711753    1740 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 11:52:16.711880    1740 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17263-1251/.minikube/bin

	I0918 11:52:16.712918    1740 out.go:303] Setting JSON to false

	I0918 11:52:16.728001    1740 start.go:128] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":1310,"bootTime":1695061826,"procs":408,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.5.2","kernelVersion":"22.6.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0918 11:52:16.728087    1740 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0918 11:52:16.732378    1740 out.go:177] * [addons-221000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	I0918 11:52:16.739454    1740 out.go:177]   - MINIKUBE_LOCATION=17263

	I0918 11:52:16.743421    1740 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 11:52:16.739507    1740 notify.go:220] Checking for updates...

	I0918 11:52:16.749403    1740 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0918 11:52:16.752377    1740 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0918 11:52:16.755381    1740 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	I0918 11:52:16.758417    1740 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0918 11:52:16.761446    1740 driver.go:373] Setting default libvirt URI to qemu:///system

	I0918 11:52:16.765371    1740 out.go:177] * Using the qemu2 driver based on user configuration

	I0918 11:52:16.777355    1740 start.go:298] selected driver: qemu2

	I0918 11:52:16.777364    1740 start.go:902] validating driver "qemu2" against <nil>

	I0918 11:52:16.777372    1740 start.go:913] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0918 11:52:16.779390    1740 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0918 11:52:16.782385    1740 out.go:177] * Automatically selected the socket_vmnet network

	I0918 11:52:16.785462    1740 start_flags.go:922] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0918 11:52:16.785488    1740 cni.go:84] Creating CNI manager for ""

	I0918 11:52:16.785496    1740 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 11:52:16.785507    1740 start_flags.go:316] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0918 11:52:16.785513    1740 start_flags.go:321] config:

	{Name:addons-221000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CR

ISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID

:0 AutoPauseInterval:1m0s}

	I0918 11:52:16.789634    1740 iso.go:125] acquiring lock: {Name:mk34e23c181861c65264ec384f06e6cfa001aa08 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0918 11:52:16.798394    1740 out.go:177] * Starting control plane node addons-221000 in cluster addons-221000

	I0918 11:52:16.802195    1740 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 11:52:16.802217    1740 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4

	I0918 11:52:16.802234    1740 cache.go:57] Caching tarball of preloaded images

	I0918 11:52:16.802301    1740 preload.go:174] Found /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0918 11:52:16.802315    1740 cache.go:60] Finished verifying existence of preloaded tar for  v1.28.2 on docker

	I0918 11:52:16.802542    1740 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/config.json ...

	I0918 11:52:16.802555    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/config.json: {Name:mk6624c585fbc7911138df2cd59d1f2e10251cf1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:16.802799    1740 start.go:365] acquiring machines lock for addons-221000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 11:52:16.802873    1740 start.go:369] acquired machines lock for "addons-221000" in 68.417µs

	I0918 11:52:16.802886    1740 start.go:93] Provisioning new machine with config: &{Name:addons-221000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfi

g:{KubernetesVersion:v1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262

144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 11:52:16.802925    1740 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 11:52:16.810242    1740 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...

	I0918 11:52:17.161676    1740 start.go:159] libmachine.API.Create for "addons-221000" (driver="qemu2")

	I0918 11:52:17.161722    1740 client.go:168] LocalClient.Create starting

	I0918 11:52:17.161932    1740 main.go:141] libmachine: Creating CA: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 11:52:17.253776    1740 main.go:141] libmachine: Creating client certificate: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 11:52:17.312301    1740 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 11:52:17.776256    1740 main.go:141] libmachine: Creating SSH key...

	I0918 11:52:17.897328    1740 main.go:141] libmachine: Creating Disk image...

	I0918 11:52:17.897334    1740 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 11:52:17.897524    1740 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2

	I0918 11:52:17.933044    1740 main.go:141] libmachine: STDOUT: 

	I0918 11:52:17.933072    1740 main.go:141] libmachine: STDERR: 

	I0918 11:52:17.933136    1740 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2 +20000M

	I0918 11:52:17.940597    1740 main.go:141] libmachine: STDOUT: Image resized.

	I0918 11:52:17.940609    1740 main.go:141] libmachine: STDERR: 

	I0918 11:52:17.940623    1740 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2

	I0918 11:52:17.940628    1740 main.go:141] libmachine: Starting QEMU VM...

	I0918 11:52:17.940657    1740 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 4000 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/qemu.pid -device virtio-net-pci,netdev=net0,mac=ce:ae:e8:0a:fd:16 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/disk.qcow2

	I0918 11:52:18.008779    1740 main.go:141] libmachine: STDOUT: 

	I0918 11:52:18.008804    1740 main.go:141] libmachine: STDERR: 

	I0918 11:52:18.008808    1740 main.go:141] libmachine: Attempt 0

	I0918 11:52:18.008820    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:20.011046    1740 main.go:141] libmachine: Attempt 1

	I0918 11:52:20.011124    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:22.013479    1740 main.go:141] libmachine: Attempt 2

	I0918 11:52:22.013559    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:24.015632    1740 main.go:141] libmachine: Attempt 3

	I0918 11:52:24.015645    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:26.017675    1740 main.go:141] libmachine: Attempt 4

	I0918 11:52:26.017681    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:28.018761    1740 main.go:141] libmachine: Attempt 5

	I0918 11:52:28.018782    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:30.020886    1740 main.go:141] libmachine: Attempt 6

	I0918 11:52:30.020920    1740 main.go:141] libmachine: Searching for ce:ae:e8:a:fd:16 in /var/db/dhcpd_leases ...

	I0918 11:52:30.021070    1740 main.go:141] libmachine: Found 1 entries in /var/db/dhcpd_leases!

	I0918 11:52:30.021123    1740 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:ce:ae:e8:a:fd:16 ID:1,ce:ae:e8:a:fd:16 Lease:0x6509edec}

	I0918 11:52:30.021130    1740 main.go:141] libmachine: Found match: ce:ae:e8:a:fd:16

	I0918 11:52:30.021140    1740 main.go:141] libmachine: IP: 192.168.105.2

	I0918 11:52:30.021147    1740 main.go:141] libmachine: Waiting for VM to start (ssh -p 22 docker@192.168.105.2)...

	I0918 11:52:31.026067    1740 machine.go:88] provisioning docker machine ...

	I0918 11:52:31.026085    1740 buildroot.go:166] provisioning hostname "addons-221000"

	I0918 11:52:31.026964    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.027231    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.027237    1740 main.go:141] libmachine: About to run SSH command:

	sudo hostname addons-221000 && echo "addons-221000" | sudo tee /etc/hostname

	I0918 11:52:31.084404    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: addons-221000

	I0918 11:52:31.084473    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.084732    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.084740    1740 main.go:141] libmachine: About to run SSH command:

			if ! grep -xq '.*\saddons-221000' /etc/hosts; then

				if grep -xq '127.0.1.1\s.*' /etc/hosts; then

					sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 addons-221000/g' /etc/hosts;

				else 

					echo '127.0.1.1 addons-221000' | sudo tee -a /etc/hosts; 

				fi

			fi

	I0918 11:52:31.144009    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	I0918 11:52:31.144022    1740 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/17263-1251/.minikube CaCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/17263-1251/.minikube}

	I0918 11:52:31.144033    1740 buildroot.go:174] setting up certificates

	I0918 11:52:31.144038    1740 provision.go:83] configureAuth start

	I0918 11:52:31.144042    1740 provision.go:138] copyHostCerts

	I0918 11:52:31.144138    1740 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.pem (1082 bytes)

	I0918 11:52:31.144342    1740 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/cert.pem (1123 bytes)

	I0918 11:52:31.144435    1740 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/key.pem (1679 bytes)

	I0918 11:52:31.144503    1740 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem org=jenkins.addons-221000 san=[192.168.105.2 192.168.105.2 localhost 127.0.0.1 minikube addons-221000]

	I0918 11:52:31.225327    1740 provision.go:172] copyRemoteCerts

	I0918 11:52:31.225385    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker

	I0918 11:52:31.225394    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.256352    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)

	I0918 11:52:31.263330    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem --> /etc/docker/server.pem (1216 bytes)

	I0918 11:52:31.270197    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)

	I0918 11:52:31.276715    1740 provision.go:86] duration metric: configureAuth took 132.670667ms

	I0918 11:52:31.276723    1740 buildroot.go:189] setting minikube options for container-runtime

	I0918 11:52:31.276820    1740 config.go:182] Loaded profile config "addons-221000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 11:52:31.276857    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.277075    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.277080    1740 main.go:141] libmachine: About to run SSH command:

	df --output=fstype / | tail -n 1

	I0918 11:52:31.337901    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs

	I0918 11:52:31.337910    1740 buildroot.go:70] root file system type: tmpfs

	I0918 11:52:31.337970    1740 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...

	I0918 11:52:31.338012    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.338275    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.338315    1740 main.go:141] libmachine: About to run SSH command:

	sudo mkdir -p /lib/systemd/system && printf %!s(MISSING) "[Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP \$MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	" | sudo tee /lib/systemd/system/docker.service.new

	I0918 11:52:31.400816    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP $MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	I0918 11:52:31.400863    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.401116    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.401126    1740 main.go:141] libmachine: About to run SSH command:

	sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }

	I0918 11:52:31.746670    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory

	Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

	I0918 11:52:31.746684    1740 machine.go:91] provisioned docker machine in 720.613ms

	I0918 11:52:31.746690    1740 client.go:171] LocalClient.Create took 14.585099291s

	I0918 11:52:31.746703    1740 start.go:167] duration metric: libmachine.API.Create for "addons-221000" took 14.585173417s

	I0918 11:52:31.746707    1740 start.go:300] post-start starting for "addons-221000" (driver="qemu2")

	I0918 11:52:31.746711    1740 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]

	I0918 11:52:31.746780    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs

	I0918 11:52:31.746790    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.775601    1740 ssh_runner.go:195] Run: cat /etc/os-release

	I0918 11:52:31.776975    1740 info.go:137] Remote host: Buildroot 2021.02.12

	I0918 11:52:31.776983    1740 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17263-1251/.minikube/addons for local assets ...

	I0918 11:52:31.777055    1740 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17263-1251/.minikube/files for local assets ...

	I0918 11:52:31.777083    1740 start.go:303] post-start completed in 30.374292ms

	I0918 11:52:31.777437    1740 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/config.json ...

	I0918 11:52:31.777603    1740 start.go:128] duration metric: createHost completed in 14.974815417s

	I0918 11:52:31.777667    1740 main.go:141] libmachine: Using SSH client type: native

	I0918 11:52:31.777884    1740 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100c60760] 0x100c62ed0 <nil>  [] 0s} 192.168.105.2 22 <nil> <nil>}

	I0918 11:52:31.777888    1740 main.go:141] libmachine: About to run SSH command:

	date +%!s(MISSING).%!N(MISSING)

	I0918 11:52:31.833629    1740 main.go:141] libmachine: SSH cmd err, output: <nil>: 1695063151.431021085

	I0918 11:52:31.833635    1740 fix.go:206] guest clock: 1695063151.431021085

	I0918 11:52:31.833638    1740 fix.go:219] Guest: 2023-09-18 11:52:31.431021085 -0700 PDT Remote: 2023-09-18 11:52:31.777608 -0700 PDT m=+15.083726834 (delta=-346.586915ms)

	I0918 11:52:31.833654    1740 fix.go:190] guest clock delta is within tolerance: -346.586915ms

	I0918 11:52:31.833656    1740 start.go:83] releasing machines lock for "addons-221000", held for 15.0309195s

	I0918 11:52:31.833905    1740 ssh_runner.go:195] Run: cat /version.json

	I0918 11:52:31.833915    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.833930    1740 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/

	I0918 11:52:31.833973    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:52:31.902596    1740 ssh_runner.go:195] Run: systemctl --version

	I0918 11:52:31.904777    1740 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"

	W0918 11:52:31.906638    1740 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found

	I0918 11:52:31.906668    1740 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;

	I0918 11:52:31.911697    1740 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)

	I0918 11:52:31.911704    1740 start.go:469] detecting cgroup driver to use...

	I0918 11:52:31.911799    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock

	" | sudo tee /etc/crictl.yaml"

	I0918 11:52:31.917320    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"

	I0918 11:52:31.920500    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"

	I0918 11:52:31.923811    1740 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...

	I0918 11:52:31.923843    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"

	I0918 11:52:31.926950    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0918 11:52:31.929666    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"

	I0918 11:52:31.932680    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0918 11:52:31.936002    1740 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"

	I0918 11:52:31.939362    1740 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"

	I0918 11:52:31.942186    1740 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables

	I0918 11:52:31.944734    1740 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

	I0918 11:52:31.947664    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:32.027464    1740 ssh_runner.go:195] Run: sudo systemctl restart containerd

	I0918 11:52:32.036559    1740 start.go:469] detecting cgroup driver to use...

	I0918 11:52:32.036614    1740 ssh_runner.go:195] Run: sudo systemctl cat docker.service

	I0918 11:52:32.042440    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0918 11:52:32.047583    1740 ssh_runner.go:195] Run: sudo systemctl stop -f containerd

	I0918 11:52:32.053840    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0918 11:52:32.058225    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I0918 11:52:32.062305    1740 ssh_runner.go:195] Run: sudo systemctl stop -f crio

	I0918 11:52:32.098720    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I0918 11:52:32.103440    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock

	" | sudo tee /etc/crictl.yaml"

	I0918 11:52:32.108844    1740 ssh_runner.go:195] Run: which cri-dockerd

	I0918 11:52:32.110173    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d

	I0918 11:52:32.112731    1740 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)

	I0918 11:52:32.117532    1740 ssh_runner.go:195] Run: sudo systemctl unmask docker.service

	I0918 11:52:32.194769    1740 ssh_runner.go:195] Run: sudo systemctl enable docker.socket

	I0918 11:52:32.269401    1740 docker.go:535] configuring docker to use "cgroupfs" as cgroup driver...

	I0918 11:52:32.269417    1740 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)

	I0918 11:52:32.274373    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:32.355030    1740 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0918 11:52:33.517984    1740 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.1629475s)

	I0918 11:52:33.518044    1740 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I0918 11:52:33.595160    1740 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket

	I0918 11:52:33.670332    1740 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I0918 11:52:33.746578    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:33.822625    1740 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket

	I0918 11:52:33.829925    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:33.909957    1740 ssh_runner.go:195] Run: sudo systemctl restart cri-docker

	I0918 11:52:33.933398    1740 start.go:516] Will wait 60s for socket path /var/run/cri-dockerd.sock

	I0918 11:52:33.933487    1740 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock

	I0918 11:52:33.935757    1740 start.go:537] Will wait 60s for crictl version

	I0918 11:52:33.935801    1740 ssh_runner.go:195] Run: which crictl

	I0918 11:52:33.937147    1740 ssh_runner.go:195] Run: sudo /usr/bin/crictl version

	I0918 11:52:33.952602    1740 start.go:553] Version:  0.1.0

	RuntimeName:  docker

	RuntimeVersion:  24.0.6

	RuntimeApiVersion:  v1alpha2

	I0918 11:52:33.952673    1740 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0918 11:52:33.962082    1740 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0918 11:52:33.975334    1740 out.go:204] * Preparing Kubernetes v1.28.2 on Docker 24.0.6 ...

	I0918 11:52:33.975416    1740 ssh_runner.go:195] Run: grep 192.168.105.1	host.minikube.internal$ /etc/hosts

	I0918 11:52:33.976970    1740 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.105.1	host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0918 11:52:33.980853    1740 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 11:52:33.980897    1740 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0918 11:52:33.986139    1740 docker.go:636] Got preloaded images: 

	I0918 11:52:33.986147    1740 docker.go:642] registry.k8s.io/kube-apiserver:v1.28.2 wasn't preloaded

	I0918 11:52:33.986189    1740 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0918 11:52:33.988954    1740 ssh_runner.go:195] Run: which lz4

	I0918 11:52:33.990479    1740 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4

	I0918 11:52:33.991766    1740 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4: Process exited with status 1

	stdout:

	stderr:

	stat: cannot statx '/preloaded.tar.lz4': No such file or directory

	I0918 11:52:33.991780    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 --> /preloaded.tar.lz4 (356993689 bytes)

	I0918 11:52:35.310513    1740 docker.go:600] Took 1.320057 seconds to copy over tarball

	I0918 11:52:35.310582    1740 ssh_runner.go:195] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4

	I0918 11:52:36.348518    1740 ssh_runner.go:235] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (1.03793175s)

	I0918 11:52:36.348535    1740 ssh_runner.go:146] rm: /preloaded.tar.lz4

	I0918 11:52:36.364745    1740 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0918 11:52:36.368295    1740 ssh_runner.go:362] scp memory --> /var/lib/docker/image/overlay2/repositories.json (2629 bytes)

	I0918 11:52:36.373429    1740 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 11:52:36.450305    1740 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0918 11:52:38.940530    1740 ssh_runner.go:235] Completed: sudo systemctl restart docker: (2.490231167s)

	I0918 11:52:38.940627    1740 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0918 11:52:38.946699    1740 docker.go:636] Got preloaded images: -- stdout --

	registry.k8s.io/kube-apiserver:v1.28.2

	registry.k8s.io/kube-scheduler:v1.28.2

	registry.k8s.io/kube-proxy:v1.28.2

	registry.k8s.io/kube-controller-manager:v1.28.2

	registry.k8s.io/etcd:3.5.9-0

	registry.k8s.io/coredns/coredns:v1.10.1

	registry.k8s.io/pause:3.9

	gcr.io/k8s-minikube/storage-provisioner:v5

	-- /stdout --

	I0918 11:52:38.946709    1740 cache_images.go:84] Images are preloaded, skipping loading

	I0918 11:52:38.946766    1740 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}

	I0918 11:52:38.954428    1740 cni.go:84] Creating CNI manager for ""

	I0918 11:52:38.954439    1740 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 11:52:38.954458    1740 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16

	I0918 11:52:38.954467    1740 kubeadm.go:176] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.105.2 APIServerPort:8443 KubernetesVersion:v1.28.2 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:addons-221000 NodeName:addons-221000 DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.105.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.105.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/ku

bernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}

	I0918 11:52:38.954540    1740 kubeadm.go:181] kubeadm config:

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: InitConfiguration

	localAPIEndpoint:

	  advertiseAddress: 192.168.105.2

	  bindPort: 8443

	bootstrapTokens:

	  - groups:

	      - system:bootstrappers:kubeadm:default-node-token

	    ttl: 24h0m0s

	    usages:

	      - signing

	      - authentication

	nodeRegistration:

	  criSocket: unix:///var/run/cri-dockerd.sock

	  name: "addons-221000"

	  kubeletExtraArgs:

	    node-ip: 192.168.105.2

	  taints: []

	---

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: ClusterConfiguration

	apiServer:

	  certSANs: ["127.0.0.1", "localhost", "192.168.105.2"]

	  extraArgs:

	    enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"

	controllerManager:

	  extraArgs:

	    allocate-node-cidrs: "true"

	    leader-elect: "false"

	scheduler:

	  extraArgs:

	    leader-elect: "false"

	certificatesDir: /var/lib/minikube/certs

	clusterName: mk

	controlPlaneEndpoint: control-plane.minikube.internal:8443

	etcd:

	  local:

	    dataDir: /var/lib/minikube/etcd

	    extraArgs:

	      proxy-refresh-interval: "70000"

	kubernetesVersion: v1.28.2

	networking:

	  dnsDomain: cluster.local

	  podSubnet: "10.244.0.0/16"

	  serviceSubnet: 10.96.0.0/12

	---

	apiVersion: kubelet.config.k8s.io/v1beta1

	kind: KubeletConfiguration

	authentication:

	  x509:

	    clientCAFile: /var/lib/minikube/certs/ca.crt

	cgroupDriver: cgroupfs

	hairpinMode: hairpin-veth

	runtimeRequestTimeout: 15m

	clusterDomain: "cluster.local"

	# disable disk resource management by default

	imageGCHighThresholdPercent: 100

	evictionHard:

	  nodefs.available: "0%!"(MISSING)

	  nodefs.inodesFree: "0%!"(MISSING)

	  imagefs.available: "0%!"(MISSING)

	failSwapOn: false

	staticPodPath: /etc/kubernetes/manifests

	---

	apiVersion: kubeproxy.config.k8s.io/v1alpha1

	kind: KubeProxyConfiguration

	clusterCIDR: "10.244.0.0/16"

	metricsBindAddress: 0.0.0.0:10249

	conntrack:

	  maxPerCore: 0

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"

	  tcpEstablishedTimeout: 0s

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"

	  tcpCloseWaitTimeout: 0s

	I0918 11:52:38.954592    1740 kubeadm.go:976] kubelet [Unit]

	Wants=docker.socket

	[Service]

	ExecStart=

	ExecStart=/var/lib/minikube/binaries/v1.28.2/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --hostname-override=addons-221000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.105.2

	[Install]

	 config:

	{KubernetesVersion:v1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}

	I0918 11:52:38.954661    1740 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.2

	I0918 11:52:38.957531    1740 binaries.go:44] Found k8s binaries, skipping transfer

	I0918 11:52:38.957562    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube

	I0918 11:52:38.960385    1740 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (375 bytes)

	I0918 11:52:38.965592    1740 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)

	I0918 11:52:38.970301    1740 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2099 bytes)

	I0918 11:52:38.975165    1740 ssh_runner.go:195] Run: grep 192.168.105.2	control-plane.minikube.internal$ /etc/hosts

	I0918 11:52:38.976400    1740 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.105.2	control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0918 11:52:38.980255    1740 certs.go:56] Setting up /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000 for IP: 192.168.105.2

	I0918 11:52:38.980276    1740 certs.go:190] acquiring lock for shared ca certs: {Name:mkfac81ee65979b8c4f5ece6243c3a0190531689 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:38.980470    1740 certs.go:204] generating minikubeCA CA: /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key

	I0918 11:52:39.170828    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt ...

	I0918 11:52:39.170844    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt: {Name:mk0f303ee67627c25d1d04e1887861f15cdad763 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.171150    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key ...

	I0918 11:52:39.171155    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key: {Name:mkc5e20e8161cfdcfc3d5dcd8300765ea2c12112 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.171271    1740 certs.go:204] generating proxyClientCA CA: /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key

	I0918 11:52:39.287022    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.crt ...

	I0918 11:52:39.287027    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.crt: {Name:mk54c49c3c44ff09930e6c0f57238b89cff4c5e9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.287171    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key ...

	I0918 11:52:39.287173    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key: {Name:mk05faae5769358f82565f32c1f37a244f2478c1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.287315    1740 certs.go:319] generating minikube-user signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.key

	I0918 11:52:39.287337    1740 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.crt with IP's: []

	I0918 11:52:39.376234    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.crt ...

	I0918 11:52:39.376241    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.crt: {Name:mkc8e654c6f2522197f557cb47d266f15eebaadd Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.376467    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.key ...

	I0918 11:52:39.376471    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.key: {Name:mkf345dd56f86115b31ecd965617f4c21d6a0cd8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.376571    1740 certs.go:319] generating minikube signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969

	I0918 11:52:39.376580    1740 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969 with IP's: [192.168.105.2 10.96.0.1 127.0.0.1 10.0.0.1]

	I0918 11:52:39.429944    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969 ...

	I0918 11:52:39.429952    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969: {Name:mkd69eb587bd0dc6ccdbaa88b78f4f92f2b47b12 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.430095    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969 ...

	I0918 11:52:39.430098    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969: {Name:mk76ea3a7fbbef2305f74e52afdf06cda921c8bb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.430199    1740 certs.go:337] copying /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt.96055969 -> /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt

	I0918 11:52:39.430382    1740 certs.go:341] copying /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key.96055969 -> /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key

	I0918 11:52:39.430499    1740 certs.go:319] generating aggregator signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key

	I0918 11:52:39.430509    1740 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt with IP's: []

	I0918 11:52:39.698555    1740 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt ...

	I0918 11:52:39.698563    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt: {Name:mk6d7a924ed10f0012b290ec4e0ea6bf1b7bfc02 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.698767    1740 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key ...

	I0918 11:52:39.698773    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key: {Name:mk8d78e9179e4c57e4602e98d4fc6a37885b4d01 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:52:39.699037    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem (1679 bytes)

	I0918 11:52:39.699062    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem (1082 bytes)

	I0918 11:52:39.699081    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem (1123 bytes)

	I0918 11:52:39.699100    1740 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem (1679 bytes)

	I0918 11:52:39.699418    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)

	I0918 11:52:39.707305    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)

	I0918 11:52:39.713956    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)

	I0918 11:52:39.720614    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)

	I0918 11:52:39.727643    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)

	I0918 11:52:39.734609    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)

	I0918 11:52:39.741305    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)

	I0918 11:52:39.748141    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)

	I0918 11:52:39.755277    1740 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)

	I0918 11:52:39.762025    1740 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)

	I0918 11:52:39.767701    1740 ssh_runner.go:195] Run: openssl version

	I0918 11:52:39.769871    1740 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"

	I0918 11:52:39.772909    1740 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem

	I0918 11:52:39.774536    1740 certs.go:480] hashing: -rw-r--r-- 1 root root 1111 Sep 18 18:52 /usr/share/ca-certificates/minikubeCA.pem

	I0918 11:52:39.774555    1740 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem

	I0918 11:52:39.776417    1740 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"

	I0918 11:52:39.779698    1740 ssh_runner.go:195] Run: ls /var/lib/minikube/certs/etcd

	I0918 11:52:39.781162    1740 certs.go:353] certs directory doesn't exist, likely first start: ls /var/lib/minikube/certs/etcd: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/var/lib/minikube/certs/etcd': No such file or directory

	I0918 11:52:39.781200    1740 kubeadm.go:404] StartCluster: {Name:addons-221000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v

1.28.2 ClusterName:addons-221000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.105.2 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 Mount

Options:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 11:52:39.781263    1740 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}

	I0918 11:52:39.787256    1740 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd

	I0918 11:52:39.790167    1740 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml

	I0918 11:52:39.792879    1740 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf

	I0918 11:52:39.795890    1740 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory

	I0918 11:52:39.795906    1740 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"

	I0918 11:52:39.820130    1740 kubeadm.go:322] [init] Using Kubernetes version: v1.28.2

	I0918 11:52:39.820157    1740 kubeadm.go:322] [preflight] Running pre-flight checks

	I0918 11:52:39.874262    1740 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster

	I0918 11:52:39.874320    1740 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection

	I0918 11:52:39.874401    1740 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	I0918 11:52:39.936649    1740 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"

	I0918 11:52:39.946863    1740 out.go:204]   - Generating certificates and keys ...

	I0918 11:52:39.946906    1740 kubeadm.go:322] [certs] Using existing ca certificate authority

	I0918 11:52:39.946940    1740 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk

	I0918 11:52:40.057135    1740 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key

	I0918 11:52:40.267412    1740 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key

	I0918 11:52:40.415260    1740 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key

	I0918 11:52:40.592293    1740 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key

	I0918 11:52:40.714190    1740 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key

	I0918 11:52:40.714252    1740 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [addons-221000 localhost] and IPs [192.168.105.2 127.0.0.1 ::1]

	I0918 11:52:40.818329    1740 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key

	I0918 11:52:40.818397    1740 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [addons-221000 localhost] and IPs [192.168.105.2 127.0.0.1 ::1]

	I0918 11:52:41.068370    1740 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key

	I0918 11:52:41.110794    1740 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key

	I0918 11:52:41.218301    1740 kubeadm.go:322] [certs] Generating "sa" key and public key

	I0918 11:52:41.218335    1740 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	I0918 11:52:41.282421    1740 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file

	I0918 11:52:41.650315    1740 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file

	I0918 11:52:41.733907    1740 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file

	I0918 11:52:41.925252    1740 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file

	I0918 11:52:41.925561    1740 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	I0918 11:52:41.927413    1740 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"

	I0918 11:52:41.931680    1740 out.go:204]   - Booting up control plane ...

	I0918 11:52:41.931754    1740 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"

	I0918 11:52:41.931794    1740 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"

	I0918 11:52:41.931831    1740 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"

	I0918 11:52:41.935171    1740 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	I0918 11:52:41.935565    1740 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	I0918 11:52:41.935586    1740 kubeadm.go:322] [kubelet-start] Starting the kubelet

	I0918 11:52:42.024365    1740 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	I0918 11:52:45.527756    1740 kubeadm.go:322] [apiclient] All control plane components are healthy after 3.503453 seconds

	I0918 11:52:45.527852    1740 kubeadm.go:322] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace

	I0918 11:52:45.533290    1740 kubeadm.go:322] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster

	I0918 11:52:46.043984    1740 kubeadm.go:322] [upload-certs] Skipping phase. Please see --upload-certs

	I0918 11:52:46.044088    1740 kubeadm.go:322] [mark-control-plane] Marking the node addons-221000 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]

	I0918 11:52:46.548611    1740 kubeadm.go:322] [bootstrap-token] Using token: 0otx18.vbdfa1zgl84pbc1n

	I0918 11:52:46.552403    1740 out.go:204]   - Configuring RBAC rules ...

	I0918 11:52:46.552463    1740 kubeadm.go:322] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles

	I0918 11:52:46.553348    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes

	I0918 11:52:46.557357    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials

	I0918 11:52:46.558552    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token

	I0918 11:52:46.559879    1740 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster

	I0918 11:52:46.560890    1740 kubeadm.go:322] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace

	I0918 11:52:46.567944    1740 kubeadm.go:322] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key

	I0918 11:52:46.739677    1740 kubeadm.go:322] [addons] Applied essential addon: CoreDNS

	I0918 11:52:46.956246    1740 kubeadm.go:322] [addons] Applied essential addon: kube-proxy

	I0918 11:52:46.958050    1740 kubeadm.go:322] 

	I0918 11:52:46.958085    1740 kubeadm.go:322] Your Kubernetes control-plane has initialized successfully!

	I0918 11:52:46.958096    1740 kubeadm.go:322] 

	I0918 11:52:46.958138    1740 kubeadm.go:322] To start using your cluster, you need to run the following as a regular user:

	I0918 11:52:46.958143    1740 kubeadm.go:322] 

	I0918 11:52:46.958157    1740 kubeadm.go:322]   mkdir -p $HOME/.kube

	I0918 11:52:46.958186    1740 kubeadm.go:322]   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

	I0918 11:52:46.958221    1740 kubeadm.go:322]   sudo chown $(id -u):$(id -g) $HOME/.kube/config

	I0918 11:52:46.958226    1740 kubeadm.go:322] 

	I0918 11:52:46.958261    1740 kubeadm.go:322] Alternatively, if you are the root user, you can run:

	I0918 11:52:46.958267    1740 kubeadm.go:322] 

	I0918 11:52:46.958291    1740 kubeadm.go:322]   export KUBECONFIG=/etc/kubernetes/admin.conf

	I0918 11:52:46.958294    1740 kubeadm.go:322] 

	I0918 11:52:46.958317    1740 kubeadm.go:322] You should now deploy a pod network to the cluster.

	I0918 11:52:46.958375    1740 kubeadm.go:322] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

	I0918 11:52:46.958411    1740 kubeadm.go:322]   https://kubernetes.io/docs/concepts/cluster-administration/addons/

	I0918 11:52:46.958416    1740 kubeadm.go:322] 

	I0918 11:52:46.958458    1740 kubeadm.go:322] You can now join any number of control-plane nodes by copying certificate authorities

	I0918 11:52:46.958503    1740 kubeadm.go:322] and service account keys on each node and then running the following as root:

	I0918 11:52:46.958507    1740 kubeadm.go:322] 

	I0918 11:52:46.958562    1740 kubeadm.go:322]   kubeadm join control-plane.minikube.internal:8443 --token 0otx18.vbdfa1zgl84pbc1n \

	I0918 11:52:46.958623    1740 kubeadm.go:322] 	--discovery-token-ca-cert-hash sha256:ba7ba5242719fff40b98ade8a053fc0c6dded3185e28e2742ca7444c7c25a7a5 \

	I0918 11:52:46.958634    1740 kubeadm.go:322] 	--control-plane 

	I0918 11:52:46.958636    1740 kubeadm.go:322] 

	I0918 11:52:46.958676    1740 kubeadm.go:322] Then you can join any number of worker nodes by running the following on each as root:

	I0918 11:52:46.958681    1740 kubeadm.go:322] 

	I0918 11:52:46.958735    1740 kubeadm.go:322] kubeadm join control-plane.minikube.internal:8443 --token 0otx18.vbdfa1zgl84pbc1n \

	I0918 11:52:46.958805    1740 kubeadm.go:322] 	--discovery-token-ca-cert-hash sha256:ba7ba5242719fff40b98ade8a053fc0c6dded3185e28e2742ca7444c7c25a7a5 

	I0918 11:52:46.958862    1740 kubeadm.go:322] 	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	I0918 11:52:46.958868    1740 cni.go:84] Creating CNI manager for ""

	I0918 11:52:46.958880    1740 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 11:52:46.967096    1740 out.go:177] * Configuring bridge CNI (Container Networking Interface) ...

	I0918 11:52:46.970213    1740 ssh_runner.go:195] Run: sudo mkdir -p /etc/cni/net.d

	I0918 11:52:46.973479    1740 ssh_runner.go:362] scp memory --> /etc/cni/net.d/1-k8s.conflist (457 bytes)

	I0918 11:52:46.977991    1740 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"

	I0918 11:52:46.978031    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:46.978044    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl label nodes minikube.k8s.io/version=v1.31.2 minikube.k8s.io/commit=ae2adeb20cc3a4b5f1decc7c8f53736ec04c4a36 minikube.k8s.io/name=addons-221000 minikube.k8s.io/updated_at=2023_09_18T11_52_46_0700 minikube.k8s.io/primary=true --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:47.038282    1740 ops.go:34] apiserver oom_adj: -16

	I0918 11:52:47.038334    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:47.073908    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:47.620721    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:48.118781    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:48.620650    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:49.120663    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:49.620741    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:50.120032    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:50.620638    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:51.118777    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:51.618788    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:52.120234    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:52.619043    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:53.120606    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:53.619266    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:54.118979    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:54.618916    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:55.120620    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:55.620599    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:56.120585    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:56.618629    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:57.120587    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:57.618783    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:58.120645    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:58.620559    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:59.120556    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:52:59.619252    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:53:00.118796    1740 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 11:53:00.169662    1740 kubeadm.go:1081] duration metric: took 13.191788791s to wait for elevateKubeSystemPrivileges.

	I0918 11:53:00.169677    1740 kubeadm.go:406] StartCluster complete in 20.38867075s

	I0918 11:53:00.169687    1740 settings.go:142] acquiring lock: {Name:mke420f28dda4f7a752738b3e6d571dc4216779e Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:53:00.169849    1740 settings.go:150] Updating kubeconfig:  /Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 11:53:00.170110    1740 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/kubeconfig: {Name:mk07020c5b974cf07ca0cda25f72a521eb245fa1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 11:53:00.170308    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"

	I0918 11:53:00.170433    1740 config.go:182] Loaded profile config "addons-221000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 11:53:00.170379    1740 addons.go:499] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:true csi-hostpath-driver:true dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:true gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:true ingress-dns:true inspektor-gadget:true istio:false istio-provisioner:false kong:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:true registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false volumesnapshots:true]

	I0918 11:53:00.170512    1740 addons.go:69] Setting ingress=true in profile "addons-221000"

	I0918 11:53:00.170517    1740 addons.go:69] Setting ingress-dns=true in profile "addons-221000"

	I0918 11:53:00.170520    1740 addons.go:231] Setting addon ingress=true in "addons-221000"

	I0918 11:53:00.170523    1740 addons.go:231] Setting addon ingress-dns=true in "addons-221000"

	I0918 11:53:00.170533    1740 addons.go:69] Setting metrics-server=true in profile "addons-221000"

	I0918 11:53:00.170538    1740 addons.go:231] Setting addon metrics-server=true in "addons-221000"

	I0918 11:53:00.170552    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170556    1740 addons.go:69] Setting inspektor-gadget=true in profile "addons-221000"

	I0918 11:53:00.170560    1740 addons.go:69] Setting csi-hostpath-driver=true in profile "addons-221000"

	I0918 11:53:00.170564    1740 addons.go:69] Setting gcp-auth=true in profile "addons-221000"

	I0918 11:53:00.170569    1740 mustload.go:65] Loading cluster: addons-221000

	I0918 11:53:00.170573    1740 addons.go:69] Setting registry=true in profile "addons-221000"

	I0918 11:53:00.170577    1740 addons.go:231] Setting addon registry=true in "addons-221000"

	I0918 11:53:00.170587    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170595    1740 addons.go:69] Setting default-storageclass=true in profile "addons-221000"

	I0918 11:53:00.170618    1740 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "addons-221000"

	I0918 11:53:00.170629    1740 addons.go:69] Setting storage-provisioner=true in profile "addons-221000"

	I0918 11:53:00.170639    1740 config.go:182] Loaded profile config "addons-221000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 11:53:00.170657    1740 addons.go:231] Setting addon storage-provisioner=true in "addons-221000"

	I0918 11:53:00.170705    1740 host.go:66] Checking if "addons-221000" exists ...

	W0918 11:53:00.170820    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	W0918 11:53:00.170825    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	W0918 11:53:00.170829    1740 addons.go:277] "addons-221000" is not running, setting ingress-dns=true and skipping enablement (err=<nil>)

	W0918 11:53:00.170831    1740 addons.go:277] "addons-221000" is not running, setting registry=true and skipping enablement (err=<nil>)

	I0918 11:53:00.170552    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170834    1740 addons.go:467] Verifying addon registry=true in "addons-221000"

	I0918 11:53:00.170556    1740 addons.go:69] Setting cloud-spanner=true in profile "addons-221000"

	I0918 11:53:00.175435    1740 out.go:177] * Verifying registry addon...

	I0918 11:53:00.170560    1740 addons.go:231] Setting addon inspektor-gadget=true in "addons-221000"

	I0918 11:53:00.170569    1740 addons.go:231] Setting addon csi-hostpath-driver=true in "addons-221000"

	I0918 11:53:00.170865    1740 addons.go:231] Setting addon cloud-spanner=true in "addons-221000"

	I0918 11:53:00.170553    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.170512    1740 addons.go:69] Setting volumesnapshots=true in profile "addons-221000"

	W0918 11:53:00.171149    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	W0918 11:53:00.171162    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	I0918 11:53:00.171465    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.188489    1740 out.go:177]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5

	I0918 11:53:00.182591    1740 host.go:66] Checking if "addons-221000" exists ...

	W0918 11:53:00.182599    1740 addons.go:277] "addons-221000" is not running, setting ingress=true and skipping enablement (err=<nil>)

	W0918 11:53:00.182603    1740 addons_storage_classes.go:55] "addons-221000" is not running, writing default-storageclass=true to disk and skipping enablement

	I0918 11:53:00.182614    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.182623    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.182641    1740 addons.go:231] Setting addon volumesnapshots=true in "addons-221000"

	I0918 11:53:00.183250    1740 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=registry" in ns "kube-system" ...

	I0918 11:53:00.196542    1740 out.go:177]   - Using image registry.k8s.io/metrics-server/metrics-server:v0.6.4

	I0918 11:53:00.192833    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:00.192879    1740 addons.go:423] installing /etc/kubernetes/addons/storage-provisioner.yaml

	I0918 11:53:00.192896    1740 addons.go:467] Verifying addon ingress=true in "addons-221000"

	I0918 11:53:00.192903    1740 addons.go:231] Setting addon default-storageclass=true in "addons-221000"

	W0918 11:53:00.193202    1740 host.go:54] host status for "addons-221000" returned error: state: connect: dial unix /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/monitor: connect: connection refused

	I0918 11:53:00.196510    1740 kapi.go:248] "coredns" deployment in "kube-system" namespace and "addons-221000" context rescaled to 1 replicas

	I0918 11:53:00.199509    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)

	I0918 11:53:00.199541    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-apiservice.yaml

	I0918 11:53:00.199554    1740 host.go:66] Checking if "addons-221000" exists ...

	W0918 11:53:00.199564    1740 addons.go:277] "addons-221000" is not running, setting cloud-spanner=true and skipping enablement (err=<nil>)

	I0918 11:53:00.206530    1740 out.go:177]   - Using image ghcr.io/inspektor-gadget/inspektor-gadget:v0.20.0

	I0918 11:53:00.210356    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)

	I0918 11:53:00.210367    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.210376    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/livenessprobe:v2.8.0

	I0918 11:53:00.210391    1740 start.go:223] Will wait 6m0s for node &{Name: IP:192.168.105.2 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 11:53:00.211130    1740 addons.go:423] installing /etc/kubernetes/addons/storageclass.yaml

	I0918 11:53:00.213481    1740 out.go:177] * Verifying ingress addon...

	I0918 11:53:00.214506    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/snapshot-controller:v6.1.0

	I0918 11:53:00.214532    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.214630    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)

	I0918 11:53:00.218827    1740 out.go:177] * Verifying Kubernetes components...

	I0918 11:53:00.221560    1740 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^        forward . \/etc\/resolv.conf.*/i \        hosts {\n           192.168.105.1 host.minikube.internal\n           fallthrough\n        }' -e '/^        errors *$/i \        log' | sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"

	I0918 11:53:00.218836    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.229444    1740 addons.go:423] installing /etc/kubernetes/addons/ig-namespace.yaml

	I0918 11:53:00.229456    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-namespace.yaml (55 bytes)

	I0918 11:53:00.217515    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml

	I0918 11:53:00.229466    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml (934 bytes)

	I0918 11:53:00.229467    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.229472    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.220086    1740 kapi.go:86] Found 0 Pods for label selector kubernetes.io/minikube-addons=registry

	I0918 11:53:00.221623    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I0918 11:53:00.236499    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-resizer:v1.6.0

	I0918 11:53:00.234112    1740 kapi.go:75] Waiting for pod with label "app.kubernetes.io/name=ingress-nginx" in ns "ingress-nginx" ...

	I0918 11:53:00.242380    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0

	I0918 11:53:00.251491    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-provisioner:v3.3.0

	I0918 11:53:00.252703    1740 kapi.go:86] Found 0 Pods for label selector app.kubernetes.io/name=ingress-nginx

	I0918 11:53:00.260396    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-attacher:v4.0.0

	I0918 11:53:00.267323    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.7.0

	I0918 11:53:00.274307    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.6.0

	I0918 11:53:00.284458    1740 out.go:177]   - Using image registry.k8s.io/sig-storage/hostpathplugin:v1.9.0

	I0918 11:53:00.287512    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-attacher.yaml

	I0918 11:53:00.287521    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-attacher.yaml (3073 bytes)

	I0918 11:53:00.287531    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:00.322513    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-server-deployment.yaml

	I0918 11:53:00.322523    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1907 bytes)

	I0918 11:53:00.336060    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml

	I0918 11:53:00.340017    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml

	I0918 11:53:00.346686    1740 addons.go:423] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml

	I0918 11:53:00.346693    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml (6471 bytes)

	I0918 11:53:00.362466    1740 addons.go:423] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml

	I0918 11:53:00.362477    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml (23126 bytes)

	I0918 11:53:00.363923    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-server-rbac.yaml

	I0918 11:53:00.363928    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)

	I0918 11:53:00.368153    1740 addons.go:423] installing /etc/kubernetes/addons/ig-serviceaccount.yaml

	I0918 11:53:00.368161    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-serviceaccount.yaml (80 bytes)

	I0918 11:53:00.376728    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-hostpath.yaml

	I0918 11:53:00.376740    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-hostpath.yaml (4266 bytes)

	I0918 11:53:00.395920    1740 addons.go:423] installing /etc/kubernetes/addons/metrics-server-service.yaml

	I0918 11:53:00.395931    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)

	I0918 11:53:00.403313    1740 addons.go:423] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml

	I0918 11:53:00.403320    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml (19582 bytes)

	I0918 11:53:00.404660    1740 addons.go:423] installing /etc/kubernetes/addons/ig-role.yaml

	I0918 11:53:00.404665    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-role.yaml (210 bytes)

	I0918 11:53:00.430003    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml

	I0918 11:53:00.430014    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml (3038 bytes)

	I0918 11:53:00.492665    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-provisioner.yaml

	I0918 11:53:00.492677    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-provisioner.yaml (4442 bytes)

	I0918 11:53:00.495149    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml

	I0918 11:53:00.495157    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml (3545 bytes)

	I0918 11:53:00.500095    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml

	I0918 11:53:00.522789    1740 addons.go:423] installing /etc/kubernetes/addons/ig-rolebinding.yaml

	I0918 11:53:00.522799    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-rolebinding.yaml (244 bytes)

	I0918 11:53:00.561433    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-resizer.yaml

	I0918 11:53:00.561445    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-resizer.yaml (2943 bytes)

	I0918 11:53:00.561578    1740 addons.go:423] installing /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0918 11:53:00.561584    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml (1475 bytes)

	I0918 11:53:00.579630    1740 addons.go:423] installing /etc/kubernetes/addons/ig-clusterrole.yaml

	I0918 11:53:00.579641    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-clusterrole.yaml (1485 bytes)

	I0918 11:53:00.604202    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0918 11:53:00.607378    1740 addons.go:423] installing /etc/kubernetes/addons/rbac-external-snapshotter.yaml

	I0918 11:53:00.607389    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-snapshotter.yaml (3149 bytes)

	I0918 11:53:00.624742    1740 addons.go:423] installing /etc/kubernetes/addons/ig-clusterrolebinding.yaml

	I0918 11:53:00.624755    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-clusterrolebinding.yaml (274 bytes)

	I0918 11:53:00.641432    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-attacher.yaml

	I0918 11:53:00.641441    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-attacher.yaml (2143 bytes)

	I0918 11:53:00.675716    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml

	I0918 11:53:00.675728    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml (1274 bytes)

	I0918 11:53:00.684962    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-plugin.yaml

	I0918 11:53:00.684971    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-plugin.yaml (8201 bytes)

	I0918 11:53:00.690643    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-resizer.yaml

	I0918 11:53:00.690652    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-resizer.yaml (2191 bytes)

	I0918 11:53:00.691887    1740 addons.go:423] installing /etc/kubernetes/addons/ig-crd.yaml

	I0918 11:53:00.691893    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-crd.yaml (5216 bytes)

	I0918 11:53:00.701987    1740 addons.go:423] installing /etc/kubernetes/addons/csi-hostpath-storageclass.yaml

	I0918 11:53:00.701999    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-storageclass.yaml (846 bytes)

	I0918 11:53:00.719094    1740 addons.go:423] installing /etc/kubernetes/addons/ig-daemonset.yaml

	I0918 11:53:00.719103    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-daemonset.yaml (7741 bytes)

	I0918 11:53:00.821368    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/ig-namespace.yaml -f /etc/kubernetes/addons/ig-serviceaccount.yaml -f /etc/kubernetes/addons/ig-role.yaml -f /etc/kubernetes/addons/ig-rolebinding.yaml -f /etc/kubernetes/addons/ig-clusterrole.yaml -f /etc/kubernetes/addons/ig-clusterrolebinding.yaml -f /etc/kubernetes/addons/ig-crd.yaml -f /etc/kubernetes/addons/ig-daemonset.yaml

	I0918 11:53:00.824330    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml

	I0918 11:53:01.167637    1740 start.go:917] {"host.minikube.internal": 192.168.105.1} host record injected into CoreDNS's ConfigMap

	I0918 11:53:01.168100    1740 node_ready.go:35] waiting up to 6m0s for node "addons-221000" to be "Ready" ...

	I0918 11:53:01.169940    1740 node_ready.go:49] node "addons-221000" has status "Ready":"True"

	I0918 11:53:01.169963    1740 node_ready.go:38] duration metric: took 1.836333ms waiting for node "addons-221000" to be "Ready" ...

	I0918 11:53:01.169968    1740 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I0918 11:53:01.172987    1740 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5dd5756b68-mbgns" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:01.659785    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (1.159683334s)

	I0918 11:53:01.659805    1740 addons.go:467] Verifying addon metrics-server=true in "addons-221000"

	I0918 11:53:01.659835    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: (1.055627667s)

	W0918 11:53:01.659861    1740 addons.go:449] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1

	stdout:

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created

	serviceaccount/snapshot-controller created

	clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created

	clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created

	role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	deployment.apps/snapshot-controller created

	stderr:

	error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"

	ensure CRDs are installed first

	I0918 11:53:01.659882    1740 retry.go:31] will retry after 288.841008ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1

	stdout:

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created

	serviceaccount/snapshot-controller created

	clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created

	clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created

	role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	deployment.apps/snapshot-controller created

	stderr:

	error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"

	ensure CRDs are installed first

	I0918 11:53:01.660144    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (1.320131291s)

	I0918 11:53:01.949280    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply --force -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0918 11:53:02.183526    1740 pod_ready.go:92] pod "coredns-5dd5756b68-mbgns" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.183539    1740 pod_ready.go:81] duration metric: took 1.010553542s waiting for pod "coredns-5dd5756b68-mbgns" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.183545    1740 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.184891    1740 pod_ready.go:97] error getting pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace (skipping!): pods "coredns-5dd5756b68-z4cmf" not found

	I0918 11:53:02.184904    1740 pod_ready.go:81] duration metric: took 1.354709ms waiting for pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace to be "Ready" ...

	E0918 11:53:02.184909    1740 pod_ready.go:66] WaitExtra: waitPodCondition: error getting pod "coredns-5dd5756b68-z4cmf" in "kube-system" namespace (skipping!): pods "coredns-5dd5756b68-z4cmf" not found

	I0918 11:53:02.184914    1740 pod_ready.go:78] waiting up to 6m0s for pod "etcd-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.187700    1740 pod_ready.go:92] pod "etcd-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.187709    1740 pod_ready.go:81] duration metric: took 2.791875ms waiting for pod "etcd-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.187714    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.190228    1740 pod_ready.go:92] pod "kube-apiserver-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.190236    1740 pod_ready.go:81] duration metric: took 2.518208ms waiting for pod "kube-apiserver-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.190240    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.571748    1740 pod_ready.go:92] pod "kube-controller-manager-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.571761    1740 pod_ready.go:81] duration metric: took 381.518375ms waiting for pod "kube-controller-manager-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.571765    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-q7gqn" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.823208    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml: (1.998874458s)

	I0918 11:53:02.823230    1740 addons.go:467] Verifying addon csi-hostpath-driver=true in "addons-221000"

	I0918 11:53:02.828329    1740 out.go:177] * Verifying csi-hostpath-driver addon...

	I0918 11:53:02.838784    1740 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=csi-hostpath-driver" in ns "kube-system" ...

	I0918 11:53:02.842760    1740 kapi.go:86] Found 3 Pods for label selector kubernetes.io/minikube-addons=csi-hostpath-driver

	I0918 11:53:02.842767    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:02.847417    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:02.972129    1740 pod_ready.go:92] pod "kube-proxy-q7gqn" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:02.972138    1740 pod_ready.go:81] duration metric: took 400.3735ms waiting for pod "kube-proxy-q7gqn" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:02.972143    1740 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:03.351863    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:03.371796    1740 pod_ready.go:92] pod "kube-scheduler-addons-221000" in "kube-system" namespace has status "Ready":"True"

	I0918 11:53:03.371806    1740 pod_ready.go:81] duration metric: took 399.662875ms waiting for pod "kube-scheduler-addons-221000" in "kube-system" namespace to be "Ready" ...

	I0918 11:53:03.371810    1740 pod_ready.go:38] duration metric: took 2.201856875s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I0918 11:53:03.371820    1740 api_server.go:52] waiting for apiserver process to appear ...

	I0918 11:53:03.371875    1740 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I0918 11:53:03.851785    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:04.352157    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:04.679993    1740 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply --force -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: (2.73071775s)

	I0918 11:53:04.680002    1740 ssh_runner.go:235] Completed: sudo pgrep -xnf kube-apiserver.*minikube.*: (1.308127625s)

	I0918 11:53:04.680021    1740 api_server.go:72] duration metric: took 4.46545525s to wait for apiserver process to appear ...

	I0918 11:53:04.680025    1740 api_server.go:88] waiting for apiserver healthz status ...

	I0918 11:53:04.680031    1740 api_server.go:253] Checking apiserver healthz at https://192.168.105.2:8443/healthz ...

	I0918 11:53:04.683995    1740 api_server.go:279] https://192.168.105.2:8443/healthz returned 200:

	ok

	I0918 11:53:04.684750    1740 api_server.go:141] control plane version: v1.28.2

	I0918 11:53:04.684756    1740 api_server.go:131] duration metric: took 4.728917ms to wait for apiserver health ...

	I0918 11:53:04.684760    1740 system_pods.go:43] waiting for kube-system pods to appear ...

	I0918 11:53:04.689157    1740 system_pods.go:59] 13 kube-system pods found

	I0918 11:53:04.689166    1740 system_pods.go:61] "coredns-5dd5756b68-mbgns" [376db80e-bef7-49a8-805c-d250bbb5ddc5] Running

	I0918 11:53:04.689171    1740 system_pods.go:61] "csi-hostpath-attacher-0" [b3eb2340-f156-4127-b844-79013849b5d5] Pending / Ready:ContainersNotReady (containers with unready status: [csi-attacher]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-attacher])

	I0918 11:53:04.689175    1740 system_pods.go:61] "csi-hostpath-resizer-0" [a39fc7e1-21e6-43e0-8a71-76fc3122aa67] Pending / Ready:ContainersNotReady (containers with unready status: [csi-resizer]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-resizer])

	I0918 11:53:04.689182    1740 system_pods.go:61] "csi-hostpathplugin-s878j" [f7db805b-46b1-4d4f-b620-4bb9732a0ba0] Pending / Ready:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter])

	I0918 11:53:04.689185    1740 system_pods.go:61] "etcd-addons-221000" [4500c338-1b0c-4d39-b5b6-76d42cf285f5] Running

	I0918 11:53:04.689188    1740 system_pods.go:61] "kube-apiserver-addons-221000" [93a9cc2e-3463-4332-a37c-86437106ed5e] Running

	I0918 11:53:04.689190    1740 system_pods.go:61] "kube-controller-manager-addons-221000" [eaeef50f-2b51-43b2-91c9-a4f97f5460ae] Running

	I0918 11:53:04.689193    1740 system_pods.go:61] "kube-proxy-q7gqn" [e971c33c-7d1b-47b7-9ff5-3a629f12fb57] Running

	I0918 11:53:04.689195    1740 system_pods.go:61] "kube-scheduler-addons-221000" [9d39af63-6f0a-4855-9435-f8e7af26869e] Running

	I0918 11:53:04.689199    1740 system_pods.go:61] "metrics-server-7c66d45ddc-ph4qt" [193ff040-48cb-4429-8aa3-4065ecb5241d] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I0918 11:53:04.689205    1740 system_pods.go:61] "snapshot-controller-58dbcc7b99-89j9m" [d9a96c2a-2231-4dea-abbf-16875dd2b1d7] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.689210    1740 system_pods.go:61] "snapshot-controller-58dbcc7b99-xwwxn" [5a80446c-a3a8-4ce7-8ac4-2894087691fb] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.689214    1740 system_pods.go:61] "storage-provisioner" [88ff8527-97e2-4317-8d5b-a2502e8cb7f7] Running

	I0918 11:53:04.689217    1740 system_pods.go:74] duration metric: took 4.45475ms to wait for pod list to return data ...

	I0918 11:53:04.689220    1740 default_sa.go:34] waiting for default service account to be created ...

	I0918 11:53:04.690951    1740 default_sa.go:45] found service account: "default"

	I0918 11:53:04.690958    1740 default_sa.go:55] duration metric: took 1.736083ms for default service account to be created ...

	I0918 11:53:04.690961    1740 system_pods.go:116] waiting for k8s-apps to be running ...

	I0918 11:53:04.694962    1740 system_pods.go:86] 13 kube-system pods found

	I0918 11:53:04.694971    1740 system_pods.go:89] "coredns-5dd5756b68-mbgns" [376db80e-bef7-49a8-805c-d250bbb5ddc5] Running

	I0918 11:53:04.694976    1740 system_pods.go:89] "csi-hostpath-attacher-0" [b3eb2340-f156-4127-b844-79013849b5d5] Pending / Ready:ContainersNotReady (containers with unready status: [csi-attacher]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-attacher])

	I0918 11:53:04.694979    1740 system_pods.go:89] "csi-hostpath-resizer-0" [a39fc7e1-21e6-43e0-8a71-76fc3122aa67] Pending / Ready:ContainersNotReady (containers with unready status: [csi-resizer]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-resizer])

	I0918 11:53:04.694983    1740 system_pods.go:89] "csi-hostpathplugin-s878j" [f7db805b-46b1-4d4f-b620-4bb9732a0ba0] Pending / Ready:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter]) / ContainersReady:ContainersNotReady (containers with unready status: [csi-external-health-monitor-controller node-driver-registrar hostpath liveness-probe csi-provisioner csi-snapshotter])

	I0918 11:53:04.694986    1740 system_pods.go:89] "etcd-addons-221000" [4500c338-1b0c-4d39-b5b6-76d42cf285f5] Running

	I0918 11:53:04.694988    1740 system_pods.go:89] "kube-apiserver-addons-221000" [93a9cc2e-3463-4332-a37c-86437106ed5e] Running

	I0918 11:53:04.694990    1740 system_pods.go:89] "kube-controller-manager-addons-221000" [eaeef50f-2b51-43b2-91c9-a4f97f5460ae] Running

	I0918 11:53:04.694994    1740 system_pods.go:89] "kube-proxy-q7gqn" [e971c33c-7d1b-47b7-9ff5-3a629f12fb57] Running

	I0918 11:53:04.694996    1740 system_pods.go:89] "kube-scheduler-addons-221000" [9d39af63-6f0a-4855-9435-f8e7af26869e] Running

	I0918 11:53:04.694999    1740 system_pods.go:89] "metrics-server-7c66d45ddc-ph4qt" [193ff040-48cb-4429-8aa3-4065ecb5241d] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I0918 11:53:04.695003    1740 system_pods.go:89] "snapshot-controller-58dbcc7b99-89j9m" [d9a96c2a-2231-4dea-abbf-16875dd2b1d7] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.695006    1740 system_pods.go:89] "snapshot-controller-58dbcc7b99-xwwxn" [5a80446c-a3a8-4ce7-8ac4-2894087691fb] Pending / Ready:ContainersNotReady (containers with unready status: [volume-snapshot-controller]) / ContainersReady:ContainersNotReady (containers with unready status: [volume-snapshot-controller])

	I0918 11:53:04.695009    1740 system_pods.go:89] "storage-provisioner" [88ff8527-97e2-4317-8d5b-a2502e8cb7f7] Running

	I0918 11:53:04.695012    1740 system_pods.go:126] duration metric: took 4.049ms to wait for k8s-apps to be running ...

	I0918 11:53:04.695014    1740 system_svc.go:44] waiting for kubelet service to be running ....

	I0918 11:53:04.695074    1740 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I0918 11:53:04.700754    1740 system_svc.go:56] duration metric: took 5.736541ms WaitForService to wait for kubelet.

	I0918 11:53:04.700761    1740 kubeadm.go:581] duration metric: took 4.486195916s to wait for : map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] ...

	I0918 11:53:04.700771    1740 node_conditions.go:102] verifying NodePressure condition ...

	I0918 11:53:04.702224    1740 node_conditions.go:122] node storage ephemeral capacity is 17784760Ki

	I0918 11:53:04.702234    1740 node_conditions.go:123] node cpu capacity is 2

	I0918 11:53:04.702239    1740 node_conditions.go:105] duration metric: took 1.466083ms to run NodePressure ...

	I0918 11:53:04.702244    1740 start.go:228] waiting for startup goroutines ...

	I0918 11:53:04.851794    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:05.352908    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:05.851843    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:06.351684    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:06.787954    1740 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_application_credentials.json (162 bytes)

	I0918 11:53:06.787972    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:06.819465    1740 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_cloud_project (12 bytes)

	I0918 11:53:06.824867    1740 addons.go:231] Setting addon gcp-auth=true in "addons-221000"

	I0918 11:53:06.824887    1740 host.go:66] Checking if "addons-221000" exists ...

	I0918 11:53:06.825624    1740 ssh_runner.go:195] Run: cat /var/lib/minikube/google_application_credentials.json

	I0918 11:53:06.825631    1740 sshutil.go:53] new ssh client: &{IP:192.168.105.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/addons-221000/id_rsa Username:docker}

	I0918 11:53:06.851875    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:06.860327    1740 out.go:177]   - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20230407

	I0918 11:53:06.868318    1740 out.go:177]   - Using image gcr.io/k8s-minikube/gcp-auth-webhook:v0.1.0

	I0918 11:53:06.871302    1740 addons.go:423] installing /etc/kubernetes/addons/gcp-auth-ns.yaml

	I0918 11:53:06.871308    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-ns.yaml (700 bytes)

	I0918 11:53:06.876186    1740 addons.go:423] installing /etc/kubernetes/addons/gcp-auth-service.yaml

	I0918 11:53:06.876191    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-service.yaml (788 bytes)

	I0918 11:53:06.881437    1740 addons.go:423] installing /etc/kubernetes/addons/gcp-auth-webhook.yaml

	I0918 11:53:06.881443    1740 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-webhook.yaml (5412 bytes)

	I0918 11:53:06.887709    1740 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/gcp-auth-ns.yaml -f /etc/kubernetes/addons/gcp-auth-service.yaml -f /etc/kubernetes/addons/gcp-auth-webhook.yaml

	I0918 11:53:07.124491    1740 addons.go:467] Verifying addon gcp-auth=true in "addons-221000"

	I0918 11:53:07.129030    1740 out.go:177] * Verifying gcp-auth addon...

	I0918 11:53:07.137330    1740 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=gcp-auth" in ns "gcp-auth" ...

	I0918 11:53:07.139319    1740 kapi.go:86] Found 1 Pods for label selector kubernetes.io/minikube-addons=gcp-auth

	I0918 11:53:07.139325    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:07.142056    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:07.351786    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:07.647787    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:07.851746    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:08.144923    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:08.351959    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:08.646023    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:08.851939    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:09.146053    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:09.352896    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:09.645776    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:09.851792    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:10.146843    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:10.351428    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:10.645894    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:10.852236    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:11.145951    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:11.352216    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:11.645774    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:11.852232    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:12.145929    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:12.355119    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:12.840629    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:12.851603    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:13.145852    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:13.351962    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:13.646245    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:13.852112    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:14.144894    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:14.351204    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:14.646046    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:15.059939    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:15.145929    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:15.351906    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:15.646033    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:15.852022    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:16.145893    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:16.351976    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:16.645740    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:16.852007    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:17.145547    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:17.352003    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:17.646147    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:17.853011    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:18.145960    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:18.353519    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:18.646257    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:18.851829    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:19.143968    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:19.351778    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:19.645563    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:19.851678    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:20.145637    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:20.351425    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:20.645727    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:20.852080    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:21.145053    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:21.352010    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:21.645631    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:21.851983    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:22.146102    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:22.351559    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:22.645364    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:22.851995    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:23.145511    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:23.351664    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:23.646427    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:23.851813    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:24.144653    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:24.351670    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:24.645732    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:24.851659    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:25.145755    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:25.350286    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:25.645572    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:25.851968    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:26.145692    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:26.352042    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:26.645575    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:26.852307    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:27.145498    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:27.352114    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:27.645719    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:27.851965    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:28.145987    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:28.351772    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:28.645594    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:28.851993    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:29.145747    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:29.351502    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:29.645697    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:29.851953    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:30.145495    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:30.351423    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:30.645330    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:30.852142    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:31.145422    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:31.352025    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:31.645572    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:31.852021    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:32.145742    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:32.351596    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:32.645761    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:32.851846    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:33.145666    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:33.351607    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:33.646262    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:33.852115    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:34.145556    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:34.353136    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:34.644719    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:34.852049    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:35.145343    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:35.351551    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:35.645361    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:35.851476    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:36.143766    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:36.351679    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:36.645788    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:36.851584    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:37.145933    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:37.351445    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0918 11:53:37.646034    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:37.851557    1740 kapi.go:107] duration metric: took 35.013102459s to wait for kubernetes.io/minikube-addons=csi-hostpath-driver ...

	I0918 11:53:38.145661    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:38.645759    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:39.145382    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:39.645858    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:40.145851    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:40.645842    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:41.145511    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:41.646176    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:42.145937    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:42.645521    1740 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0918 11:53:43.145380    1740 kapi.go:107] duration metric: took 36.008387458s to wait for kubernetes.io/minikube-addons=gcp-auth ...

	I0918 11:53:43.148984    1740 out.go:177] * Your GCP credentials will now be mounted into every pod created in the addons-221000 cluster.

	I0918 11:53:43.152958    1740 out.go:177] * If you don't want your credentials mounted into a specific pod, add a label with the `gcp-auth-skip-secret` key to your pod configuration.

	I0918 11:53:43.156871    1740 out.go:177] * If you want existing pods to be mounted with credentials, either recreate them or rerun addons enable with --refresh.

	I0918 11:59:00.190235    1740 kapi.go:107] duration metric: took 6m0.011650291s to wait for kubernetes.io/minikube-addons=registry ...

	W0918 11:59:00.190354    1740 out.go:239] ! Enabling 'registry' returned an error: running callbacks: [waiting for kubernetes.io/minikube-addons=registry pods: context deadline exceeded]

	I0918 11:59:00.236861    1740 kapi.go:107] duration metric: took 6m0.007418791s to wait for app.kubernetes.io/name=ingress-nginx ...

	W0918 11:59:00.236887    1740 out.go:239] ! Enabling 'ingress' returned an error: running callbacks: [waiting for app.kubernetes.io/name=ingress-nginx pods: context deadline exceeded]

	I0918 11:59:00.245622    1740 out.go:177] * Enabled addons: ingress-dns, cloud-spanner, default-storageclass, metrics-server, storage-provisioner, inspektor-gadget, volumesnapshots, csi-hostpath-driver, gcp-auth

	I0918 11:59:00.252559    1740 addons.go:502] enable addons completed in 6m0.086876584s: enabled=[ingress-dns cloud-spanner default-storageclass metrics-server storage-provisioner inspektor-gadget volumesnapshots csi-hostpath-driver gcp-auth]

	I0918 11:59:00.252570    1740 start.go:233] waiting for cluster config update ...

	I0918 11:59:00.252577    1740 start.go:242] writing updated cluster config ...

	I0918 11:59:00.253046    1740 ssh_runner.go:195] Run: rm -f paused

	I0918 11:59:00.282509    1740 start.go:600] kubectl: 1.27.2, cluster: 1.28.2 (minor skew: 1)

	I0918 11:59:00.285533    1740 out.go:177] * Done! kubectl is now configured to use "addons-221000" cluster and "default" namespace by default

	* 

	* ==> Docker <==

	* -- Journal begins at Mon 2023-09-18 18:52:28 UTC, ends at Mon 2023-09-18 19:12:24 UTC. --

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.378956054Z" level=info msg="shim disconnected" id=db4658d155f2578fc668ba94399fd4b64164a06274f25bb873a3470c521d2b3b namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.378988221Z" level=warning msg="cleaning up after shim disconnected" id=db4658d155f2578fc668ba94399fd4b64164a06274f25bb873a3470c521d2b3b namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.378992929Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1100]: time="2023-09-18T19:12:04.436636476Z" level=info msg="ignoring event" container=f31f590bd070326784aa894bb3b130e04a21f13ff6b7835f98e9c9c994be5c89 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.437103975Z" level=info msg="shim disconnected" id=f31f590bd070326784aa894bb3b130e04a21f13ff6b7835f98e9c9c994be5c89 namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.437132058Z" level=warning msg="cleaning up after shim disconnected" id=f31f590bd070326784aa894bb3b130e04a21f13ff6b7835f98e9c9c994be5c89 namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.437137642Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:04 addons-221000 dockerd[1106]: time="2023-09-18T19:12:04.441266547Z" level=warning msg="cleanup warnings time=\"2023-09-18T19:12:04Z\" level=warning msg=\"failed to remove runc container\" error=\"runc did not terminate successfully: exit status 255: \" runtime=io.containerd.runc.v2\n" namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1100]: time="2023-09-18T19:12:10.222057782Z" level=info msg="ignoring event" container=ab33a0f8756c5abdbe52f99802252c9d88620a35e7026b8d48ddc961d9fd608a module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.222121782Z" level=info msg="shim disconnected" id=ab33a0f8756c5abdbe52f99802252c9d88620a35e7026b8d48ddc961d9fd608a namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.222148615Z" level=warning msg="cleaning up after shim disconnected" id=ab33a0f8756c5abdbe52f99802252c9d88620a35e7026b8d48ddc961d9fd608a namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.222152574Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1100]: time="2023-09-18T19:12:10.284872898Z" level=info msg="ignoring event" container=b759b3bc87e672ab6709e17bb94b6e35eee5335882a2a4303af5bb7802b4d324 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.284977606Z" level=info msg="shim disconnected" id=b759b3bc87e672ab6709e17bb94b6e35eee5335882a2a4303af5bb7802b4d324 namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.285020689Z" level=warning msg="cleaning up after shim disconnected" id=b759b3bc87e672ab6709e17bb94b6e35eee5335882a2a4303af5bb7802b4d324 namespace=moby

	Sep 18 19:12:10 addons-221000 dockerd[1106]: time="2023-09-18T19:12:10.285025106Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:10 addons-221000 cri-dockerd[995]: time="2023-09-18T19:12:10Z" level=error msg="EOF Failed to get stats from container ab33a0f8756c5abdbe52f99802252c9d88620a35e7026b8d48ddc961d9fd608a"

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.398430134Z" level=info msg="shim disconnected" id=afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1100]: time="2023-09-18T19:12:14.398477592Z" level=info msg="ignoring event" container=afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.398687758Z" level=warning msg="cleaning up after shim disconnected" id=afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.398698092Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.419717366Z" level=info msg="shim disconnected" id=a7e6c06852560a1d0bdec45aab8087733475e5299415ad38f796c38431947c7a namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.419822949Z" level=warning msg="cleaning up after shim disconnected" id=a7e6c06852560a1d0bdec45aab8087733475e5299415ad38f796c38431947c7a namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1106]: time="2023-09-18T19:12:14.419832116Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:12:14 addons-221000 dockerd[1100]: time="2023-09-18T19:12:14.420080948Z" level=info msg="ignoring event" container=a7e6c06852560a1d0bdec45aab8087733475e5299415ad38f796c38431947c7a module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	* 

	* ==> container status <==

	* CONTAINER           IMAGE                                                                                                          CREATED              STATE               NAME                      ATTEMPT             POD ID

	e55be7fdce01f       ghcr.io/headlamp-k8s/headlamp@sha256:498ea22dc5acadaa4015e7a50335d21fdce45d9e8f1f8adf29c2777da4182f98          About a minute ago   Running             headlamp                  0                   6eea9635aa263

	6009e365438d8       gcr.io/k8s-minikube/gcp-auth-webhook@sha256:d730651bb6584f969d95d8279a754cf9d8d31b5055c43dbdb8d7363a8c6371cf   18 minutes ago       Running             gcp-auth                  0                   e321131e2d88d

	96b1e19d34d0f       ba04bb24b9575                                                                                                  19 minutes ago       Running             storage-provisioner       0                   31030bebae5dd

	3397ed73112e1       97e04611ad434                                                                                                  19 minutes ago       Running             coredns                   0                   ecf214ed85c34

	3b8b236037bf7       7da62c127fc0f                                                                                                  19 minutes ago       Running             kube-proxy                0                   efd5b0f304a7c

	17d16f9191cb9       64fc40cee3716                                                                                                  19 minutes ago       Running             kube-scheduler            0                   6dcf2ed48fa0d

	cb85fb8fd00cf       89d57b83c1786                                                                                                  19 minutes ago       Running             kube-controller-manager   0                   d0674da883f4f

	4a8eb16a561d8       30bb499447fe1                                                                                                  19 minutes ago       Running             kube-apiserver            0                   2a8be15cc8448

	2a7ec1fe69df2       9cdd6470f48c8                                                                                                  19 minutes ago       Running             etcd                      0                   0f317e031c491

	* 

	* ==> coredns [3397ed73112e] <==

	* .:53

	[INFO] plugin/reload: Running configuration SHA512 = 591cf328cccc12bc490481273e738df59329c62c0b729d94e8b61db9961c2fa5f046dd37f1cf888b953814040d180f52594972691cd6ff41be96639138a43908

	CoreDNS-1.10.1

	linux/arm64, go1.20, 055b2c3

	[INFO] Reloading

	[INFO] plugin/reload: Running configuration SHA512 = ea7a0d73d9d208f758b1f67640ef03c58089b9d9366cf3478df3bb369b210e39f213811b46224f8a04380814b6e0890ccd358f5b5e8c80bc22ac19c8601ee35b

	[INFO] Reloading complete

	[INFO] 127.0.0.1:51523 - 34394 "HINFO IN 2890994018648264973.7586751697303130781. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.046989834s

	[INFO] 10.244.0.11:51746 - 24273 "AAAA IN storage.googleapis.com.gcp-auth.svc.cluster.local. udp 78 false 1232" NXDOMAIN qr,aa,rd 160 0.000125498s

	[INFO] 10.244.0.11:38492 - 22517 "A IN storage.googleapis.com.gcp-auth.svc.cluster.local. udp 78 false 1232" NXDOMAIN qr,aa,rd 160 0.000252786s

	[INFO] 10.244.0.11:44956 - 48338 "A IN storage.googleapis.com.svc.cluster.local. udp 69 false 1232" NXDOMAIN qr,aa,rd 151 0.000035291s

	[INFO] 10.244.0.11:34979 - 63104 "AAAA IN storage.googleapis.com.svc.cluster.local. udp 69 false 1232" NXDOMAIN qr,aa,rd 151 0.000059957s

	[INFO] 10.244.0.11:38551 - 60811 "A IN storage.googleapis.com.cluster.local. udp 65 false 1232" NXDOMAIN qr,aa,rd 147 0.000037916s

	[INFO] 10.244.0.11:44454 - 13651 "AAAA IN storage.googleapis.com.cluster.local. udp 65 false 1232" NXDOMAIN qr,aa,rd 147 0.000042082s

	[INFO] 10.244.0.11:41401 - 32727 "AAAA IN storage.googleapis.com. udp 51 false 1232" NOERROR qr,rd,ra 240 0.001047395s

	[INFO] 10.244.0.11:40924 - 8451 "A IN storage.googleapis.com. udp 51 false 1232" NOERROR qr,rd,ra 496 0.001040478s

	* 

	* ==> describe nodes <==

	* Name:               addons-221000

	Roles:              control-plane

	Labels:             beta.kubernetes.io/arch=arm64

	                    beta.kubernetes.io/os=linux

	                    kubernetes.io/arch=arm64

	                    kubernetes.io/hostname=addons-221000

	                    kubernetes.io/os=linux

	                    minikube.k8s.io/commit=ae2adeb20cc3a4b5f1decc7c8f53736ec04c4a36

	                    minikube.k8s.io/name=addons-221000

	                    minikube.k8s.io/primary=true

	                    minikube.k8s.io/updated_at=2023_09_18T11_52_46_0700

	                    minikube.k8s.io/version=v1.31.2

	                    node-role.kubernetes.io/control-plane=

	                    node.kubernetes.io/exclude-from-external-load-balancers=

	                    topology.hostpath.csi/node=addons-221000

	Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock

	                    node.alpha.kubernetes.io/ttl: 0

	                    volumes.kubernetes.io/controller-managed-attach-detach: true

	CreationTimestamp:  Mon, 18 Sep 2023 18:52:43 +0000

	Taints:             <none>

	Unschedulable:      false

	Lease:

	  HolderIdentity:  addons-221000

	  AcquireTime:     <unset>

	  RenewTime:       Mon, 18 Sep 2023 19:12:19 +0000

	Conditions:

	  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message

	  ----             ------  -----------------                 ------------------                ------                       -------

	  MemoryPressure   False   Mon, 18 Sep 2023 19:11:52 +0000   Mon, 18 Sep 2023 18:52:43 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available

	  DiskPressure     False   Mon, 18 Sep 2023 19:11:52 +0000   Mon, 18 Sep 2023 18:52:43 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure

	  PIDPressure      False   Mon, 18 Sep 2023 19:11:52 +0000   Mon, 18 Sep 2023 18:52:43 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available

	  Ready            True    Mon, 18 Sep 2023 19:11:52 +0000   Mon, 18 Sep 2023 18:52:48 +0000   KubeletReady                 kubelet is posting ready status

	Addresses:

	  InternalIP:  192.168.105.2

	  Hostname:    addons-221000

	Capacity:

	  cpu:                2

	  ephemeral-storage:  17784760Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3905012Ki

	  pods:               110

	Allocatable:

	  cpu:                2

	  ephemeral-storage:  17784760Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3905012Ki

	  pods:               110

	System Info:

	  Machine ID:                 605bc2fc72a045ae88e907db961da3d3

	  System UUID:                605bc2fc72a045ae88e907db961da3d3

	  Boot ID:                    6a9990c2-fe5e-48d8-97ca-ea50d8c8e3b8

	  Kernel Version:             5.10.57

	  OS Image:                   Buildroot 2021.02.12

	  Operating System:           linux

	  Architecture:               arm64

	  Container Runtime Version:  docker://24.0.6

	  Kubelet Version:            v1.28.2

	  Kube-Proxy Version:         v1.28.2

	PodCIDR:                      10.244.0.0/24

	PodCIDRs:                     10.244.0.0/24

	Non-terminated Pods:          (9 in total)

	  Namespace                   Name                                     CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age

	  ---------                   ----                                     ------------  ----------  ---------------  -------------  ---

	  gcp-auth                    gcp-auth-d4c87556c-2vm8d                 0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	  headlamp                    headlamp-699c48fb74-6fg5z                0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         84s

	  kube-system                 coredns-5dd5756b68-mbgns                 100m (5%!)(MISSING)     0 (0%!)(MISSING)      70Mi (1%!)(MISSING)        170Mi (4%!)(MISSING)     19m

	  kube-system                 etcd-addons-221000                       100m (5%!)(MISSING)     0 (0%!)(MISSING)      100Mi (2%!)(MISSING)       0 (0%!)(MISSING)         19m

	  kube-system                 kube-apiserver-addons-221000             250m (12%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	  kube-system                 kube-controller-manager-addons-221000    200m (10%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	  kube-system                 kube-proxy-q7gqn                         0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	  kube-system                 kube-scheduler-addons-221000             100m (5%!)(MISSING)     0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	  kube-system                 storage-provisioner                      0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         19m

	Allocated resources:

	  (Total limits may be over 100 percent, i.e., overcommitted.)

	  Resource           Requests    Limits

	  --------           --------    ------

	  cpu                750m (37%!)(MISSING)  0 (0%!)(MISSING)

	  memory             170Mi (4%!)(MISSING)  170Mi (4%!)(MISSING)

	  ephemeral-storage  0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-1Gi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-2Mi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-32Mi     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-64Ki     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	Events:

	  Type    Reason                   Age   From             Message

	  ----    ------                   ----  ----             -------

	  Normal  Starting                 19m   kube-proxy       

	  Normal  Starting                 19m   kubelet          Starting kubelet.

	  Normal  NodeAllocatableEnforced  19m   kubelet          Updated Node Allocatable limit across pods

	  Normal  NodeHasSufficientMemory  19m   kubelet          Node addons-221000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    19m   kubelet          Node addons-221000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     19m   kubelet          Node addons-221000 status is now: NodeHasSufficientPID

	  Normal  NodeReady                19m   kubelet          Node addons-221000 status is now: NodeReady

	  Normal  RegisteredNode           19m   node-controller  Node addons-221000 event: Registered Node addons-221000 in Controller

	* 

	* ==> dmesg <==

	* [  +0.641705] EINJ: EINJ table not found.

	[  +0.512113] systemd-fstab-generator[117]: Ignoring "noauto" for root device

	[  +0.044199] systemd[1]: systemd-journald.service: unit configures an IP firewall, but the local system does not support BPF/cgroup firewalling.

	[  +0.000795] systemd[1]: (This warning is only shown for the first unit using IP firewalling.)

	[  +3.076142] systemd-fstab-generator[480]: Ignoring "noauto" for root device

	[  +0.068934] systemd-fstab-generator[491]: Ignoring "noauto" for root device

	[  +0.415221] systemd-fstab-generator[666]: Ignoring "noauto" for root device

	[  +0.169313] systemd-fstab-generator[702]: Ignoring "noauto" for root device

	[  +0.075007] systemd-fstab-generator[713]: Ignoring "noauto" for root device

	[  +0.084392] systemd-fstab-generator[726]: Ignoring "noauto" for root device

	[  +1.144805] kauditd_printk_skb: 17 callbacks suppressed

	[  +0.094980] systemd-fstab-generator[914]: Ignoring "noauto" for root device

	[  +0.076494] systemd-fstab-generator[925]: Ignoring "noauto" for root device

	[  +0.074272] systemd-fstab-generator[936]: Ignoring "noauto" for root device

	[  +0.076208] systemd-fstab-generator[947]: Ignoring "noauto" for root device

	[  +0.088287] systemd-fstab-generator[988]: Ignoring "noauto" for root device

	[  +2.538937] systemd-fstab-generator[1093]: Ignoring "noauto" for root device

	[  +2.473246] kauditd_printk_skb: 29 callbacks suppressed

	[  +3.092849] systemd-fstab-generator[1411]: Ignoring "noauto" for root device

	[  +4.633357] systemd-fstab-generator[2291]: Ignoring "noauto" for root device

	[Sep18 18:53] kauditd_printk_skb: 41 callbacks suppressed

	[  +1.646092] TCP: eth0: Driver has suspect GRO implementation, TCP performance may be compromised.

	[  +4.523270] kauditd_printk_skb: 55 callbacks suppressed

	[ +11.548658] kauditd_printk_skb: 8 callbacks suppressed

	[Sep18 19:12] kauditd_printk_skb: 12 callbacks suppressed

	* 

	* ==> etcd [2a7ec1fe69df] <==

	* {"level":"info","ts":"2023-09-18T18:52:43.023404Z","caller":"etcdserver/server.go:2571","msg":"setting up initial cluster version using v2 API","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:52:43.023573Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2023-09-18T18:52:43.02399Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.105.2:2379"}

	{"level":"info","ts":"2023-09-18T18:52:43.024241Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}

	{"level":"info","ts":"2023-09-18T18:52:43.02431Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}

	{"level":"info","ts":"2023-09-18T18:52:43.024433Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"6e03e7863b4f9c54","local-member-id":"c46d288d2fcb0590","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:52:43.024491Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:52:43.025343Z","caller":"etcdserver/server.go:2595","msg":"cluster version is updated","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T18:53:12.839558Z","caller":"traceutil/trace.go:171","msg":"trace[499305584] linearizableReadLoop","detail":"{readStateIndex:693; appliedIndex:692; }","duration":"193.895541ms","start":"2023-09-18T18:53:12.645653Z","end":"2023-09-18T18:53:12.839549Z","steps":["trace[499305584] 'read index received'  (duration: 193.791791ms)","trace[499305584] 'applied index is now lower than readState.Index'  (duration: 103.417µs)"],"step_count":2}

	{"level":"info","ts":"2023-09-18T18:53:12.839607Z","caller":"traceutil/trace.go:171","msg":"trace[593448589] transaction","detail":"{read_only:false; response_revision:671; number_of_response:1; }","duration":"309.762625ms","start":"2023-09-18T18:53:12.529838Z","end":"2023-09-18T18:53:12.839601Z","steps":["trace[593448589] 'process raft request'  (duration: 309.63075ms)"],"step_count":1}

	{"level":"warn","ts":"2023-09-18T18:53:12.839655Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"193.989167ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/gcp-auth/\" range_end:\"/registry/pods/gcp-auth0\" ","response":"range_response_count:3 size:10532"}

	{"level":"info","ts":"2023-09-18T18:53:12.839671Z","caller":"traceutil/trace.go:171","msg":"trace[1312627899] range","detail":"{range_begin:/registry/pods/gcp-auth/; range_end:/registry/pods/gcp-auth0; response_count:3; response_revision:671; }","duration":"194.031ms","start":"2023-09-18T18:53:12.645637Z","end":"2023-09-18T18:53:12.839668Z","steps":["trace[1312627899] 'agreement among raft nodes before linearized reading'  (duration: 193.963833ms)"],"step_count":1}

	{"level":"warn","ts":"2023-09-18T18:53:12.8398Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2023-09-18T18:53:12.529832Z","time spent":"309.7915ms","remote":"127.0.0.1:54556","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1098,"response count":0,"response size":39,"request content":"compare:<target:MOD key:\"/registry/services/endpoints/kube-system/k8s.io-minikube-hostpath\" mod_revision:669 > success:<request_put:<key:\"/registry/services/endpoints/kube-system/k8s.io-minikube-hostpath\" value_size:1025 >> failure:<request_range:<key:\"/registry/services/endpoints/kube-system/k8s.io-minikube-hostpath\" > >"}

	{"level":"info","ts":"2023-09-18T18:53:15.058529Z","caller":"traceutil/trace.go:171","msg":"trace[42488278] linearizableReadLoop","detail":"{readStateIndex:694; appliedIndex:693; }","duration":"207.837416ms","start":"2023-09-18T18:53:14.850683Z","end":"2023-09-18T18:53:15.05852Z","steps":["trace[42488278] 'read index received'  (duration: 207.76525ms)","trace[42488278] 'applied index is now lower than readState.Index'  (duration: 71.708µs)"],"step_count":2}

	{"level":"warn","ts":"2023-09-18T18:53:15.058651Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"207.966125ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/\" range_end:\"/registry/pods/kube-system0\" ","response":"range_response_count:13 size:63849"}

	{"level":"info","ts":"2023-09-18T18:53:15.058696Z","caller":"traceutil/trace.go:171","msg":"trace[2008509018] range","detail":"{range_begin:/registry/pods/kube-system/; range_end:/registry/pods/kube-system0; response_count:13; response_revision:672; }","duration":"208.018666ms","start":"2023-09-18T18:53:14.850673Z","end":"2023-09-18T18:53:15.058691Z","steps":["trace[2008509018] 'agreement among raft nodes before linearized reading'  (duration: 207.886666ms)"],"step_count":1}

	{"level":"info","ts":"2023-09-18T18:53:15.058824Z","caller":"traceutil/trace.go:171","msg":"trace[1590636669] transaction","detail":"{read_only:false; response_revision:672; number_of_response:1; }","duration":"214.775084ms","start":"2023-09-18T18:53:14.844046Z","end":"2023-09-18T18:53:15.058821Z","steps":["trace[1590636669] 'process raft request'  (duration: 214.424584ms)"],"step_count":1}

	{"level":"info","ts":"2023-09-18T19:02:43.443637Z","caller":"mvcc/index.go:214","msg":"compact tree index","revision":1094}

	{"level":"info","ts":"2023-09-18T19:02:43.458272Z","caller":"mvcc/kvstore_compaction.go:66","msg":"finished scheduled compaction","compact-revision":1094,"took":"14.26046ms","hash":1108450172}

	{"level":"info","ts":"2023-09-18T19:02:43.458292Z","caller":"mvcc/hash.go:137","msg":"storing new hash","hash":1108450172,"revision":1094,"compact-revision":-1}

	{"level":"info","ts":"2023-09-18T19:07:43.446092Z","caller":"mvcc/index.go:214","msg":"compact tree index","revision":1454}

	{"level":"info","ts":"2023-09-18T19:07:43.44678Z","caller":"mvcc/kvstore_compaction.go:66","msg":"finished scheduled compaction","compact-revision":1454,"took":"494.332µs","hash":2695871163}

	{"level":"info","ts":"2023-09-18T19:07:43.446796Z","caller":"mvcc/hash.go:137","msg":"storing new hash","hash":2695871163,"revision":1454,"compact-revision":1094}

	{"level":"info","ts":"2023-09-18T19:11:06.589478Z","caller":"traceutil/trace.go:171","msg":"trace[847243809] transaction","detail":"{read_only:false; response_revision:2088; number_of_response:1; }","duration":"105.795246ms","start":"2023-09-18T19:11:06.483671Z","end":"2023-09-18T19:11:06.589466Z","steps":["trace[847243809] 'process raft request'  (duration: 76.680577ms)","trace[847243809] 'compare'  (duration: 28.988461ms)"],"step_count":2}

	{"level":"info","ts":"2023-09-18T19:11:19.569525Z","caller":"traceutil/trace.go:171","msg":"trace[199883351] transaction","detail":"{read_only:false; response_revision:2131; number_of_response:1; }","duration":"161.881881ms","start":"2023-09-18T19:11:19.407634Z","end":"2023-09-18T19:11:19.569516Z","steps":["trace[199883351] 'process raft request'  (duration: 161.812756ms)"],"step_count":1}

	* 

	* ==> gcp-auth [6009e365438d] <==

	* 2023/09/18 18:53:42 GCP Auth Webhook started!

	2023/09/18 19:11:01 Ready to marshal response ...

	2023/09/18 19:11:01 Ready to write response ...

	2023/09/18 19:11:01 Ready to marshal response ...

	2023/09/18 19:11:01 Ready to write response ...

	2023/09/18 19:11:01 Ready to marshal response ...

	2023/09/18 19:11:01 Ready to write response ...

	2023/09/18 19:11:15 Ready to marshal response ...

	2023/09/18 19:11:15 Ready to write response ...

	2023/09/18 19:11:48 Ready to marshal response ...

	2023/09/18 19:11:48 Ready to write response ...

	* 

	* ==> kernel <==

	*  19:12:25 up 19 min,  0 users,  load average: 0.21, 0.19, 0.17

	Linux addons-221000 5.10.57 #1 SMP PREEMPT Fri Sep 15 19:03:18 UTC 2023 aarch64 GNU/Linux

	PRETTY_NAME="Buildroot 2021.02.12"

	* 

	* ==> kube-apiserver [4a8eb16a561d] <==

	* I0918 19:10:44.200176       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:11:01.618081       1 alloc.go:330] "allocated clusterIPs" service="headlamp/headlamp" clusterIPs={"IPv4":"10.105.100.41"}

	I0918 19:11:27.087628       1 controller.go:624] quota admission added evaluator for: volumesnapshots.snapshot.storage.k8s.io

	I0918 19:11:44.199807       1 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.823931       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.823948       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.825604       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.825722       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.833256       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.833284       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.846841       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.846862       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.847044       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.847095       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.856162       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.856179       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0918 19:12:03.857357       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0918 19:12:03.857367       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	W0918 19:12:04.848160       1 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io

	W0918 19:12:04.857030       1 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io

	W0918 19:12:04.865591       1 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io

	I0918 19:12:11.428741       1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Nothing (removed from the queue).

	I0918 19:12:14.330149       1 handler.go:232] Adding GroupVersion gadget.kinvolk.io v1alpha1 to ResourceManager

	I0918 19:12:14.335152       1 handler.go:232] Adding GroupVersion gadget.kinvolk.io v1alpha1 to ResourceManager

	W0918 19:12:15.339237       1 cacher.go:171] Terminating all watchers from cacher traces.gadget.kinvolk.io

	* 

	* ==> kube-controller-manager [cb85fb8fd00c] <==

	* W0918 19:12:06.404971       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:06.404987       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:07.765663       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:07.765681       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:09.044305       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:09.044327       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	I0918 19:12:09.124215       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kube-system/metrics-server-7c66d45ddc" duration="1.875µs"

	W0918 19:12:09.340711       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:09.340733       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:12.778356       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:12.778375       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:13.814310       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:13.814362       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:15.053536       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:15.053562       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:15.339981       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:16.531417       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:16.531440       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:18.180272       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:18.180294       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:20.842470       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:20.842486       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0918 19:12:22.068003       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0918 19:12:22.068025       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	I0918 19:12:24.363587       1 namespace_controller.go:182] "Namespace has been deleted" namespace="gadget"

	* 

	* ==> kube-proxy [3b8b236037bf] <==

	* I0918 18:53:00.480730       1 server_others.go:69] "Using iptables proxy"

	I0918 18:53:00.488175       1 node.go:141] Successfully retrieved node IP: 192.168.105.2

	I0918 18:53:00.508601       1 server_others.go:121] "No iptables support for family" ipFamily="IPv6"

	I0918 18:53:00.508621       1 server.go:634] "kube-proxy running in single-stack mode" ipFamily="IPv4"

	I0918 18:53:00.509405       1 server_others.go:152] "Using iptables Proxier"

	I0918 18:53:00.509431       1 proxier.go:251] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses"

	I0918 18:53:00.509518       1 server.go:846] "Version info" version="v1.28.2"

	I0918 18:53:00.509524       1 server.go:848] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0918 18:53:00.512081       1 config.go:188] "Starting service config controller"

	I0918 18:53:00.512089       1 shared_informer.go:311] Waiting for caches to sync for service config

	I0918 18:53:00.512105       1 config.go:97] "Starting endpoint slice config controller"

	I0918 18:53:00.512107       1 shared_informer.go:311] Waiting for caches to sync for endpoint slice config

	I0918 18:53:00.512284       1 config.go:315] "Starting node config controller"

	I0918 18:53:00.512287       1 shared_informer.go:311] Waiting for caches to sync for node config

	I0918 18:53:01.015326       1 shared_informer.go:318] Caches are synced for node config

	I0918 18:53:01.015354       1 shared_informer.go:318] Caches are synced for service config

	I0918 18:53:01.015384       1 shared_informer.go:318] Caches are synced for endpoint slice config

	* 

	* ==> kube-scheduler [17d16f9191cb] <==

	* W0918 18:52:43.862760       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	E0918 18:52:43.863344       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	W0918 18:52:43.862808       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	E0918 18:52:43.863392       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	W0918 18:52:43.862823       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	E0918 18:52:43.863442       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	W0918 18:52:43.862835       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	E0918 18:52:43.863451       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	W0918 18:52:43.862850       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope

	E0918 18:52:43.863466       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope

	W0918 18:52:43.862867       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	E0918 18:52:43.863530       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	W0918 18:52:43.862878       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	E0918 18:52:43.863565       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	W0918 18:52:43.862692       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	E0918 18:52:43.863575       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	W0918 18:52:43.863300       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	E0918 18:52:43.863661       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	W0918 18:52:44.751836       1 reflector.go:535] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	E0918 18:52:44.751859       1 reflector.go:147] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	W0918 18:52:44.752492       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope

	E0918 18:52:44.752505       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope

	W0918 18:52:44.760405       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	E0918 18:52:44.760417       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	I0918 18:52:46.760908       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	* 

	* ==> kubelet <==

	* -- Journal begins at Mon 2023-09-18 18:52:28 UTC, ends at Mon 2023-09-18 19:12:25 UTC. --

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524125    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-run\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524134    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-host\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524142    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"bpffs\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-bpffs\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524150    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"modules\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-modules\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524157    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"debugfs\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-debugfs\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524165    2297 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"cgroup\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-cgroup\") pod \"7eea1399-50e6-40e6-8424-bacd2f982bff\" (UID: \"7eea1399-50e6-40e6-8424-bacd2f982bff\") "

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524209    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-cgroup" (OuterVolumeSpecName: "cgroup") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "cgroup". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524422    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-bpffs" (OuterVolumeSpecName: "bpffs") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "bpffs". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524436    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-run" (OuterVolumeSpecName: "run") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "run". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524443    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-host" (OuterVolumeSpecName: "host") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "host". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524450    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-modules" (OuterVolumeSpecName: "modules") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "modules". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.524522    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-debugfs" (OuterVolumeSpecName: "debugfs") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "debugfs". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.525925    2297 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eea1399-50e6-40e6-8424-bacd2f982bff-kube-api-access-pfdln" (OuterVolumeSpecName: "kube-api-access-pfdln") pod "7eea1399-50e6-40e6-8424-bacd2f982bff" (UID: "7eea1399-50e6-40e6-8424-bacd2f982bff"). InnerVolumeSpecName "kube-api-access-pfdln". PluginName "kubernetes.io/projected", VolumeGidValue ""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625197    2297 reconciler_common.go:300] "Volume detached for volume \"bpffs\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-bpffs\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625210    2297 reconciler_common.go:300] "Volume detached for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-host\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625217    2297 reconciler_common.go:300] "Volume detached for volume \"cgroup\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-cgroup\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625222    2297 reconciler_common.go:300] "Volume detached for volume \"modules\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-modules\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625227    2297 reconciler_common.go:300] "Volume detached for volume \"debugfs\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-debugfs\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625233    2297 reconciler_common.go:300] "Volume detached for volume \"kube-api-access-pfdln\" (UniqueName: \"kubernetes.io/projected/7eea1399-50e6-40e6-8424-bacd2f982bff-kube-api-access-pfdln\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:14 addons-221000 kubelet[2297]: I0918 19:12:14.625238    2297 reconciler_common.go:300] "Volume detached for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7eea1399-50e6-40e6-8424-bacd2f982bff-run\") on node \"addons-221000\" DevicePath \"\""

	Sep 18 19:12:15 addons-221000 kubelet[2297]: I0918 19:12:15.398217    2297 scope.go:117] "RemoveContainer" containerID="afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a"

	Sep 18 19:12:15 addons-221000 kubelet[2297]: I0918 19:12:15.407228    2297 scope.go:117] "RemoveContainer" containerID="afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a"

	Sep 18 19:12:15 addons-221000 kubelet[2297]: E0918 19:12:15.407605    2297 remote_runtime.go:432] "ContainerStatus from runtime service failed" err="rpc error: code = Unknown desc = Error response from daemon: No such container: afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a" containerID="afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a"

	Sep 18 19:12:15 addons-221000 kubelet[2297]: I0918 19:12:15.407627    2297 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"docker","ID":"afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a"} err="failed to get container status \"afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a\": rpc error: code = Unknown desc = Error response from daemon: No such container: afd4040b11d6ebc91070b0c00e982de4e986fedf052239b165939eb011f4f31a"

	Sep 18 19:12:16 addons-221000 kubelet[2297]: I0918 19:12:16.788684    2297 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7eea1399-50e6-40e6-8424-bacd2f982bff" path="/var/lib/kubelet/pods/7eea1399-50e6-40e6-8424-bacd2f982bff/volumes"

	* 

	* ==> storage-provisioner [96b1e19d34d0] <==

	* I0918 18:53:02.494940       1 storage_provisioner.go:116] Initializing the minikube storage provisioner...

	I0918 18:53:02.503242       1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!

	I0918 18:53:02.503674       1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...

	I0918 18:53:02.507046       1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath

	I0918 18:53:02.507107       1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_addons-221000_d01c2a4d-820c-47b2-b527-e47d47f0ebf9!

	I0918 18:53:02.507893       1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"993ff549-fac0-4a25-b8bc-6e13c7f3eb70", APIVersion:"v1", ResourceVersion:"490", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' addons-221000_d01c2a4d-820c-47b2-b527-e47d47f0ebf9 became leader

	I0918 18:53:02.607443       1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_addons-221000_d01c2a4d-820c-47b2-b527-e47d47f0ebf9!

-- /stdout --

helpers_test.go:254: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.APIServer}} -p addons-221000 -n addons-221000

helpers_test.go:261: (dbg) Run:  kubectl --context addons-221000 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running

helpers_test.go:285: <<< TestAddons/parallel/CloudSpanner FAILED: end of post-mortem logs <<<

helpers_test.go:286: ---------------------/post-mortem---------------------------------

--- FAIL: TestAddons/parallel/CloudSpanner (805.10s)

=== RUN   TestCertOptions

=== PAUSE TestCertOptions

=== CONT  TestCertOptions

cert_options_test.go:49: (dbg) Run:  out/minikube-darwin-arm64 start -p cert-options-770000 --memory=2048 --apiserver-ips=127.0.0.1 --apiserver-ips=192.168.15.15 --apiserver-names=localhost --apiserver-names=www.google.com --apiserver-port=8555 --driver=qemu2 

E0918 12:38:38.750760    1668 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/ingress-addon-legacy-356000/client.crt: no such file or directory

cert_options_test.go:49: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p cert-options-770000 --memory=2048 --apiserver-ips=127.0.0.1 --apiserver-ips=192.168.15.15 --apiserver-names=localhost --apiserver-names=www.google.com --apiserver-port=8555 --driver=qemu2 : exit status 80 (9.7147185s)

-- stdout --

	* [cert-options-770000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	  - MINIKUBE_LOCATION=17263

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting control plane node cert-options-770000 in cluster cert-options-770000

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "cert-options-770000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p cert-options-770000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

cert_options_test.go:51: failed to start minikube with args: "out/minikube-darwin-arm64 start -p cert-options-770000 --memory=2048 --apiserver-ips=127.0.0.1 --apiserver-ips=192.168.15.15 --apiserver-names=localhost --apiserver-names=www.google.com --apiserver-port=8555 --driver=qemu2 " : exit status 80

cert_options_test.go:60: (dbg) Run:  out/minikube-darwin-arm64 -p cert-options-770000 ssh "openssl x509 -text -noout -in /var/lib/minikube/certs/apiserver.crt"

cert_options_test.go:60: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p cert-options-770000 ssh "openssl x509 -text -noout -in /var/lib/minikube/certs/apiserver.crt": exit status 89 (76.41525ms)

-- stdout --

	* The control plane node must be running for this command

	  To start a cluster, run: "minikube start -p cert-options-770000"

-- /stdout --

cert_options_test.go:62: failed to read apiserver cert inside minikube. args "out/minikube-darwin-arm64 -p cert-options-770000 ssh \"openssl x509 -text -noout -in /var/lib/minikube/certs/apiserver.crt\"": exit status 89

cert_options_test.go:69: apiserver cert does not include 127.0.0.1 in SAN.

cert_options_test.go:69: apiserver cert does not include 192.168.15.15 in SAN.

cert_options_test.go:69: apiserver cert does not include localhost in SAN.

cert_options_test.go:69: apiserver cert does not include www.google.com in SAN.

cert_options_test.go:88: (dbg) Run:  kubectl --context cert-options-770000 config view

cert_options_test.go:93: Kubeconfig apiserver server port incorrect. Output of 

'kubectl config view' = "\n-- stdout --\n\tapiVersion: v1\n\tclusters: null\n\tcontexts: null\n\tcurrent-context: \"\"\n\tkind: Config\n\tpreferences: {}\n\tusers: null\n\n-- /stdout --"

cert_options_test.go:100: (dbg) Run:  out/minikube-darwin-arm64 ssh -p cert-options-770000 -- "sudo cat /etc/kubernetes/admin.conf"

cert_options_test.go:100: (dbg) Non-zero exit: out/minikube-darwin-arm64 ssh -p cert-options-770000 -- "sudo cat /etc/kubernetes/admin.conf": exit status 89 (38.979667ms)

-- stdout --

	* The control plane node must be running for this command

	  To start a cluster, run: "minikube start -p cert-options-770000"

-- /stdout --

cert_options_test.go:102: failed to SSH to minikube with args: "out/minikube-darwin-arm64 ssh -p cert-options-770000 -- \"sudo cat /etc/kubernetes/admin.conf\"" : exit status 89

cert_options_test.go:106: Internal minikube kubeconfig (admin.conf) does not contains the right api port. 

-- stdout --

	* The control plane node must be running for this command

	  To start a cluster, run: "minikube start -p cert-options-770000"

-- /stdout --

cert_options_test.go:109: *** TestCertOptions FAILED at 2023-09-18 12:38:48.315116 -0700 PDT m=+2824.792433334

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p cert-options-770000 -n cert-options-770000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p cert-options-770000 -n cert-options-770000: exit status 7 (28.081125ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "cert-options-770000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "cert-options-770000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p cert-options-770000

--- FAIL: TestCertOptions (9.99s)

E0918 12:39:00.235656    1668 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/addons-221000/client.crt: no such file or directory

E0918 12:40:01.826050    1668 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/ingress-addon-legacy-356000/client.crt: no such file or directory

E0918 12:40:38.906552    1668 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/functional-847000/client.crt: no such file or directory

=== RUN   TestCertExpiration

=== PAUSE TestCertExpiration

=== CONT  TestCertExpiration

cert_options_test.go:123: (dbg) Run:  out/minikube-darwin-arm64 start -p cert-expiration-336000 --memory=2048 --cert-expiration=3m --driver=qemu2 

cert_options_test.go:123: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p cert-expiration-336000 --memory=2048 --cert-expiration=3m --driver=qemu2 : exit status 80 (9.695404083s)

-- stdout --

	* [cert-expiration-336000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	  - MINIKUBE_LOCATION=17263

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting control plane node cert-expiration-336000 in cluster cert-expiration-336000

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "cert-expiration-336000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p cert-expiration-336000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

cert_options_test.go:125: failed to start minikube with args: "out/minikube-darwin-arm64 start -p cert-expiration-336000 --memory=2048 --cert-expiration=3m --driver=qemu2 " : exit status 80

cert_options_test.go:131: (dbg) Run:  out/minikube-darwin-arm64 start -p cert-expiration-336000 --memory=2048 --cert-expiration=8760h --driver=qemu2 

cert_options_test.go:131: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p cert-expiration-336000 --memory=2048 --cert-expiration=8760h --driver=qemu2 : exit status 80 (5.222219292s)

-- stdout --

	* [cert-expiration-336000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	  - MINIKUBE_LOCATION=17263

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on existing profile

	* Starting control plane node cert-expiration-336000 in cluster cert-expiration-336000

	* Restarting existing qemu2 VM for "cert-expiration-336000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Restarting existing qemu2 VM for "cert-expiration-336000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p cert-expiration-336000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

cert_options_test.go:133: failed to start minikube after cert expiration: "out/minikube-darwin-arm64 start -p cert-expiration-336000 --memory=2048 --cert-expiration=8760h --driver=qemu2 " : exit status 80

cert_options_test.go:136: minikube start output did not warn about expired certs: 

-- stdout --

	* [cert-expiration-336000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	  - MINIKUBE_LOCATION=17263

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on existing profile

	* Starting control plane node cert-expiration-336000 in cluster cert-expiration-336000

	* Restarting existing qemu2 VM for "cert-expiration-336000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Restarting existing qemu2 VM for "cert-expiration-336000" ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	! StartHost failed, but will try again: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p cert-expiration-336000" may fix it: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: driver start: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* 

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

cert_options_test.go:138: *** TestCertExpiration FAILED at 2023-09-18 12:41:48.44217 -0700 PDT m=+3004.922863501

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p cert-expiration-336000 -n cert-expiration-336000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p cert-expiration-336000 -n cert-expiration-336000: exit status 7 (64.821917ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "cert-expiration-336000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "cert-expiration-336000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p cert-expiration-336000

--- FAIL: TestCertExpiration (195.09s)

=== RUN   TestDockerFlags

=== PAUSE TestDockerFlags

=== CONT  TestDockerFlags

docker_test.go:51: (dbg) Run:  out/minikube-darwin-arm64 start -p docker-flags-564000 --cache-images=false --memory=2048 --install-addons=false --wait=false --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug --docker-opt=icc=true --alsologtostderr -v=5 --driver=qemu2 

docker_test.go:51: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p docker-flags-564000 --cache-images=false --memory=2048 --install-addons=false --wait=false --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug --docker-opt=icc=true --alsologtostderr -v=5 --driver=qemu2 : exit status 80 (9.808334042s)

-- stdout --

	* [docker-flags-564000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	  - MINIKUBE_LOCATION=17263

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting control plane node docker-flags-564000 in cluster docker-flags-564000

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "docker-flags-564000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0918 12:38:28.426292    4043 out.go:296] Setting OutFile to fd 1 ...

	I0918 12:38:28.426408    4043 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:38:28.426412    4043 out.go:309] Setting ErrFile to fd 2...

	I0918 12:38:28.426415    4043 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:38:28.426542    4043 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17263-1251/.minikube/bin

	I0918 12:38:28.427556    4043 out.go:303] Setting JSON to false

	I0918 12:38:28.442529    4043 start.go:128] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":4082,"bootTime":1695061826,"procs":403,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.5.2","kernelVersion":"22.6.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0918 12:38:28.442619    4043 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0918 12:38:28.447802    4043 out.go:177] * [docker-flags-564000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	I0918 12:38:28.455774    4043 out.go:177]   - MINIKUBE_LOCATION=17263

	I0918 12:38:28.460759    4043 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 12:38:28.455861    4043 notify.go:220] Checking for updates...

	I0918 12:38:28.466683    4043 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0918 12:38:28.469742    4043 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0918 12:38:28.472710    4043 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	I0918 12:38:28.475723    4043 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0918 12:38:28.479097    4043 config.go:182] Loaded profile config "force-systemd-flag-847000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 12:38:28.479147    4043 driver.go:373] Setting default libvirt URI to qemu:///system

	I0918 12:38:28.483699    4043 out.go:177] * Using the qemu2 driver based on user configuration

	I0918 12:38:28.502732    4043 start.go:298] selected driver: qemu2

	I0918 12:38:28.502739    4043 start.go:902] validating driver "qemu2" against <nil>

	I0918 12:38:28.502745    4043 start.go:913] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0918 12:38:28.504758    4043 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0918 12:38:28.507682    4043 out.go:177] * Automatically selected the socket_vmnet network

	I0918 12:38:28.510804    4043 start_flags.go:917] Waiting for no components: map[apiserver:false apps_running:false default_sa:false extra:false kubelet:false node_ready:false system_pods:false]

	I0918 12:38:28.510832    4043 cni.go:84] Creating CNI manager for ""

	I0918 12:38:28.510847    4043 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 12:38:28.510850    4043 start_flags.go:316] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0918 12:38:28.510855    4043 start_flags.go:321] config:

	{Name:docker-flags-564000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[FOO=BAR BAZ=BAT] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[debug icc=true] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:docker-flags-564000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain

:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:false EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:false apps_running:false default_sa:false extra:false kubelet:false node_ready:false system_pods:false] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/ru

n/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 12:38:28.515030    4043 iso.go:125] acquiring lock: {Name:mk34e23c181861c65264ec384f06e6cfa001aa08 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0918 12:38:28.521714    4043 out.go:177] * Starting control plane node docker-flags-564000 in cluster docker-flags-564000

	I0918 12:38:28.525723    4043 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 12:38:28.525743    4043 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4

	I0918 12:38:28.525756    4043 cache.go:57] Caching tarball of preloaded images

	I0918 12:38:28.525833    4043 preload.go:174] Found /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0918 12:38:28.525838    4043 cache.go:60] Finished verifying existence of preloaded tar for  v1.28.2 on docker

	I0918 12:38:28.525904    4043 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/docker-flags-564000/config.json ...

	I0918 12:38:28.525917    4043 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/docker-flags-564000/config.json: {Name:mkff536612056e9c0b51f844675eae806b0747ef Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:38:28.526139    4043 start.go:365] acquiring machines lock for docker-flags-564000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 12:38:28.526171    4043 start.go:369] acquired machines lock for "docker-flags-564000" in 25.333µs

	I0918 12:38:28.526186    4043 start.go:93] Provisioning new machine with config: &{Name:docker-flags-564000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[FOO=BAR BAZ=BAT] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[debug icc=true] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root

SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:docker-flags-564000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:false EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:false apps_running:false default_sa:false extra:false kubelet:false node_ready:false system_pods:false] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion

:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 12:38:28.526222    4043 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 12:38:28.534715    4043 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0918 12:38:28.551387    4043 start.go:159] libmachine.API.Create for "docker-flags-564000" (driver="qemu2")

	I0918 12:38:28.551411    4043 client.go:168] LocalClient.Create starting

	I0918 12:38:28.551467    4043 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 12:38:28.551492    4043 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:28.551502    4043 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:28.551543    4043 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 12:38:28.551562    4043 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:28.551569    4043 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:28.551897    4043 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 12:38:28.688633    4043 main.go:141] libmachine: Creating SSH key...

	I0918 12:38:28.749372    4043 main.go:141] libmachine: Creating Disk image...

	I0918 12:38:28.749379    4043 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 12:38:28.749529    4043 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2

	I0918 12:38:28.757969    4043 main.go:141] libmachine: STDOUT: 

	I0918 12:38:28.757984    4043 main.go:141] libmachine: STDERR: 

	I0918 12:38:28.758029    4043 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2 +20000M

	I0918 12:38:28.765260    4043 main.go:141] libmachine: STDOUT: Image resized.

	I0918 12:38:28.765273    4043 main.go:141] libmachine: STDERR: 

	I0918 12:38:28.765293    4043 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2

	I0918 12:38:28.765301    4043 main.go:141] libmachine: Starting QEMU VM...

	I0918 12:38:28.765331    4043 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/qemu.pid -device virtio-net-pci,netdev=net0,mac=ca:29:d7:ca:f7:df -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2

	I0918 12:38:28.766841    4043 main.go:141] libmachine: STDOUT: 

	I0918 12:38:28.766854    4043 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0918 12:38:28.766872    4043 client.go:171] LocalClient.Create took 215.4585ms

	I0918 12:38:30.769072    4043 start.go:128] duration metric: createHost completed in 2.242808458s

	I0918 12:38:30.769131    4043 start.go:83] releasing machines lock for "docker-flags-564000", held for 2.242992958s

	W0918 12:38:30.769187    4043 start.go:688] error starting host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:30.781095    4043 out.go:177] * Deleting "docker-flags-564000" in qemu2 ...

	W0918 12:38:30.801944    4043 out.go:239] ! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:30.801970    4043 start.go:703] Will try again in 5 seconds ...

	I0918 12:38:35.804085    4043 start.go:365] acquiring machines lock for docker-flags-564000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 12:38:35.887436    4043 start.go:369] acquired machines lock for "docker-flags-564000" in 83.273458ms

	I0918 12:38:35.887593    4043 start.go:93] Provisioning new machine with config: &{Name:docker-flags-564000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[FOO=BAR BAZ=BAT] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[debug icc=true] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root

SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:docker-flags-564000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:false EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:false apps_running:false default_sa:false extra:false kubelet:false node_ready:false system_pods:false] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion

:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 12:38:35.887804    4043 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 12:38:35.896200    4043 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0918 12:38:35.942910    4043 start.go:159] libmachine.API.Create for "docker-flags-564000" (driver="qemu2")

	I0918 12:38:35.943236    4043 client.go:168] LocalClient.Create starting

	I0918 12:38:35.943495    4043 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 12:38:35.943592    4043 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:35.943630    4043 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:35.943739    4043 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 12:38:35.943790    4043 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:35.943808    4043 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:35.944661    4043 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 12:38:36.075710    4043 main.go:141] libmachine: Creating SSH key...

	I0918 12:38:36.146809    4043 main.go:141] libmachine: Creating Disk image...

	I0918 12:38:36.146815    4043 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 12:38:36.146950    4043 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2

	I0918 12:38:36.155402    4043 main.go:141] libmachine: STDOUT: 

	I0918 12:38:36.155416    4043 main.go:141] libmachine: STDERR: 

	I0918 12:38:36.155465    4043 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2 +20000M

	I0918 12:38:36.162610    4043 main.go:141] libmachine: STDOUT: Image resized.

	I0918 12:38:36.162636    4043 main.go:141] libmachine: STDERR: 

	I0918 12:38:36.162654    4043 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2

	I0918 12:38:36.162662    4043 main.go:141] libmachine: Starting QEMU VM...

	I0918 12:38:36.162710    4043 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/qemu.pid -device virtio-net-pci,netdev=net0,mac=3e:96:f3:43:87:68 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/docker-flags-564000/disk.qcow2

	I0918 12:38:36.164221    4043 main.go:141] libmachine: STDOUT: 

	I0918 12:38:36.164232    4043 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0918 12:38:36.164246    4043 client.go:171] LocalClient.Create took 220.981667ms

	I0918 12:38:38.166394    4043 start.go:128] duration metric: createHost completed in 2.278601041s

	I0918 12:38:38.166478    4043 start.go:83] releasing machines lock for "docker-flags-564000", held for 2.279058375s

	W0918 12:38:38.167098    4043 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p docker-flags-564000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p docker-flags-564000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:38.179700    4043 out.go:177] 

	W0918 12:38:38.185035    4043 out.go:239] X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0918 12:38:38.185077    4043 out.go:239] * 

	* 

	W0918 12:38:38.187667    4043 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0918 12:38:38.195665    4043 out.go:177] 

** /stderr **

docker_test.go:53: failed to start minikube with args: "out/minikube-darwin-arm64 start -p docker-flags-564000 --cache-images=false --memory=2048 --install-addons=false --wait=false --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug --docker-opt=icc=true --alsologtostderr -v=5 --driver=qemu2 " : exit status 80

docker_test.go:56: (dbg) Run:  out/minikube-darwin-arm64 -p docker-flags-564000 ssh "sudo systemctl show docker --property=Environment --no-pager"

docker_test.go:56: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p docker-flags-564000 ssh "sudo systemctl show docker --property=Environment --no-pager": exit status 89 (76.905333ms)

-- stdout --

	* The control plane node must be running for this command

	  To start a cluster, run: "minikube start -p docker-flags-564000"

-- /stdout --

docker_test.go:58: failed to 'systemctl show docker' inside minikube. args "out/minikube-darwin-arm64 -p docker-flags-564000 ssh \"sudo systemctl show docker --property=Environment --no-pager\"": exit status 89

docker_test.go:63: expected env key/value "FOO=BAR" to be passed to minikube's docker and be included in: *"* The control plane node must be running for this command\n  To start a cluster, run: \"minikube start -p docker-flags-564000\"\n"*.

docker_test.go:63: expected env key/value "BAZ=BAT" to be passed to minikube's docker and be included in: *"* The control plane node must be running for this command\n  To start a cluster, run: \"minikube start -p docker-flags-564000\"\n"*.

docker_test.go:67: (dbg) Run:  out/minikube-darwin-arm64 -p docker-flags-564000 ssh "sudo systemctl show docker --property=ExecStart --no-pager"

docker_test.go:67: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p docker-flags-564000 ssh "sudo systemctl show docker --property=ExecStart --no-pager": exit status 89 (42.506041ms)

-- stdout --

	* The control plane node must be running for this command

	  To start a cluster, run: "minikube start -p docker-flags-564000"

-- /stdout --

docker_test.go:69: failed on the second 'systemctl show docker' inside minikube. args "out/minikube-darwin-arm64 -p docker-flags-564000 ssh \"sudo systemctl show docker --property=ExecStart --no-pager\"": exit status 89

docker_test.go:73: expected "out/minikube-darwin-arm64 -p docker-flags-564000 ssh \"sudo systemctl show docker --property=ExecStart --no-pager\"" output to have include *--debug* . output: "* The control plane node must be running for this command\n  To start a cluster, run: \"minikube start -p docker-flags-564000\"\n"

panic.go:523: *** TestDockerFlags FAILED at 2023-09-18 12:38:38.33077 -0700 PDT m=+2814.807900084

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p docker-flags-564000 -n docker-flags-564000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p docker-flags-564000 -n docker-flags-564000: exit status 7 (27.617ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "docker-flags-564000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "docker-flags-564000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p docker-flags-564000

--- FAIL: TestDockerFlags (10.05s)

=== RUN   TestForceSystemdFlag

=== PAUSE TestForceSystemdFlag

=== CONT  TestForceSystemdFlag

docker_test.go:91: (dbg) Run:  out/minikube-darwin-arm64 start -p force-systemd-flag-847000 --memory=2048 --force-systemd --alsologtostderr -v=5 --driver=qemu2 

docker_test.go:91: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p force-systemd-flag-847000 --memory=2048 --force-systemd --alsologtostderr -v=5 --driver=qemu2 : exit status 80 (9.777439292s)

-- stdout --

	* [force-systemd-flag-847000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	  - MINIKUBE_LOCATION=17263

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting control plane node force-systemd-flag-847000 in cluster force-systemd-flag-847000

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "force-systemd-flag-847000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0918 12:38:23.570276    4019 out.go:296] Setting OutFile to fd 1 ...

	I0918 12:38:23.570414    4019 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:38:23.570417    4019 out.go:309] Setting ErrFile to fd 2...

	I0918 12:38:23.570420    4019 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:38:23.570577    4019 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17263-1251/.minikube/bin

	I0918 12:38:23.571584    4019 out.go:303] Setting JSON to false

	I0918 12:38:23.586508    4019 start.go:128] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":4077,"bootTime":1695061826,"procs":405,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.5.2","kernelVersion":"22.6.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0918 12:38:23.586575    4019 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0918 12:38:23.592529    4019 out.go:177] * [force-systemd-flag-847000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	I0918 12:38:23.599546    4019 out.go:177]   - MINIKUBE_LOCATION=17263

	I0918 12:38:23.603558    4019 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 12:38:23.599617    4019 notify.go:220] Checking for updates...

	I0918 12:38:23.609420    4019 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0918 12:38:23.612493    4019 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0918 12:38:23.615516    4019 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	I0918 12:38:23.618462    4019 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0918 12:38:23.621892    4019 config.go:182] Loaded profile config "force-systemd-env-724000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 12:38:23.621942    4019 driver.go:373] Setting default libvirt URI to qemu:///system

	I0918 12:38:23.626560    4019 out.go:177] * Using the qemu2 driver based on user configuration

	I0918 12:38:23.633456    4019 start.go:298] selected driver: qemu2

	I0918 12:38:23.633463    4019 start.go:902] validating driver "qemu2" against <nil>

	I0918 12:38:23.633469    4019 start.go:913] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0918 12:38:23.635524    4019 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0918 12:38:23.638550    4019 out.go:177] * Automatically selected the socket_vmnet network

	I0918 12:38:23.639927    4019 start_flags.go:904] Wait components to verify : map[apiserver:true system_pods:true]

	I0918 12:38:23.639954    4019 cni.go:84] Creating CNI manager for ""

	I0918 12:38:23.639963    4019 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 12:38:23.639967    4019 start_flags.go:316] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0918 12:38:23.639975    4019 start_flags.go:321] config:

	{Name:force-systemd-flag-847000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:force-systemd-flag-847000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Co

ntainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 12:38:23.644073    4019 iso.go:125] acquiring lock: {Name:mk34e23c181861c65264ec384f06e6cfa001aa08 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0918 12:38:23.647545    4019 out.go:177] * Starting control plane node force-systemd-flag-847000 in cluster force-systemd-flag-847000

	I0918 12:38:23.655492    4019 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 12:38:23.655508    4019 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4

	I0918 12:38:23.655518    4019 cache.go:57] Caching tarball of preloaded images

	I0918 12:38:23.655566    4019 preload.go:174] Found /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0918 12:38:23.655571    4019 cache.go:60] Finished verifying existence of preloaded tar for  v1.28.2 on docker

	I0918 12:38:23.655632    4019 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/force-systemd-flag-847000/config.json ...

	I0918 12:38:23.655644    4019 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/force-systemd-flag-847000/config.json: {Name:mk468b06b28bf0b43b6e2c1c80d7ccaba68c03d3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:38:23.655850    4019 start.go:365] acquiring machines lock for force-systemd-flag-847000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 12:38:23.655885    4019 start.go:369] acquired machines lock for "force-systemd-flag-847000" in 23.542µs

	I0918 12:38:23.655897    4019 start.go:93] Provisioning new machine with config: &{Name:force-systemd-flag-847000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kub

ernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:force-systemd-flag-847000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p Mount

UID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 12:38:23.655933    4019 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 12:38:23.664467    4019 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0918 12:38:23.679902    4019 start.go:159] libmachine.API.Create for "force-systemd-flag-847000" (driver="qemu2")

	I0918 12:38:23.679932    4019 client.go:168] LocalClient.Create starting

	I0918 12:38:23.679990    4019 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 12:38:23.680016    4019 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:23.680028    4019 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:23.680070    4019 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 12:38:23.680089    4019 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:23.680094    4019 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:23.680428    4019 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 12:38:23.796673    4019 main.go:141] libmachine: Creating SSH key...

	I0918 12:38:23.834034    4019 main.go:141] libmachine: Creating Disk image...

	I0918 12:38:23.834040    4019 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 12:38:23.834169    4019 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2

	I0918 12:38:23.842801    4019 main.go:141] libmachine: STDOUT: 

	I0918 12:38:23.842813    4019 main.go:141] libmachine: STDERR: 

	I0918 12:38:23.842864    4019 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2 +20000M

	I0918 12:38:23.849991    4019 main.go:141] libmachine: STDOUT: Image resized.

	I0918 12:38:23.850009    4019 main.go:141] libmachine: STDERR: 

	I0918 12:38:23.850023    4019 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2

	I0918 12:38:23.850029    4019 main.go:141] libmachine: Starting QEMU VM...

	I0918 12:38:23.850063    4019 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/qemu.pid -device virtio-net-pci,netdev=net0,mac=a2:a0:ca:1d:c5:83 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2

	I0918 12:38:23.851577    4019 main.go:141] libmachine: STDOUT: 

	I0918 12:38:23.851589    4019 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0918 12:38:23.851607    4019 client.go:171] LocalClient.Create took 171.673417ms

	I0918 12:38:25.853748    4019 start.go:128] duration metric: createHost completed in 2.197839291s

	I0918 12:38:25.853818    4019 start.go:83] releasing machines lock for "force-systemd-flag-847000", held for 2.19796225s

	W0918 12:38:25.853883    4019 start.go:688] error starting host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:25.870857    4019 out.go:177] * Deleting "force-systemd-flag-847000" in qemu2 ...

	W0918 12:38:25.887419    4019 out.go:239] ! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:25.887443    4019 start.go:703] Will try again in 5 seconds ...

	I0918 12:38:30.889568    4019 start.go:365] acquiring machines lock for force-systemd-flag-847000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 12:38:30.889947    4019 start.go:369] acquired machines lock for "force-systemd-flag-847000" in 274.958µs

	I0918 12:38:30.890064    4019 start.go:93] Provisioning new machine with config: &{Name:force-systemd-flag-847000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kub

ernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:force-systemd-flag-847000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p Mount

UID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 12:38:30.890263    4019 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 12:38:30.899486    4019 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0918 12:38:30.946219    4019 start.go:159] libmachine.API.Create for "force-systemd-flag-847000" (driver="qemu2")

	I0918 12:38:30.946261    4019 client.go:168] LocalClient.Create starting

	I0918 12:38:30.946380    4019 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 12:38:30.946454    4019 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:30.946494    4019 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:30.946567    4019 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 12:38:30.946612    4019 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:30.946634    4019 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:30.947365    4019 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 12:38:31.075884    4019 main.go:141] libmachine: Creating SSH key...

	I0918 12:38:31.261725    4019 main.go:141] libmachine: Creating Disk image...

	I0918 12:38:31.261735    4019 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 12:38:31.261887    4019 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2

	I0918 12:38:31.270540    4019 main.go:141] libmachine: STDOUT: 

	I0918 12:38:31.270556    4019 main.go:141] libmachine: STDERR: 

	I0918 12:38:31.270605    4019 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2 +20000M

	I0918 12:38:31.277811    4019 main.go:141] libmachine: STDOUT: Image resized.

	I0918 12:38:31.277822    4019 main.go:141] libmachine: STDERR: 

	I0918 12:38:31.277835    4019 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2

	I0918 12:38:31.277841    4019 main.go:141] libmachine: Starting QEMU VM...

	I0918 12:38:31.277886    4019 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/qemu.pid -device virtio-net-pci,netdev=net0,mac=7e:04:8c:bb:ef:1e -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-flag-847000/disk.qcow2

	I0918 12:38:31.279378    4019 main.go:141] libmachine: STDOUT: 

	I0918 12:38:31.279388    4019 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0918 12:38:31.279399    4019 client.go:171] LocalClient.Create took 333.139041ms

	I0918 12:38:33.281544    4019 start.go:128] duration metric: createHost completed in 2.391295666s

	I0918 12:38:33.281609    4019 start.go:83] releasing machines lock for "force-systemd-flag-847000", held for 2.391684125s

	W0918 12:38:33.281997    4019 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p force-systemd-flag-847000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p force-systemd-flag-847000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:33.292703    4019 out.go:177] 

	W0918 12:38:33.296798    4019 out.go:239] X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0918 12:38:33.296892    4019 out.go:239] * 

	* 

	W0918 12:38:33.299480    4019 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0918 12:38:33.308732    4019 out.go:177] 

** /stderr **

docker_test.go:93: failed to start minikube with args: "out/minikube-darwin-arm64 start -p force-systemd-flag-847000 --memory=2048 --force-systemd --alsologtostderr -v=5 --driver=qemu2 " : exit status 80

docker_test.go:110: (dbg) Run:  out/minikube-darwin-arm64 -p force-systemd-flag-847000 ssh "docker info --format {{.CgroupDriver}}"

docker_test.go:110: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p force-systemd-flag-847000 ssh "docker info --format {{.CgroupDriver}}": exit status 89 (72.748875ms)

-- stdout --

	* The control plane node must be running for this command

	  To start a cluster, run: "minikube start -p force-systemd-flag-847000"

-- /stdout --

docker_test.go:112: failed to get docker cgroup driver. args "out/minikube-darwin-arm64 -p force-systemd-flag-847000 ssh \"docker info --format {{.CgroupDriver}}\"": exit status 89

docker_test.go:106: *** TestForceSystemdFlag FAILED at 2023-09-18 12:38:33.397514 -0700 PDT m=+2809.874551959

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p force-systemd-flag-847000 -n force-systemd-flag-847000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p force-systemd-flag-847000 -n force-systemd-flag-847000: exit status 7 (36.272375ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "force-systemd-flag-847000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "force-systemd-flag-847000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p force-systemd-flag-847000

--- FAIL: TestForceSystemdFlag (9.98s)

=== RUN   TestForceSystemdEnv

=== PAUSE TestForceSystemdEnv

=== CONT  TestForceSystemdEnv

docker_test.go:155: (dbg) Run:  out/minikube-darwin-arm64 start -p force-systemd-env-724000 --memory=2048 --alsologtostderr -v=5 --driver=qemu2 

docker_test.go:155: (dbg) Non-zero exit: out/minikube-darwin-arm64 start -p force-systemd-env-724000 --memory=2048 --alsologtostderr -v=5 --driver=qemu2 : exit status 80 (9.992353375s)

-- stdout --

	* [force-systemd-env-724000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	  - MINIKUBE_LOCATION=17263

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-arm64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=true

	* Using the qemu2 driver based on user configuration

	* Automatically selected the socket_vmnet network

	* Starting control plane node force-systemd-env-724000 in cluster force-systemd-env-724000

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	* Deleting "force-systemd-env-724000" in qemu2 ...

	* Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	OUTPUT: 

	ERROR: Failed to connect to "/var/run/socket_vmnet": Connection refused

-- /stdout --

** stderr ** 

	I0918 12:38:18.229607    3983 out.go:296] Setting OutFile to fd 1 ...

	I0918 12:38:18.229736    3983 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:38:18.229739    3983 out.go:309] Setting ErrFile to fd 2...

	I0918 12:38:18.229742    3983 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:38:18.229880    3983 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17263-1251/.minikube/bin

	I0918 12:38:18.230979    3983 out.go:303] Setting JSON to false

	I0918 12:38:18.246185    3983 start.go:128] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":4072,"bootTime":1695061826,"procs":400,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.5.2","kernelVersion":"22.6.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0918 12:38:18.246282    3983 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0918 12:38:18.252086    3983 out.go:177] * [force-systemd-env-724000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	I0918 12:38:18.260095    3983 out.go:177]   - MINIKUBE_LOCATION=17263

	I0918 12:38:18.264039    3983 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 12:38:18.260167    3983 notify.go:220] Checking for updates...

	I0918 12:38:18.270016    3983 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0918 12:38:18.272997    3983 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0918 12:38:18.276105    3983 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	I0918 12:38:18.279077    3983 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=true

	I0918 12:38:18.282160    3983 driver.go:373] Setting default libvirt URI to qemu:///system

	I0918 12:38:18.286016    3983 out.go:177] * Using the qemu2 driver based on user configuration

	I0918 12:38:18.293053    3983 start.go:298] selected driver: qemu2

	I0918 12:38:18.293062    3983 start.go:902] validating driver "qemu2" against <nil>

	I0918 12:38:18.293068    3983 start.go:913] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0918 12:38:18.295097    3983 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0918 12:38:18.298007    3983 out.go:177] * Automatically selected the socket_vmnet network

	I0918 12:38:18.301163    3983 start_flags.go:904] Wait components to verify : map[apiserver:true system_pods:true]

	I0918 12:38:18.301189    3983 cni.go:84] Creating CNI manager for ""

	I0918 12:38:18.301210    3983 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 12:38:18.301214    3983 start_flags.go:316] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0918 12:38:18.301219    3983 start_flags.go:321] config:

	{Name:force-systemd-env-724000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:force-systemd-env-724000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Cont

ainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 12:38:18.305649    3983 iso.go:125] acquiring lock: {Name:mk34e23c181861c65264ec384f06e6cfa001aa08 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0918 12:38:18.313069    3983 out.go:177] * Starting control plane node force-systemd-env-724000 in cluster force-systemd-env-724000

	I0918 12:38:18.317062    3983 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 12:38:18.317086    3983 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4

	I0918 12:38:18.317093    3983 cache.go:57] Caching tarball of preloaded images

	I0918 12:38:18.317161    3983 preload.go:174] Found /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0918 12:38:18.317167    3983 cache.go:60] Finished verifying existence of preloaded tar for  v1.28.2 on docker

	I0918 12:38:18.317393    3983 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/force-systemd-env-724000/config.json ...

	I0918 12:38:18.317405    3983 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/force-systemd-env-724000/config.json: {Name:mkeac7299d2d0eac46a8751c8e3401831f90a0f5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:38:18.317624    3983 start.go:365] acquiring machines lock for force-systemd-env-724000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 12:38:18.317657    3983 start.go:369] acquired machines lock for "force-systemd-env-724000" in 24µs

	I0918 12:38:18.317668    3983 start.go:93] Provisioning new machine with config: &{Name:force-systemd-env-724000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kube

rnetesConfig:{KubernetesVersion:v1.28.2 ClusterName:force-systemd-env-724000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUI

D:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 12:38:18.317697    3983 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 12:38:18.326065    3983 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0918 12:38:18.342218    3983 start.go:159] libmachine.API.Create for "force-systemd-env-724000" (driver="qemu2")

	I0918 12:38:18.342263    3983 client.go:168] LocalClient.Create starting

	I0918 12:38:18.342322    3983 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 12:38:18.342346    3983 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:18.342360    3983 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:18.342399    3983 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 12:38:18.342418    3983 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:18.342425    3983 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:18.342747    3983 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 12:38:18.455949    3983 main.go:141] libmachine: Creating SSH key...

	I0918 12:38:18.575671    3983 main.go:141] libmachine: Creating Disk image...

	I0918 12:38:18.575681    3983 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 12:38:18.575841    3983 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2

	I0918 12:38:18.584533    3983 main.go:141] libmachine: STDOUT: 

	I0918 12:38:18.584551    3983 main.go:141] libmachine: STDERR: 

	I0918 12:38:18.584613    3983 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2 +20000M

	I0918 12:38:18.592004    3983 main.go:141] libmachine: STDOUT: Image resized.

	I0918 12:38:18.592017    3983 main.go:141] libmachine: STDERR: 

	I0918 12:38:18.592032    3983 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2

	I0918 12:38:18.592040    3983 main.go:141] libmachine: Starting QEMU VM...

	I0918 12:38:18.592076    3983 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/qemu.pid -device virtio-net-pci,netdev=net0,mac=da:8b:f0:07:b9:17 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2

	I0918 12:38:18.593642    3983 main.go:141] libmachine: STDOUT: 

	I0918 12:38:18.593659    3983 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0918 12:38:18.593685    3983 client.go:171] LocalClient.Create took 251.417833ms

	I0918 12:38:20.595726    3983 start.go:128] duration metric: createHost completed in 2.27806275s

	I0918 12:38:20.595750    3983 start.go:83] releasing machines lock for "force-systemd-env-724000", held for 2.278132042s

	W0918 12:38:20.595770    3983 start.go:688] error starting host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:20.601302    3983 out.go:177] * Deleting "force-systemd-env-724000" in qemu2 ...

	W0918 12:38:20.613131    3983 out.go:239] ! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	! StartHost failed, but will try again: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:20.613142    3983 start.go:703] Will try again in 5 seconds ...

	I0918 12:38:25.615330    3983 start.go:365] acquiring machines lock for force-systemd-env-724000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 12:38:25.853980    3983 start.go:369] acquired machines lock for "force-systemd-env-724000" in 238.507083ms

	I0918 12:38:25.854099    3983 start.go:93] Provisioning new machine with config: &{Name:force-systemd-env-724000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kube

rnetesConfig:{KubernetesVersion:v1.28.2 ClusterName:force-systemd-env-724000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUI

D:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 12:38:25.854300    3983 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 12:38:25.863807    3983 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...

	I0918 12:38:25.910255    3983 start.go:159] libmachine.API.Create for "force-systemd-env-724000" (driver="qemu2")

	I0918 12:38:25.910289    3983 client.go:168] LocalClient.Create starting

	I0918 12:38:25.910422    3983 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 12:38:25.910482    3983 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:25.910500    3983 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:25.910575    3983 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 12:38:25.910610    3983 main.go:141] libmachine: Decoding PEM data...

	I0918 12:38:25.910621    3983 main.go:141] libmachine: Parsing certificate...

	I0918 12:38:25.911080    3983 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 12:38:26.050250    3983 main.go:141] libmachine: Creating SSH key...

	I0918 12:38:26.133064    3983 main.go:141] libmachine: Creating Disk image...

	I0918 12:38:26.133069    3983 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 12:38:26.133209    3983 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2

	I0918 12:38:26.141758    3983 main.go:141] libmachine: STDOUT: 

	I0918 12:38:26.141773    3983 main.go:141] libmachine: STDERR: 

	I0918 12:38:26.141833    3983 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2 +20000M

	I0918 12:38:26.148984    3983 main.go:141] libmachine: STDOUT: Image resized.

	I0918 12:38:26.148998    3983 main.go:141] libmachine: STDERR: 

	I0918 12:38:26.149011    3983 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2

	I0918 12:38:26.149019    3983 main.go:141] libmachine: Starting QEMU VM...

	I0918 12:38:26.149060    3983 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt,highmem=off -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 2048 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/qemu.pid -device virtio-net-pci,netdev=net0,mac=5e:5a:60:38:de:d6 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/force-systemd-env-724000/disk.qcow2

	I0918 12:38:26.150652    3983 main.go:141] libmachine: STDOUT: 

	I0918 12:38:26.150673    3983 main.go:141] libmachine: STDERR: Failed to connect to "/var/run/socket_vmnet": Connection refused

	I0918 12:38:26.150685    3983 client.go:171] LocalClient.Create took 240.393291ms

	I0918 12:38:28.152843    3983 start.go:128] duration metric: createHost completed in 2.298549125s

	I0918 12:38:28.152930    3983 start.go:83] releasing machines lock for "force-systemd-env-724000", held for 2.298962667s

	W0918 12:38:28.153347    3983 out.go:239] * Failed to start qemu2 VM. Running "minikube delete -p force-systemd-env-724000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	* Failed to start qemu2 VM. Running "minikube delete -p force-systemd-env-724000" may fix it: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	I0918 12:38:28.164100    3983 out.go:177] 

	W0918 12:38:28.169254    3983 out.go:239] X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	X Exiting due to GUEST_PROVISION: error provisioning guest: Failed to start host: creating host: create: creating: Failed to connect to "/var/run/socket_vmnet": Connection refused: exit status 1

	W0918 12:38:28.169316    3983 out.go:239] * 

	* 

	W0918 12:38:28.172075    3983 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I0918 12:38:28.182160    3983 out.go:177] 

** /stderr **

docker_test.go:157: failed to start minikube with args: "out/minikube-darwin-arm64 start -p force-systemd-env-724000 --memory=2048 --alsologtostderr -v=5 --driver=qemu2 " : exit status 80

docker_test.go:110: (dbg) Run:  out/minikube-darwin-arm64 -p force-systemd-env-724000 ssh "docker info --format {{.CgroupDriver}}"

docker_test.go:110: (dbg) Non-zero exit: out/minikube-darwin-arm64 -p force-systemd-env-724000 ssh "docker info --format {{.CgroupDriver}}": exit status 89 (75.097416ms)

-- stdout --

	* The control plane node must be running for this command

	  To start a cluster, run: "minikube start -p force-systemd-env-724000"

-- /stdout --

docker_test.go:112: failed to get docker cgroup driver. args "out/minikube-darwin-arm64 -p force-systemd-env-724000 ssh \"docker info --format {{.CgroupDriver}}\"": exit status 89

docker_test.go:166: *** TestForceSystemdEnv FAILED at 2023-09-18 12:38:28.275534 -0700 PDT m=+2804.752475876

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p force-systemd-env-724000 -n force-systemd-env-724000

helpers_test.go:239: (dbg) Non-zero exit: out/minikube-darwin-arm64 status --format={{.Host}} -p force-systemd-env-724000 -n force-systemd-env-724000: exit status 7 (32.740792ms)

-- stdout --

	Stopped

-- /stdout --

helpers_test.go:239: status error: exit status 7 (may be ok)

helpers_test.go:241: "force-systemd-env-724000" host is not running, skipping log retrieval (state="Stopped")

helpers_test.go:175: Cleaning up "force-systemd-env-724000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-arm64 delete -p force-systemd-env-724000

--- FAIL: TestForceSystemdEnv (10.20s)

=== RUN   TestFunctional/parallel/ServiceCmdConnect

=== PAUSE TestFunctional/parallel/ServiceCmdConnect

=== CONT  TestFunctional/parallel/ServiceCmdConnect

functional_test.go:1626: (dbg) Run:  kubectl --context functional-847000 create deployment hello-node-connect --image=registry.k8s.io/echoserver-arm:1.8

functional_test.go:1634: (dbg) Run:  kubectl --context functional-847000 expose deployment hello-node-connect --type=NodePort --port=8080

functional_test.go:1639: (dbg) TestFunctional/parallel/ServiceCmdConnect: waiting 10m0s for pods matching "app=hello-node-connect" in namespace "default" ...

helpers_test.go:344: "hello-node-connect-7799dfb7c6-ctzmg" [8cc3e8ed-8be6-4416-87c0-3548bd1cbbe6] Pending / Ready:ContainersNotReady (containers with unready status: [echoserver-arm]) / ContainersReady:ContainersNotReady (containers with unready status: [echoserver-arm])

helpers_test.go:344: "hello-node-connect-7799dfb7c6-ctzmg" [8cc3e8ed-8be6-4416-87c0-3548bd1cbbe6] Running / Ready:ContainersNotReady (containers with unready status: [echoserver-arm]) / ContainersReady:ContainersNotReady (containers with unready status: [echoserver-arm])

functional_test.go:1639: (dbg) TestFunctional/parallel/ServiceCmdConnect: app=hello-node-connect healthy within 12.006927459s

functional_test.go:1648: (dbg) Run:  out/minikube-darwin-arm64 -p functional-847000 service hello-node-connect --url

functional_test.go:1654: found endpoint for hello-node-connect: http://192.168.105.4:32307

functional_test.go:1660: error fetching http://192.168.105.4:32307: Get "http://192.168.105.4:32307": dial tcp 192.168.105.4:32307: connect: connection refused

functional_test.go:1660: error fetching http://192.168.105.4:32307: Get "http://192.168.105.4:32307": dial tcp 192.168.105.4:32307: connect: connection refused

functional_test.go:1660: error fetching http://192.168.105.4:32307: Get "http://192.168.105.4:32307": dial tcp 192.168.105.4:32307: connect: connection refused

functional_test.go:1660: error fetching http://192.168.105.4:32307: Get "http://192.168.105.4:32307": dial tcp 192.168.105.4:32307: connect: connection refused

functional_test.go:1660: error fetching http://192.168.105.4:32307: Get "http://192.168.105.4:32307": dial tcp 192.168.105.4:32307: connect: connection refused

functional_test.go:1660: error fetching http://192.168.105.4:32307: Get "http://192.168.105.4:32307": dial tcp 192.168.105.4:32307: connect: connection refused

functional_test.go:1660: error fetching http://192.168.105.4:32307: Get "http://192.168.105.4:32307": dial tcp 192.168.105.4:32307: connect: connection refused

2023/09/18 12:16:28 [DEBUG] GET http://127.0.0.1:36195/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/

functional_test.go:1660: error fetching http://192.168.105.4:32307: Get "http://192.168.105.4:32307": dial tcp 192.168.105.4:32307: connect: connection refused

functional_test.go:1680: failed to fetch http://192.168.105.4:32307: Get "http://192.168.105.4:32307": dial tcp 192.168.105.4:32307: connect: connection refused

functional_test.go:1597: service test failed - dumping debug information

functional_test.go:1598: -----------------------service failure post-mortem--------------------------------

functional_test.go:1601: (dbg) Run:  kubectl --context functional-847000 describe po hello-node-connect

functional_test.go:1605: hello-node pod describe:

Name:             hello-node-connect-7799dfb7c6-ctzmg

Namespace:        default

Priority:         0

Service Account:  default

Node:             functional-847000/192.168.105.4

Start Time:       Mon, 18 Sep 2023 12:15:48 -0700

Labels:           app=hello-node-connect

pod-template-hash=7799dfb7c6

Annotations:      <none>

Status:           Running

IP:               10.244.0.8

IPs:

IP:           10.244.0.8

Controlled By:  ReplicaSet/hello-node-connect-7799dfb7c6

Containers:

echoserver-arm:

Container ID:   docker://9de3fa653dead4c7773edccbd3c53ddd83b403c71318f5dec4753b80bd88dca9

Image:          registry.k8s.io/echoserver-arm:1.8

Image ID:       docker-pullable://registry.k8s.io/echoserver-arm@sha256:b33d4cdf6ed097f4e9b77b135d83a596ab73c6268b0342648818eb85f5edfdb5

Port:           <none>

Host Port:      <none>

State:          Waiting

Reason:       CrashLoopBackOff

Last State:     Terminated

Reason:       Error

Exit Code:    1

Started:      Mon, 18 Sep 2023 12:16:08 -0700

Finished:     Mon, 18 Sep 2023 12:16:08 -0700

Ready:          False

Restart Count:  2

Environment:    <none>

Mounts:

/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-tgv6b (ro)

Conditions:

Type              Status

Initialized       True 

Ready             False 

ContainersReady   False 

PodScheduled      True 

Volumes:

kube-api-access-tgv6b:

Type:                    Projected (a volume that contains injected data from multiple sources)

TokenExpirationSeconds:  3607

ConfigMapName:           kube-root-ca.crt

ConfigMapOptional:       <nil>

DownwardAPI:             true

QoS Class:                   BestEffort

Node-Selectors:              <none>

Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s

node.kubernetes.io/unreachable:NoExecute op=Exists for 300s

Events:

Type     Reason     Age                From               Message

----     ------     ----               ----               -------

Normal   Scheduled  42s                default-scheduler  Successfully assigned default/hello-node-connect-7799dfb7c6-ctzmg to functional-847000

Normal   Pulling    41s                kubelet            Pulling image "registry.k8s.io/echoserver-arm:1.8"

Normal   Pulled     38s                kubelet            Successfully pulled image "registry.k8s.io/echoserver-arm:1.8" in 3.652s (3.652s including waiting)

Normal   Created    22s (x3 over 38s)  kubelet            Created container echoserver-arm

Normal   Started    22s (x3 over 37s)  kubelet            Started container echoserver-arm

Normal   Pulled     22s (x2 over 37s)  kubelet            Container image "registry.k8s.io/echoserver-arm:1.8" already present on machine

Warning  BackOff    7s (x4 over 36s)   kubelet            Back-off restarting failed container echoserver-arm in pod hello-node-connect-7799dfb7c6-ctzmg_default(8cc3e8ed-8be6-4416-87c0-3548bd1cbbe6)

functional_test.go:1607: (dbg) Run:  kubectl --context functional-847000 logs -l app=hello-node-connect

functional_test.go:1611: hello-node logs:

exec /usr/sbin/nginx: exec format error

functional_test.go:1613: (dbg) Run:  kubectl --context functional-847000 describe svc hello-node-connect

functional_test.go:1617: hello-node svc describe:

Name:                     hello-node-connect

Namespace:                default

Labels:                   app=hello-node-connect

Annotations:              <none>

Selector:                 app=hello-node-connect

Type:                     NodePort

IP Family Policy:         SingleStack

IP Families:              IPv4

IP:                       10.102.30.144

IPs:                      10.102.30.144

Port:                     <unset>  8080/TCP

TargetPort:               8080/TCP

NodePort:                 <unset>  32307/TCP

Endpoints:                

Session Affinity:         None

External Traffic Policy:  Cluster

Events:                   <none>

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p functional-847000 -n functional-847000

helpers_test.go:244: <<< TestFunctional/parallel/ServiceCmdConnect FAILED: start of post-mortem logs <<<

helpers_test.go:245: ======>  post-mortem[TestFunctional/parallel/ServiceCmdConnect]: minikube logs <======

helpers_test.go:247: (dbg) Run:  out/minikube-darwin-arm64 -p functional-847000 logs -n 25

helpers_test.go:252: TestFunctional/parallel/ServiceCmdConnect logs: 

-- stdout --

	* 

	* ==> Audit <==

	* |-----------|----------------------------------------------------------------------------------------------------------------------|-------------------|---------|---------|---------------------|---------------------|

	|  Command  |                                                         Args                                                         |      Profile      |  User   | Version |     Start Time      |      End Time       |

	|-----------|----------------------------------------------------------------------------------------------------------------------|-------------------|---------|---------|---------------------|---------------------|

	| ssh       | functional-847000 ssh findmnt                                                                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | -T /mount-9p | grep 9p                                                                                               |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh -- ls                                                                                          | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | -la /mount-9p                                                                                                        |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh cat                                                                                            | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | /mount-9p/test-1695064572470330000                                                                                   |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh stat                                                                                           | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | /mount-9p/created-by-test                                                                                            |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh stat                                                                                           | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | /mount-9p/created-by-pod                                                                                             |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh sudo                                                                                           | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | umount -f /mount-9p                                                                                                  |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh findmnt                                                                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | -T /mount-9p | grep 9p                                                                                               |                   |         |         |                     |                     |

	| mount     | -p functional-847000                                                                                                 | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/TestFunctionalparallelMountCmdspecific-port1700026312/001:/mount-9p |                   |         |         |                     |                     |

	|           | --alsologtostderr -v=1 --port 46464                                                                                  |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh findmnt                                                                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | -T /mount-9p | grep 9p                                                                                               |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh -- ls                                                                                          | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | -la /mount-9p                                                                                                        |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh sudo                                                                                           | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | umount -f /mount-9p                                                                                                  |                   |         |         |                     |                     |

	| mount     | -p functional-847000                                                                                                 | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/TestFunctionalparallelMountCmdVerifyCleanup817082744/001:/mount2    |                   |         |         |                     |                     |

	|           | --alsologtostderr -v=1                                                                                               |                   |         |         |                     |                     |

	| mount     | -p functional-847000                                                                                                 | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/TestFunctionalparallelMountCmdVerifyCleanup817082744/001:/mount1    |                   |         |         |                     |                     |

	|           | --alsologtostderr -v=1                                                                                               |                   |         |         |                     |                     |

	| mount     | -p functional-847000                                                                                                 | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | /var/folders/2k/p0kjt1w95hl7b54xjlcc45ph0000gp/T/TestFunctionalparallelMountCmdVerifyCleanup817082744/001:/mount3    |                   |         |         |                     |                     |

	|           | --alsologtostderr -v=1                                                                                               |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh findmnt                                                                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | -T /mount1                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh findmnt                                                                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | -T /mount1                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh findmnt                                                                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | -T /mount2                                                                                                           |                   |         |         |                     |                     |

	| ssh       | functional-847000 ssh findmnt                                                                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | -T /mount3                                                                                                           |                   |         |         |                     |                     |

	| mount     | -p functional-847000                                                                                                 | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | --kill=true                                                                                                          |                   |         |         |                     |                     |

	| start     | -p functional-847000                                                                                                 | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | --dry-run --memory                                                                                                   |                   |         |         |                     |                     |

	|           | 250MB --alsologtostderr                                                                                              |                   |         |         |                     |                     |

	|           | --driver=qemu2                                                                                                       |                   |         |         |                     |                     |

	| start     | -p functional-847000                                                                                                 | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | --dry-run --memory                                                                                                   |                   |         |         |                     |                     |

	|           | 250MB --alsologtostderr                                                                                              |                   |         |         |                     |                     |

	|           | --driver=qemu2                                                                                                       |                   |         |         |                     |                     |

	| start     | -p functional-847000 --dry-run                                                                                       | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | --alsologtostderr -v=1                                                                                               |                   |         |         |                     |                     |

	|           | --driver=qemu2                                                                                                       |                   |         |         |                     |                     |

	| dashboard | --url --port 36195                                                                                                   | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|           | -p functional-847000                                                                                                 |                   |         |         |                     |                     |

	|           | --alsologtostderr -v=1                                                                                               |                   |         |         |                     |                     |

	| license   |                                                                                                                      | minikube          | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	| ssh       | functional-847000 ssh sudo                                                                                           | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|           | systemctl is-active crio                                                                                             |                   |         |         |                     |                     |

	|-----------|----------------------------------------------------------------------------------------------------------------------|-------------------|---------|---------|---------------------|---------------------|

	* 

	* ==> Last Start <==

	* Log file created at: 2023/09/18 12:16:18

	Running on machine: MacOS-M1-Agent-1

	Binary: Built with gc go1.21.1 for darwin/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0918 12:16:18.704945    2569 out.go:296] Setting OutFile to fd 1 ...

	I0918 12:16:18.705092    2569 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:16:18.705095    2569 out.go:309] Setting ErrFile to fd 2...

	I0918 12:16:18.705098    2569 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:16:18.705242    2569 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17263-1251/.minikube/bin

	I0918 12:16:18.706244    2569 out.go:303] Setting JSON to false

	I0918 12:16:18.721978    2569 start.go:128] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":2752,"bootTime":1695061826,"procs":406,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.5.2","kernelVersion":"22.6.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0918 12:16:18.722081    2569 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0918 12:16:18.725185    2569 out.go:177] * [functional-847000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	I0918 12:16:18.733510    2569 out.go:177]   - MINIKUBE_LOCATION=17263

	I0918 12:16:18.737483    2569 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 12:16:18.733542    2569 notify.go:220] Checking for updates...

	I0918 12:16:18.744197    2569 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0918 12:16:18.745685    2569 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0918 12:16:18.749190    2569 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	I0918 12:16:18.752202    2569 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0918 12:16:18.755464    2569 config.go:182] Loaded profile config "functional-847000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 12:16:18.755732    2569 driver.go:373] Setting default libvirt URI to qemu:///system

	I0918 12:16:18.760190    2569 out.go:177] * Using the qemu2 driver based on existing profile

	I0918 12:16:18.767263    2569 start.go:298] selected driver: qemu2

	I0918 12:16:18.767270    2569 start.go:902] validating driver "qemu2" against &{Name:functional-847000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConf

ig:{KubernetesVersion:v1.28.2 ClusterName:functional-847000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:apiserver Key:enable-admission-plugins Value:NamespaceAutoProvision}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8441 NodeName:} Nodes:[{Name: IP:192.168.105.4 Port:8441 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false E

xtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 12:16:18.767321    2569 start.go:913] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0918 12:16:18.769198    2569 cni.go:84] Creating CNI manager for ""

	I0918 12:16:18.769212    2569 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 12:16:18.769219    2569 start_flags.go:321] config:

	{Name:functional-847000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:functional-847000 Namespace:default A

PIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:apiserver Key:enable-admission-plugins Value:NamespaceAutoProvision}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8441 NodeName:} Nodes:[{Name: IP:192.168.105.4 Port:8441 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-

host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 12:16:18.781237    2569 out.go:177] * dry-run validation complete!

	* 

	* ==> Docker <==

	* -- Journal begins at Mon 2023-09-18 19:13:39 UTC, ends at Mon 2023-09-18 19:16:30 UTC. --

	Sep 18 19:16:18 functional-847000 dockerd[6288]: time="2023-09-18T19:16:18.468349722Z" level=info msg="shim disconnected" id=bc135da570b0af6add9587d560db445997346ae3dee21f532c91798b013c9592 namespace=moby

	Sep 18 19:16:18 functional-847000 dockerd[6288]: time="2023-09-18T19:16:18.468388846Z" level=warning msg="cleaning up after shim disconnected" id=bc135da570b0af6add9587d560db445997346ae3dee21f532c91798b013c9592 namespace=moby

	Sep 18 19:16:18 functional-847000 dockerd[6288]: time="2023-09-18T19:16:18.468392930Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:16:19 functional-847000 dockerd[6288]: time="2023-09-18T19:16:19.645211785Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 19:16:19 functional-847000 dockerd[6288]: time="2023-09-18T19:16:19.645244119Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:16:19 functional-847000 dockerd[6288]: time="2023-09-18T19:16:19.645587785Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 19:16:19 functional-847000 dockerd[6288]: time="2023-09-18T19:16:19.645603451Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:16:19 functional-847000 dockerd[6288]: time="2023-09-18T19:16:19.655433927Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 19:16:19 functional-847000 dockerd[6288]: time="2023-09-18T19:16:19.655507260Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:16:19 functional-847000 dockerd[6288]: time="2023-09-18T19:16:19.655527677Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 19:16:19 functional-847000 dockerd[6288]: time="2023-09-18T19:16:19.655539593Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:16:19 functional-847000 cri-dockerd[6554]: time="2023-09-18T19:16:19Z" level=info msg="Will attempt to re-write config file /var/lib/docker/containers/281e3eb380fd9854508d0bc8275a35bc6ebec1699788fabbe4d9ba6fe06f7fff/resolv.conf as [nameserver 10.96.0.10 search kubernetes-dashboard.svc.cluster.local svc.cluster.local cluster.local options ndots:5]"

	Sep 18 19:16:19 functional-847000 cri-dockerd[6554]: time="2023-09-18T19:16:19Z" level=info msg="Will attempt to re-write config file /var/lib/docker/containers/510d95a6aae7fde8cd365289bda69f0f727b4e2fdedbe28029f252f202282aa2/resolv.conf as [nameserver 10.96.0.10 search kubernetes-dashboard.svc.cluster.local svc.cluster.local cluster.local options ndots:5]"

	Sep 18 19:16:20 functional-847000 dockerd[6282]: time="2023-09-18T19:16:20.039331800Z" level=warning msg="reference for unknown type: " digest="sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c" remote="docker.io/kubernetesui/metrics-scraper@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c"

	Sep 18 19:16:22 functional-847000 cri-dockerd[6554]: time="2023-09-18T19:16:22Z" level=info msg="Stop pulling image docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c: Status: Downloaded newer image for kubernetesui/metrics-scraper@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c"

	Sep 18 19:16:22 functional-847000 dockerd[6288]: time="2023-09-18T19:16:22.197294064Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 19:16:22 functional-847000 dockerd[6288]: time="2023-09-18T19:16:22.197323314Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:16:22 functional-847000 dockerd[6288]: time="2023-09-18T19:16:22.197332273Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 19:16:22 functional-847000 dockerd[6288]: time="2023-09-18T19:16:22.197338231Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:16:22 functional-847000 dockerd[6282]: time="2023-09-18T19:16:22.388564061Z" level=warning msg="reference for unknown type: " digest="sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93" remote="docker.io/kubernetesui/dashboard@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93"

	Sep 18 19:16:26 functional-847000 cri-dockerd[6554]: time="2023-09-18T19:16:26Z" level=info msg="Stop pulling image docker.io/kubernetesui/dashboard:v2.7.0@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93: Status: Downloaded newer image for kubernetesui/dashboard@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93"

	Sep 18 19:16:26 functional-847000 dockerd[6288]: time="2023-09-18T19:16:26.748651353Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 19:16:26 functional-847000 dockerd[6288]: time="2023-09-18T19:16:26.748687270Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:16:26 functional-847000 dockerd[6288]: time="2023-09-18T19:16:26.748693145Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 19:16:26 functional-847000 dockerd[6288]: time="2023-09-18T19:16:26.748697270Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	* 

	* ==> container status <==

	* CONTAINER           IMAGE                                                                                                  CREATED              STATE               NAME                        ATTEMPT             POD ID

	4840d9fd28aeb       kubernetesui/dashboard@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93         4 seconds ago        Running             kubernetes-dashboard        0                   510d95a6aae7f

	2d04daa021856       kubernetesui/metrics-scraper@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c   8 seconds ago        Running             dashboard-metrics-scraper   0                   281e3eb380fd9

	bc135da570b0a       72565bf5bbedf                                                                                          12 seconds ago       Exited              echoserver-arm              2                   84aaf3fc7a947

	6b5be5d04c686       gcr.io/k8s-minikube/busybox@sha256:2d03e6ceeb99250061dd110530b0ece7998cd84121f952adef120ea7c5a6f00e    16 seconds ago       Exited              mount-munger                0                   d719925ca4d12

	9de3fa653dead       72565bf5bbedf                                                                                          22 seconds ago       Exited              echoserver-arm              2                   5526a0bfe9493

	65c90fd7ca398       nginx@sha256:6926dd802f40e5e7257fded83e0d8030039642e4e10c4a98a6478e9c6fe06153                          32 seconds ago       Running             myfrontend                  0                   fc644d8627dcb

	e0f02a896b25a       nginx@sha256:16164a43b5faec40adb521e98272edc528e74f31c1352719132b8f7e53418d70                          48 seconds ago       Running             nginx                       0                   18e7595c39b8e

	cdd0803f0b0cb       ba04bb24b9575                                                                                          About a minute ago   Running             storage-provisioner         1                   4fb03cbb860de

	cdac16e16be0b       97e04611ad434                                                                                          About a minute ago   Running             coredns                     2                   e6387c0a9569d

	c89a5dd67e174       7da62c127fc0f                                                                                          About a minute ago   Running             kube-proxy                  2                   3489bd34acf67

	43c1216be4bde       30bb499447fe1                                                                                          About a minute ago   Running             kube-apiserver              0                   bfafc2d037cd4

	f56043fcfb982       89d57b83c1786                                                                                          About a minute ago   Running             kube-controller-manager     2                   cb591155c26d4

	3262d7c8a5d7f       9cdd6470f48c8                                                                                          About a minute ago   Running             etcd                        2                   7eebd5bd3002a

	f67b1852a8d9e       64fc40cee3716                                                                                          About a minute ago   Running             kube-scheduler              2                   615816f6bc980

	5388c197829fe       ba04bb24b9575                                                                                          About a minute ago   Exited              storage-provisioner         0                   42c69ab27645b

	6e029fe12af83       97e04611ad434                                                                                          2 minutes ago        Exited              coredns                     1                   cb7a7275af7b0

	94ddf0564ae6d       64fc40cee3716                                                                                          2 minutes ago        Exited              kube-scheduler              1                   bfaa09371b4f2

	4c7ca1c5d88fc       7da62c127fc0f                                                                                          2 minutes ago        Exited              kube-proxy                  1                   4e318fcf7ecda

	1943ec7156082       9cdd6470f48c8                                                                                          2 minutes ago        Exited              etcd                        1                   014a9aec26cc4

	0c9d5dcc35153       89d57b83c1786                                                                                          2 minutes ago        Exited              kube-controller-manager     1                   ad6652113e368

	* 

	* ==> coredns [6e029fe12af8] <==

	* [INFO] plugin/ready: Still waiting on: "kubernetes"

	[INFO] plugin/kubernetes: waiting for Kubernetes API before starting server

	[INFO] plugin/kubernetes: waiting for Kubernetes API before starting server

	[INFO] plugin/ready: Still waiting on: "kubernetes"

	[INFO] plugin/ready: Still waiting on: "kubernetes"

	[INFO] plugin/kubernetes: waiting for Kubernetes API before starting server

	[INFO] plugin/kubernetes: waiting for Kubernetes API before starting server

	[INFO] plugin/kubernetes: waiting for Kubernetes API before starting server

	.:53

	[INFO] plugin/reload: Running configuration SHA512 = ea7a0d73d9d208f758b1f67640ef03c58089b9d9366cf3478df3bb369b210e39f213811b46224f8a04380814b6e0890ccd358f5b5e8c80bc22ac19c8601ee35b

	CoreDNS-1.10.1

	linux/arm64, go1.20, 055b2c3

	[INFO] 127.0.0.1:35090 - 22301 "HINFO IN 8790299216817915349.1978943822594942572. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.007422328s

	[INFO] SIGTERM: Shutting down servers then terminating

	[INFO] plugin/health: Going into lameduck mode for 5s

	* 

	* ==> coredns [cdac16e16be0] <==

	* .:53

	[INFO] plugin/reload: Running configuration SHA512 = ea7a0d73d9d208f758b1f67640ef03c58089b9d9366cf3478df3bb369b210e39f213811b46224f8a04380814b6e0890ccd358f5b5e8c80bc22ac19c8601ee35b

	CoreDNS-1.10.1

	linux/arm64, go1.20, 055b2c3

	[INFO] 127.0.0.1:55974 - 60985 "HINFO IN 4813087894308010664.2847401593487084594. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.004188822s

	[INFO] 10.244.0.1:6029 - 45163 "A IN nginx-svc.default.svc.cluster.local. udp 64 false 4096" NOERROR qr,aa,rd 104 0.00010891s

	[INFO] 10.244.0.1:39807 - 40062 "AAAA IN nginx-svc.default.svc.cluster.local. udp 53 false 512" NOERROR qr,aa,rd 146 0.000067954s

	[INFO] 10.244.0.1:48535 - 64447 "A IN nginx-svc.default.svc.cluster.local. udp 53 false 512" NOERROR qr,aa,rd 104 0.000028914s

	[INFO] 10.244.0.1:20601 - 59450 "SVCB IN _dns.resolver.arpa. udp 36 false 512" NXDOMAIN qr,rd,ra 116 0.000991357s

	[INFO] 10.244.0.1:32674 - 7105 "AAAA IN nginx-svc.default.svc.cluster.local. udp 64 false 1232" NOERROR qr,aa,rd 146 0.000074079s

	[INFO] 10.244.0.1:36055 - 1271 "A IN nginx-svc.default.svc.cluster.local. udp 64 false 1232" NOERROR qr,aa,rd 104 0.000126784s

	* 

	* ==> describe nodes <==

	* Name:               functional-847000

	Roles:              control-plane

	Labels:             beta.kubernetes.io/arch=arm64

	                    beta.kubernetes.io/os=linux

	                    kubernetes.io/arch=arm64

	                    kubernetes.io/hostname=functional-847000

	                    kubernetes.io/os=linux

	                    minikube.k8s.io/commit=ae2adeb20cc3a4b5f1decc7c8f53736ec04c4a36

	                    minikube.k8s.io/name=functional-847000

	                    minikube.k8s.io/primary=true

	                    minikube.k8s.io/updated_at=2023_09_18T12_13_56_0700

	                    minikube.k8s.io/version=v1.31.2

	                    node-role.kubernetes.io/control-plane=

	                    node.kubernetes.io/exclude-from-external-load-balancers=

	Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock

	                    node.alpha.kubernetes.io/ttl: 0

	                    volumes.kubernetes.io/controller-managed-attach-detach: true

	CreationTimestamp:  Mon, 18 Sep 2023 19:13:53 +0000

	Taints:             <none>

	Unschedulable:      false

	Lease:

	  HolderIdentity:  functional-847000

	  AcquireTime:     <unset>

	  RenewTime:       Mon, 18 Sep 2023 19:16:23 +0000

	Conditions:

	  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message

	  ----             ------  -----------------                 ------------------                ------                       -------

	  MemoryPressure   False   Mon, 18 Sep 2023 19:16:13 +0000   Mon, 18 Sep 2023 19:13:53 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available

	  DiskPressure     False   Mon, 18 Sep 2023 19:16:13 +0000   Mon, 18 Sep 2023 19:13:53 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure

	  PIDPressure      False   Mon, 18 Sep 2023 19:16:13 +0000   Mon, 18 Sep 2023 19:13:53 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available

	  Ready            True    Mon, 18 Sep 2023 19:16:13 +0000   Mon, 18 Sep 2023 19:14:00 +0000   KubeletReady                 kubelet is posting ready status

	Addresses:

	  InternalIP:  192.168.105.4

	  Hostname:    functional-847000

	Capacity:

	  cpu:                2

	  ephemeral-storage:  17784760Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3905012Ki

	  pods:               110

	Allocatable:

	  cpu:                2

	  ephemeral-storage:  17784760Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3905012Ki

	  pods:               110

	System Info:

	  Machine ID:                 0e7b3e9f722f4b68bf45c5b4f96a0739

	  System UUID:                0e7b3e9f722f4b68bf45c5b4f96a0739

	  Boot ID:                    22f7494a-0f91-428b-a098-43d2bb63156b

	  Kernel Version:             5.10.57

	  OS Image:                   Buildroot 2021.02.12

	  Operating System:           linux

	  Architecture:               arm64

	  Container Runtime Version:  docker://24.0.6

	  Kubelet Version:            v1.28.2

	  Kube-Proxy Version:         v1.28.2

	PodCIDR:                      10.244.0.0/24

	PodCIDRs:                     10.244.0.0/24

	Non-terminated Pods:          (13 in total)

	  Namespace                   Name                                          CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age

	  ---------                   ----                                          ------------  ----------  ---------------  -------------  ---

	  default                     hello-node-759d89bdcc-8g5cr                   0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         26s

	  default                     hello-node-connect-7799dfb7c6-ctzmg           0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         42s

	  default                     nginx-svc                                     0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         51s

	  default                     sp-pod                                        0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         34s

	  kube-system                 coredns-5dd5756b68-tgwk6                      100m (5%!)(MISSING)     0 (0%!)(MISSING)      70Mi (1%!)(MISSING)        170Mi (4%!)(MISSING)     2m20s

	  kube-system                 etcd-functional-847000                        100m (5%!)(MISSING)     0 (0%!)(MISSING)      100Mi (2%!)(MISSING)       0 (0%!)(MISSING)         2m34s

	  kube-system                 kube-apiserver-functional-847000              250m (12%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         78s

	  kube-system                 kube-controller-manager-functional-847000     200m (10%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         2m34s

	  kube-system                 kube-proxy-sb8tc                              0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         2m20s

	  kube-system                 kube-scheduler-functional-847000              100m (5%!)(MISSING)     0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         2m34s

	  kube-system                 storage-provisioner                           0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         117s

	  kubernetes-dashboard        dashboard-metrics-scraper-7fd5cb4ddc-922ns    0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         11s

	  kubernetes-dashboard        kubernetes-dashboard-8694d4445c-fvx7h         0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         11s

	Allocated resources:

	  (Total limits may be over 100 percent, i.e., overcommitted.)

	  Resource           Requests    Limits

	  --------           --------    ------

	  cpu                750m (37%!)(MISSING)  0 (0%!)(MISSING)

	  memory             170Mi (4%!)(MISSING)  170Mi (4%!)(MISSING)

	  ephemeral-storage  0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-1Gi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-2Mi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-32Mi     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-64Ki     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	Events:

	  Type    Reason                   Age                From             Message

	  ----    ------                   ----               ----             -------

	  Normal  Starting                 2m19s              kube-proxy       

	  Normal  Starting                 77s                kube-proxy       

	  Normal  Starting                 118s               kube-proxy       

	  Normal  Starting                 2m34s              kubelet          Starting kubelet.

	  Normal  NodeAllocatableEnforced  2m34s              kubelet          Updated Node Allocatable limit across pods

	  Normal  NodeHasSufficientMemory  2m34s              kubelet          Node functional-847000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    2m34s              kubelet          Node functional-847000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     2m34s              kubelet          Node functional-847000 status is now: NodeHasSufficientPID

	  Normal  NodeReady                2m30s              kubelet          Node functional-847000 status is now: NodeReady

	  Normal  RegisteredNode           2m21s              node-controller  Node functional-847000 event: Registered Node functional-847000 in Controller

	  Normal  NodeNotReady             2m13s              kubelet          Node functional-847000 status is now: NodeNotReady

	  Normal  RegisteredNode           106s               node-controller  Node functional-847000 event: Registered Node functional-847000 in Controller

	  Normal  Starting                 82s                kubelet          Starting kubelet.

	  Normal  NodeHasSufficientMemory  82s (x8 over 82s)  kubelet          Node functional-847000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    82s (x8 over 82s)  kubelet          Node functional-847000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     82s (x7 over 82s)  kubelet          Node functional-847000 status is now: NodeHasSufficientPID

	  Normal  NodeAllocatableEnforced  82s                kubelet          Updated Node Allocatable limit across pods

	  Normal  RegisteredNode           66s                node-controller  Node functional-847000 event: Registered Node functional-847000 in Controller

	* 

	* ==> dmesg <==

	* [  +5.031913] kauditd_printk_skb: 23 callbacks suppressed

	[  +6.297452] systemd-fstab-generator[4047]: Ignoring "noauto" for root device

	[  +0.083604] systemd-fstab-generator[4058]: Ignoring "noauto" for root device

	[  +0.080464] systemd-fstab-generator[4069]: Ignoring "noauto" for root device

	[  +0.084761] systemd-fstab-generator[4080]: Ignoring "noauto" for root device

	[  +0.076893] systemd-fstab-generator[4146]: Ignoring "noauto" for root device

	[  +5.097153] kauditd_printk_skb: 34 callbacks suppressed

	[ +23.043707] systemd-fstab-generator[5825]: Ignoring "noauto" for root device

	[  +0.150003] systemd-fstab-generator[5858]: Ignoring "noauto" for root device

	[  +0.097832] systemd-fstab-generator[5869]: Ignoring "noauto" for root device

	[  +0.103226] systemd-fstab-generator[5882]: Ignoring "noauto" for root device

	[Sep18 19:15] systemd-fstab-generator[6434]: Ignoring "noauto" for root device

	[  +0.085823] systemd-fstab-generator[6445]: Ignoring "noauto" for root device

	[  +0.084394] systemd-fstab-generator[6466]: Ignoring "noauto" for root device

	[  +0.090378] systemd-fstab-generator[6477]: Ignoring "noauto" for root device

	[  +0.075345] systemd-fstab-generator[6537]: Ignoring "noauto" for root device

	[  +0.959073] systemd-fstab-generator[6795]: Ignoring "noauto" for root device

	[  +4.565477] kauditd_printk_skb: 29 callbacks suppressed

	[ +26.440071] kauditd_printk_skb: 9 callbacks suppressed

	[  +5.004772] kauditd_printk_skb: 1 callbacks suppressed

	[  +5.431270] TCP: eth0: Driver has suspect GRO implementation, TCP performance may be compromised.

	[  +4.075329] kauditd_printk_skb: 6 callbacks suppressed

	[Sep18 19:16] kauditd_printk_skb: 1 callbacks suppressed

	[  +9.468564] kauditd_printk_skb: 1 callbacks suppressed

	[ +13.444299] kauditd_printk_skb: 8 callbacks suppressed

	* 

	* ==> etcd [1943ec715608] <==

	* {"level":"info","ts":"2023-09-18T19:14:29.480034Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T19:14:31.362213Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 is starting a new election at term 2"}

	{"level":"info","ts":"2023-09-18T19:14:31.362355Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became pre-candidate at term 2"}

	{"level":"info","ts":"2023-09-18T19:14:31.362397Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 received MsgPreVoteResp from 7520ddf439b1d16 at term 2"}

	{"level":"info","ts":"2023-09-18T19:14:31.362428Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became candidate at term 3"}

	{"level":"info","ts":"2023-09-18T19:14:31.362443Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 received MsgVoteResp from 7520ddf439b1d16 at term 3"}

	{"level":"info","ts":"2023-09-18T19:14:31.362469Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became leader at term 3"}

	{"level":"info","ts":"2023-09-18T19:14:31.362523Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: 7520ddf439b1d16 elected leader 7520ddf439b1d16 at term 3"}

	{"level":"info","ts":"2023-09-18T19:14:31.365358Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2023-09-18T19:14:31.365846Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2023-09-18T19:14:31.365364Z","caller":"etcdserver/server.go:2062","msg":"published local member to cluster through raft","local-member-id":"7520ddf439b1d16","local-member-attributes":"{Name:functional-847000 ClientURLs:[https://192.168.105.4:2379]}","request-path":"/0/members/7520ddf439b1d16/attributes","cluster-id":"80e92d98c466b02f","publish-timeout":"7s"}

	{"level":"info","ts":"2023-09-18T19:14:31.369096Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"127.0.0.1:2379"}

	{"level":"info","ts":"2023-09-18T19:14:31.370513Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}

	{"level":"info","ts":"2023-09-18T19:14:31.370751Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}

	{"level":"info","ts":"2023-09-18T19:14:31.371498Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.105.4:2379"}

	{"level":"info","ts":"2023-09-18T19:14:55.651174Z","caller":"osutil/interrupt_unix.go:64","msg":"received signal; shutting down","signal":"terminated"}

	{"level":"info","ts":"2023-09-18T19:14:55.651198Z","caller":"embed/etcd.go:376","msg":"closing etcd server","name":"functional-847000","data-dir":"/var/lib/minikube/etcd","advertise-peer-urls":["https://192.168.105.4:2380"],"advertise-client-urls":["https://192.168.105.4:2379"]}

	{"level":"warn","ts":"2023-09-18T19:14:55.65124Z","caller":"embed/serve.go:212","msg":"stopping secure grpc server due to error","error":"accept tcp 127.0.0.1:2379: use of closed network connection"}

	{"level":"warn","ts":"2023-09-18T19:14:55.651273Z","caller":"embed/serve.go:214","msg":"stopped secure grpc server due to error","error":"accept tcp 127.0.0.1:2379: use of closed network connection"}

	{"level":"warn","ts":"2023-09-18T19:14:55.66043Z","caller":"embed/serve.go:212","msg":"stopping secure grpc server due to error","error":"accept tcp 192.168.105.4:2379: use of closed network connection"}

	{"level":"warn","ts":"2023-09-18T19:14:55.66045Z","caller":"embed/serve.go:214","msg":"stopped secure grpc server due to error","error":"accept tcp 192.168.105.4:2379: use of closed network connection"}

	{"level":"info","ts":"2023-09-18T19:14:55.662153Z","caller":"etcdserver/server.go:1465","msg":"skipped leadership transfer for single voting member cluster","local-member-id":"7520ddf439b1d16","current-leader-member-id":"7520ddf439b1d16"}

	{"level":"info","ts":"2023-09-18T19:14:55.667315Z","caller":"embed/etcd.go:579","msg":"stopping serving peer traffic","address":"192.168.105.4:2380"}

	{"level":"info","ts":"2023-09-18T19:14:55.667368Z","caller":"embed/etcd.go:584","msg":"stopped serving peer traffic","address":"192.168.105.4:2380"}

	{"level":"info","ts":"2023-09-18T19:14:55.667376Z","caller":"embed/etcd.go:378","msg":"closed etcd server","name":"functional-847000","data-dir":"/var/lib/minikube/etcd","advertise-peer-urls":["https://192.168.105.4:2380"],"advertise-client-urls":["https://192.168.105.4:2379"]}

	* 

	* ==> etcd [3262d7c8a5d7] <==

	* {"level":"info","ts":"2023-09-18T19:15:09.259215Z","caller":"embed/etcd.go:855","msg":"serving metrics","address":"http://127.0.0.1:2381"}

	{"level":"info","ts":"2023-09-18T19:15:09.259229Z","caller":"etcdserver/server.go:754","msg":"starting initial election tick advance","election-ticks":10}

	{"level":"info","ts":"2023-09-18T19:15:09.259324Z","caller":"embed/etcd.go:597","msg":"serving peer traffic","address":"192.168.105.4:2380"}

	{"level":"info","ts":"2023-09-18T19:15:09.259328Z","caller":"embed/etcd.go:569","msg":"cmux::serve","address":"192.168.105.4:2380"}

	{"level":"info","ts":"2023-09-18T19:15:09.259535Z","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"/var/lib/minikube/etcd/member/snap","suffix":"snap.db","max":5,"interval":"30s"}

	{"level":"info","ts":"2023-09-18T19:15:09.259543Z","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"/var/lib/minikube/etcd/member/snap","suffix":"snap","max":5,"interval":"30s"}

	{"level":"info","ts":"2023-09-18T19:15:09.259547Z","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"/var/lib/minikube/etcd/member/wal","suffix":"wal","max":5,"interval":"30s"}

	{"level":"info","ts":"2023-09-18T19:15:09.259646Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 switched to configuration voters=(527499358918876438)"}

	{"level":"info","ts":"2023-09-18T19:15:09.259667Z","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"80e92d98c466b02f","local-member-id":"7520ddf439b1d16","added-peer-id":"7520ddf439b1d16","added-peer-peer-urls":["https://192.168.105.4:2380"]}

	{"level":"info","ts":"2023-09-18T19:15:09.259704Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"80e92d98c466b02f","local-member-id":"7520ddf439b1d16","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T19:15:09.259715Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T19:15:10.829503Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 is starting a new election at term 3"}

	{"level":"info","ts":"2023-09-18T19:15:10.829636Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became pre-candidate at term 3"}

	{"level":"info","ts":"2023-09-18T19:15:10.82971Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 received MsgPreVoteResp from 7520ddf439b1d16 at term 3"}

	{"level":"info","ts":"2023-09-18T19:15:10.829744Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became candidate at term 4"}

	{"level":"info","ts":"2023-09-18T19:15:10.82976Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 received MsgVoteResp from 7520ddf439b1d16 at term 4"}

	{"level":"info","ts":"2023-09-18T19:15:10.829785Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"7520ddf439b1d16 became leader at term 4"}

	{"level":"info","ts":"2023-09-18T19:15:10.829807Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: 7520ddf439b1d16 elected leader 7520ddf439b1d16 at term 4"}

	{"level":"info","ts":"2023-09-18T19:15:10.832211Z","caller":"etcdserver/server.go:2062","msg":"published local member to cluster through raft","local-member-id":"7520ddf439b1d16","local-member-attributes":"{Name:functional-847000 ClientURLs:[https://192.168.105.4:2379]}","request-path":"/0/members/7520ddf439b1d16/attributes","cluster-id":"80e92d98c466b02f","publish-timeout":"7s"}

	{"level":"info","ts":"2023-09-18T19:15:10.832237Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2023-09-18T19:15:10.832471Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2023-09-18T19:15:10.834444Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"127.0.0.1:2379"}

	{"level":"info","ts":"2023-09-18T19:15:10.834495Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.105.4:2379"}

	{"level":"info","ts":"2023-09-18T19:15:10.834849Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}

	{"level":"info","ts":"2023-09-18T19:15:10.834875Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}

	* 

	* ==> kernel <==

	*  19:16:30 up 2 min,  0 users,  load average: 0.70, 0.41, 0.16

	Linux functional-847000 5.10.57 #1 SMP PREEMPT Fri Sep 15 19:03:18 UTC 2023 aarch64 GNU/Linux

	PRETTY_NAME="Buildroot 2021.02.12"

	* 

	* ==> kube-apiserver [43c1216be4bd] <==

	* I0918 19:15:11.498165       1 cache.go:39] Caches are synced for APIServiceRegistrationController controller

	I0918 19:15:11.513410       1 shared_informer.go:318] Caches are synced for crd-autoregister

	I0918 19:15:11.513436       1 aggregator.go:166] initial CRD sync complete...

	I0918 19:15:11.513441       1 autoregister_controller.go:141] Starting autoregister controller

	I0918 19:15:11.513444       1 cache.go:32] Waiting for caches to sync for autoregister controller

	I0918 19:15:11.513447       1 cache.go:39] Caches are synced for autoregister controller

	I0918 19:15:11.549261       1 shared_informer.go:318] Caches are synced for node_authorizer

	I0918 19:15:11.570417       1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io

	I0918 19:15:11.570417       1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io

	I0918 19:15:12.401267       1 storage_scheduling.go:111] all system priority classes are created successfully or already exist.

	I0918 19:15:12.539596       1 controller.go:624] quota admission added evaluator for: serviceaccounts

	I0918 19:15:12.542612       1 controller.go:624] quota admission added evaluator for: deployments.apps

	I0918 19:15:12.553080       1 controller.go:624] quota admission added evaluator for: daemonsets.apps

	I0918 19:15:12.560476       1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io

	I0918 19:15:12.562589       1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io

	I0918 19:15:24.236838       1 controller.go:624] quota admission added evaluator for: endpoints

	I0918 19:15:24.387273       1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io

	I0918 19:15:34.139655       1 alloc.go:330] "allocated clusterIPs" service="default/invalid-svc" clusterIPs={"IPv4":"10.98.235.204"}

	I0918 19:15:39.279016       1 alloc.go:330] "allocated clusterIPs" service="default/nginx-svc" clusterIPs={"IPv4":"10.104.79.9"}

	I0918 19:15:48.665428       1 controller.go:624] quota admission added evaluator for: replicasets.apps

	I0918 19:15:48.707889       1 alloc.go:330] "allocated clusterIPs" service="default/hello-node-connect" clusterIPs={"IPv4":"10.102.30.144"}

	I0918 19:16:04.086904       1 alloc.go:330] "allocated clusterIPs" service="default/hello-node" clusterIPs={"IPv4":"10.102.94.41"}

	I0918 19:16:19.236786       1 controller.go:624] quota admission added evaluator for: namespaces

	I0918 19:16:19.312973       1 alloc.go:330] "allocated clusterIPs" service="kubernetes-dashboard/kubernetes-dashboard" clusterIPs={"IPv4":"10.101.160.101"}

	I0918 19:16:19.333831       1 alloc.go:330] "allocated clusterIPs" service="kubernetes-dashboard/dashboard-metrics-scraper" clusterIPs={"IPv4":"10.106.213.236"}

	* 

	* ==> kube-controller-manager [0c9d5dcc3515] <==

	* I0918 19:14:44.841562       1 shared_informer.go:318] Caches are synced for persistent volume

	I0918 19:14:44.841675       1 shared_informer.go:318] Caches are synced for GC

	I0918 19:14:44.841706       1 shared_informer.go:318] Caches are synced for ReplicaSet

	I0918 19:14:44.841952       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kube-system/coredns-5dd5756b68" duration="149.892µs"

	I0918 19:14:44.842373       1 shared_informer.go:318] Caches are synced for daemon sets

	I0918 19:14:44.843282       1 shared_informer.go:318] Caches are synced for HPA

	I0918 19:14:44.846434       1 shared_informer.go:318] Caches are synced for node

	I0918 19:14:44.846507       1 range_allocator.go:174] "Sending events to api server"

	I0918 19:14:44.846532       1 range_allocator.go:178] "Starting range CIDR allocator"

	I0918 19:14:44.846550       1 shared_informer.go:311] Waiting for caches to sync for cidrallocator

	I0918 19:14:44.846570       1 shared_informer.go:318] Caches are synced for cidrallocator

	I0918 19:14:44.847055       1 shared_informer.go:318] Caches are synced for PVC protection

	I0918 19:14:44.848169       1 shared_informer.go:318] Caches are synced for ephemeral

	I0918 19:14:44.849279       1 shared_informer.go:318] Caches are synced for endpoint_slice

	I0918 19:14:44.849495       1 shared_informer.go:318] Caches are synced for expand

	I0918 19:14:44.899791       1 shared_informer.go:318] Caches are synced for deployment

	I0918 19:14:44.913909       1 shared_informer.go:318] Caches are synced for disruption

	I0918 19:14:44.915125       1 shared_informer.go:318] Caches are synced for ClusterRoleAggregator

	I0918 19:14:44.959948       1 shared_informer.go:318] Caches are synced for resource quota

	I0918 19:14:44.991815       1 shared_informer.go:318] Caches are synced for endpoint

	I0918 19:14:45.016461       1 shared_informer.go:318] Caches are synced for endpoint_slice_mirroring

	I0918 19:14:45.026557       1 shared_informer.go:318] Caches are synced for resource quota

	I0918 19:14:45.371567       1 shared_informer.go:318] Caches are synced for garbage collector

	I0918 19:14:45.443296       1 shared_informer.go:318] Caches are synced for garbage collector

	I0918 19:14:45.443345       1 garbagecollector.go:166] "All resource monitors have synced. Proceeding to collect garbage"

	* 

	* ==> kube-controller-manager [f56043fcfb98] <==

	* E0918 19:16:19.276844       1 replica_set.go:557] sync "kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" failed with pods "dashboard-metrics-scraper-7fd5cb4ddc-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I0918 19:16:19.277212       1 event.go:307] "Event occurred" object="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"kubernetes-dashboard-8694d4445c-\" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount \"kubernetes-dashboard\" not found"

	I0918 19:16:19.277245       1 event.go:307] "Event occurred" object="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"dashboard-metrics-scraper-7fd5cb4ddc-\" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount \"kubernetes-dashboard\" not found"

	I0918 19:16:19.280534       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="9.261518ms"

	E0918 19:16:19.280626       1 replica_set.go:557] sync "kubernetes-dashboard/kubernetes-dashboard-8694d4445c" failed with pods "kubernetes-dashboard-8694d4445c-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I0918 19:16:19.283196       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="2.52316ms"

	E0918 19:16:19.283276       1 replica_set.go:557] sync "kubernetes-dashboard/kubernetes-dashboard-8694d4445c" failed with pods "kubernetes-dashboard-8694d4445c-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I0918 19:16:19.283310       1 event.go:307] "Event occurred" object="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"kubernetes-dashboard-8694d4445c-\" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount \"kubernetes-dashboard\" not found"

	I0918 19:16:19.288340       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="2.595119ms"

	E0918 19:16:19.288462       1 replica_set.go:557] sync "kubernetes-dashboard/kubernetes-dashboard-8694d4445c" failed with pods "kubernetes-dashboard-8694d4445c-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I0918 19:16:19.288619       1 event.go:307] "Event occurred" object="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"kubernetes-dashboard-8694d4445c-\" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount \"kubernetes-dashboard\" not found"

	I0918 19:16:19.300404       1 event.go:307] "Event occurred" object="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: dashboard-metrics-scraper-7fd5cb4ddc-922ns"

	I0918 19:16:19.304321       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="6.494608ms"

	I0918 19:16:19.314310       1 event.go:307] "Event occurred" object="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kubernetes-dashboard-8694d4445c-fvx7h"

	I0918 19:16:19.321022       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="16.675917ms"

	I0918 19:16:19.321045       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="9.208µs"

	I0918 19:16:19.321063       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="12.453844ms"

	I0918 19:16:19.327894       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="6.812941ms"

	I0918 19:16:19.328034       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="121.542µs"

	I0918 19:16:19.332221       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="24.75µs"

	I0918 19:16:22.983867       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="3.529878ms"

	I0918 19:16:22.983898       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc" duration="12.834µs"

	I0918 19:16:23.404948       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-node-connect-7799dfb7c6" duration="24.125µs"

	I0918 19:16:27.002245       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="3.64135ms"

	I0918 19:16:27.002267       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="kubernetes-dashboard/kubernetes-dashboard-8694d4445c" duration="10.792µs"

	* 

	* ==> kube-proxy [4c7ca1c5d88f] <==

	* I0918 19:14:29.746748       1 server_others.go:69] "Using iptables proxy"

	I0918 19:14:32.037782       1 node.go:141] Successfully retrieved node IP: 192.168.105.4

	I0918 19:14:32.051409       1 server_others.go:121] "No iptables support for family" ipFamily="IPv6"

	I0918 19:14:32.051421       1 server.go:634] "kube-proxy running in single-stack mode" ipFamily="IPv4"

	I0918 19:14:32.052685       1 server_others.go:152] "Using iptables Proxier"

	I0918 19:14:32.052719       1 proxier.go:251] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses"

	I0918 19:14:32.052805       1 server.go:846] "Version info" version="v1.28.2"

	I0918 19:14:32.052864       1 server.go:848] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0918 19:14:32.053165       1 config.go:188] "Starting service config controller"

	I0918 19:14:32.053184       1 shared_informer.go:311] Waiting for caches to sync for service config

	I0918 19:14:32.053196       1 config.go:97] "Starting endpoint slice config controller"

	I0918 19:14:32.053205       1 shared_informer.go:311] Waiting for caches to sync for endpoint slice config

	I0918 19:14:32.053401       1 config.go:315] "Starting node config controller"

	I0918 19:14:32.053413       1 shared_informer.go:311] Waiting for caches to sync for node config

	I0918 19:14:32.153628       1 shared_informer.go:318] Caches are synced for node config

	I0918 19:14:32.153673       1 shared_informer.go:318] Caches are synced for service config

	I0918 19:14:32.153688       1 shared_informer.go:318] Caches are synced for endpoint slice config

	* 

	* ==> kube-proxy [c89a5dd67e17] <==

	* I0918 19:15:12.969026       1 server_others.go:69] "Using iptables proxy"

	I0918 19:15:12.973211       1 node.go:141] Successfully retrieved node IP: 192.168.105.4

	I0918 19:15:12.981282       1 server_others.go:121] "No iptables support for family" ipFamily="IPv6"

	I0918 19:15:12.981294       1 server.go:634] "kube-proxy running in single-stack mode" ipFamily="IPv4"

	I0918 19:15:12.981941       1 server_others.go:152] "Using iptables Proxier"

	I0918 19:15:12.981984       1 proxier.go:251] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses"

	I0918 19:15:12.982069       1 server.go:846] "Version info" version="v1.28.2"

	I0918 19:15:12.982080       1 server.go:848] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0918 19:15:12.982427       1 config.go:188] "Starting service config controller"

	I0918 19:15:12.982438       1 shared_informer.go:311] Waiting for caches to sync for service config

	I0918 19:15:12.982482       1 config.go:97] "Starting endpoint slice config controller"

	I0918 19:15:12.982487       1 shared_informer.go:311] Waiting for caches to sync for endpoint slice config

	I0918 19:15:12.983193       1 config.go:315] "Starting node config controller"

	I0918 19:15:12.983197       1 shared_informer.go:311] Waiting for caches to sync for node config

	I0918 19:15:13.082751       1 shared_informer.go:318] Caches are synced for endpoint slice config

	I0918 19:15:13.082773       1 shared_informer.go:318] Caches are synced for service config

	I0918 19:15:13.083287       1 shared_informer.go:318] Caches are synced for node config

	* 

	* ==> kube-scheduler [94ddf0564ae6] <==

	* I0918 19:14:29.961365       1 serving.go:348] Generated self-signed cert in-memory

	W0918 19:14:31.994863       1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system.  Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'

	W0918 19:14:31.994878       1 authentication.go:368] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"

	W0918 19:14:31.994883       1 authentication.go:369] Continuing without authentication configuration. This may treat all requests as anonymous.

	W0918 19:14:31.994886       1 authentication.go:370] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false

	I0918 19:14:32.023179       1 server.go:154] "Starting Kubernetes Scheduler" version="v1.28.2"

	I0918 19:14:32.023626       1 server.go:156] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0918 19:14:32.025637       1 secure_serving.go:210] Serving securely on 127.0.0.1:10259

	I0918 19:14:32.025788       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"

	I0918 19:14:32.025852       1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	I0918 19:14:32.026712       1 tlsconfig.go:240] "Starting DynamicServingCertificateController"

	I0918 19:14:32.126964       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	I0918 19:14:55.656804       1 secure_serving.go:255] Stopped listening on 127.0.0.1:10259

	I0918 19:14:55.656826       1 tlsconfig.go:255] "Shutting down DynamicServingCertificateController"

	E0918 19:14:55.656917       1 run.go:74] "command failed" err="finished without leader elect"

	* 

	* ==> kube-scheduler [f67b1852a8d9] <==

	* I0918 19:15:11.471066       1 secure_serving.go:210] Serving securely on 127.0.0.1:10259

	I0918 19:15:11.471852       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"

	I0918 19:15:11.471864       1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	I0918 19:15:11.471871       1 tlsconfig.go:240] "Starting DynamicServingCertificateController"

	W0918 19:15:11.475302       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope

	E0918 19:15:11.475322       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope

	W0918 19:15:11.475411       1 reflector.go:535] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	E0918 19:15:11.475421       1 reflector.go:147] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	W0918 19:15:11.475486       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	E0918 19:15:11.475494       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	W0918 19:15:11.475538       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope

	E0918 19:15:11.475546       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope

	W0918 19:15:11.475583       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	E0918 19:15:11.475592       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	W0918 19:15:11.475639       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	E0918 19:15:11.475647       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	W0918 19:15:11.478247       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	E0918 19:15:11.478259       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	W0918 19:15:11.478310       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	E0918 19:15:11.478317       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	W0918 19:15:11.478558       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	E0918 19:15:11.478602       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	W0918 19:15:11.478686       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	E0918 19:15:11.478715       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	I0918 19:15:12.872217       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	* 

	* ==> kubelet <==

	* -- Journal begins at Mon 2023-09-18 19:13:39 UTC, ends at Mon 2023-09-18 19:16:31 UTC. --

	Sep 18 19:16:13 functional-847000 kubelet[6801]: I0918 19:16:13.117692    6801 topology_manager.go:215] "Topology Admit Handler" podUID="ebf89895-9ac9-4a77-ad15-d68c1c62a5c2" podNamespace="default" podName="busybox-mount"

	Sep 18 19:16:13 functional-847000 kubelet[6801]: I0918 19:16:13.239334    6801 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhdcw\" (UniqueName: \"kubernetes.io/projected/ebf89895-9ac9-4a77-ad15-d68c1c62a5c2-kube-api-access-zhdcw\") pod \"busybox-mount\" (UID: \"ebf89895-9ac9-4a77-ad15-d68c1c62a5c2\") " pod="default/busybox-mount"

	Sep 18 19:16:13 functional-847000 kubelet[6801]: I0918 19:16:13.239390    6801 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-volume\" (UniqueName: \"kubernetes.io/host-path/ebf89895-9ac9-4a77-ad15-d68c1c62a5c2-test-volume\") pod \"busybox-mount\" (UID: \"ebf89895-9ac9-4a77-ad15-d68c1c62a5c2\") " pod="default/busybox-mount"

	Sep 18 19:16:16 functional-847000 kubelet[6801]: I0918 19:16:16.157655    6801 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"test-volume\" (UniqueName: \"kubernetes.io/host-path/ebf89895-9ac9-4a77-ad15-d68c1c62a5c2-test-volume\") pod \"ebf89895-9ac9-4a77-ad15-d68c1c62a5c2\" (UID: \"ebf89895-9ac9-4a77-ad15-d68c1c62a5c2\") "

	Sep 18 19:16:16 functional-847000 kubelet[6801]: I0918 19:16:16.157697    6801 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhdcw\" (UniqueName: \"kubernetes.io/projected/ebf89895-9ac9-4a77-ad15-d68c1c62a5c2-kube-api-access-zhdcw\") pod \"ebf89895-9ac9-4a77-ad15-d68c1c62a5c2\" (UID: \"ebf89895-9ac9-4a77-ad15-d68c1c62a5c2\") "

	Sep 18 19:16:16 functional-847000 kubelet[6801]: I0918 19:16:16.157794    6801 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/ebf89895-9ac9-4a77-ad15-d68c1c62a5c2-test-volume" (OuterVolumeSpecName: "test-volume") pod "ebf89895-9ac9-4a77-ad15-d68c1c62a5c2" (UID: "ebf89895-9ac9-4a77-ad15-d68c1c62a5c2"). InnerVolumeSpecName "test-volume". PluginName "kubernetes.io/host-path", VolumeGidValue ""

	Sep 18 19:16:16 functional-847000 kubelet[6801]: I0918 19:16:16.161238    6801 operation_generator.go:878] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf89895-9ac9-4a77-ad15-d68c1c62a5c2-kube-api-access-zhdcw" (OuterVolumeSpecName: "kube-api-access-zhdcw") pod "ebf89895-9ac9-4a77-ad15-d68c1c62a5c2" (UID: "ebf89895-9ac9-4a77-ad15-d68c1c62a5c2"). InnerVolumeSpecName "kube-api-access-zhdcw". PluginName "kubernetes.io/projected", VolumeGidValue ""

	Sep 18 19:16:16 functional-847000 kubelet[6801]: I0918 19:16:16.258101    6801 reconciler_common.go:300] "Volume detached for volume \"kube-api-access-zhdcw\" (UniqueName: \"kubernetes.io/projected/ebf89895-9ac9-4a77-ad15-d68c1c62a5c2-kube-api-access-zhdcw\") on node \"functional-847000\" DevicePath \"\""

	Sep 18 19:16:16 functional-847000 kubelet[6801]: I0918 19:16:16.258127    6801 reconciler_common.go:300] "Volume detached for volume \"test-volume\" (UniqueName: \"kubernetes.io/host-path/ebf89895-9ac9-4a77-ad15-d68c1c62a5c2-test-volume\") on node \"functional-847000\" DevicePath \"\""

	Sep 18 19:16:16 functional-847000 kubelet[6801]: I0918 19:16:16.934016    6801 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d719925ca4d1215e8e4dba4fd9c42dd5d4f50f6ea0e88c697f6c8e6bf819170e"

	Sep 18 19:16:18 functional-847000 kubelet[6801]: I0918 19:16:18.398146    6801 scope.go:117] "RemoveContainer" containerID="4da4c99c7fd674aa14c1e28ab025d1240e6d23aa9a84e5bf15f2c40a0b660ff8"

	Sep 18 19:16:18 functional-847000 kubelet[6801]: I0918 19:16:18.946932    6801 scope.go:117] "RemoveContainer" containerID="4da4c99c7fd674aa14c1e28ab025d1240e6d23aa9a84e5bf15f2c40a0b660ff8"

	Sep 18 19:16:18 functional-847000 kubelet[6801]: I0918 19:16:18.947106    6801 scope.go:117] "RemoveContainer" containerID="bc135da570b0af6add9587d560db445997346ae3dee21f532c91798b013c9592"

	Sep 18 19:16:18 functional-847000 kubelet[6801]: E0918 19:16:18.947332    6801 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"echoserver-arm\" with CrashLoopBackOff: \"back-off 20s restarting failed container=echoserver-arm pod=hello-node-759d89bdcc-8g5cr_default(751e17c6-7b8b-4bf6-bb0d-97f745627ff3)\"" pod="default/hello-node-759d89bdcc-8g5cr" podUID="751e17c6-7b8b-4bf6-bb0d-97f745627ff3"

	Sep 18 19:16:19 functional-847000 kubelet[6801]: I0918 19:16:19.302720    6801 topology_manager.go:215] "Topology Admit Handler" podUID="c04d1d8a-74c2-4b78-9540-f0b80e57ea5c" podNamespace="kubernetes-dashboard" podName="dashboard-metrics-scraper-7fd5cb4ddc-922ns"

	Sep 18 19:16:19 functional-847000 kubelet[6801]: E0918 19:16:19.302754    6801 cpu_manager.go:395] "RemoveStaleState: removing container" podUID="ebf89895-9ac9-4a77-ad15-d68c1c62a5c2" containerName="mount-munger"

	Sep 18 19:16:19 functional-847000 kubelet[6801]: I0918 19:16:19.302773    6801 memory_manager.go:346] "RemoveStaleState removing state" podUID="ebf89895-9ac9-4a77-ad15-d68c1c62a5c2" containerName="mount-munger"

	Sep 18 19:16:19 functional-847000 kubelet[6801]: I0918 19:16:19.317717    6801 topology_manager.go:215] "Topology Admit Handler" podUID="333556e9-a33c-4d7a-a9d5-c7a61c11f2cf" podNamespace="kubernetes-dashboard" podName="kubernetes-dashboard-8694d4445c-fvx7h"

	Sep 18 19:16:19 functional-847000 kubelet[6801]: I0918 19:16:19.473260    6801 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-volume\" (UniqueName: \"kubernetes.io/empty-dir/333556e9-a33c-4d7a-a9d5-c7a61c11f2cf-tmp-volume\") pod \"kubernetes-dashboard-8694d4445c-fvx7h\" (UID: \"333556e9-a33c-4d7a-a9d5-c7a61c11f2cf\") " pod="kubernetes-dashboard/kubernetes-dashboard-8694d4445c-fvx7h"

	Sep 18 19:16:19 functional-847000 kubelet[6801]: I0918 19:16:19.473488    6801 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d46fl\" (UniqueName: \"kubernetes.io/projected/c04d1d8a-74c2-4b78-9540-f0b80e57ea5c-kube-api-access-d46fl\") pod \"dashboard-metrics-scraper-7fd5cb4ddc-922ns\" (UID: \"c04d1d8a-74c2-4b78-9540-f0b80e57ea5c\") " pod="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc-922ns"

	Sep 18 19:16:19 functional-847000 kubelet[6801]: I0918 19:16:19.473507    6801 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnkd8\" (UniqueName: \"kubernetes.io/projected/333556e9-a33c-4d7a-a9d5-c7a61c11f2cf-kube-api-access-nnkd8\") pod \"kubernetes-dashboard-8694d4445c-fvx7h\" (UID: \"333556e9-a33c-4d7a-a9d5-c7a61c11f2cf\") " pod="kubernetes-dashboard/kubernetes-dashboard-8694d4445c-fvx7h"

	Sep 18 19:16:19 functional-847000 kubelet[6801]: I0918 19:16:19.473518    6801 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-volume\" (UniqueName: \"kubernetes.io/empty-dir/c04d1d8a-74c2-4b78-9540-f0b80e57ea5c-tmp-volume\") pod \"dashboard-metrics-scraper-7fd5cb4ddc-922ns\" (UID: \"c04d1d8a-74c2-4b78-9540-f0b80e57ea5c\") " pod="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc-922ns"

	Sep 18 19:16:22 functional-847000 kubelet[6801]: I0918 19:16:22.980270    6801 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="kubernetes-dashboard/dashboard-metrics-scraper-7fd5cb4ddc-922ns" podStartSLOduration=1.636377989 podCreationTimestamp="2023-09-18 19:16:19 +0000 UTC" firstStartedPulling="2023-09-18 19:16:19.809756169 +0000 UTC m=+71.488512821" lastFinishedPulling="2023-09-18 19:16:22.153622995 +0000 UTC m=+73.832379688" observedRunningTime="2023-09-18 19:16:22.980092773 +0000 UTC m=+74.658849467" watchObservedRunningTime="2023-09-18 19:16:22.980244856 +0000 UTC m=+74.659001550"

	Sep 18 19:16:23 functional-847000 kubelet[6801]: I0918 19:16:23.398433    6801 scope.go:117] "RemoveContainer" containerID="9de3fa653dead4c7773edccbd3c53ddd83b403c71318f5dec4753b80bd88dca9"

	Sep 18 19:16:23 functional-847000 kubelet[6801]: E0918 19:16:23.398546    6801 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"echoserver-arm\" with CrashLoopBackOff: \"back-off 20s restarting failed container=echoserver-arm pod=hello-node-connect-7799dfb7c6-ctzmg_default(8cc3e8ed-8be6-4416-87c0-3548bd1cbbe6)\"" pod="default/hello-node-connect-7799dfb7c6-ctzmg" podUID="8cc3e8ed-8be6-4416-87c0-3548bd1cbbe6"

	* 

	* ==> kubernetes-dashboard [4840d9fd28ae] <==

	* 2023/09/18 19:16:26 Using namespace: kubernetes-dashboard

	2023/09/18 19:16:26 Using in-cluster config to connect to apiserver

	2023/09/18 19:16:26 Using secret token for csrf signing

	2023/09/18 19:16:26 Initializing csrf token from kubernetes-dashboard-csrf secret

	2023/09/18 19:16:26 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf

	2023/09/18 19:16:26 Successful initial request to the apiserver, version: v1.28.2

	2023/09/18 19:16:26 Generating JWE encryption key

	2023/09/18 19:16:26 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting

	2023/09/18 19:16:26 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard

	2023/09/18 19:16:26 Initializing JWE encryption key from synchronized object

	2023/09/18 19:16:26 Creating in-cluster Sidecar client

	2023/09/18 19:16:26 Successful request to sidecar

	2023/09/18 19:16:26 Serving insecurely on HTTP port: 9090

	2023/09/18 19:16:26 Starting overwatch

	* 

	* ==> storage-provisioner [5388c197829f] <==

	* I0918 19:14:36.205039       1 storage_provisioner.go:116] Initializing the minikube storage provisioner...

	I0918 19:14:36.209325       1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!

	I0918 19:14:36.209341       1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...

	I0918 19:14:36.212653       1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath

	I0918 19:14:36.212760       1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_functional-847000_9f2ffb87-fca2-4a54-9f0e-bbedd64a022c!

	I0918 19:14:36.212982       1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"48555537-faa4-450e-9382-be08f24094ee", APIVersion:"v1", ResourceVersion:"439", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' functional-847000_9f2ffb87-fca2-4a54-9f0e-bbedd64a022c became leader

	I0918 19:14:36.312990       1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_functional-847000_9f2ffb87-fca2-4a54-9f0e-bbedd64a022c!

	* 

	* ==> storage-provisioner [cdd0803f0b0c] <==

	* I0918 19:15:12.950236       1 storage_provisioner.go:116] Initializing the minikube storage provisioner...

	I0918 19:15:12.956497       1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!

	I0918 19:15:12.956586       1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...

	I0918 19:15:30.342345       1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath

	I0918 19:15:30.342438       1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_functional-847000_132c0f0e-bd4c-49ea-aaa6-3323a3935e5b!

	I0918 19:15:30.342849       1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"48555537-faa4-450e-9382-be08f24094ee", APIVersion:"v1", ResourceVersion:"558", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' functional-847000_132c0f0e-bd4c-49ea-aaa6-3323a3935e5b became leader

	I0918 19:15:30.443295       1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_functional-847000_132c0f0e-bd4c-49ea-aaa6-3323a3935e5b!

	I0918 19:15:44.099278       1 controller.go:1332] provision "default/myclaim" class "standard": started

	I0918 19:15:44.099358       1 storage_provisioner.go:61] Provisioning volume {&StorageClass{ObjectMeta:{standard    d032ad6f-165d-4083-8a88-fed37e50afe1 346 0 2023-09-18 19:14:10 +0000 UTC <nil> <nil> map[addonmanager.kubernetes.io/mode:EnsureExists] map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"},"labels":{"addonmanager.kubernetes.io/mode":"EnsureExists"},"name":"standard"},"provisioner":"k8s.io/minikube-hostpath"}

	 storageclass.kubernetes.io/is-default-class:true] [] []  [{kubectl-client-side-apply Update storage.k8s.io/v1 2023-09-18 19:14:10 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{},"f:storageclass.kubernetes.io/is-default-class":{}},"f:labels":{".":{},"f:addonmanager.kubernetes.io/mode":{}}},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}}}]},Provisioner:k8s.io/minikube-hostpath,Parameters:map[string]string{},ReclaimPolicy:*Delete,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*Immediate,AllowedTopologies:[]TopologySelectorTerm{},} pvc-928b0ae6-6aab-4910-ba6d-2f276dad7fcb &PersistentVolumeClaim{ObjectMeta:{myclaim  default  928b0ae6-6aab-4910-ba6d-2f276dad7fcb 623 0 2023-09-18 19:15:44 +0000 UTC <nil> <nil> map[] map[kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"name":"myclaim","namespace":"default"},"spec":{"accessModes":["Rea

dWriteOnce"],"resources":{"requests":{"storage":"500Mi"}},"volumeMode":"Filesystem"}}

	 volume.beta.kubernetes.io/storage-provisioner:k8s.io/minikube-hostpath volume.kubernetes.io/storage-provisioner:k8s.io/minikube-hostpath] [] [kubernetes.io/pvc-protection]  [{kube-controller-manager Update v1 2023-09-18 19:15:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:volume.beta.kubernetes.io/storage-provisioner":{},"f:volume.kubernetes.io/storage-provisioner":{}}}}} {kubectl-client-side-apply Update v1 2023-09-18 19:15:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:volumeMode":{}}}}]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{524288000 0} {<nil>} 500Mi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*standard,VolumeMode:*Filesystem,DataSource:nil,},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:

ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} nil} to /tmp/hostpath-provisioner/default/myclaim

	I0918 19:15:44.100007       1 controller.go:1439] provision "default/myclaim" class "standard": volume "pvc-928b0ae6-6aab-4910-ba6d-2f276dad7fcb" provisioned

	I0918 19:15:44.100228       1 controller.go:1456] provision "default/myclaim" class "standard": succeeded

	I0918 19:15:44.100264       1 volume_store.go:212] Trying to save persistentvolume "pvc-928b0ae6-6aab-4910-ba6d-2f276dad7fcb"

	I0918 19:15:44.100898       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"myclaim", UID:"928b0ae6-6aab-4910-ba6d-2f276dad7fcb", APIVersion:"v1", ResourceVersion:"623", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "default/myclaim"

	I0918 19:15:44.106613       1 volume_store.go:219] persistentvolume "pvc-928b0ae6-6aab-4910-ba6d-2f276dad7fcb" saved

	I0918 19:15:44.106780       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"myclaim", UID:"928b0ae6-6aab-4910-ba6d-2f276dad7fcb", APIVersion:"v1", ResourceVersion:"623", FieldPath:""}): type: 'Normal' reason: 'ProvisioningSucceeded' Successfully provisioned volume pvc-928b0ae6-6aab-4910-ba6d-2f276dad7fcb

-- /stdout --

helpers_test.go:254: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.APIServer}} -p functional-847000 -n functional-847000

helpers_test.go:261: (dbg) Run:  kubectl --context functional-847000 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running

helpers_test.go:272: non-running pods: busybox-mount

helpers_test.go:274: ======> post-mortem[TestFunctional/parallel/ServiceCmdConnect]: describe non-running pods <======

helpers_test.go:277: (dbg) Run:  kubectl --context functional-847000 describe pod busybox-mount

helpers_test.go:282: (dbg) kubectl --context functional-847000 describe pod busybox-mount:

-- stdout --

	Name:             busybox-mount

	Namespace:        default

	Priority:         0

	Service Account:  default

	Node:             functional-847000/192.168.105.4

	Start Time:       Mon, 18 Sep 2023 12:16:13 -0700

	Labels:           integration-test=busybox-mount

	Annotations:      <none>

	Status:           Succeeded

	IP:               10.244.0.11

	IPs:

	  IP:  10.244.0.11

	Containers:

	  mount-munger:

	    Container ID:  docker://6b5be5d04c6864dfb8d865d0ab0170f5c83b9d1f8f2e563210b87bf66219f30e

	    Image:         gcr.io/k8s-minikube/busybox:1.28.4-glibc

	    Image ID:      docker-pullable://gcr.io/k8s-minikube/busybox@sha256:2d03e6ceeb99250061dd110530b0ece7998cd84121f952adef120ea7c5a6f00e

	    Port:          <none>

	    Host Port:     <none>

	    Command:

	      /bin/sh

	      -c

	      --

	    Args:

	      cat /mount-9p/created-by-test; echo test > /mount-9p/created-by-pod; rm /mount-9p/created-by-test-removed-by-pod; echo test > /mount-9p/created-by-pod-removed-by-test date >> /mount-9p/pod-dates

	    State:          Terminated

	      Reason:       Completed

	      Exit Code:    0

	      Started:      Mon, 18 Sep 2023 12:16:14 -0700

	      Finished:     Mon, 18 Sep 2023 12:16:14 -0700

	    Ready:          False

	    Restart Count:  0

	    Environment:    <none>

	    Mounts:

	      /mount-9p from test-volume (rw)

	      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zhdcw (ro)

	Conditions:

	  Type              Status

	  Initialized       True 

	  Ready             False 

	  ContainersReady   False 

	  PodScheduled      True 

	Volumes:

	  test-volume:

	    Type:          HostPath (bare host directory volume)

	    Path:          /mount-9p

	    HostPathType:  

	  kube-api-access-zhdcw:

	    Type:                    Projected (a volume that contains injected data from multiple sources)

	    TokenExpirationSeconds:  3607

	    ConfigMapName:           kube-root-ca.crt

	    ConfigMapOptional:       <nil>

	    DownwardAPI:             true

	QoS Class:                   BestEffort

	Node-Selectors:              <none>

	Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s

	                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s

	Events:

	  Type    Reason     Age   From               Message

	  ----    ------     ----  ----               -------

	  Normal  Scheduled  18s   default-scheduler  Successfully assigned default/busybox-mount to functional-847000

	  Normal  Pulling    18s   kubelet            Pulling image "gcr.io/k8s-minikube/busybox:1.28.4-glibc"

	  Normal  Pulled     17s   kubelet            Successfully pulled image "gcr.io/k8s-minikube/busybox:1.28.4-glibc" in 1.052s (1.052s including waiting)

	  Normal  Created    17s   kubelet            Created container mount-munger

	  Normal  Started    17s   kubelet            Started container mount-munger

-- /stdout --

helpers_test.go:285: <<< TestFunctional/parallel/ServiceCmdConnect FAILED: end of post-mortem logs <<<

helpers_test.go:286: ---------------------/post-mortem---------------------------------

--- FAIL: TestFunctional/parallel/ServiceCmdConnect (42.71s)

=== RUN   TestImageBuild/serial/BuildWithBuildArg

image_test.go:99: (dbg) Run:  out/minikube-darwin-arm64 image build -t aaa:latest --build-opt=build-arg=ENV_A=test_env_str --build-opt=no-cache ./testdata/image-build/test-arg -p image-438000

image_test.go:105: failed to pass build-args with args: "out/minikube-darwin-arm64 image build -t aaa:latest --build-opt=build-arg=ENV_A=test_env_str --build-opt=no-cache ./testdata/image-build/test-arg -p image-438000" : 

-- stdout --

	Sending build context to Docker daemon  2.048kB

	Step 1/5 : FROM gcr.io/google-containers/alpine-with-bash:1.0

	 ---> 822c13824dc2

	Step 2/5 : ARG ENV_A

	 ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

	 ---> Running in 3d847d91d804

	Removing intermediate container 3d847d91d804

	 ---> cec5f152cdf4

	Step 3/5 : ARG ENV_B

	 ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

	 ---> Running in c8ca0e1b6638

	Removing intermediate container c8ca0e1b6638

	 ---> 1a3849f84aaa

	Step 4/5 : RUN echo "test-build-arg" $ENV_A $ENV_B

	 ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

	 ---> Running in a6a0272fed0e

	exec /bin/sh: exec format error

	

-- /stdout --

** stderr ** 

	DEPRECATED: The legacy builder is deprecated and will be removed in a future release.

	            Install the buildx component to build images with BuildKit:

	            https://docs.docker.com/go/buildx/

	The command '/bin/sh -c echo "test-build-arg" $ENV_A $ENV_B' returned a non-zero code: 1

** /stderr **

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.Host}} -p image-438000 -n image-438000

helpers_test.go:244: <<< TestImageBuild/serial/BuildWithBuildArg FAILED: start of post-mortem logs <<<

helpers_test.go:245: ======>  post-mortem[TestImageBuild/serial/BuildWithBuildArg]: minikube logs <======

helpers_test.go:247: (dbg) Run:  out/minikube-darwin-arm64 -p image-438000 logs -n 25

helpers_test.go:252: TestImageBuild/serial/BuildWithBuildArg logs: 

-- stdout --

	* 

	* ==> Audit <==

	* |----------------|----------------------------------------------------------|-------------------|---------|---------|---------------------|---------------------|

	|    Command     |                           Args                           |      Profile      |  User   | Version |     Start Time      |      End Time       |

	|----------------|----------------------------------------------------------|-------------------|---------|---------|---------------------|---------------------|

	| update-context | functional-847000                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | update-context                                           |                   |         |         |                     |                     |

	|                | --alsologtostderr -v=2                                   |                   |         |         |                     |                     |

	| update-context | functional-847000                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | update-context                                           |                   |         |         |                     |                     |

	|                | --alsologtostderr -v=2                                   |                   |         |         |                     |                     |

	| update-context | functional-847000                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | update-context                                           |                   |         |         |                     |                     |

	|                | --alsologtostderr -v=2                                   |                   |         |         |                     |                     |

	| image          | functional-847000 image ls                               | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	| image          | functional-847000 image load --daemon                    | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | gcr.io/google-containers/addon-resizer:functional-847000 |                   |         |         |                     |                     |

	|                | --alsologtostderr                                        |                   |         |         |                     |                     |

	| image          | functional-847000 image ls                               | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	| image          | functional-847000 image load --daemon                    | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | gcr.io/google-containers/addon-resizer:functional-847000 |                   |         |         |                     |                     |

	|                | --alsologtostderr                                        |                   |         |         |                     |                     |

	| image          | functional-847000 image ls                               | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	| image          | functional-847000 image save                             | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | gcr.io/google-containers/addon-resizer:functional-847000 |                   |         |         |                     |                     |

	|                | /Users/jenkins/workspace/addon-resizer-save.tar          |                   |         |         |                     |                     |

	|                | --alsologtostderr                                        |                   |         |         |                     |                     |

	| image          | functional-847000 image rm                               | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | gcr.io/google-containers/addon-resizer:functional-847000 |                   |         |         |                     |                     |

	|                | --alsologtostderr                                        |                   |         |         |                     |                     |

	| image          | functional-847000 image ls                               | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	| image          | functional-847000 image load                             | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | /Users/jenkins/workspace/addon-resizer-save.tar          |                   |         |         |                     |                     |

	|                | --alsologtostderr                                        |                   |         |         |                     |                     |

	| image          | functional-847000 image ls                               | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	| image          | functional-847000 image save --daemon                    | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | gcr.io/google-containers/addon-resizer:functional-847000 |                   |         |         |                     |                     |

	|                | --alsologtostderr                                        |                   |         |         |                     |                     |

	| image          | functional-847000                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|                | image ls --format yaml                                   |                   |         |         |                     |                     |

	|                | --alsologtostderr                                        |                   |         |         |                     |                     |

	| image          | functional-847000                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | image ls --format short                                  |                   |         |         |                     |                     |

	|                | --alsologtostderr                                        |                   |         |         |                     |                     |

	| ssh            | functional-847000 ssh pgrep                              | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT |                     |

	|                | buildkitd                                                |                   |         |         |                     |                     |

	| image          | functional-847000                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | image ls --format json                                   |                   |         |         |                     |                     |

	|                | --alsologtostderr                                        |                   |         |         |                     |                     |

	| image          | functional-847000 image build -t                         | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | localhost/my-image:functional-847000                     |                   |         |         |                     |                     |

	|                | testdata/build --alsologtostderr                         |                   |         |         |                     |                     |

	| image          | functional-847000                                        | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	|                | image ls --format table                                  |                   |         |         |                     |                     |

	|                | --alsologtostderr                                        |                   |         |         |                     |                     |

	| image          | functional-847000 image ls                               | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	| delete         | -p functional-847000                                     | functional-847000 | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:16 PDT |

	| start          | -p image-438000 --driver=qemu2                           | image-438000      | jenkins | v1.31.2 | 18 Sep 23 12:16 PDT | 18 Sep 23 12:17 PDT |

	|                |                                                          |                   |         |         |                     |                     |

	| image          | build -t aaa:latest                                      | image-438000      | jenkins | v1.31.2 | 18 Sep 23 12:17 PDT | 18 Sep 23 12:17 PDT |

	|                | ./testdata/image-build/test-normal                       |                   |         |         |                     |                     |

	|                | -p image-438000                                          |                   |         |         |                     |                     |

	| image          | build -t aaa:latest                                      | image-438000      | jenkins | v1.31.2 | 18 Sep 23 12:17 PDT | 18 Sep 23 12:17 PDT |

	|                | --build-opt=build-arg=ENV_A=test_env_str                 |                   |         |         |                     |                     |

	|                | --build-opt=no-cache                                     |                   |         |         |                     |                     |

	|                | ./testdata/image-build/test-arg -p                       |                   |         |         |                     |                     |

	|                | image-438000                                             |                   |         |         |                     |                     |

	|----------------|----------------------------------------------------------|-------------------|---------|---------|---------------------|---------------------|

	* 

	* ==> Last Start <==

	* Log file created at: 2023/09/18 12:16:42

	Running on machine: MacOS-M1-Agent-1

	Binary: Built with gc go1.21.1 for darwin/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0918 12:16:42.389494    2777 out.go:296] Setting OutFile to fd 1 ...

	I0918 12:16:42.389618    2777 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:16:42.389619    2777 out.go:309] Setting ErrFile to fd 2...

	I0918 12:16:42.389621    2777 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I0918 12:16:42.389736    2777 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17263-1251/.minikube/bin

	I0918 12:16:42.390782    2777 out.go:303] Setting JSON to false

	I0918 12:16:42.406947    2777 start.go:128] hostinfo: {"hostname":"MacOS-M1-Agent-1.local","uptime":2776,"bootTime":1695061826,"procs":421,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.5.2","kernelVersion":"22.6.0","kernelArch":"arm64","virtualizationSystem":"","virtualizationRole":"","hostId":"5b726c3a-f72c-561b-b03e-814251f12bfa"}

	W0918 12:16:42.407001    2777 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I0918 12:16:42.411291    2777 out.go:177] * [image-438000] minikube v1.31.2 on Darwin 13.5.2 (arm64)

	I0918 12:16:42.418288    2777 out.go:177]   - MINIKUBE_LOCATION=17263

	I0918 12:16:42.418339    2777 notify.go:220] Checking for updates...

	I0918 12:16:42.422314    2777 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 12:16:42.423272    2777 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-arm64

	I0918 12:16:42.426241    2777 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0918 12:16:42.429253    2777 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17263-1251/.minikube

	I0918 12:16:42.432281    2777 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0918 12:16:42.435494    2777 driver.go:373] Setting default libvirt URI to qemu:///system

	I0918 12:16:42.439258    2777 out.go:177] * Using the qemu2 driver based on user configuration

	I0918 12:16:42.446186    2777 start.go:298] selected driver: qemu2

	I0918 12:16:42.446190    2777 start.go:902] validating driver "qemu2" against <nil>

	I0918 12:16:42.446195    2777 start.go:913] status for qemu2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0918 12:16:42.446246    2777 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I0918 12:16:42.449233    2777 out.go:177] * Automatically selected the socket_vmnet network

	I0918 12:16:42.454519    2777 start_flags.go:384] Using suggested 4000MB memory alloc based on sys=16384MB, container=0MB

	I0918 12:16:42.454618    2777 start_flags.go:904] Wait components to verify : map[apiserver:true system_pods:true]

	I0918 12:16:42.454634    2777 cni.go:84] Creating CNI manager for ""

	I0918 12:16:42.454646    2777 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 12:16:42.454654    2777 start_flags.go:316] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I0918 12:16:42.454659    2777 start_flags.go:321] config:

	{Name:image-438000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:image-438000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRIS

ocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 12:16:42.459047    2777 iso.go:125] acquiring lock: {Name:mk34e23c181861c65264ec384f06e6cfa001aa08 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0918 12:16:42.463292    2777 out.go:177] * Starting control plane node image-438000 in cluster image-438000

	I0918 12:16:42.471237    2777 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 12:16:42.471263    2777 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4

	I0918 12:16:42.471271    2777 cache.go:57] Caching tarball of preloaded images

	I0918 12:16:42.471324    2777 preload.go:174] Found /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 in cache, skipping download

	I0918 12:16:42.471328    2777 cache.go:60] Finished verifying existence of preloaded tar for  v1.28.2 on docker

	I0918 12:16:42.471512    2777 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/config.json ...

	I0918 12:16:42.471523    2777 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/config.json: {Name:mk97d7d776b3a89c6280fcfd709e3983c8c42e77 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:16:42.471697    2777 start.go:365] acquiring machines lock for image-438000: {Name:mk4b1be517888c87845a1fd1bcf71e9e6d2854bb Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I0918 12:16:42.471725    2777 start.go:369] acquired machines lock for "image-438000" in 24.917µs

	I0918 12:16:42.471734    2777 start.go:93] Provisioning new machine with config: &{Name:image-438000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig

:{KubernetesVersion:v1.28.2 ClusterName:image-438000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: D

isableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 12:16:42.471758    2777 start.go:125] createHost starting for "" (driver="qemu2")

	I0918 12:16:42.490219    2777 out.go:204] * Creating qemu2 VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...

	I0918 12:16:42.512727    2777 start.go:159] libmachine.API.Create for "image-438000" (driver="qemu2")

	I0918 12:16:42.512747    2777 client.go:168] LocalClient.Create starting

	I0918 12:16:42.512813    2777 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem

	I0918 12:16:42.512836    2777 main.go:141] libmachine: Decoding PEM data...

	I0918 12:16:42.512844    2777 main.go:141] libmachine: Parsing certificate...

	I0918 12:16:42.512878    2777 main.go:141] libmachine: Reading certificate data from /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem

	I0918 12:16:42.512894    2777 main.go:141] libmachine: Decoding PEM data...

	I0918 12:16:42.512900    2777 main.go:141] libmachine: Parsing certificate...

	I0918 12:16:42.513191    2777 main.go:141] libmachine: Downloading /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17263-1251/.minikube/cache/iso/arm64/minikube-v1.31.0-1694798110-17250-arm64.iso...

	I0918 12:16:42.654104    2777 main.go:141] libmachine: Creating SSH key...

	I0918 12:16:42.725079    2777 main.go:141] libmachine: Creating Disk image...

	I0918 12:16:42.725083    2777 main.go:141] libmachine: Creating 20000 MB hard disk image...

	I0918 12:16:42.725223    2777 main.go:141] libmachine: executing: qemu-img convert -f raw -O qcow2 /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/disk.qcow2.raw /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/disk.qcow2

	I0918 12:16:42.746898    2777 main.go:141] libmachine: STDOUT: 

	I0918 12:16:42.746909    2777 main.go:141] libmachine: STDERR: 

	I0918 12:16:42.746963    2777 main.go:141] libmachine: executing: qemu-img resize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/disk.qcow2 +20000M

	I0918 12:16:42.754107    2777 main.go:141] libmachine: STDOUT: Image resized.

	I0918 12:16:42.754126    2777 main.go:141] libmachine: STDERR: 

	I0918 12:16:42.754144    2777 main.go:141] libmachine: DONE writing to /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/disk.qcow2.raw and /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/disk.qcow2

	I0918 12:16:42.754151    2777 main.go:141] libmachine: Starting QEMU VM...

	I0918 12:16:42.754184    2777 main.go:141] libmachine: executing: /opt/socket_vmnet/bin/socket_vmnet_client /var/run/socket_vmnet qemu-system-aarch64 -M virt -cpu host -drive file=/opt/homebrew/opt/qemu/share/qemu/edk2-aarch64-code.fd,readonly=on,format=raw,if=pflash -display none -accel hvf -m 4000 -smp 2 -boot d -cdrom /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/boot2docker.iso -qmp unix:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/monitor,server,nowait -pidfile /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/qemu.pid -device virtio-net-pci,netdev=net0,mac=86:c6:2b:22:62:a1 -netdev socket,id=net0,fd=3 -daemonize /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/disk.qcow2

	I0918 12:16:42.806234    2777 main.go:141] libmachine: STDOUT: 

	I0918 12:16:42.806252    2777 main.go:141] libmachine: STDERR: 

	I0918 12:16:42.806256    2777 main.go:141] libmachine: Attempt 0

	I0918 12:16:42.806270    2777 main.go:141] libmachine: Searching for 86:c6:2b:22:62:a1 in /var/db/dhcpd_leases ...

	I0918 12:16:42.806336    2777 main.go:141] libmachine: Found 3 entries in /var/db/dhcpd_leases!

	I0918 12:16:42.806358    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.4 HWAddress:72:e4:5a:e6:f2:f3 ID:1,72:e4:5a:e6:f2:f3 Lease:0x6509f2e3}

	I0918 12:16:42.806362    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.3 HWAddress:7a:5a:5:1b:96:65 ID:1,7a:5a:5:1b:96:65 Lease:0x6508a157}

	I0918 12:16:42.806366    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:ce:ae:e8:a:fd:16 ID:1,ce:ae:e8:a:fd:16 Lease:0x6508a134}

	I0918 12:16:44.808559    2777 main.go:141] libmachine: Attempt 1

	I0918 12:16:44.808608    2777 main.go:141] libmachine: Searching for 86:c6:2b:22:62:a1 in /var/db/dhcpd_leases ...

	I0918 12:16:44.808892    2777 main.go:141] libmachine: Found 3 entries in /var/db/dhcpd_leases!

	I0918 12:16:44.808935    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.4 HWAddress:72:e4:5a:e6:f2:f3 ID:1,72:e4:5a:e6:f2:f3 Lease:0x6509f2e3}

	I0918 12:16:44.808963    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.3 HWAddress:7a:5a:5:1b:96:65 ID:1,7a:5a:5:1b:96:65 Lease:0x6508a157}

	I0918 12:16:44.809020    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:ce:ae:e8:a:fd:16 ID:1,ce:ae:e8:a:fd:16 Lease:0x6508a134}

	I0918 12:16:46.811166    2777 main.go:141] libmachine: Attempt 2

	I0918 12:16:46.811181    2777 main.go:141] libmachine: Searching for 86:c6:2b:22:62:a1 in /var/db/dhcpd_leases ...

	I0918 12:16:46.811381    2777 main.go:141] libmachine: Found 3 entries in /var/db/dhcpd_leases!

	I0918 12:16:46.811419    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.4 HWAddress:72:e4:5a:e6:f2:f3 ID:1,72:e4:5a:e6:f2:f3 Lease:0x6509f2e3}

	I0918 12:16:46.811429    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.3 HWAddress:7a:5a:5:1b:96:65 ID:1,7a:5a:5:1b:96:65 Lease:0x6508a157}

	I0918 12:16:46.811439    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:ce:ae:e8:a:fd:16 ID:1,ce:ae:e8:a:fd:16 Lease:0x6508a134}

	I0918 12:16:48.813482    2777 main.go:141] libmachine: Attempt 3

	I0918 12:16:48.813486    2777 main.go:141] libmachine: Searching for 86:c6:2b:22:62:a1 in /var/db/dhcpd_leases ...

	I0918 12:16:48.813533    2777 main.go:141] libmachine: Found 3 entries in /var/db/dhcpd_leases!

	I0918 12:16:48.813539    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.4 HWAddress:72:e4:5a:e6:f2:f3 ID:1,72:e4:5a:e6:f2:f3 Lease:0x6509f2e3}

	I0918 12:16:48.813543    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.3 HWAddress:7a:5a:5:1b:96:65 ID:1,7a:5a:5:1b:96:65 Lease:0x6508a157}

	I0918 12:16:48.813547    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:ce:ae:e8:a:fd:16 ID:1,ce:ae:e8:a:fd:16 Lease:0x6508a134}

	I0918 12:16:50.815553    2777 main.go:141] libmachine: Attempt 4

	I0918 12:16:50.815556    2777 main.go:141] libmachine: Searching for 86:c6:2b:22:62:a1 in /var/db/dhcpd_leases ...

	I0918 12:16:50.815582    2777 main.go:141] libmachine: Found 3 entries in /var/db/dhcpd_leases!

	I0918 12:16:50.815587    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.4 HWAddress:72:e4:5a:e6:f2:f3 ID:1,72:e4:5a:e6:f2:f3 Lease:0x6509f2e3}

	I0918 12:16:50.815591    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.3 HWAddress:7a:5a:5:1b:96:65 ID:1,7a:5a:5:1b:96:65 Lease:0x6508a157}

	I0918 12:16:50.815595    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:ce:ae:e8:a:fd:16 ID:1,ce:ae:e8:a:fd:16 Lease:0x6508a134}

	I0918 12:16:52.816038    2777 main.go:141] libmachine: Attempt 5

	I0918 12:16:52.816050    2777 main.go:141] libmachine: Searching for 86:c6:2b:22:62:a1 in /var/db/dhcpd_leases ...

	I0918 12:16:52.816137    2777 main.go:141] libmachine: Found 3 entries in /var/db/dhcpd_leases!

	I0918 12:16:52.816145    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.4 HWAddress:72:e4:5a:e6:f2:f3 ID:1,72:e4:5a:e6:f2:f3 Lease:0x6509f2e3}

	I0918 12:16:52.816149    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.3 HWAddress:7a:5a:5:1b:96:65 ID:1,7a:5a:5:1b:96:65 Lease:0x6508a157}

	I0918 12:16:52.816153    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.2 HWAddress:ce:ae:e8:a:fd:16 ID:1,ce:ae:e8:a:fd:16 Lease:0x6508a134}

	I0918 12:16:54.818234    2777 main.go:141] libmachine: Attempt 6

	I0918 12:16:54.818250    2777 main.go:141] libmachine: Searching for 86:c6:2b:22:62:a1 in /var/db/dhcpd_leases ...

	I0918 12:16:54.818343    2777 main.go:141] libmachine: Found 4 entries in /var/db/dhcpd_leases!

	I0918 12:16:54.818356    2777 main.go:141] libmachine: dhcp entry: {Name:minikube IPAddress:192.168.105.5 HWAddress:86:c6:2b:22:62:a1 ID:1,86:c6:2b:22:62:a1 Lease:0x6509f3a5}

	I0918 12:16:54.818360    2777 main.go:141] libmachine: Found match: 86:c6:2b:22:62:a1

	I0918 12:16:54.818369    2777 main.go:141] libmachine: IP: 192.168.105.5

	I0918 12:16:54.818374    2777 main.go:141] libmachine: Waiting for VM to start (ssh -p 22 docker@192.168.105.5)...

	I0918 12:16:55.824111    2777 machine.go:88] provisioning docker machine ...

	I0918 12:16:55.824124    2777 buildroot.go:166] provisioning hostname "image-438000"

	I0918 12:16:55.824164    2777 main.go:141] libmachine: Using SSH client type: native

	I0918 12:16:55.824419    2777 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100820760] 0x100822ed0 <nil>  [] 0s} 192.168.105.5 22 <nil> <nil>}

	I0918 12:16:55.824423    2777 main.go:141] libmachine: About to run SSH command:

	sudo hostname image-438000 && echo "image-438000" | sudo tee /etc/hostname

	I0918 12:16:55.904568    2777 main.go:141] libmachine: SSH cmd err, output: <nil>: image-438000

	I0918 12:16:55.904621    2777 main.go:141] libmachine: Using SSH client type: native

	I0918 12:16:55.904868    2777 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100820760] 0x100822ed0 <nil>  [] 0s} 192.168.105.5 22 <nil> <nil>}

	I0918 12:16:55.904875    2777 main.go:141] libmachine: About to run SSH command:

			if ! grep -xq '.*\simage-438000' /etc/hosts; then

				if grep -xq '127.0.1.1\s.*' /etc/hosts; then

					sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 image-438000/g' /etc/hosts;

				else 

					echo '127.0.1.1 image-438000' | sudo tee -a /etc/hosts; 

				fi

			fi

	I0918 12:16:55.979378    2777 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	I0918 12:16:55.979386    2777 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/17263-1251/.minikube CaCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/17263-1251/.minikube}

	I0918 12:16:55.979392    2777 buildroot.go:174] setting up certificates

	I0918 12:16:55.979398    2777 provision.go:83] configureAuth start

	I0918 12:16:55.979401    2777 provision.go:138] copyHostCerts

	I0918 12:16:55.979476    2777 exec_runner.go:144] found /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.pem, removing ...

	I0918 12:16:55.979480    2777 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.pem

	I0918 12:16:55.979597    2777 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.pem (1082 bytes)

	I0918 12:16:55.979799    2777 exec_runner.go:144] found /Users/jenkins/minikube-integration/17263-1251/.minikube/cert.pem, removing ...

	I0918 12:16:55.979801    2777 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17263-1251/.minikube/cert.pem

	I0918 12:16:55.979847    2777 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/cert.pem (1123 bytes)

	I0918 12:16:55.979945    2777 exec_runner.go:144] found /Users/jenkins/minikube-integration/17263-1251/.minikube/key.pem, removing ...

	I0918 12:16:55.979947    2777 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17263-1251/.minikube/key.pem

	I0918 12:16:55.979994    2777 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/17263-1251/.minikube/key.pem (1679 bytes)

	I0918 12:16:55.980088    2777 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem org=jenkins.image-438000 san=[192.168.105.5 192.168.105.5 localhost 127.0.0.1 minikube image-438000]

	I0918 12:16:56.071897    2777 provision.go:172] copyRemoteCerts

	I0918 12:16:56.071921    2777 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker

	I0918 12:16:56.071926    2777 sshutil.go:53] new ssh client: &{IP:192.168.105.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/id_rsa Username:docker}

	I0918 12:16:56.111841    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)

	I0918 12:16:56.119004    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server.pem --> /etc/docker/server.pem (1216 bytes)

	I0918 12:16:56.125794    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)

	I0918 12:16:56.132420    2777 provision.go:86] duration metric: configureAuth took 153.015916ms

	I0918 12:16:56.132425    2777 buildroot.go:189] setting minikube options for container-runtime

	I0918 12:16:56.132525    2777 config.go:182] Loaded profile config "image-438000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 12:16:56.132563    2777 main.go:141] libmachine: Using SSH client type: native

	I0918 12:16:56.132771    2777 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100820760] 0x100822ed0 <nil>  [] 0s} 192.168.105.5 22 <nil> <nil>}

	I0918 12:16:56.132774    2777 main.go:141] libmachine: About to run SSH command:

	df --output=fstype / | tail -n 1

	I0918 12:16:56.207807    2777 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs

	I0918 12:16:56.207812    2777 buildroot.go:70] root file system type: tmpfs

	I0918 12:16:56.207866    2777 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...

	I0918 12:16:56.207931    2777 main.go:141] libmachine: Using SSH client type: native

	I0918 12:16:56.208198    2777 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100820760] 0x100822ed0 <nil>  [] 0s} 192.168.105.5 22 <nil> <nil>}

	I0918 12:16:56.208233    2777 main.go:141] libmachine: About to run SSH command:

	sudo mkdir -p /lib/systemd/system && printf %!s(MISSING) "[Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP \$MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	" | sudo tee /lib/systemd/system/docker.service.new

	I0918 12:16:56.289157    2777 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=qemu2 --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP $MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	I0918 12:16:56.289205    2777 main.go:141] libmachine: Using SSH client type: native

	I0918 12:16:56.289486    2777 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100820760] 0x100822ed0 <nil>  [] 0s} 192.168.105.5 22 <nil> <nil>}

	I0918 12:16:56.289495    2777 main.go:141] libmachine: About to run SSH command:

	sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }

	I0918 12:16:56.617108    2777 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory

	Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

	I0918 12:16:56.617117    2777 machine.go:91] provisioned docker machine in 793.007583ms

	I0918 12:16:56.617121    2777 client.go:171] LocalClient.Create took 14.10451825s

	I0918 12:16:56.617133    2777 start.go:167] duration metric: libmachine.API.Create for "image-438000" took 14.104565083s

	I0918 12:16:56.617140    2777 start.go:300] post-start starting for "image-438000" (driver="qemu2")

	I0918 12:16:56.617144    2777 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]

	I0918 12:16:56.617229    2777 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs

	I0918 12:16:56.617237    2777 sshutil.go:53] new ssh client: &{IP:192.168.105.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/id_rsa Username:docker}

	I0918 12:16:56.655936    2777 ssh_runner.go:195] Run: cat /etc/os-release

	I0918 12:16:56.657321    2777 info.go:137] Remote host: Buildroot 2021.02.12

	I0918 12:16:56.657326    2777 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17263-1251/.minikube/addons for local assets ...

	I0918 12:16:56.657391    2777 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17263-1251/.minikube/files for local assets ...

	I0918 12:16:56.657488    2777 filesync.go:149] local asset: /Users/jenkins/minikube-integration/17263-1251/.minikube/files/etc/ssl/certs/16682.pem -> 16682.pem in /etc/ssl/certs

	I0918 12:16:56.657597    2777 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs

	I0918 12:16:56.660322    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/files/etc/ssl/certs/16682.pem --> /etc/ssl/certs/16682.pem (1708 bytes)

	I0918 12:16:56.667753    2777 start.go:303] post-start completed in 50.602666ms

	I0918 12:16:56.668106    2777 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/config.json ...

	I0918 12:16:56.668260    2777 start.go:128] duration metric: createHost completed in 14.196646625s

	I0918 12:16:56.668294    2777 main.go:141] libmachine: Using SSH client type: native

	I0918 12:16:56.668512    2777 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x100820760] 0x100822ed0 <nil>  [] 0s} 192.168.105.5 22 <nil> <nil>}

	I0918 12:16:56.668515    2777 main.go:141] libmachine: About to run SSH command:

	date +%!s(MISSING).%!N(MISSING)

	I0918 12:16:56.742155    2777 main.go:141] libmachine: SSH cmd err, output: <nil>: 1695064616.543014793

	I0918 12:16:56.742159    2777 fix.go:206] guest clock: 1695064616.543014793

	I0918 12:16:56.742163    2777 fix.go:219] Guest: 2023-09-18 12:16:56.543014793 -0700 PDT Remote: 2023-09-18 12:16:56.668261 -0700 PDT m=+14.298587835 (delta=-125.246207ms)

	I0918 12:16:56.742172    2777 fix.go:190] guest clock delta is within tolerance: -125.246207ms

	I0918 12:16:56.742174    2777 start.go:83] releasing machines lock for "image-438000", held for 14.270593958s

	I0918 12:16:56.742462    2777 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/

	I0918 12:16:56.742462    2777 ssh_runner.go:195] Run: cat /version.json

	I0918 12:16:56.742468    2777 sshutil.go:53] new ssh client: &{IP:192.168.105.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/id_rsa Username:docker}

	I0918 12:16:56.742479    2777 sshutil.go:53] new ssh client: &{IP:192.168.105.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/id_rsa Username:docker}

	I0918 12:16:56.826265    2777 ssh_runner.go:195] Run: systemctl --version

	I0918 12:16:56.828523    2777 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"

	W0918 12:16:56.830780    2777 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found

	I0918 12:16:56.830812    2777 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;

	I0918 12:16:56.836363    2777 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)

	I0918 12:16:56.836368    2777 start.go:469] detecting cgroup driver to use...

	I0918 12:16:56.836435    2777 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock

	" | sudo tee /etc/crictl.yaml"

	I0918 12:16:56.842590    2777 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"

	I0918 12:16:56.845979    2777 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"

	I0918 12:16:56.849182    2777 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...

	I0918 12:16:56.849200    2777 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"

	I0918 12:16:56.852307    2777 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0918 12:16:56.855600    2777 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"

	I0918 12:16:56.859074    2777 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I0918 12:16:56.862191    2777 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"

	I0918 12:16:56.865003    2777 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"

	I0918 12:16:56.868278    2777 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables

	I0918 12:16:56.871576    2777 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

	I0918 12:16:56.874741    2777 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 12:16:56.934308    2777 ssh_runner.go:195] Run: sudo systemctl restart containerd

	I0918 12:16:56.942745    2777 start.go:469] detecting cgroup driver to use...

	I0918 12:16:56.942802    2777 ssh_runner.go:195] Run: sudo systemctl cat docker.service

	I0918 12:16:56.950513    2777 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0918 12:16:56.955181    2777 ssh_runner.go:195] Run: sudo systemctl stop -f containerd

	I0918 12:16:56.962774    2777 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0918 12:16:56.967366    2777 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I0918 12:16:56.972011    2777 ssh_runner.go:195] Run: sudo systemctl stop -f crio

	I0918 12:16:57.008219    2777 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I0918 12:16:57.013543    2777 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock

	" | sudo tee /etc/crictl.yaml"

	I0918 12:16:57.018876    2777 ssh_runner.go:195] Run: which cri-dockerd

	I0918 12:16:57.020097    2777 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d

	I0918 12:16:57.022786    2777 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)

	I0918 12:16:57.028033    2777 ssh_runner.go:195] Run: sudo systemctl unmask docker.service

	I0918 12:16:57.094029    2777 ssh_runner.go:195] Run: sudo systemctl enable docker.socket

	I0918 12:16:57.154562    2777 docker.go:535] configuring docker to use "cgroupfs" as cgroup driver...

	I0918 12:16:57.154579    2777 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)

	I0918 12:16:57.159814    2777 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 12:16:57.216793    2777 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0918 12:16:58.383612    2777 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.166818792s)

	I0918 12:16:58.383673    2777 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I0918 12:16:58.444379    2777 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket

	I0918 12:16:58.506169    2777 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I0918 12:16:58.566311    2777 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 12:16:58.627692    2777 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket

	I0918 12:16:58.635469    2777 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 12:16:58.699681    2777 ssh_runner.go:195] Run: sudo systemctl restart cri-docker

	I0918 12:16:58.723197    2777 start.go:516] Will wait 60s for socket path /var/run/cri-dockerd.sock

	I0918 12:16:58.723271    2777 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock

	I0918 12:16:58.725503    2777 start.go:537] Will wait 60s for crictl version

	I0918 12:16:58.725540    2777 ssh_runner.go:195] Run: which crictl

	I0918 12:16:58.726909    2777 ssh_runner.go:195] Run: sudo /usr/bin/crictl version

	I0918 12:16:58.742110    2777 start.go:553] Version:  0.1.0

	RuntimeName:  docker

	RuntimeVersion:  24.0.6

	RuntimeApiVersion:  v1alpha2

	I0918 12:16:58.742173    2777 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0918 12:16:58.751883    2777 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I0918 12:16:58.763129    2777 out.go:204] * Preparing Kubernetes v1.28.2 on Docker 24.0.6 ...

	I0918 12:16:58.763202    2777 ssh_runner.go:195] Run: grep 192.168.105.1	host.minikube.internal$ /etc/hosts

	I0918 12:16:58.764554    2777 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.105.1	host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0918 12:16:58.768260    2777 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I0918 12:16:58.768301    2777 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0918 12:16:58.773415    2777 docker.go:636] Got preloaded images: 

	I0918 12:16:58.773419    2777 docker.go:642] registry.k8s.io/kube-apiserver:v1.28.2 wasn't preloaded

	I0918 12:16:58.773453    2777 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0918 12:16:58.776471    2777 ssh_runner.go:195] Run: which lz4

	I0918 12:16:58.778035    2777 ssh_runner.go:195] Run: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4

	I0918 12:16:58.779316    2777 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%!s(MISSING) %!y(MISSING)" /preloaded.tar.lz4: Process exited with status 1

	stdout:

	stderr:

	stat: cannot statx '/preloaded.tar.lz4': No such file or directory

	I0918 12:16:58.779325    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-arm64.tar.lz4 --> /preloaded.tar.lz4 (356993689 bytes)

	I0918 12:17:00.108721    2777 docker.go:600] Took 1.330732 seconds to copy over tarball

	I0918 12:17:00.108770    2777 ssh_runner.go:195] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4

	I0918 12:17:01.122008    2777 ssh_runner.go:235] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (1.013224833s)

	I0918 12:17:01.122023    2777 ssh_runner.go:146] rm: /preloaded.tar.lz4

	I0918 12:17:01.137485    2777 ssh_runner.go:195] Run: sudo cat /var/lib/docker/image/overlay2/repositories.json

	I0918 12:17:01.140664    2777 ssh_runner.go:362] scp memory --> /var/lib/docker/image/overlay2/repositories.json (2629 bytes)

	I0918 12:17:01.145934    2777 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0918 12:17:01.211417    2777 ssh_runner.go:195] Run: sudo systemctl restart docker

	I0918 12:17:02.755354    2777 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.543939166s)

	I0918 12:17:02.755437    2777 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I0918 12:17:02.761716    2777 docker.go:636] Got preloaded images: -- stdout --

	registry.k8s.io/kube-apiserver:v1.28.2

	registry.k8s.io/kube-proxy:v1.28.2

	registry.k8s.io/kube-controller-manager:v1.28.2

	registry.k8s.io/kube-scheduler:v1.28.2

	registry.k8s.io/etcd:3.5.9-0

	registry.k8s.io/coredns/coredns:v1.10.1

	registry.k8s.io/pause:3.9

	gcr.io/k8s-minikube/storage-provisioner:v5

	-- /stdout --

	I0918 12:17:02.761725    2777 cache_images.go:84] Images are preloaded, skipping loading

	I0918 12:17:02.761800    2777 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}

	I0918 12:17:02.772524    2777 cni.go:84] Creating CNI manager for ""

	I0918 12:17:02.772534    2777 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 12:17:02.772541    2777 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16

	I0918 12:17:02.772549    2777 kubeadm.go:176] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.105.5 APIServerPort:8443 KubernetesVersion:v1.28.2 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:image-438000 NodeName:image-438000 DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.105.5"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.105.5 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kube

rnetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}

	I0918 12:17:02.772619    2777 kubeadm.go:181] kubeadm config:

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: InitConfiguration

	localAPIEndpoint:

	  advertiseAddress: 192.168.105.5

	  bindPort: 8443

	bootstrapTokens:

	  - groups:

	      - system:bootstrappers:kubeadm:default-node-token

	    ttl: 24h0m0s

	    usages:

	      - signing

	      - authentication

	nodeRegistration:

	  criSocket: unix:///var/run/cri-dockerd.sock

	  name: "image-438000"

	  kubeletExtraArgs:

	    node-ip: 192.168.105.5

	  taints: []

	---

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: ClusterConfiguration

	apiServer:

	  certSANs: ["127.0.0.1", "localhost", "192.168.105.5"]

	  extraArgs:

	    enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"

	controllerManager:

	  extraArgs:

	    allocate-node-cidrs: "true"

	    leader-elect: "false"

	scheduler:

	  extraArgs:

	    leader-elect: "false"

	certificatesDir: /var/lib/minikube/certs

	clusterName: mk

	controlPlaneEndpoint: control-plane.minikube.internal:8443

	etcd:

	  local:

	    dataDir: /var/lib/minikube/etcd

	    extraArgs:

	      proxy-refresh-interval: "70000"

	kubernetesVersion: v1.28.2

	networking:

	  dnsDomain: cluster.local

	  podSubnet: "10.244.0.0/16"

	  serviceSubnet: 10.96.0.0/12

	---

	apiVersion: kubelet.config.k8s.io/v1beta1

	kind: KubeletConfiguration

	authentication:

	  x509:

	    clientCAFile: /var/lib/minikube/certs/ca.crt

	cgroupDriver: cgroupfs

	hairpinMode: hairpin-veth

	runtimeRequestTimeout: 15m

	clusterDomain: "cluster.local"

	# disable disk resource management by default

	imageGCHighThresholdPercent: 100

	evictionHard:

	  nodefs.available: "0%!"(MISSING)

	  nodefs.inodesFree: "0%!"(MISSING)

	  imagefs.available: "0%!"(MISSING)

	failSwapOn: false

	staticPodPath: /etc/kubernetes/manifests

	---

	apiVersion: kubeproxy.config.k8s.io/v1alpha1

	kind: KubeProxyConfiguration

	clusterCIDR: "10.244.0.0/16"

	metricsBindAddress: 0.0.0.0:10249

	conntrack:

	  maxPerCore: 0

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"

	  tcpEstablishedTimeout: 0s

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"

	  tcpCloseWaitTimeout: 0s

	I0918 12:17:02.772656    2777 kubeadm.go:976] kubelet [Unit]

	Wants=docker.socket

	[Service]

	ExecStart=

	ExecStart=/var/lib/minikube/binaries/v1.28.2/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --hostname-override=image-438000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.105.5

	[Install]

	 config:

	{KubernetesVersion:v1.28.2 ClusterName:image-438000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}

	I0918 12:17:02.772709    2777 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.2

	I0918 12:17:02.776155    2777 binaries.go:44] Found k8s binaries, skipping transfer

	I0918 12:17:02.776179    2777 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube

	I0918 12:17:02.779405    2777 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (374 bytes)

	I0918 12:17:02.784646    2777 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)

	I0918 12:17:02.789904    2777 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2098 bytes)

	I0918 12:17:02.794763    2777 ssh_runner.go:195] Run: grep 192.168.105.5	control-plane.minikube.internal$ /etc/hosts

	I0918 12:17:02.796052    2777 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.105.5	control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0918 12:17:02.800084    2777 certs.go:56] Setting up /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000 for IP: 192.168.105.5

	I0918 12:17:02.800091    2777 certs.go:190] acquiring lock for shared ca certs: {Name:mkfac81ee65979b8c4f5ece6243c3a0190531689 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:17:02.800224    2777 certs.go:199] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key

	I0918 12:17:02.800268    2777 certs.go:199] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key

	I0918 12:17:02.800291    2777 certs.go:319] generating minikube-user signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/client.key

	I0918 12:17:02.800298    2777 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/client.crt with IP's: []

	I0918 12:17:02.903595    2777 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/client.crt ...

	I0918 12:17:02.903601    2777 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/client.crt: {Name:mk142f03069fc218854f90a3783cbf80eaa79196 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:17:02.903857    2777 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/client.key ...

	I0918 12:17:02.903859    2777 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/client.key: {Name:mk03e2cd3e41a4b62e7d8da6e4b98e1208fb3701 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:17:02.903962    2777 certs.go:319] generating minikube signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.key.e69b33ca

	I0918 12:17:02.903970    2777 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.crt.e69b33ca with IP's: [192.168.105.5 10.96.0.1 127.0.0.1 10.0.0.1]

	I0918 12:17:02.940725    2777 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.crt.e69b33ca ...

	I0918 12:17:02.940727    2777 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.crt.e69b33ca: {Name:mk9ca6a9c22b344015776609d2e05ea1b0cf4938 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:17:02.940873    2777 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.key.e69b33ca ...

	I0918 12:17:02.940876    2777 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.key.e69b33ca: {Name:mkdf5dd48cc36d16afa3dfc0dc716a932a01c5ee Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:17:02.940981    2777 certs.go:337] copying /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.crt.e69b33ca -> /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.crt

	I0918 12:17:02.941067    2777 certs.go:341] copying /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.key.e69b33ca -> /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.key

	I0918 12:17:02.941148    2777 certs.go:319] generating aggregator signed cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/proxy-client.key

	I0918 12:17:02.941155    2777 crypto.go:68] Generating cert /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/proxy-client.crt with IP's: []

	I0918 12:17:03.059910    2777 crypto.go:156] Writing cert to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/proxy-client.crt ...

	I0918 12:17:03.059915    2777 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/proxy-client.crt: {Name:mkfa338318b5b18faada0e424b1758376f761ce2 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:17:03.060135    2777 crypto.go:164] Writing key to /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/proxy-client.key ...

	I0918 12:17:03.060138    2777 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/proxy-client.key: {Name:mkbbfc94decd1624e5f3c29601dd6e201020225a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:17:03.060372    2777 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/1668.pem (1338 bytes)

	W0918 12:17:03.060399    2777 certs.go:433] ignoring /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/1668_empty.pem, impossibly tiny 0 bytes

	I0918 12:17:03.060404    2777 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca-key.pem (1679 bytes)

	I0918 12:17:03.060422    2777 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/ca.pem (1082 bytes)

	I0918 12:17:03.060441    2777 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/cert.pem (1123 bytes)

	I0918 12:17:03.060457    2777 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/certs/key.pem (1679 bytes)

	I0918 12:17:03.060492    2777 certs.go:437] found cert: /Users/jenkins/minikube-integration/17263-1251/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/17263-1251/.minikube/files/etc/ssl/certs/16682.pem (1708 bytes)

	I0918 12:17:03.060784    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)

	I0918 12:17:03.068420    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)

	I0918 12:17:03.075877    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)

	I0918 12:17:03.083111    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/profiles/image-438000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)

	I0918 12:17:03.090222    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)

	I0918 12:17:03.096943    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)

	I0918 12:17:03.104062    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)

	I0918 12:17:03.111655    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)

	I0918 12:17:03.119147    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)

	I0918 12:17:03.126418    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/certs/1668.pem --> /usr/share/ca-certificates/1668.pem (1338 bytes)

	I0918 12:17:03.133107    2777 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17263-1251/.minikube/files/etc/ssl/certs/16682.pem --> /usr/share/ca-certificates/16682.pem (1708 bytes)

	I0918 12:17:03.139969    2777 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)

	I0918 12:17:03.145126    2777 ssh_runner.go:195] Run: openssl version

	I0918 12:17:03.147130    2777 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"

	I0918 12:17:03.150079    2777 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem

	I0918 12:17:03.151576    2777 certs.go:480] hashing: -rw-r--r-- 1 root root 1111 Sep 18 18:52 /usr/share/ca-certificates/minikubeCA.pem

	I0918 12:17:03.151596    2777 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem

	I0918 12:17:03.153593    2777 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"

	I0918 12:17:03.156557    2777 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1668.pem && ln -fs /usr/share/ca-certificates/1668.pem /etc/ssl/certs/1668.pem"

	I0918 12:17:03.159880    2777 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1668.pem

	I0918 12:17:03.161417    2777 certs.go:480] hashing: -rw-r--r-- 1 root root 1338 Sep 18 19:13 /usr/share/ca-certificates/1668.pem

	I0918 12:17:03.161436    2777 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1668.pem

	I0918 12:17:03.163317    2777 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1668.pem /etc/ssl/certs/51391683.0"

	I0918 12:17:03.166404    2777 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/16682.pem && ln -fs /usr/share/ca-certificates/16682.pem /etc/ssl/certs/16682.pem"

	I0918 12:17:03.169339    2777 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/16682.pem

	I0918 12:17:03.170822    2777 certs.go:480] hashing: -rw-r--r-- 1 root root 1708 Sep 18 19:13 /usr/share/ca-certificates/16682.pem

	I0918 12:17:03.170841    2777 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/16682.pem

	I0918 12:17:03.172679    2777 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/16682.pem /etc/ssl/certs/3ec20f2e.0"

	I0918 12:17:03.176035    2777 ssh_runner.go:195] Run: ls /var/lib/minikube/certs/etcd

	I0918 12:17:03.177449    2777 certs.go:353] certs directory doesn't exist, likely first start: ls /var/lib/minikube/certs/etcd: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/var/lib/minikube/certs/etcd': No such file or directory

	I0918 12:17:03.177475    2777 kubeadm.go:404] StartCluster: {Name:image-438000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17250/minikube-v1.31.0-1694798110-17250-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1694798187-17250@sha256:8d9a070cda8e1b1082ed355bde1aaf66fbf63d64fa6e9f553f449efc74157fe3 Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:qemu2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1

.28.2 ClusterName:image-438000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.105.5 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network:socket_vmnet Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOpt

imizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I0918 12:17:03.177534    2777 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}

	I0918 12:17:03.182896    2777 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd

	I0918 12:17:03.186005    2777 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml

	I0918 12:17:03.188694    2777 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf

	I0918 12:17:03.192002    2777 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory

	I0918 12:17:03.192013    2777 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem"

	I0918 12:17:03.215594    2777 kubeadm.go:322] [init] Using Kubernetes version: v1.28.2

	I0918 12:17:03.215615    2777 kubeadm.go:322] [preflight] Running pre-flight checks

	I0918 12:17:03.274022    2777 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster

	I0918 12:17:03.274080    2777 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection

	I0918 12:17:03.274123    2777 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	I0918 12:17:03.333691    2777 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"

	I0918 12:17:03.342894    2777 out.go:204]   - Generating certificates and keys ...

	I0918 12:17:03.342937    2777 kubeadm.go:322] [certs] Using existing ca certificate authority

	I0918 12:17:03.342968    2777 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk

	I0918 12:17:03.414666    2777 kubeadm.go:322] [certs] Generating "apiserver-kubelet-client" certificate and key

	I0918 12:17:03.548242    2777 kubeadm.go:322] [certs] Generating "front-proxy-ca" certificate and key

	I0918 12:17:03.685234    2777 kubeadm.go:322] [certs] Generating "front-proxy-client" certificate and key

	I0918 12:17:03.835433    2777 kubeadm.go:322] [certs] Generating "etcd/ca" certificate and key

	I0918 12:17:03.885433    2777 kubeadm.go:322] [certs] Generating "etcd/server" certificate and key

	I0918 12:17:03.885490    2777 kubeadm.go:322] [certs] etcd/server serving cert is signed for DNS names [image-438000 localhost] and IPs [192.168.105.5 127.0.0.1 ::1]

	I0918 12:17:03.941032    2777 kubeadm.go:322] [certs] Generating "etcd/peer" certificate and key

	I0918 12:17:03.941102    2777 kubeadm.go:322] [certs] etcd/peer serving cert is signed for DNS names [image-438000 localhost] and IPs [192.168.105.5 127.0.0.1 ::1]

	I0918 12:17:04.068600    2777 kubeadm.go:322] [certs] Generating "etcd/healthcheck-client" certificate and key

	I0918 12:17:04.107092    2777 kubeadm.go:322] [certs] Generating "apiserver-etcd-client" certificate and key

	I0918 12:17:04.235762    2777 kubeadm.go:322] [certs] Generating "sa" key and public key

	I0918 12:17:04.235795    2777 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	I0918 12:17:04.326236    2777 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file

	I0918 12:17:04.415832    2777 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file

	I0918 12:17:04.503126    2777 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file

	I0918 12:17:04.679114    2777 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file

	I0918 12:17:04.679333    2777 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	I0918 12:17:04.681195    2777 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"

	I0918 12:17:04.689409    2777 out.go:204]   - Booting up control plane ...

	I0918 12:17:04.689532    2777 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"

	I0918 12:17:04.689580    2777 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"

	I0918 12:17:04.689606    2777 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"

	I0918 12:17:04.689658    2777 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	I0918 12:17:04.689694    2777 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	I0918 12:17:04.689711    2777 kubeadm.go:322] [kubelet-start] Starting the kubelet

	I0918 12:17:04.763428    2777 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	I0918 12:17:08.266972    2777 kubeadm.go:322] [apiclient] All control plane components are healthy after 3.503804 seconds

	I0918 12:17:08.267039    2777 kubeadm.go:322] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace

	I0918 12:17:08.272091    2777 kubeadm.go:322] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster

	I0918 12:17:08.783138    2777 kubeadm.go:322] [upload-certs] Skipping phase. Please see --upload-certs

	I0918 12:17:08.783290    2777 kubeadm.go:322] [mark-control-plane] Marking the node image-438000 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]

	I0918 12:17:09.288325    2777 kubeadm.go:322] [bootstrap-token] Using token: d56bv3.xfz54degysdiooht

	I0918 12:17:09.294708    2777 out.go:204]   - Configuring RBAC rules ...

	I0918 12:17:09.294768    2777 kubeadm.go:322] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles

	I0918 12:17:09.296238    2777 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes

	I0918 12:17:09.303786    2777 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials

	I0918 12:17:09.304910    2777 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token

	I0918 12:17:09.306108    2777 kubeadm.go:322] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster

	I0918 12:17:09.307244    2777 kubeadm.go:322] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace

	I0918 12:17:09.311601    2777 kubeadm.go:322] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key

	I0918 12:17:09.470842    2777 kubeadm.go:322] [addons] Applied essential addon: CoreDNS

	I0918 12:17:09.698860    2777 kubeadm.go:322] [addons] Applied essential addon: kube-proxy

	I0918 12:17:09.699255    2777 kubeadm.go:322] 

	I0918 12:17:09.699279    2777 kubeadm.go:322] Your Kubernetes control-plane has initialized successfully!

	I0918 12:17:09.699281    2777 kubeadm.go:322] 

	I0918 12:17:09.699316    2777 kubeadm.go:322] To start using your cluster, you need to run the following as a regular user:

	I0918 12:17:09.699317    2777 kubeadm.go:322] 

	I0918 12:17:09.699332    2777 kubeadm.go:322]   mkdir -p $HOME/.kube

	I0918 12:17:09.699366    2777 kubeadm.go:322]   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

	I0918 12:17:09.699391    2777 kubeadm.go:322]   sudo chown $(id -u):$(id -g) $HOME/.kube/config

	I0918 12:17:09.699393    2777 kubeadm.go:322] 

	I0918 12:17:09.699417    2777 kubeadm.go:322] Alternatively, if you are the root user, you can run:

	I0918 12:17:09.699418    2777 kubeadm.go:322] 

	I0918 12:17:09.699438    2777 kubeadm.go:322]   export KUBECONFIG=/etc/kubernetes/admin.conf

	I0918 12:17:09.699439    2777 kubeadm.go:322] 

	I0918 12:17:09.699463    2777 kubeadm.go:322] You should now deploy a pod network to the cluster.

	I0918 12:17:09.699499    2777 kubeadm.go:322] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

	I0918 12:17:09.699529    2777 kubeadm.go:322]   https://kubernetes.io/docs/concepts/cluster-administration/addons/

	I0918 12:17:09.699531    2777 kubeadm.go:322] 

	I0918 12:17:09.699569    2777 kubeadm.go:322] You can now join any number of control-plane nodes by copying certificate authorities

	I0918 12:17:09.699599    2777 kubeadm.go:322] and service account keys on each node and then running the following as root:

	I0918 12:17:09.699601    2777 kubeadm.go:322] 

	I0918 12:17:09.699640    2777 kubeadm.go:322]   kubeadm join control-plane.minikube.internal:8443 --token d56bv3.xfz54degysdiooht \

	I0918 12:17:09.699693    2777 kubeadm.go:322] 	--discovery-token-ca-cert-hash sha256:ba7ba5242719fff40b98ade8a053fc0c6dded3185e28e2742ca7444c7c25a7a5 \

	I0918 12:17:09.699703    2777 kubeadm.go:322] 	--control-plane 

	I0918 12:17:09.699704    2777 kubeadm.go:322] 

	I0918 12:17:09.699744    2777 kubeadm.go:322] Then you can join any number of worker nodes by running the following on each as root:

	I0918 12:17:09.699746    2777 kubeadm.go:322] 

	I0918 12:17:09.699788    2777 kubeadm.go:322] kubeadm join control-plane.minikube.internal:8443 --token d56bv3.xfz54degysdiooht \

	I0918 12:17:09.699835    2777 kubeadm.go:322] 	--discovery-token-ca-cert-hash sha256:ba7ba5242719fff40b98ade8a053fc0c6dded3185e28e2742ca7444c7c25a7a5 

	I0918 12:17:09.699966    2777 kubeadm.go:322] 	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	I0918 12:17:09.699972    2777 cni.go:84] Creating CNI manager for ""

	I0918 12:17:09.699980    2777 cni.go:158] "qemu2" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I0918 12:17:09.705002    2777 out.go:177] * Configuring bridge CNI (Container Networking Interface) ...

	I0918 12:17:09.708099    2777 ssh_runner.go:195] Run: sudo mkdir -p /etc/cni/net.d

	I0918 12:17:09.711431    2777 ssh_runner.go:362] scp memory --> /etc/cni/net.d/1-k8s.conflist (457 bytes)

	I0918 12:17:09.716259    2777 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"

	I0918 12:17:09.716297    2777 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl label nodes minikube.k8s.io/version=v1.31.2 minikube.k8s.io/commit=ae2adeb20cc3a4b5f1decc7c8f53736ec04c4a36 minikube.k8s.io/name=image-438000 minikube.k8s.io/updated_at=2023_09_18T12_17_09_0700 minikube.k8s.io/primary=true --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 12:17:09.716298    2777 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.2/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig

	I0918 12:17:09.791675    2777 kubeadm.go:1081] duration metric: took 75.408583ms to wait for elevateKubeSystemPrivileges.

	I0918 12:17:09.791684    2777 ops.go:34] apiserver oom_adj: -16

	I0918 12:17:09.791688    2777 kubeadm.go:406] StartCluster complete in 6.614282375s

	I0918 12:17:09.791698    2777 settings.go:142] acquiring lock: {Name:mke420f28dda4f7a752738b3e6d571dc4216779e Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:17:09.791777    2777 settings.go:150] Updating kubeconfig:  /Users/jenkins/minikube-integration/17263-1251/kubeconfig

	I0918 12:17:09.792145    2777 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17263-1251/kubeconfig: {Name:mk07020c5b974cf07ca0cda25f72a521eb245fa1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0918 12:17:09.792317    2777 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"

	I0918 12:17:09.792362    2777 addons.go:499] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false volumesnapshots:false]

	I0918 12:17:09.792396    2777 addons.go:69] Setting storage-provisioner=true in profile "image-438000"

	I0918 12:17:09.792400    2777 addons.go:69] Setting default-storageclass=true in profile "image-438000"

	I0918 12:17:09.792401    2777 addons.go:231] Setting addon storage-provisioner=true in "image-438000"

	I0918 12:17:09.792406    2777 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "image-438000"

	I0918 12:17:09.792420    2777 host.go:66] Checking if "image-438000" exists ...

	I0918 12:17:09.792576    2777 config.go:182] Loaded profile config "image-438000": Driver=qemu2, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I0918 12:17:09.798018    2777 out.go:177]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5

	I0918 12:17:09.802047    2777 addons.go:423] installing /etc/kubernetes/addons/storage-provisioner.yaml

	I0918 12:17:09.802052    2777 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)

	I0918 12:17:09.802060    2777 sshutil.go:53] new ssh client: &{IP:192.168.105.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/id_rsa Username:docker}

	I0918 12:17:09.806025    2777 addons.go:231] Setting addon default-storageclass=true in "image-438000"

	I0918 12:17:09.806039    2777 host.go:66] Checking if "image-438000" exists ...

	I0918 12:17:09.806752    2777 addons.go:423] installing /etc/kubernetes/addons/storageclass.yaml

	I0918 12:17:09.806756    2777 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)

	I0918 12:17:09.806761    2777 sshutil.go:53] new ssh client: &{IP:192.168.105.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17263-1251/.minikube/machines/image-438000/id_rsa Username:docker}

	I0918 12:17:09.809693    2777 kapi.go:248] "coredns" deployment in "kube-system" namespace and "image-438000" context rescaled to 1 replicas

	I0918 12:17:09.809706    2777 start.go:223] Will wait 6m0s for node &{Name: IP:192.168.105.5 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I0918 12:17:09.816959    2777 out.go:177] * Verifying Kubernetes components...

	I0918 12:17:09.820034    2777 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I0918 12:17:09.847513    2777 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml

	I0918 12:17:09.853866    2777 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml

	I0918 12:17:09.855861    2777 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^        forward . \/etc\/resolv.conf.*/i \        hosts {\n           192.168.105.1 host.minikube.internal\n           fallthrough\n        }' -e '/^        errors *$/i \        log' | sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"

	I0918 12:17:09.856038    2777 api_server.go:52] waiting for apiserver process to appear ...

	I0918 12:17:09.856062    2777 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I0918 12:17:10.358533    2777 start.go:917] {"host.minikube.internal": 192.168.105.1} host record injected into CoreDNS's ConfigMap

	I0918 12:17:10.358541    2777 api_server.go:72] duration metric: took 548.829959ms to wait for apiserver process to appear ...

	I0918 12:17:10.364332    2777 out.go:177] * Enabled addons: storage-provisioner, default-storageclass

	I0918 12:17:10.358545    2777 api_server.go:88] waiting for apiserver healthz status ...

	I0918 12:17:10.368359    2777 api_server.go:253] Checking apiserver healthz at https://192.168.105.5:8443/healthz ...

	I0918 12:17:10.368381    2777 addons.go:502] enable addons completed in 576.038916ms: enabled=[storage-provisioner default-storageclass]

	I0918 12:17:10.371793    2777 api_server.go:279] https://192.168.105.5:8443/healthz returned 200:

	ok

	I0918 12:17:10.372493    2777 api_server.go:141] control plane version: v1.28.2

	I0918 12:17:10.372496    2777 api_server.go:131] duration metric: took 4.145959ms to wait for apiserver health ...

	I0918 12:17:10.372499    2777 system_pods.go:43] waiting for kube-system pods to appear ...

	I0918 12:17:10.374999    2777 system_pods.go:59] 5 kube-system pods found

	I0918 12:17:10.375005    2777 system_pods.go:61] "etcd-image-438000" [abf4c201-4256-40e1-92aa-fcc7f7739cc8] Pending

	I0918 12:17:10.375007    2777 system_pods.go:61] "kube-apiserver-image-438000" [909da55d-9501-4bb0-ad21-694c92c9cb2e] Pending

	I0918 12:17:10.375009    2777 system_pods.go:61] "kube-controller-manager-image-438000" [dcad0e8f-797f-4cb1-971b-a710af56fcd7] Pending

	I0918 12:17:10.375011    2777 system_pods.go:61] "kube-scheduler-image-438000" [4ba6e54b-00d8-494e-afbf-72244630f478] Pending

	I0918 12:17:10.375012    2777 system_pods.go:61] "storage-provisioner" [5f3e6e9a-7709-4e2f-bc4a-954becbeb730] Pending

	I0918 12:17:10.375013    2777 system_pods.go:74] duration metric: took 2.512667ms to wait for pod list to return data ...

	I0918 12:17:10.375016    2777 kubeadm.go:581] duration metric: took 565.30725ms to wait for : map[apiserver:true system_pods:true] ...

	I0918 12:17:10.375020    2777 node_conditions.go:102] verifying NodePressure condition ...

	I0918 12:17:10.376371    2777 node_conditions.go:122] node storage ephemeral capacity is 17784760Ki

	I0918 12:17:10.376376    2777 node_conditions.go:123] node cpu capacity is 2

	I0918 12:17:10.376381    2777 node_conditions.go:105] duration metric: took 1.358958ms to run NodePressure ...

	I0918 12:17:10.376385    2777 start.go:228] waiting for startup goroutines ...

	I0918 12:17:10.376387    2777 start.go:233] waiting for cluster config update ...

	I0918 12:17:10.376391    2777 start.go:242] writing updated cluster config ...

	I0918 12:17:10.376620    2777 ssh_runner.go:195] Run: rm -f paused

	I0918 12:17:10.404035    2777 start.go:600] kubectl: 1.27.2, cluster: 1.28.2 (minor skew: 1)

	I0918 12:17:10.408298    2777 out.go:177] * Done! kubectl is now configured to use "image-438000" cluster and "default" namespace by default

	* 

	* ==> Docker <==

	* -- Journal begins at Mon 2023-09-18 19:16:53 UTC, ends at Mon 2023-09-18 19:17:11 UTC. --

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.506294214Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.509459547Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.509587339Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.509621506Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.509643506Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.513056839Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.513097631Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.513109172Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.513115547Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.572777547Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.572833214Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.572850464Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 19:17:05 image-438000 dockerd[1107]: time="2023-09-18T19:17:05.572862422Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:17:11 image-438000 dockerd[1101]: time="2023-09-18T19:17:11.278169758Z" level=info msg="Layer sha256:5e5d01bb2a8d3e34816f24ff1a055b5d084e5a5a1919cd77684120916d61c3eb cleaned up"

	Sep 18 19:17:11 image-438000 dockerd[1101]: time="2023-09-18T19:17:11.406799175Z" level=info msg="Layer sha256:5e5d01bb2a8d3e34816f24ff1a055b5d084e5a5a1919cd77684120916d61c3eb cleaned up"

	Sep 18 19:17:11 image-438000 dockerd[1101]: time="2023-09-18T19:17:11.422048925Z" level=info msg="Layer sha256:5e5d01bb2a8d3e34816f24ff1a055b5d084e5a5a1919cd77684120916d61c3eb cleaned up"

	Sep 18 19:17:11 image-438000 dockerd[1107]: time="2023-09-18T19:17:11.462978842Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Sep 18 19:17:11 image-438000 dockerd[1107]: time="2023-09-18T19:17:11.463012925Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:17:11 image-438000 dockerd[1107]: time="2023-09-18T19:17:11.463325800Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Sep 18 19:17:11 image-438000 dockerd[1107]: time="2023-09-18T19:17:11.463372467Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Sep 18 19:17:11 image-438000 dockerd[1101]: time="2023-09-18T19:17:11.597920050Z" level=info msg="ignoring event" container=a6a0272fed0e37de65910732f43e22c40a7630cf08867a9129a1237acf0816af module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Sep 18 19:17:11 image-438000 dockerd[1107]: time="2023-09-18T19:17:11.597978217Z" level=info msg="shim disconnected" id=a6a0272fed0e37de65910732f43e22c40a7630cf08867a9129a1237acf0816af namespace=moby

	Sep 18 19:17:11 image-438000 dockerd[1107]: time="2023-09-18T19:17:11.598004092Z" level=warning msg="cleaning up after shim disconnected" id=a6a0272fed0e37de65910732f43e22c40a7630cf08867a9129a1237acf0816af namespace=moby

	Sep 18 19:17:11 image-438000 dockerd[1107]: time="2023-09-18T19:17:11.598008009Z" level=info msg="cleaning up dead shim" namespace=moby

	Sep 18 19:17:11 image-438000 dockerd[1107]: time="2023-09-18T19:17:11.605344509Z" level=warning msg="cleanup warnings time=\"2023-09-18T19:17:11Z\" level=warning msg=\"failed to remove runc container\" error=\"runc did not terminate successfully: exit status 255: \" runtime=io.containerd.runc.v2\n" namespace=moby

	* 

	* ==> container status <==

	* CONTAINER           IMAGE               CREATED             STATE               NAME                      ATTEMPT             POD ID

	ab71429bd8195       9cdd6470f48c8       6 seconds ago       Running             etcd                      0                   900bb9d69314e

	4325f96e9eab0       64fc40cee3716       6 seconds ago       Running             kube-scheduler            0                   bbdd1cef0bf46

	996710bd53049       89d57b83c1786       6 seconds ago       Running             kube-controller-manager   0                   9f51e4e0f5ba6

	f498fa724b4d9       30bb499447fe1       6 seconds ago       Running             kube-apiserver            0                   785ef9b260df8

	* 

	* ==> describe nodes <==

	* Name:               image-438000

	Roles:              control-plane

	Labels:             beta.kubernetes.io/arch=arm64

	                    beta.kubernetes.io/os=linux

	                    kubernetes.io/arch=arm64

	                    kubernetes.io/hostname=image-438000

	                    kubernetes.io/os=linux

	                    minikube.k8s.io/commit=ae2adeb20cc3a4b5f1decc7c8f53736ec04c4a36

	                    minikube.k8s.io/name=image-438000

	                    minikube.k8s.io/primary=true

	                    minikube.k8s.io/updated_at=2023_09_18T12_17_09_0700

	                    minikube.k8s.io/version=v1.31.2

	                    node-role.kubernetes.io/control-plane=

	                    node.kubernetes.io/exclude-from-external-load-balancers=

	Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock

	                    volumes.kubernetes.io/controller-managed-attach-detach: true

	CreationTimestamp:  Mon, 18 Sep 2023 19:17:06 +0000

	Taints:             node.kubernetes.io/not-ready:NoSchedule

	Unschedulable:      false

	Lease:

	  HolderIdentity:  image-438000

	  AcquireTime:     <unset>

	  RenewTime:       Mon, 18 Sep 2023 19:17:09 +0000

	Conditions:

	  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message

	  ----             ------  -----------------                 ------------------                ------                       -------

	  MemoryPressure   False   Mon, 18 Sep 2023 19:17:09 +0000   Mon, 18 Sep 2023 19:17:06 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available

	  DiskPressure     False   Mon, 18 Sep 2023 19:17:09 +0000   Mon, 18 Sep 2023 19:17:06 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure

	  PIDPressure      False   Mon, 18 Sep 2023 19:17:09 +0000   Mon, 18 Sep 2023 19:17:06 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available

	  Ready            False   Mon, 18 Sep 2023 19:17:09 +0000   Mon, 18 Sep 2023 19:17:06 +0000   KubeletNotReady              container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

	Addresses:

	  InternalIP:  192.168.105.5

	  Hostname:    image-438000

	Capacity:

	  cpu:                2

	  ephemeral-storage:  17784760Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3905012Ki

	  pods:               110

	Allocatable:

	  cpu:                2

	  ephemeral-storage:  17784760Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             3905012Ki

	  pods:               110

	System Info:

	  Machine ID:                 588ea76bfe9647f2a635df94e1a36725

	  System UUID:                588ea76bfe9647f2a635df94e1a36725

	  Boot ID:                    f44f265d-c40b-4b99-b9b7-6cafcb8d10ca

	  Kernel Version:             5.10.57

	  OS Image:                   Buildroot 2021.02.12

	  Operating System:           linux

	  Architecture:               arm64

	  Container Runtime Version:  docker://24.0.6

	  Kubelet Version:            v1.28.2

	  Kube-Proxy Version:         v1.28.2

	Non-terminated Pods:          (4 in total)

	  Namespace                   Name                                    CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age

	  ---------                   ----                                    ------------  ----------  ---------------  -------------  ---

	  kube-system                 etcd-image-438000                       100m (5%!)(MISSING)     0 (0%!)(MISSING)      100Mi (2%!)(MISSING)       0 (0%!)(MISSING)         3s

	  kube-system                 kube-apiserver-image-438000             250m (12%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         3s

	  kube-system                 kube-controller-manager-image-438000    200m (10%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         3s

	  kube-system                 kube-scheduler-image-438000             100m (5%!)(MISSING)     0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         3s

	Allocated resources:

	  (Total limits may be over 100 percent, i.e., overcommitted.)

	  Resource           Requests    Limits

	  --------           --------    ------

	  cpu                650m (32%!)(MISSING)  0 (0%!)(MISSING)

	  memory             100Mi (2%!)(MISSING)  0 (0%!)(MISSING)

	  ephemeral-storage  0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-1Gi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-2Mi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-32Mi     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-64Ki     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	Events:

	  Type    Reason                   Age   From     Message

	  ----    ------                   ----  ----     -------

	  Normal  Starting                 3s    kubelet  Starting kubelet.

	  Normal  NodeAllocatableEnforced  3s    kubelet  Updated Node Allocatable limit across pods

	  Normal  NodeHasSufficientMemory  3s    kubelet  Node image-438000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    3s    kubelet  Node image-438000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     3s    kubelet  Node image-438000 status is now: NodeHasSufficientPID

	* 

	* ==> dmesg <==

	* [Sep18 19:16] ACPI: SRAT not present

	[  +0.000000] KASLR disabled due to lack of seed

	[  +0.643461] EINJ: EINJ table not found.

	[  +0.527717] systemd-fstab-generator[117]: Ignoring "noauto" for root device

	[  +0.043487] systemd[1]: systemd-journald.service: unit configures an IP firewall, but the local system does not support BPF/cgroup firewalling.

	[  +0.000796] systemd[1]: (This warning is only shown for the first unit using IP firewalling.)

	[  +3.151019] systemd-fstab-generator[480]: Ignoring "noauto" for root device

	[  +0.059597] systemd-fstab-generator[491]: Ignoring "noauto" for root device

	[  +0.447077] systemd-fstab-generator[669]: Ignoring "noauto" for root device

	[  +0.160652] systemd-fstab-generator[704]: Ignoring "noauto" for root device

	[  +0.059300] systemd-fstab-generator[715]: Ignoring "noauto" for root device

	[  +0.060951] systemd-fstab-generator[728]: Ignoring "noauto" for root device

	[  +1.147898] kauditd_printk_skb: 17 callbacks suppressed

	[  +0.076231] systemd-fstab-generator[901]: Ignoring "noauto" for root device

	[  +0.066091] systemd-fstab-generator[927]: Ignoring "noauto" for root device

	[  +0.059203] systemd-fstab-generator[938]: Ignoring "noauto" for root device

	[  +0.061356] systemd-fstab-generator[949]: Ignoring "noauto" for root device

	[  +0.070383] systemd-fstab-generator[985]: Ignoring "noauto" for root device

	[Sep18 19:17] systemd-fstab-generator[1094]: Ignoring "noauto" for root device

	[  +3.540816] systemd-fstab-generator[1430]: Ignoring "noauto" for root device

	[  +0.238814] kauditd_printk_skb: 68 callbacks suppressed

	[  +4.384565] systemd-fstab-generator[2310]: Ignoring "noauto" for root device

	[  +2.302002] kauditd_printk_skb: 2 callbacks suppressed

	* 

	* ==> etcd [ab71429bd819] <==

	* {"level":"info","ts":"2023-09-18T19:17:05.77162Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"58de0efec1d86300 switched to configuration voters=(6403572207504089856)"}

	{"level":"info","ts":"2023-09-18T19:17:05.77177Z","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"cd5c0afff2184bea","local-member-id":"58de0efec1d86300","added-peer-id":"58de0efec1d86300","added-peer-peer-urls":["https://192.168.105.5:2380"]}

	{"level":"info","ts":"2023-09-18T19:17:05.772569Z","caller":"embed/etcd.go:726","msg":"starting with client TLS","tls-info":"cert = /var/lib/minikube/certs/etcd/server.crt, key = /var/lib/minikube/certs/etcd/server.key, client-cert=, client-key=, trusted-ca = /var/lib/minikube/certs/etcd/ca.crt, client-cert-auth = true, crl-file = ","cipher-suites":[]}

	{"level":"info","ts":"2023-09-18T19:17:05.774852Z","caller":"embed/etcd.go:597","msg":"serving peer traffic","address":"192.168.105.5:2380"}

	{"level":"info","ts":"2023-09-18T19:17:05.775513Z","caller":"embed/etcd.go:569","msg":"cmux::serve","address":"192.168.105.5:2380"}

	{"level":"info","ts":"2023-09-18T19:17:05.775785Z","caller":"embed/etcd.go:278","msg":"now serving peer/client/metrics","local-member-id":"58de0efec1d86300","initial-advertise-peer-urls":["https://192.168.105.5:2380"],"listen-peer-urls":["https://192.168.105.5:2380"],"advertise-client-urls":["https://192.168.105.5:2379"],"listen-client-urls":["https://127.0.0.1:2379","https://192.168.105.5:2379"],"listen-metrics-urls":["http://127.0.0.1:2381"]}

	{"level":"info","ts":"2023-09-18T19:17:05.775863Z","caller":"embed/etcd.go:855","msg":"serving metrics","address":"http://127.0.0.1:2381"}

	{"level":"info","ts":"2023-09-18T19:17:05.958644Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"58de0efec1d86300 is starting a new election at term 1"}

	{"level":"info","ts":"2023-09-18T19:17:05.958755Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"58de0efec1d86300 became pre-candidate at term 1"}

	{"level":"info","ts":"2023-09-18T19:17:05.958792Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"58de0efec1d86300 received MsgPreVoteResp from 58de0efec1d86300 at term 1"}

	{"level":"info","ts":"2023-09-18T19:17:05.958819Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"58de0efec1d86300 became candidate at term 2"}

	{"level":"info","ts":"2023-09-18T19:17:05.95885Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"58de0efec1d86300 received MsgVoteResp from 58de0efec1d86300 at term 2"}

	{"level":"info","ts":"2023-09-18T19:17:05.958873Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"58de0efec1d86300 became leader at term 2"}

	{"level":"info","ts":"2023-09-18T19:17:05.958896Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: 58de0efec1d86300 elected leader 58de0efec1d86300 at term 2"}

	{"level":"info","ts":"2023-09-18T19:17:05.959593Z","caller":"etcdserver/server.go:2062","msg":"published local member to cluster through raft","local-member-id":"58de0efec1d86300","local-member-attributes":"{Name:image-438000 ClientURLs:[https://192.168.105.5:2379]}","request-path":"/0/members/58de0efec1d86300/attributes","cluster-id":"cd5c0afff2184bea","publish-timeout":"7s"}

	{"level":"info","ts":"2023-09-18T19:17:05.959631Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2023-09-18T19:17:05.960094Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"127.0.0.1:2379"}

	{"level":"info","ts":"2023-09-18T19:17:05.960166Z","caller":"etcdserver/server.go:2571","msg":"setting up initial cluster version using v2 API","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T19:17:05.960489Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2023-09-18T19:17:05.96083Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.105.5:2379"}

	{"level":"info","ts":"2023-09-18T19:17:05.960966Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"cd5c0afff2184bea","local-member-id":"58de0efec1d86300","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T19:17:05.96318Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T19:17:05.963204Z","caller":"etcdserver/server.go:2595","msg":"cluster version is updated","cluster-version":"3.5"}

	{"level":"info","ts":"2023-09-18T19:17:05.963101Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}

	{"level":"info","ts":"2023-09-18T19:17:05.963219Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}

	* 

	* ==> kernel <==

	*  19:17:12 up 0 min,  0 users,  load average: 0.65, 0.15, 0.05

	Linux image-438000 5.10.57 #1 SMP PREEMPT Fri Sep 15 19:03:18 UTC 2023 aarch64 GNU/Linux

	PRETTY_NAME="Buildroot 2021.02.12"

	* 

	* ==> kube-apiserver [f498fa724b4d] <==

	* I0918 19:17:06.773175       1 cache.go:39] Caches are synced for AvailableConditionController controller

	I0918 19:17:06.773297       1 apf_controller.go:377] Running API Priority and Fairness config worker

	I0918 19:17:06.773305       1 apf_controller.go:380] Running API Priority and Fairness periodic rebalancing process

	I0918 19:17:06.773340       1 shared_informer.go:318] Caches are synced for configmaps

	I0918 19:17:06.774094       1 controller.go:624] quota admission added evaluator for: namespaces

	I0918 19:17:06.776839       1 cache.go:39] Caches are synced for APIServiceRegistrationController controller

	I0918 19:17:06.788576       1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io

	I0918 19:17:06.795037       1 shared_informer.go:318] Caches are synced for crd-autoregister

	I0918 19:17:06.795119       1 aggregator.go:166] initial CRD sync complete...

	I0918 19:17:06.795143       1 autoregister_controller.go:141] Starting autoregister controller

	I0918 19:17:06.795164       1 cache.go:32] Waiting for caches to sync for autoregister controller

	I0918 19:17:06.795180       1 cache.go:39] Caches are synced for autoregister controller

	I0918 19:17:07.677566       1 storage_scheduling.go:95] created PriorityClass system-node-critical with value 2000001000

	I0918 19:17:07.679025       1 storage_scheduling.go:95] created PriorityClass system-cluster-critical with value 2000000000

	I0918 19:17:07.679033       1 storage_scheduling.go:111] all system priority classes are created successfully or already exist.

	I0918 19:17:07.817857       1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io

	I0918 19:17:07.828677       1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io

	I0918 19:17:07.883049       1 alloc.go:330] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.96.0.1"}

	W0918 19:17:07.885782       1 lease.go:263] Resetting endpoints for master service "kubernetes" to [192.168.105.5]

	I0918 19:17:07.886178       1 controller.go:624] quota admission added evaluator for: endpoints

	I0918 19:17:07.887589       1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io

	I0918 19:17:08.704841       1 controller.go:624] quota admission added evaluator for: serviceaccounts

	I0918 19:17:09.267436       1 controller.go:624] quota admission added evaluator for: deployments.apps

	I0918 19:17:09.271301       1 alloc.go:330] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.96.0.10"}

	I0918 19:17:09.274916       1 controller.go:624] quota admission added evaluator for: daemonsets.apps

	* 

	* ==> kube-controller-manager [996710bd5304] <==

	* I0918 19:17:08.958480       1 controllermanager.go:642] "Started controller" controller="namespace-controller"

	I0918 19:17:08.958512       1 namespace_controller.go:197] "Starting namespace controller"

	I0918 19:17:08.958519       1 shared_informer.go:311] Waiting for caches to sync for namespace

	I0918 19:17:09.104562       1 controllermanager.go:642] "Started controller" controller="replicaset-controller"

	I0918 19:17:09.104606       1 replica_set.go:214] "Starting controller" name="replicaset"

	I0918 19:17:09.104610       1 shared_informer.go:311] Waiting for caches to sync for ReplicaSet

	I0918 19:17:09.403842       1 controllermanager.go:642] "Started controller" controller="horizontal-pod-autoscaler-controller"

	I0918 19:17:09.403875       1 horizontal.go:200] "Starting HPA controller"

	I0918 19:17:09.403883       1 shared_informer.go:311] Waiting for caches to sync for HPA

	I0918 19:17:09.554189       1 controllermanager.go:642] "Started controller" controller="token-cleaner-controller"

	I0918 19:17:09.554217       1 tokencleaner.go:112] "Starting token cleaner controller"

	I0918 19:17:09.554221       1 shared_informer.go:311] Waiting for caches to sync for token_cleaner

	I0918 19:17:09.554225       1 shared_informer.go:318] Caches are synced for token_cleaner

	I0918 19:17:09.704630       1 controllermanager.go:642] "Started controller" controller="persistentvolumeclaim-protection-controller"

	I0918 19:17:09.704666       1 pvc_protection_controller.go:102] "Starting PVC protection controller"

	I0918 19:17:09.704671       1 shared_informer.go:311] Waiting for caches to sync for PVC protection

	I0918 19:17:09.855142       1 controllermanager.go:642] "Started controller" controller="endpointslice-mirroring-controller"

	I0918 19:17:09.855206       1 endpointslicemirroring_controller.go:223] "Starting EndpointSliceMirroring controller"

	I0918 19:17:09.855214       1 shared_informer.go:311] Waiting for caches to sync for endpoint_slice_mirroring

	I0918 19:17:10.104056       1 controllermanager.go:642] "Started controller" controller="garbage-collector-controller"

	I0918 19:17:10.104758       1 garbagecollector.go:155] "Starting controller" controller="garbagecollector"

	I0918 19:17:10.104766       1 shared_informer.go:311] Waiting for caches to sync for garbage collector

	I0918 19:17:10.104827       1 graph_builder.go:294] "Running" component="GraphBuilder"

	I0918 19:17:10.154146       1 controllermanager.go:642] "Started controller" controller="certificatesigningrequest-cleaner-controller"

	I0918 19:17:10.154184       1 cleaner.go:83] "Starting CSR cleaner controller"

	* 

	* ==> kube-scheduler [4325f96e9eab] <==

	* W0918 19:17:06.740001       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	E0918 19:17:06.740690       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	W0918 19:17:06.740166       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	E0918 19:17:06.740731       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	W0918 19:17:06.740182       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	E0918 19:17:06.740791       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	W0918 19:17:06.740194       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	E0918 19:17:06.740947       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	W0918 19:17:06.740030       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	E0918 19:17:06.741006       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	W0918 19:17:06.739862       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	W0918 19:17:06.739987       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope

	E0918 19:17:06.741103       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope

	E0918 19:17:06.741095       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	W0918 19:17:07.551364       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	E0918 19:17:07.551381       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	W0918 19:17:07.576789       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	E0918 19:17:07.576806       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	W0918 19:17:07.604508       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	E0918 19:17:07.604517       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	W0918 19:17:07.696599       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	E0918 19:17:07.696615       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	W0918 19:17:07.716903       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	E0918 19:17:07.717013       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	I0918 19:17:08.425313       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	* 

	* ==> kubelet <==

	* -- Journal begins at Mon 2023-09-18 19:16:53 UTC, ends at Mon 2023-09-18 19:17:12 UTC. --

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.356321    2329 plugin_manager.go:118] "Starting Kubelet Plugin Manager"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.426812    2329 kubelet_node_status.go:70] "Attempting to register node" node="image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.430247    2329 topology_manager.go:215] "Topology Admit Handler" podUID="23a1f2e5d3b4ea2a46557aaadb138e83" podNamespace="kube-system" podName="etcd-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.430306    2329 topology_manager.go:215] "Topology Admit Handler" podUID="2d403184dbd531fd9dade79127282944" podNamespace="kube-system" podName="kube-apiserver-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.430326    2329 topology_manager.go:215] "Topology Admit Handler" podUID="a09e2858012c56ffbeb865d4589112e9" podNamespace="kube-system" podName="kube-controller-manager-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.430338    2329 topology_manager.go:215] "Topology Admit Handler" podUID="3e7747dc3cfd98c20fdc8d35328f31d7" podNamespace="kube-system" podName="kube-scheduler-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.430790    2329 kubelet_node_status.go:108] "Node was previously registered" node="image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.430821    2329 kubelet_node_status.go:73] "Successfully registered node" node="image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.624995    2329 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"k8s-certs\" (UniqueName: \"kubernetes.io/host-path/a09e2858012c56ffbeb865d4589112e9-k8s-certs\") pod \"kube-controller-manager-image-438000\" (UID: \"a09e2858012c56ffbeb865d4589112e9\") " pod="kube-system/kube-controller-manager-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.625015    2329 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/3e7747dc3cfd98c20fdc8d35328f31d7-kubeconfig\") pod \"kube-scheduler-image-438000\" (UID: \"3e7747dc3cfd98c20fdc8d35328f31d7\") " pod="kube-system/kube-scheduler-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.625026    2329 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etcd-data\" (UniqueName: \"kubernetes.io/host-path/23a1f2e5d3b4ea2a46557aaadb138e83-etcd-data\") pod \"etcd-image-438000\" (UID: \"23a1f2e5d3b4ea2a46557aaadb138e83\") " pod="kube-system/etcd-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.625034    2329 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-certs\" (UniqueName: \"kubernetes.io/host-path/2d403184dbd531fd9dade79127282944-ca-certs\") pod \"kube-apiserver-image-438000\" (UID: \"2d403184dbd531fd9dade79127282944\") " pod="kube-system/kube-apiserver-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.625048    2329 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"usr-share-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/2d403184dbd531fd9dade79127282944-usr-share-ca-certificates\") pod \"kube-apiserver-image-438000\" (UID: \"2d403184dbd531fd9dade79127282944\") " pod="kube-system/kube-apiserver-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.625057    2329 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-certs\" (UniqueName: \"kubernetes.io/host-path/a09e2858012c56ffbeb865d4589112e9-ca-certs\") pod \"kube-controller-manager-image-438000\" (UID: \"a09e2858012c56ffbeb865d4589112e9\") " pod="kube-system/kube-controller-manager-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.625066    2329 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"flexvolume-dir\" (UniqueName: \"kubernetes.io/host-path/a09e2858012c56ffbeb865d4589112e9-flexvolume-dir\") pod \"kube-controller-manager-image-438000\" (UID: \"a09e2858012c56ffbeb865d4589112e9\") " pod="kube-system/kube-controller-manager-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.625076    2329 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/a09e2858012c56ffbeb865d4589112e9-kubeconfig\") pod \"kube-controller-manager-image-438000\" (UID: \"a09e2858012c56ffbeb865d4589112e9\") " pod="kube-system/kube-controller-manager-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.625087    2329 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"usr-share-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/a09e2858012c56ffbeb865d4589112e9-usr-share-ca-certificates\") pod \"kube-controller-manager-image-438000\" (UID: \"a09e2858012c56ffbeb865d4589112e9\") " pod="kube-system/kube-controller-manager-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.625096    2329 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etcd-certs\" (UniqueName: \"kubernetes.io/host-path/23a1f2e5d3b4ea2a46557aaadb138e83-etcd-certs\") pod \"etcd-image-438000\" (UID: \"23a1f2e5d3b4ea2a46557aaadb138e83\") " pod="kube-system/etcd-image-438000"

	Sep 18 19:17:09 image-438000 kubelet[2329]: I0918 19:17:09.625105    2329 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"k8s-certs\" (UniqueName: \"kubernetes.io/host-path/2d403184dbd531fd9dade79127282944-k8s-certs\") pod \"kube-apiserver-image-438000\" (UID: \"2d403184dbd531fd9dade79127282944\") " pod="kube-system/kube-apiserver-image-438000"

	Sep 18 19:17:10 image-438000 kubelet[2329]: I0918 19:17:10.308130    2329 apiserver.go:52] "Watching apiserver"

	Sep 18 19:17:10 image-438000 kubelet[2329]: I0918 19:17:10.324767    2329 desired_state_of_world_populator.go:159] "Finished populating initial desired state of world"

	Sep 18 19:17:10 image-438000 kubelet[2329]: I0918 19:17:10.359764    2329 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="kube-system/kube-apiserver-image-438000" podStartSLOduration=1.359732008 podCreationTimestamp="2023-09-18 19:17:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2023-09-18 19:17:10.35481005 +0000 UTC m=+1.096183210" watchObservedRunningTime="2023-09-18 19:17:10.359732008 +0000 UTC m=+1.101105127"

	Sep 18 19:17:10 image-438000 kubelet[2329]: I0918 19:17:10.363914    2329 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="kube-system/kube-controller-manager-image-438000" podStartSLOduration=1.36389205 podCreationTimestamp="2023-09-18 19:17:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2023-09-18 19:17:10.359897425 +0000 UTC m=+1.101270544" watchObservedRunningTime="2023-09-18 19:17:10.36389205 +0000 UTC m=+1.105265210"

	Sep 18 19:17:10 image-438000 kubelet[2329]: I0918 19:17:10.363942    2329 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="kube-system/kube-scheduler-image-438000" podStartSLOduration=1.363935758 podCreationTimestamp="2023-09-18 19:17:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2023-09-18 19:17:10.3639278 +0000 UTC m=+1.105300960" watchObservedRunningTime="2023-09-18 19:17:10.363935758 +0000 UTC m=+1.105308919"

	Sep 18 19:17:10 image-438000 kubelet[2329]: I0918 19:17:10.367419    2329 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="kube-system/etcd-image-438000" podStartSLOduration=1.367404841 podCreationTimestamp="2023-09-18 19:17:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2023-09-18 19:17:10.367225966 +0000 UTC m=+1.108599127" watchObservedRunningTime="2023-09-18 19:17:10.367404841 +0000 UTC m=+1.108778002"

-- /stdout --

helpers_test.go:254: (dbg) Run:  out/minikube-darwin-arm64 status --format={{.APIServer}} -p image-438000 -n image-438000

helpers_test.go:261: (dbg) Run:  kubectl --context image-438000 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running

helpers_test.go:272: non-running pods: storage-provisioner

helpers_test.go:274: ======> post-mortem[TestImageBuild/serial/BuildWithBuildArg]: describe non-running pods <======

helpers_test.go:277: (dbg) Run:  kubectl --context image-438000 describe pod storage-provisioner

helpers_test.go:277: (dbg) Non-zero exit: kubectl --context image-438000 describe pod storage-provisioner: exit status 1 (37.505708ms)

** stderr ** 

	Error from server (NotFound): pods "storage-provisioner" not found

** /stderr **

helpers_test.go:279: kubectl --context image-438000 describe pod storage-provisioner: exit status 1

--- FAIL: TestImageBuild/serial/BuildWithBuildArg (1.08s)