=== RUN   TestAddons/Setup

addons_test.go:89: (dbg) Run:  out/minikube-darwin-amd64 start -p addons-076000 --wait=true --memory=4000 --alsologtostderr --addons=registry --addons=metrics-server --addons=volumesnapshots --addons=csi-hostpath-driver --addons=gcp-auth --addons=cloud-spanner --addons=inspektor-gadget --addons=storage-provisioner-rancher --driver=hyperkit  --addons=ingress --addons=ingress-dns --addons=helm-tiller

addons_test.go:89: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p addons-076000 --wait=true --memory=4000 --alsologtostderr --addons=registry --addons=metrics-server --addons=volumesnapshots --addons=csi-hostpath-driver --addons=gcp-auth --addons=cloud-spanner --addons=inspektor-gadget --addons=storage-provisioner-rancher --driver=hyperkit  --addons=ingress --addons=ingress-dns --addons=helm-tiller: exit status 90 (15.170443388s)

-- stdout --

	* [addons-076000] minikube v1.31.2 on Darwin 14.0

	  - MINIKUBE_LOCATION=17345

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17345-936/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17345-936/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the hyperkit driver based on user configuration

	* Starting control plane node addons-076000 in cluster addons-076000

	* Creating hyperkit VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...

-- /stdout --

** stderr ** 

	I1003 17:03:41.002457    1517 out.go:296] Setting OutFile to fd 1 ...

	I1003 17:03:41.002714    1517 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:03:41.002721    1517 out.go:309] Setting ErrFile to fd 2...

	I1003 17:03:41.002725    1517 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:03:41.002897    1517 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17345-936/.minikube/bin

	I1003 17:03:41.004360    1517 out.go:303] Setting JSON to false

	I1003 17:03:41.026268    1517 start.go:128] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":189,"bootTime":1696377632,"procs":416,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.0","kernelVersion":"23.0.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}

	W1003 17:03:41.026373    1517 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I1003 17:03:41.048548    1517 out.go:177] * [addons-076000] minikube v1.31.2 on Darwin 14.0

	I1003 17:03:41.092328    1517 out.go:177]   - MINIKUBE_LOCATION=17345

	I1003 17:03:41.092475    1517 notify.go:220] Checking for updates...

	I1003 17:03:41.115305    1517 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17345-936/kubeconfig

	I1003 17:03:41.137456    1517 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-amd64

	I1003 17:03:41.159042    1517 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I1003 17:03:41.180326    1517 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17345-936/.minikube

	I1003 17:03:41.201277    1517 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I1003 17:03:41.223548    1517 driver.go:373] Setting default libvirt URI to qemu:///system

	I1003 17:03:41.253451    1517 out.go:177] * Using the hyperkit driver based on user configuration

	I1003 17:03:41.296284    1517 start.go:298] selected driver: hyperkit

	I1003 17:03:41.296341    1517 start.go:902] validating driver "hyperkit" against <nil>

	I1003 17:03:41.296360    1517 start.go:913] status for hyperkit: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I1003 17:03:41.300831    1517 install.go:52] acquiring lock: {Name:mk4023283b30b374c3f04c8805d539e68824c0b8 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I1003 17:03:41.300930    1517 install.go:117] Validating docker-machine-driver-hyperkit, PATH=/Users/jenkins/minikube-integration/17345-936/.minikube/bin:/Users/jenkins/workspace/out/:/usr/bin:/bin:/usr/sbin:/sbin:/Users/jenkins/google-cloud-sdk/bin:/usr/local/bin/:/usr/local/go/bin/:/Users/jenkins/go/bin

	I1003 17:03:41.308432    1517 install.go:137] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit version is 1.31.2

	I1003 17:03:41.312076    1517 install.go:79] stdout: /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:03:41.312097    1517 install.go:81] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit looks good

	I1003 17:03:41.312123    1517 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I1003 17:03:41.312321    1517 start_flags.go:923] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I1003 17:03:41.312349    1517 cni.go:84] Creating CNI manager for ""

	I1003 17:03:41.312362    1517 cni.go:158] "hyperkit" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I1003 17:03:41.312369    1517 start_flags.go:316] Found "bridge CNI" CNI - setting NetworkPlugin=cni

	I1003 17:03:41.312377    1517 start_flags.go:321] config:

	{Name:addons-076000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1696360059-17345@sha256:76d99edd1576614d5c20a839dd16ae1d7c810f3b909a01797063d483159ea3ae Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:addons-076000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker

CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I1003 17:03:41.312504    1517 iso.go:125] acquiring lock: {Name:mk09a17bc6b7923952debc9d2a1712efcb7ec756 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I1003 17:03:41.354254    1517 out.go:177] * Starting control plane node addons-076000 in cluster addons-076000

	I1003 17:03:41.376200    1517 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I1003 17:03:41.376292    1517 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17345-936/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-amd64.tar.lz4

	I1003 17:03:41.376321    1517 cache.go:57] Caching tarball of preloaded images

	I1003 17:03:41.376509    1517 preload.go:174] Found /Users/jenkins/minikube-integration/17345-936/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download

	I1003 17:03:41.376530    1517 cache.go:60] Finished verifying existence of preloaded tar for  v1.28.2 on docker

	I1003 17:03:41.377019    1517 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/addons-076000/config.json ...

	I1003 17:03:41.377057    1517 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/addons-076000/config.json: {Name:mk2259f69c5af3ed0b147fef6258b3a0c1de9f85 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I1003 17:03:41.377857    1517 start.go:365] acquiring machines lock for addons-076000: {Name:mk2907765d495f362bb545719659eca626cfc4d0 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I1003 17:03:41.378047    1517 start.go:369] acquired machines lock for "addons-076000" in 169.32µs

	I1003 17:03:41.378099    1517 start.go:93] Provisioning new machine with config: &{Name:addons-076000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17240/minikube-v1.31.0-1695060926-17240-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1696360059-17345@sha256:76d99edd1576614d5c20a839dd16ae1d7c810f3b909a01797063d483159ea3ae Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesCo

nfig:{KubernetesVersion:v1.28.2 ClusterName:addons-076000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 Mount

Options:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I1003 17:03:41.378176    1517 start.go:125] createHost starting for "" (driver="hyperkit")

	I1003 17:03:41.400516    1517 out.go:204] * Creating hyperkit VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...

	I1003 17:03:41.400915    1517 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:03:41.400992    1517 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:03:41.409945    1517 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:49346

	I1003 17:03:41.410329    1517 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:03:41.410760    1517 main.go:141] libmachine: Using API Version  1

	I1003 17:03:41.410770    1517 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:03:41.410965    1517 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:03:41.411072    1517 main.go:141] libmachine: (addons-076000) Calling .GetMachineName

	I1003 17:03:41.411168    1517 main.go:141] libmachine: (addons-076000) Calling .DriverName

	I1003 17:03:41.411269    1517 start.go:159] libmachine.API.Create for "addons-076000" (driver="hyperkit")

	I1003 17:03:41.411299    1517 client.go:168] LocalClient.Create starting

	I1003 17:03:41.411334    1517 main.go:141] libmachine: Creating CA: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca.pem

	I1003 17:03:41.609055    1517 main.go:141] libmachine: Creating client certificate: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/cert.pem

	I1003 17:03:41.777781    1517 main.go:141] libmachine: Running pre-create checks...

	I1003 17:03:41.777793    1517 main.go:141] libmachine: (addons-076000) Calling .PreCreateCheck

	I1003 17:03:41.778065    1517 main.go:141] libmachine: (addons-076000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:03:41.778250    1517 main.go:141] libmachine: (addons-076000) Calling .GetConfigRaw

	I1003 17:03:41.778741    1517 main.go:141] libmachine: Creating machine...

	I1003 17:03:41.778750    1517 main.go:141] libmachine: (addons-076000) Calling .Create

	I1003 17:03:41.778881    1517 main.go:141] libmachine: (addons-076000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:03:41.779121    1517 main.go:141] libmachine: (addons-076000) DBG | I1003 17:03:41.778882    1525 common.go:145] Making disk image using store path: /Users/jenkins/minikube-integration/17345-936/.minikube

	I1003 17:03:41.779215    1517 main.go:141] libmachine: (addons-076000) Downloading /Users/jenkins/minikube-integration/17345-936/.minikube/cache/boot2docker.iso from file:///Users/jenkins/minikube-integration/17345-936/.minikube/cache/iso/amd64/minikube-v1.31.0-1695060926-17240-amd64.iso...

	I1003 17:03:41.995627    1517 main.go:141] libmachine: (addons-076000) DBG | I1003 17:03:41.995533    1525 common.go:152] Creating ssh key: /Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/id_rsa...

	I1003 17:03:42.139006    1517 main.go:141] libmachine: (addons-076000) DBG | I1003 17:03:42.138893    1525 common.go:158] Creating raw disk image: /Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/addons-076000.rawdisk...

	I1003 17:03:42.139020    1517 main.go:141] libmachine: (addons-076000) DBG | Writing magic tar header

	I1003 17:03:42.139030    1517 main.go:141] libmachine: (addons-076000) DBG | Writing SSH key tar header

	I1003 17:03:42.139950    1517 main.go:141] libmachine: (addons-076000) DBG | I1003 17:03:42.139823    1525 common.go:172] Fixing permissions on /Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000 ...

	I1003 17:03:42.495370    1517 main.go:141] libmachine: (addons-076000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:03:42.495387    1517 main.go:141] libmachine: (addons-076000) DBG | clean start, hyperkit pid file doesn't exist: /Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/hyperkit.pid

	I1003 17:03:42.495432    1517 main.go:141] libmachine: (addons-076000) DBG | Using UUID 79ffcae8-6249-11ee-8f5c-149d997cd0f1

	I1003 17:03:42.561187    1517 main.go:141] libmachine: (addons-076000) DBG | Generated MAC 12:d6:1e:c9:6f:b1

	I1003 17:03:42.561226    1517 main.go:141] libmachine: (addons-076000) DBG | Starting with cmdline: loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=addons-076000

	I1003 17:03:42.561292    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 DEBUG: hyperkit: Start &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"79ffcae8-6249-11ee-8f5c-149d997cd0f1", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc00011b1d0)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/bzimage", Initrd:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/initrd", Bootrom:"", CPUs:2, Memory:4000, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os.Process)(nil)}

	I1003 17:03:42.561349    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 DEBUG: hyperkit: check &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"79ffcae8-6249-11ee-8f5c-149d997cd0f1", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc00011b1d0)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/bzimage", Initrd:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/initrd", Bootrom:"", CPUs:2, Memory:4000, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os.Process)(nil)}

	I1003 17:03:42.561405    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 DEBUG: hyperkit: Arguments: []string{"-A", "-u", "-F", "/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/hyperkit.pid", "-c", "2", "-m", "4000M", "-s", "0:0,hostbridge", "-s", "31,lpc", "-s", "1:0,virtio-net", "-U", "79ffcae8-6249-11ee-8f5c-149d997cd0f1", "-s", "2:0,virtio-blk,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/addons-076000.rawdisk", "-s", "3,ahci-cd,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/boot2docker.iso", "-s", "4,virtio-rnd", "-l", "com1,autopty=/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/tty,log=/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/console-ring", "-f", "kexec,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/bzimage,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addon

s-076000/initrd,earlyprintk=serial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=addons-076000"}

	I1003 17:03:42.561479    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 DEBUG: hyperkit: CmdLine: "/usr/local/bin/hyperkit -A -u -F /Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/hyperkit.pid -c 2 -m 4000M -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-net -U 79ffcae8-6249-11ee-8f5c-149d997cd0f1 -s 2:0,virtio-blk,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/addons-076000.rawdisk -s 3,ahci-cd,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/boot2docker.iso -s 4,virtio-rnd -l com1,autopty=/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/tty,log=/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/console-ring -f kexec,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/bzimage,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/initrd,earlyprintk=serial loglevel=3 console=ttyS0 co

nsole=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=addons-076000"

	I1003 17:03:42.561498    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 DEBUG: hyperkit: Redirecting stdout/stderr to logger

	I1003 17:03:42.564602    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 DEBUG: hyperkit: Pid is 1530

	I1003 17:03:42.565175    1517 main.go:141] libmachine: (addons-076000) DBG | Attempt 0

	I1003 17:03:42.565195    1517 main.go:141] libmachine: (addons-076000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:03:42.565312    1517 main.go:141] libmachine: (addons-076000) DBG | hyperkit pid from json: 1530

	I1003 17:03:42.566611    1517 main.go:141] libmachine: (addons-076000) DBG | Searching for 12:d6:1e:c9:6f:b1 in /var/db/dhcpd_leases ...

	I1003 17:03:42.566654    1517 main.go:141] libmachine: (addons-076000) DBG | Found 1 entries in /var/db/dhcpd_leases!

	I1003 17:03:42.566667    1517 main.go:141] libmachine: (addons-076000) DBG | dhcp entry: {Name: IPAddress:192.168.86.2 HWAddress:f2:21:c3:3b:c7:2c ID:1,f2:21:c3:3b:c7:2c Lease:0x651dfcb3}

	I1003 17:03:42.586277    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 INFO : hyperkit: stderr: Using fd 5 for I/O notifications

	I1003 17:03:42.617719    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 INFO : hyperkit: stderr: /Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/boot2docker.iso: fcntl(F_PUNCHHOLE) Operation not permitted: block device will not support TRIM/DISCARD

	I1003 17:03:42.618421    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0

	I1003 17:03:42.618438    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0

	I1003 17:03:42.618446    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0

	I1003 17:03:42.618454    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:42 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0

	I1003 17:03:43.128000    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:43 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 0

	I1003 17:03:43.128017    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:43 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 0

	I1003 17:03:43.232996    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:43 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0

	I1003 17:03:43.233025    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:43 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0

	I1003 17:03:43.233038    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:43 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0

	I1003 17:03:43.233067    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:43 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0

	I1003 17:03:43.233911    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:43 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 1

	I1003 17:03:43.233929    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:43 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 1

	I1003 17:03:44.566832    1517 main.go:141] libmachine: (addons-076000) DBG | Attempt 1

	I1003 17:03:44.566849    1517 main.go:141] libmachine: (addons-076000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:03:44.566924    1517 main.go:141] libmachine: (addons-076000) DBG | hyperkit pid from json: 1530

	I1003 17:03:44.567721    1517 main.go:141] libmachine: (addons-076000) DBG | Searching for 12:d6:1e:c9:6f:b1 in /var/db/dhcpd_leases ...

	I1003 17:03:44.567753    1517 main.go:141] libmachine: (addons-076000) DBG | Found 1 entries in /var/db/dhcpd_leases!

	I1003 17:03:44.567767    1517 main.go:141] libmachine: (addons-076000) DBG | dhcp entry: {Name: IPAddress:192.168.86.2 HWAddress:f2:21:c3:3b:c7:2c ID:1,f2:21:c3:3b:c7:2c Lease:0x651dfcb3}

	I1003 17:03:46.569153    1517 main.go:141] libmachine: (addons-076000) DBG | Attempt 2

	I1003 17:03:46.569188    1517 main.go:141] libmachine: (addons-076000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:03:46.569289    1517 main.go:141] libmachine: (addons-076000) DBG | hyperkit pid from json: 1530

	I1003 17:03:46.570126    1517 main.go:141] libmachine: (addons-076000) DBG | Searching for 12:d6:1e:c9:6f:b1 in /var/db/dhcpd_leases ...

	I1003 17:03:46.570148    1517 main.go:141] libmachine: (addons-076000) DBG | Found 1 entries in /var/db/dhcpd_leases!

	I1003 17:03:46.570163    1517 main.go:141] libmachine: (addons-076000) DBG | dhcp entry: {Name: IPAddress:192.168.86.2 HWAddress:f2:21:c3:3b:c7:2c ID:1,f2:21:c3:3b:c7:2c Lease:0x651dfcb3}

	I1003 17:03:48.135569    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:48 INFO : hyperkit: stderr: rdmsr to register 0x64d on vcpu 1

	I1003 17:03:48.135636    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:48 INFO : hyperkit: stderr: rdmsr to register 0x64e on vcpu 1

	I1003 17:03:48.135646    1517 main.go:141] libmachine: (addons-076000) DBG | 2023/10/03 17:03:48 INFO : hyperkit: stderr: rdmsr to register 0x34 on vcpu 1

	I1003 17:03:48.570859    1517 main.go:141] libmachine: (addons-076000) DBG | Attempt 3

	I1003 17:03:48.570874    1517 main.go:141] libmachine: (addons-076000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:03:48.570931    1517 main.go:141] libmachine: (addons-076000) DBG | hyperkit pid from json: 1530

	I1003 17:03:48.571673    1517 main.go:141] libmachine: (addons-076000) DBG | Searching for 12:d6:1e:c9:6f:b1 in /var/db/dhcpd_leases ...

	I1003 17:03:48.571712    1517 main.go:141] libmachine: (addons-076000) DBG | Found 1 entries in /var/db/dhcpd_leases!

	I1003 17:03:48.571723    1517 main.go:141] libmachine: (addons-076000) DBG | dhcp entry: {Name: IPAddress:192.168.86.2 HWAddress:f2:21:c3:3b:c7:2c ID:1,f2:21:c3:3b:c7:2c Lease:0x651dfcb3}

	I1003 17:03:50.572722    1517 main.go:141] libmachine: (addons-076000) DBG | Attempt 4

	I1003 17:03:50.572737    1517 main.go:141] libmachine: (addons-076000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:03:50.572836    1517 main.go:141] libmachine: (addons-076000) DBG | hyperkit pid from json: 1530

	I1003 17:03:50.573638    1517 main.go:141] libmachine: (addons-076000) DBG | Searching for 12:d6:1e:c9:6f:b1 in /var/db/dhcpd_leases ...

	I1003 17:03:50.573666    1517 main.go:141] libmachine: (addons-076000) DBG | Found 1 entries in /var/db/dhcpd_leases!

	I1003 17:03:50.573680    1517 main.go:141] libmachine: (addons-076000) DBG | dhcp entry: {Name: IPAddress:192.168.86.2 HWAddress:f2:21:c3:3b:c7:2c ID:1,f2:21:c3:3b:c7:2c Lease:0x651dfcb3}

	I1003 17:03:52.575274    1517 main.go:141] libmachine: (addons-076000) DBG | Attempt 5

	I1003 17:03:52.575305    1517 main.go:141] libmachine: (addons-076000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:03:52.575415    1517 main.go:141] libmachine: (addons-076000) DBG | hyperkit pid from json: 1530

	I1003 17:03:52.576768    1517 main.go:141] libmachine: (addons-076000) DBG | Searching for 12:d6:1e:c9:6f:b1 in /var/db/dhcpd_leases ...

	I1003 17:03:52.576817    1517 main.go:141] libmachine: (addons-076000) DBG | Found 2 entries in /var/db/dhcpd_leases!

	I1003 17:03:52.576869    1517 main.go:141] libmachine: (addons-076000) DBG | dhcp entry: {Name:minikube IPAddress:192.168.87.2 HWAddress:12:d6:1e:c9:6f:b1 ID:1,12:d6:1e:c9:6f:b1 Lease:0x651dfd66}

	I1003 17:03:52.576898    1517 main.go:141] libmachine: (addons-076000) DBG | Found match: 12:d6:1e:c9:6f:b1

	I1003 17:03:52.576915    1517 main.go:141] libmachine: (addons-076000) DBG | IP: 192.168.87.2

	I1003 17:03:52.576971    1517 main.go:141] libmachine: (addons-076000) Calling .GetConfigRaw

	I1003 17:03:52.577765    1517 main.go:141] libmachine: (addons-076000) Calling .DriverName

	I1003 17:03:52.577914    1517 main.go:141] libmachine: (addons-076000) Calling .DriverName

	I1003 17:03:52.578053    1517 main.go:141] libmachine: Waiting for machine to be running, this may take a few minutes...

	I1003 17:03:52.578070    1517 main.go:141] libmachine: (addons-076000) Calling .GetState

	I1003 17:03:52.578180    1517 main.go:141] libmachine: (addons-076000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:03:52.578248    1517 main.go:141] libmachine: (addons-076000) DBG | hyperkit pid from json: 1530

	I1003 17:03:52.579219    1517 main.go:141] libmachine: Detecting operating system of created instance...

	I1003 17:03:52.579238    1517 main.go:141] libmachine: Waiting for SSH to be available...

	I1003 17:03:52.579249    1517 main.go:141] libmachine: Getting to WaitForSSH function...

	I1003 17:03:52.579257    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:52.579387    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:52.579505    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:52.579593    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:52.579683    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:52.580428    1517 main.go:141] libmachine: Using SSH client type: native

	I1003 17:03:52.580717    1517 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.87.2 22 <nil> <nil>}

	I1003 17:03:52.580725    1517 main.go:141] libmachine: About to run SSH command:

	exit 0

	I1003 17:03:52.639989    1517 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	I1003 17:03:52.640004    1517 main.go:141] libmachine: Detecting the provisioner...

	I1003 17:03:52.640014    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:52.640146    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:52.640230    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:52.640318    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:52.640395    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:52.640518    1517 main.go:141] libmachine: Using SSH client type: native

	I1003 17:03:52.640769    1517 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.87.2 22 <nil> <nil>}

	I1003 17:03:52.640777    1517 main.go:141] libmachine: About to run SSH command:

	cat /etc/os-release

	I1003 17:03:52.697354    1517 main.go:141] libmachine: SSH cmd err, output: <nil>: NAME=Buildroot

	VERSION=2021.02.12-1-gb090841-dirty

	ID=buildroot

	VERSION_ID=2021.02.12

	PRETTY_NAME="Buildroot 2021.02.12"

	I1003 17:03:52.697404    1517 main.go:141] libmachine: found compatible host: buildroot

	I1003 17:03:52.697411    1517 main.go:141] libmachine: Provisioning with buildroot...

	I1003 17:03:52.697417    1517 main.go:141] libmachine: (addons-076000) Calling .GetMachineName

	I1003 17:03:52.697551    1517 buildroot.go:166] provisioning hostname "addons-076000"

	I1003 17:03:52.697562    1517 main.go:141] libmachine: (addons-076000) Calling .GetMachineName

	I1003 17:03:52.697658    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:52.697758    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:52.697847    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:52.697947    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:52.698080    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:52.698218    1517 main.go:141] libmachine: Using SSH client type: native

	I1003 17:03:52.698471    1517 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.87.2 22 <nil> <nil>}

	I1003 17:03:52.698480    1517 main.go:141] libmachine: About to run SSH command:

	sudo hostname addons-076000 && echo "addons-076000" | sudo tee /etc/hostname

	I1003 17:03:52.764134    1517 main.go:141] libmachine: SSH cmd err, output: <nil>: addons-076000

	I1003 17:03:52.764159    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:52.764304    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:52.764402    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:52.764482    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:52.764573    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:52.764701    1517 main.go:141] libmachine: Using SSH client type: native

	I1003 17:03:52.764944    1517 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.87.2 22 <nil> <nil>}

	I1003 17:03:52.764958    1517 main.go:141] libmachine: About to run SSH command:

			if ! grep -xq '.*\saddons-076000' /etc/hosts; then

				if grep -xq '127.0.1.1\s.*' /etc/hosts; then

					sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 addons-076000/g' /etc/hosts;

				else 

					echo '127.0.1.1 addons-076000' | sudo tee -a /etc/hosts; 

				fi

			fi

	I1003 17:03:52.828255    1517 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	I1003 17:03:52.828276    1517 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/17345-936/.minikube CaCertPath:/Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/17345-936/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/17345-936/.minikube}

	I1003 17:03:52.828291    1517 buildroot.go:174] setting up certificates

	I1003 17:03:52.828301    1517 provision.go:83] configureAuth start

	I1003 17:03:52.828308    1517 main.go:141] libmachine: (addons-076000) Calling .GetMachineName

	I1003 17:03:52.828443    1517 main.go:141] libmachine: (addons-076000) Calling .GetIP

	I1003 17:03:52.828551    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:52.828645    1517 provision.go:138] copyHostCerts

	I1003 17:03:52.828719    1517 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/17345-936/.minikube/cert.pem (1123 bytes)

	I1003 17:03:52.829039    1517 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/17345-936/.minikube/key.pem (1675 bytes)

	I1003 17:03:52.829244    1517 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/17345-936/.minikube/ca.pem (1078 bytes)

	I1003 17:03:52.829378    1517 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/17345-936/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca-key.pem org=jenkins.addons-076000 san=[192.168.87.2 192.168.87.2 localhost 127.0.0.1 minikube addons-076000]

	I1003 17:03:52.904151    1517 provision.go:172] copyRemoteCerts

	I1003 17:03:52.904210    1517 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker

	I1003 17:03:52.904226    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:52.904351    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:52.904441    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:52.904521    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:52.904618    1517 sshutil.go:53] new ssh client: &{IP:192.168.87.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/id_rsa Username:docker}

	I1003 17:03:52.939596    1517 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)

	I1003 17:03:52.955886    1517 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/machines/server.pem --> /etc/docker/server.pem (1216 bytes)

	I1003 17:03:52.971631    1517 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)

	I1003 17:03:52.987347    1517 provision.go:86] duration metric: configureAuth took 159.026941ms

	I1003 17:03:52.987360    1517 buildroot.go:189] setting minikube options for container-runtime

	I1003 17:03:52.987493    1517 config.go:182] Loaded profile config "addons-076000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I1003 17:03:52.987507    1517 main.go:141] libmachine: (addons-076000) Calling .DriverName

	I1003 17:03:52.987640    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:52.987721    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:52.987814    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:52.987895    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:52.987978    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:52.988095    1517 main.go:141] libmachine: Using SSH client type: native

	I1003 17:03:52.988334    1517 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.87.2 22 <nil> <nil>}

	I1003 17:03:52.988343    1517 main.go:141] libmachine: About to run SSH command:

	df --output=fstype / | tail -n 1

	I1003 17:03:53.045005    1517 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs

	I1003 17:03:53.045018    1517 buildroot.go:70] root file system type: tmpfs

	I1003 17:03:53.045099    1517 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...

	I1003 17:03:53.045112    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:53.045241    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:53.045338    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:53.045421    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:53.045505    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:53.045637    1517 main.go:141] libmachine: Using SSH client type: native

	I1003 17:03:53.045907    1517 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.87.2 22 <nil> <nil>}

	I1003 17:03:53.045955    1517 main.go:141] libmachine: About to run SSH command:

	sudo mkdir -p /lib/systemd/system && printf %s "[Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP \$MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	" | sudo tee /lib/systemd/system/docker.service.new

	I1003 17:03:53.109831    1517 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP $MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	I1003 17:03:53.109855    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:53.109992    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:53.110082    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:53.110179    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:53.110288    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:53.110428    1517 main.go:141] libmachine: Using SSH client type: native

	I1003 17:03:53.110695    1517 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.87.2 22 <nil> <nil>}

	I1003 17:03:53.110708    1517 main.go:141] libmachine: About to run SSH command:

	sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }

	I1003 17:03:53.607386    1517 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory

	Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

	I1003 17:03:53.607409    1517 main.go:141] libmachine: Checking connection to Docker...

	I1003 17:03:53.607416    1517 main.go:141] libmachine: (addons-076000) Calling .GetURL

	I1003 17:03:53.607547    1517 main.go:141] libmachine: Docker is up and running!

	I1003 17:03:53.607554    1517 main.go:141] libmachine: Reticulating splines...

	I1003 17:03:53.607558    1517 client.go:171] LocalClient.Create took 12.195552837s

	I1003 17:03:53.607570    1517 start.go:167] duration metric: libmachine.API.Create for "addons-076000" took 12.195600242s

	I1003 17:03:53.607580    1517 start.go:300] post-start starting for "addons-076000" (driver="hyperkit")

	I1003 17:03:53.607589    1517 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]

	I1003 17:03:53.607599    1517 main.go:141] libmachine: (addons-076000) Calling .DriverName

	I1003 17:03:53.607744    1517 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs

	I1003 17:03:53.607758    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:53.607839    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:53.607923    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:53.607997    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:53.608084    1517 sshutil.go:53] new ssh client: &{IP:192.168.87.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/id_rsa Username:docker}

	I1003 17:03:53.640747    1517 ssh_runner.go:195] Run: cat /etc/os-release

	I1003 17:03:53.643456    1517 info.go:137] Remote host: Buildroot 2021.02.12

	I1003 17:03:53.643469    1517 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17345-936/.minikube/addons for local assets ...

	I1003 17:03:53.643557    1517 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17345-936/.minikube/files for local assets ...

	I1003 17:03:53.643604    1517 start.go:303] post-start completed in 36.017016ms

	I1003 17:03:53.643621    1517 main.go:141] libmachine: (addons-076000) Calling .GetConfigRaw

	I1003 17:03:53.644169    1517 main.go:141] libmachine: (addons-076000) Calling .GetIP

	I1003 17:03:53.644303    1517 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/addons-076000/config.json ...

	I1003 17:03:53.644640    1517 start.go:128] duration metric: createHost completed in 12.265746775s

	I1003 17:03:53.644656    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:53.644747    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:53.644838    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:53.644926    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:53.645007    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:53.645112    1517 main.go:141] libmachine: Using SSH client type: native

	I1003 17:03:53.645355    1517 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.87.2 22 <nil> <nil>}

	I1003 17:03:53.645363    1517 main.go:141] libmachine: About to run SSH command:

	date +%s.%N

	I1003 17:03:53.702129    1517 main.go:141] libmachine: SSH cmd err, output: <nil>: 1696377833.591583409

	I1003 17:03:53.702145    1517 fix.go:206] guest clock: 1696377833.591583409

	I1003 17:03:53.702153    1517 fix.go:219] Guest: 2023-10-03 17:03:53.591583409 -0700 PDT Remote: 2023-10-03 17:03:53.644648 -0700 PDT m=+12.672681521 (delta=-53.064591ms)

	I1003 17:03:53.702170    1517 fix.go:190] guest clock delta is within tolerance: -53.064591ms

	I1003 17:03:53.702173    1517 start.go:83] releasing machines lock for "addons-076000", held for 12.323407042s

	I1003 17:03:53.702191    1517 main.go:141] libmachine: (addons-076000) Calling .DriverName

	I1003 17:03:53.702322    1517 main.go:141] libmachine: (addons-076000) Calling .GetIP

	I1003 17:03:53.702417    1517 main.go:141] libmachine: (addons-076000) Calling .DriverName

	I1003 17:03:53.702703    1517 main.go:141] libmachine: (addons-076000) Calling .DriverName

	I1003 17:03:53.702805    1517 main.go:141] libmachine: (addons-076000) Calling .DriverName

	I1003 17:03:53.702870    1517 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/

	I1003 17:03:53.702902    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:53.702943    1517 ssh_runner.go:195] Run: cat /version.json

	I1003 17:03:53.702955    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHHostname

	I1003 17:03:53.702996    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:53.703048    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHPort

	I1003 17:03:53.703098    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:53.703134    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHKeyPath

	I1003 17:03:53.703222    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:53.703246    1517 main.go:141] libmachine: (addons-076000) Calling .GetSSHUsername

	I1003 17:03:53.703302    1517 sshutil.go:53] new ssh client: &{IP:192.168.87.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/id_rsa Username:docker}

	I1003 17:03:53.703332    1517 sshutil.go:53] new ssh client: &{IP:192.168.87.2 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/addons-076000/id_rsa Username:docker}

	I1003 17:03:53.735066    1517 ssh_runner.go:195] Run: systemctl --version

	I1003 17:03:53.738885    1517 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"

	W1003 17:03:53.785904    1517 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found

	I1003 17:03:53.785983    1517 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;

	I1003 17:03:53.797328    1517 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)

	I1003 17:03:53.797342    1517 start.go:469] detecting cgroup driver to use...

	I1003 17:03:53.797445    1517 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock

	" | sudo tee /etc/crictl.yaml"

	I1003 17:03:53.809597    1517 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"

	I1003 17:03:53.816148    1517 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"

	I1003 17:03:53.822652    1517 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...

	I1003 17:03:53.822689    1517 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"

	I1003 17:03:53.829262    1517 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I1003 17:03:53.835849    1517 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"

	I1003 17:03:53.842411    1517 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I1003 17:03:53.848924    1517 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"

	I1003 17:03:53.855725    1517 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"

	I1003 17:03:53.862318    1517 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables

	I1003 17:03:53.868167    1517 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

	I1003 17:03:53.873982    1517 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I1003 17:03:53.955911    1517 ssh_runner.go:195] Run: sudo systemctl restart containerd

	I1003 17:03:53.968573    1517 start.go:469] detecting cgroup driver to use...

	I1003 17:03:53.968643    1517 ssh_runner.go:195] Run: sudo systemctl cat docker.service

	I1003 17:03:53.982119    1517 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I1003 17:03:53.994721    1517 ssh_runner.go:195] Run: sudo systemctl stop -f containerd

	I1003 17:03:54.011251    1517 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I1003 17:03:54.020089    1517 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I1003 17:03:54.028816    1517 ssh_runner.go:195] Run: sudo systemctl stop -f crio

	I1003 17:03:54.049618    1517 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I1003 17:03:54.058839    1517 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/cri-dockerd.sock

	" | sudo tee /etc/crictl.yaml"

	I1003 17:03:54.070965    1517 ssh_runner.go:195] Run: which cri-dockerd

	I1003 17:03:54.073298    1517 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d

	I1003 17:03:54.078781    1517 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)

	I1003 17:03:54.089951    1517 ssh_runner.go:195] Run: sudo systemctl unmask docker.service

	I1003 17:03:54.172860    1517 ssh_runner.go:195] Run: sudo systemctl enable docker.socket

	I1003 17:03:54.265760    1517 docker.go:554] configuring docker to use "cgroupfs" as cgroup driver...

	I1003 17:03:54.265844    1517 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)

	I1003 17:03:54.277393    1517 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I1003 17:03:54.361585    1517 ssh_runner.go:195] Run: sudo systemctl restart docker

	I1003 17:03:55.642861    1517 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.281174494s)

	I1003 17:03:55.642919    1517 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I1003 17:03:55.722842    1517 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket

	I1003 17:03:55.806759    1517 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I1003 17:03:55.895428    1517 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I1003 17:03:55.979036    1517 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket

	I1003 17:03:56.018062    1517 out.go:177] 

	W1003 17:03:56.042176    1517 out.go:239] X Exiting due to RUNTIME_ENABLE: Failed to enable container runtime: sudo systemctl restart cri-docker.socket: Process exited with status 1

	stdout:

	stderr:

	Job failed. See "journalctl -xe" for details.

	X Exiting due to RUNTIME_ENABLE: Failed to enable container runtime: sudo systemctl restart cri-docker.socket: Process exited with status 1

	stdout:

	stderr:

	Job failed. See "journalctl -xe" for details.

	W1003 17:03:56.042199    1517 out.go:239] * 

	* 

	W1003 17:03:56.043372    1517 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	╭─────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                             │

	│    * If the above advice does not help, please let us know:                                 │

	│      https://github.com/kubernetes/minikube/issues/new/choose                               │

	│                                                                                             │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │

	│                                                                                             │

	╰─────────────────────────────────────────────────────────────────────────────────────────────╯

	I1003 17:03:56.087878    1517 out.go:177] 

** /stderr **

addons_test.go:91: out/minikube-darwin-amd64 start -p addons-076000 --wait=true --memory=4000 --alsologtostderr --addons=registry --addons=metrics-server --addons=volumesnapshots --addons=csi-hostpath-driver --addons=gcp-auth --addons=cloud-spanner --addons=inspektor-gadget --addons=storage-provisioner-rancher --driver=hyperkit  --addons=ingress --addons=ingress-dns --addons=helm-tiller failed: exit status 90

--- FAIL: TestAddons/Setup (15.18s)

=== RUN   TestStartStop/group/old-k8s-version/serial/VerifyKubernetesImages

start_stop_delete_test.go:304: (dbg) Run:  out/minikube-darwin-amd64 ssh -p old-k8s-version-999000 "sudo crictl images -o json"

start_stop_delete_test.go:304: (dbg) Non-zero exit: out/minikube-darwin-amd64 ssh -p old-k8s-version-999000 "sudo crictl images -o json": exit status 1 (125.755632ms)

-- stdout --

	FATA[0000] validate service connection: validate CRI v1 image API for endpoint "unix:///var/run/dockershim.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.ImageService 

-- /stdout --

** stderr ** 

	ssh: Process exited with status 1

** /stderr **

start_stop_delete_test.go:304: failed to get images inside minikube. args "out/minikube-darwin-amd64 ssh -p old-k8s-version-999000 \"sudo crictl images -o json\"": exit status 1

start_stop_delete_test.go:304: failed to decode images json invalid character '\x1b' looking for beginning of value. output:

FATA[0000] validate service connection: validate CRI v1 image API for endpoint "unix:///var/run/dockershim.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.ImageService 

start_stop_delete_test.go:304: v1.16.0 images missing (-want +got):

[]string{

- 	"k8s.gcr.io/coredns:1.6.2",

- 	"k8s.gcr.io/etcd:3.3.15-0",

- 	"k8s.gcr.io/k8s-minikube/storage-provisioner:v5",

- 	"k8s.gcr.io/kube-apiserver:v1.16.0",

- 	"k8s.gcr.io/kube-controller-manager:v1.16.0",

- 	"k8s.gcr.io/kube-proxy:v1.16.0",

- 	"k8s.gcr.io/kube-scheduler:v1.16.0",

- 	"k8s.gcr.io/pause:3.1",

}

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:239: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.Host}} -p old-k8s-version-999000 -n old-k8s-version-999000

helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/VerifyKubernetesImages FAILED: start of post-mortem logs <<<

helpers_test.go:245: ======>  post-mortem[TestStartStop/group/old-k8s-version/serial/VerifyKubernetesImages]: minikube logs <======

helpers_test.go:247: (dbg) Run:  out/minikube-darwin-amd64 -p old-k8s-version-999000 logs -n 25

helpers_test.go:247: (dbg) Done: out/minikube-darwin-amd64 -p old-k8s-version-999000 logs -n 25: (2.111738237s)

helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/VerifyKubernetesImages logs: 

-- stdout --

	* 

	* ==> Audit <==

	* |---------|--------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|

	| Command |                          Args                          |        Profile         |  User   | Version |     Start Time      |      End Time       |

	|---------|--------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|

	| ssh     | -p false-312000 sudo systemctl                         | false-312000           | jenkins | v1.31.2 | 03 Oct 23 17:45 PDT |                     |

	|         | status crio --all --full                               |                        |         |         |                     |                     |

	|         | --no-pager                                             |                        |         |         |                     |                     |

	| ssh     | -p false-312000 sudo systemctl                         | false-312000           | jenkins | v1.31.2 | 03 Oct 23 17:45 PDT | 03 Oct 23 17:45 PDT |

	|         | cat crio --no-pager                                    |                        |         |         |                     |                     |

	| ssh     | -p false-312000 sudo find                              | false-312000           | jenkins | v1.31.2 | 03 Oct 23 17:45 PDT | 03 Oct 23 17:45 PDT |

	|         | /etc/crio -type f -exec sh -c                          |                        |         |         |                     |                     |

	|         | 'echo {}; cat {}' \;                                   |                        |         |         |                     |                     |

	| ssh     | -p false-312000 sudo crio                              | false-312000           | jenkins | v1.31.2 | 03 Oct 23 17:45 PDT | 03 Oct 23 17:45 PDT |

	|         | config                                                 |                        |         |         |                     |                     |

	| delete  | -p false-312000                                        | false-312000           | jenkins | v1.31.2 | 03 Oct 23 17:45 PDT | 03 Oct 23 17:45 PDT |

	| start   | -p no-preload-232000                                   | no-preload-232000      | jenkins | v1.31.2 | 03 Oct 23 17:45 PDT | 03 Oct 23 17:46 PDT |

	|         | --memory=2200                                          |                        |         |         |                     |                     |

	|         | --alsologtostderr                                      |                        |         |         |                     |                     |

	|         | --wait=true --preload=false                            |                        |         |         |                     |                     |

	|         | --driver=hyperkit                                      |                        |         |         |                     |                     |

	|         | --kubernetes-version=v1.28.2                           |                        |         |         |                     |                     |

	| addons  | enable metrics-server -p no-preload-232000             | no-preload-232000      | jenkins | v1.31.2 | 03 Oct 23 17:46 PDT | 03 Oct 23 17:46 PDT |

	|         | --images=MetricsServer=registry.k8s.io/echoserver:1.4  |                        |         |         |                     |                     |

	|         | --registries=MetricsServer=fake.domain                 |                        |         |         |                     |                     |

	| stop    | -p no-preload-232000                                   | no-preload-232000      | jenkins | v1.31.2 | 03 Oct 23 17:46 PDT | 03 Oct 23 17:46 PDT |

	|         | --alsologtostderr -v=3                                 |                        |         |         |                     |                     |

	| addons  | enable dashboard -p no-preload-232000                  | no-preload-232000      | jenkins | v1.31.2 | 03 Oct 23 17:46 PDT | 03 Oct 23 17:46 PDT |

	|         | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 |                        |         |         |                     |                     |

	| start   | -p no-preload-232000                                   | no-preload-232000      | jenkins | v1.31.2 | 03 Oct 23 17:46 PDT | 03 Oct 23 17:51 PDT |

	|         | --memory=2200                                          |                        |         |         |                     |                     |

	|         | --alsologtostderr                                      |                        |         |         |                     |                     |

	|         | --wait=true --preload=false                            |                        |         |         |                     |                     |

	|         | --driver=hyperkit                                      |                        |         |         |                     |                     |

	|         | --kubernetes-version=v1.28.2                           |                        |         |         |                     |                     |

	| addons  | enable metrics-server -p old-k8s-version-999000        | old-k8s-version-999000 | jenkins | v1.31.2 | 03 Oct 23 17:47 PDT | 03 Oct 23 17:47 PDT |

	|         | --images=MetricsServer=registry.k8s.io/echoserver:1.4  |                        |         |         |                     |                     |

	|         | --registries=MetricsServer=fake.domain                 |                        |         |         |                     |                     |

	| stop    | -p old-k8s-version-999000                              | old-k8s-version-999000 | jenkins | v1.31.2 | 03 Oct 23 17:47 PDT | 03 Oct 23 17:47 PDT |

	|         | --alsologtostderr -v=3                                 |                        |         |         |                     |                     |

	| addons  | enable dashboard -p old-k8s-version-999000             | old-k8s-version-999000 | jenkins | v1.31.2 | 03 Oct 23 17:47 PDT | 03 Oct 23 17:47 PDT |

	|         | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 |                        |         |         |                     |                     |

	| start   | -p old-k8s-version-999000                              | old-k8s-version-999000 | jenkins | v1.31.2 | 03 Oct 23 17:47 PDT | 03 Oct 23 17:55 PDT |

	|         | --memory=2200                                          |                        |         |         |                     |                     |

	|         | --alsologtostderr --wait=true                          |                        |         |         |                     |                     |

	|         | --kvm-network=default                                  |                        |         |         |                     |                     |

	|         | --kvm-qemu-uri=qemu:///system                          |                        |         |         |                     |                     |

	|         | --disable-driver-mounts                                |                        |         |         |                     |                     |

	|         | --keep-context=false                                   |                        |         |         |                     |                     |

	|         | --driver=hyperkit                                      |                        |         |         |                     |                     |

	|         | --kubernetes-version=v1.16.0                           |                        |         |         |                     |                     |

	| ssh     | -p no-preload-232000 sudo                              | no-preload-232000      | jenkins | v1.31.2 | 03 Oct 23 17:51 PDT | 03 Oct 23 17:51 PDT |

	|         | crictl images -o json                                  |                        |         |         |                     |                     |

	| pause   | -p no-preload-232000                                   | no-preload-232000      | jenkins | v1.31.2 | 03 Oct 23 17:51 PDT | 03 Oct 23 17:51 PDT |

	|         | --alsologtostderr -v=1                                 |                        |         |         |                     |                     |

	| unpause | -p no-preload-232000                                   | no-preload-232000      | jenkins | v1.31.2 | 03 Oct 23 17:51 PDT | 03 Oct 23 17:51 PDT |

	|         | --alsologtostderr -v=1                                 |                        |         |         |                     |                     |

	| delete  | -p no-preload-232000                                   | no-preload-232000      | jenkins | v1.31.2 | 03 Oct 23 17:51 PDT | 03 Oct 23 17:51 PDT |

	| delete  | -p no-preload-232000                                   | no-preload-232000      | jenkins | v1.31.2 | 03 Oct 23 17:51 PDT | 03 Oct 23 17:51 PDT |

	| start   | -p embed-certs-266000                                  | embed-certs-266000     | jenkins | v1.31.2 | 03 Oct 23 17:51 PDT | 03 Oct 23 17:52 PDT |

	|         | --memory=2200                                          |                        |         |         |                     |                     |

	|         | --alsologtostderr                                      |                        |         |         |                     |                     |

	|         | --wait=true --embed-certs                              |                        |         |         |                     |                     |

	|         | --driver=hyperkit                                      |                        |         |         |                     |                     |

	|         | --kubernetes-version=v1.28.2                           |                        |         |         |                     |                     |

	| addons  | enable metrics-server -p embed-certs-266000            | embed-certs-266000     | jenkins | v1.31.2 | 03 Oct 23 17:52 PDT | 03 Oct 23 17:52 PDT |

	|         | --images=MetricsServer=registry.k8s.io/echoserver:1.4  |                        |         |         |                     |                     |

	|         | --registries=MetricsServer=fake.domain                 |                        |         |         |                     |                     |

	| stop    | -p embed-certs-266000                                  | embed-certs-266000     | jenkins | v1.31.2 | 03 Oct 23 17:52 PDT | 03 Oct 23 17:52 PDT |

	|         | --alsologtostderr -v=3                                 |                        |         |         |                     |                     |

	| addons  | enable dashboard -p embed-certs-266000                 | embed-certs-266000     | jenkins | v1.31.2 | 03 Oct 23 17:52 PDT | 03 Oct 23 17:52 PDT |

	|         | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 |                        |         |         |                     |                     |

	| start   | -p embed-certs-266000                                  | embed-certs-266000     | jenkins | v1.31.2 | 03 Oct 23 17:52 PDT |                     |

	|         | --memory=2200                                          |                        |         |         |                     |                     |

	|         | --alsologtostderr                                      |                        |         |         |                     |                     |

	|         | --wait=true --embed-certs                              |                        |         |         |                     |                     |

	|         | --driver=hyperkit                                      |                        |         |         |                     |                     |

	|         | --kubernetes-version=v1.28.2                           |                        |         |         |                     |                     |

	| ssh     | -p old-k8s-version-999000 sudo                         | old-k8s-version-999000 | jenkins | v1.31.2 | 03 Oct 23 17:55 PDT |                     |

	|         | crictl images -o json                                  |                        |         |         |                     |                     |

	|---------|--------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|

	* 

	* ==> Last Start <==

	* Log file created at: 2023/10/03 17:52:58

	Running on machine: MacOS-Agent-1

	Binary: Built with gc go1.21.1 for darwin/amd64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I1003 17:52:58.285294    8925 out.go:296] Setting OutFile to fd 1 ...

	I1003 17:52:58.285584    8925 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:52:58.285589    8925 out.go:309] Setting ErrFile to fd 2...

	I1003 17:52:58.285593    8925 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:52:58.285769    8925 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17345-936/.minikube/bin

	I1003 17:52:58.287237    8925 out.go:303] Setting JSON to false

	I1003 17:52:58.309205    8925 start.go:128] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":3146,"bootTime":1696377632,"procs":444,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.0","kernelVersion":"23.0.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}

	W1003 17:52:58.309316    8925 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I1003 17:52:58.331776    8925 out.go:177] * [embed-certs-266000] minikube v1.31.2 on Darwin 14.0

	I1003 17:52:55.679373    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:52:58.179619    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:52:58.353282    8925 notify.go:220] Checking for updates...

	I1003 17:52:58.373975    8925 out.go:177]   - MINIKUBE_LOCATION=17345

	I1003 17:52:58.438938    8925 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17345-936/kubeconfig

	I1003 17:52:58.460348    8925 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-amd64

	I1003 17:52:58.482180    8925 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I1003 17:52:58.503168    8925 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17345-936/.minikube

	I1003 17:52:58.524351    8925 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I1003 17:52:58.546590    8925 config.go:182] Loaded profile config "embed-certs-266000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I1003 17:52:58.546970    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:52:58.547025    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:52:58.555188    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56172

	I1003 17:52:58.555544    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:52:58.555990    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:52:58.556001    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:52:58.556212    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:52:58.556308    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:52:58.556495    8925 driver.go:373] Setting default libvirt URI to qemu:///system

	I1003 17:52:58.556734    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:52:58.556757    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:52:58.564382    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56174

	I1003 17:52:58.564704    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:52:58.565074    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:52:58.565094    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:52:58.565290    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:52:58.565387    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:52:58.594060    8925 out.go:177] * Using the hyperkit driver based on existing profile

	I1003 17:52:58.636328    8925 start.go:298] selected driver: hyperkit

	I1003 17:52:58.636357    8925 start.go:902] validating driver "hyperkit" against &{Name:embed-certs-266000 KeepContext:false EmbedCerts:true MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17240/minikube-v1.31.0-1695060926-17240-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1696360059-17345@sha256:76d99edd1576614d5c20a839dd16ae1d7c810f3b909a01797063d483159ea3ae Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kubernet

esConfig:{KubernetesVersion:v1.28.2 ClusterName:embed-certs-266000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.89.28 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedP

orts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I1003 17:52:58.636558    8925 start.go:913] status for hyperkit: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I1003 17:52:58.640769    8925 install.go:52] acquiring lock: {Name:mk4023283b30b374c3f04c8805d539e68824c0b8 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I1003 17:52:58.640866    8925 install.go:117] Validating docker-machine-driver-hyperkit, PATH=/Users/jenkins/minikube-integration/17345-936/.minikube/bin:/Users/jenkins/workspace/out/:/usr/bin:/bin:/usr/sbin:/sbin:/Users/jenkins/google-cloud-sdk/bin:/usr/local/bin/:/usr/local/go/bin/:/Users/jenkins/go/bin

	I1003 17:52:58.648522    8925 install.go:137] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit version is 1.31.2

	I1003 17:52:58.652348    8925 install.go:79] stdout: /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:52:58.652373    8925 install.go:81] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit looks good

	I1003 17:52:58.652514    8925 start_flags.go:923] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I1003 17:52:58.652549    8925 cni.go:84] Creating CNI manager for ""

	I1003 17:52:58.652561    8925 cni.go:158] "hyperkit" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I1003 17:52:58.652571    8925 start_flags.go:321] config:

	{Name:embed-certs-266000 KeepContext:false EmbedCerts:true MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17240/minikube-v1.31.0-1695060926-17240-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1696360059-17345@sha256:76d99edd1576614d5c20a839dd16ae1d7c810f3b909a01797063d483159ea3ae Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:embed-certs-266000 Namespace:defau

lt APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.89.28 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 Cert

Expiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I1003 17:52:58.652720    8925 iso.go:125] acquiring lock: {Name:mk09a17bc6b7923952debc9d2a1712efcb7ec756 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I1003 17:52:58.711285    8925 out.go:177] * Starting control plane node embed-certs-266000 in cluster embed-certs-266000

	I1003 17:52:58.733261    8925 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I1003 17:52:58.733359    8925 preload.go:148] Found local preload: /Users/jenkins/minikube-integration/17345-936/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-amd64.tar.lz4

	I1003 17:52:58.733387    8925 cache.go:57] Caching tarball of preloaded images

	I1003 17:52:58.733593    8925 preload.go:174] Found /Users/jenkins/minikube-integration/17345-936/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download

	I1003 17:52:58.733617    8925 cache.go:60] Finished verifying existence of preloaded tar for  v1.28.2 on docker

	I1003 17:52:58.733803    8925 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/embed-certs-266000/config.json ...

	I1003 17:52:58.734893    8925 start.go:365] acquiring machines lock for embed-certs-266000: {Name:mk2907765d495f362bb545719659eca626cfc4d0 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}

	I1003 17:52:58.735006    8925 start.go:369] acquired machines lock for "embed-certs-266000" in 88.096µs

	I1003 17:52:58.735040    8925 start.go:96] Skipping create...Using existing machine configuration

	I1003 17:52:58.735068    8925 fix.go:54] fixHost starting: 

	I1003 17:52:58.735466    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:52:58.735495    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:52:58.743854    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56176

	I1003 17:52:58.744194    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:52:58.744528    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:52:58.744556    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:52:58.744792    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:52:58.744910    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:52:58.745015    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetState

	I1003 17:52:58.745100    8925 main.go:141] libmachine: (embed-certs-266000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:52:58.745169    8925 main.go:141] libmachine: (embed-certs-266000) DBG | hyperkit pid from json: 8877

	I1003 17:52:58.746127    8925 main.go:141] libmachine: (embed-certs-266000) DBG | hyperkit pid 8877 missing from process table

	I1003 17:52:58.746191    8925 fix.go:102] recreateIfNeeded on embed-certs-266000: state=Stopped err=<nil>

	I1003 17:52:58.746216    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	W1003 17:52:58.746310    8925 fix.go:128] unexpected machine state, will restart: <nil>

	I1003 17:52:58.789113    8925 out.go:177] * Restarting existing hyperkit VM for "embed-certs-266000" ...

	I1003 17:52:58.812109    8925 main.go:141] libmachine: (embed-certs-266000) Calling .Start

	I1003 17:52:58.812293    8925 main.go:141] libmachine: (embed-certs-266000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:52:58.812335    8925 main.go:141] libmachine: (embed-certs-266000) minikube might have been shutdown in an unclean way, the hyperkit pid file still exists: /Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/hyperkit.pid

	I1003 17:52:58.813355    8925 main.go:141] libmachine: (embed-certs-266000) DBG | hyperkit pid 8877 missing from process table

	I1003 17:52:58.813369    8925 main.go:141] libmachine: (embed-certs-266000) DBG | pid 8877 is in state "Stopped"

	I1003 17:52:58.813381    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Removing stale pid file /Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/hyperkit.pid...

	I1003 17:52:58.813499    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Using UUID 34969534-6250-11ee-97fb-149d997cd0f1

	I1003 17:52:58.829749    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Generated MAC 9e:e8:4f:d4:fd:22

	I1003 17:52:58.829772    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Starting with cmdline: loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=embed-certs-266000

	I1003 17:52:58.829955    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 DEBUG: hyperkit: Start &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"34969534-6250-11ee-97fb-149d997cd0f1", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc0003e91d0)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/bzimage", Initrd:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os

.Process)(nil)}

	I1003 17:52:58.829988    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 DEBUG: hyperkit: check &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"34969534-6250-11ee-97fb-149d997cd0f1", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc0003e91d0)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/bzimage", Initrd:"/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os

.Process)(nil)}

	I1003 17:52:58.830044    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 DEBUG: hyperkit: Arguments: []string{"-A", "-u", "-F", "/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/hyperkit.pid", "-c", "2", "-m", "2200M", "-s", "0:0,hostbridge", "-s", "31,lpc", "-s", "1:0,virtio-net", "-U", "34969534-6250-11ee-97fb-149d997cd0f1", "-s", "2:0,virtio-blk,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/embed-certs-266000.rawdisk", "-s", "3,ahci-cd,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/boot2docker.iso", "-s", "4,virtio-rnd", "-l", "com1,autopty=/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/tty,log=/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/console-ring", "-f", "kexec,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/bzimage,/Users/jenkins/minikube-integr

ation/17345-936/.minikube/machines/embed-certs-266000/initrd,earlyprintk=serial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=embed-certs-266000"}

	I1003 17:52:58.830097    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 DEBUG: hyperkit: CmdLine: "/usr/local/bin/hyperkit -A -u -F /Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/hyperkit.pid -c 2 -m 2200M -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-net -U 34969534-6250-11ee-97fb-149d997cd0f1 -s 2:0,virtio-blk,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/embed-certs-266000.rawdisk -s 3,ahci-cd,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/boot2docker.iso -s 4,virtio-rnd -l com1,autopty=/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/tty,log=/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/console-ring -f kexec,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/bzimage,/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/initrd,e

arlyprintk=serial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=embed-certs-266000"

	I1003 17:52:58.830113    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 DEBUG: hyperkit: Redirecting stdout/stderr to logger

	I1003 17:52:58.831685    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 DEBUG: hyperkit: Pid is 8936

	I1003 17:52:58.832296    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Attempt 0

	I1003 17:52:58.832333    8925 main.go:141] libmachine: (embed-certs-266000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:52:58.832390    8925 main.go:141] libmachine: (embed-certs-266000) DBG | hyperkit pid from json: 8936

	I1003 17:52:58.834070    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Searching for 9e:e8:4f:d4:fd:22 in /var/db/dhcpd_leases ...

	I1003 17:52:58.834206    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Found 43 entries in /var/db/dhcpd_leases!

	I1003 17:52:58.834223    8925 main.go:141] libmachine: (embed-certs-266000) DBG | dhcp entry: {Name:minikube IPAddress:192.168.89.28 HWAddress:9e:e8:4f:d4:fd:22 ID:1,9e:e8:4f:d4:fd:22 Lease:0x651e08b0}

	I1003 17:52:58.834268    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Found match: 9e:e8:4f:d4:fd:22

	I1003 17:52:58.834286    8925 main.go:141] libmachine: (embed-certs-266000) DBG | IP: 192.168.89.28

	I1003 17:52:58.834362    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetConfigRaw

	I1003 17:52:58.835044    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetIP

	I1003 17:52:58.835237    8925 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/embed-certs-266000/config.json ...

	I1003 17:52:58.835617    8925 machine.go:88] provisioning docker machine ...

	I1003 17:52:58.835628    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:52:58.835751    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetMachineName

	I1003 17:52:58.835859    8925 buildroot.go:166] provisioning hostname "embed-certs-266000"

	I1003 17:52:58.835870    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetMachineName

	I1003 17:52:58.835982    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:52:58.836079    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:52:58.836164    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:52:58.836256    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:52:58.836349    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:52:58.836498    8925 main.go:141] libmachine: Using SSH client type: native

	I1003 17:52:58.836817    8925 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.89.28 22 <nil> <nil>}

	I1003 17:52:58.836830    8925 main.go:141] libmachine: About to run SSH command:

	sudo hostname embed-certs-266000 && echo "embed-certs-266000" | sudo tee /etc/hostname

	I1003 17:52:58.839740    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 INFO : hyperkit: stderr: Using fd 5 for I/O notifications

	I1003 17:52:58.847962    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 INFO : hyperkit: stderr: /Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/boot2docker.iso: fcntl(F_PUNCHHOLE) Operation not permitted: block device will not support TRIM/DISCARD

	I1003 17:52:58.848915    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0

	I1003 17:52:58.848941    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0

	I1003 17:52:58.848958    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0

	I1003 17:52:58.848968    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:58 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0

	I1003 17:52:59.218064    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:59 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 0

	I1003 17:52:59.218079    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:59 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 0

	I1003 17:52:59.322108    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:59 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0

	I1003 17:52:59.322140    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:59 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0

	I1003 17:52:59.322166    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:59 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0

	I1003 17:52:59.322186    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:59 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0

	I1003 17:52:59.322958    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:59 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 1

	I1003 17:52:59.322968    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:52:59 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 1

	I1003 17:53:00.676821    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:02.678005    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:04.244926    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:53:04 INFO : hyperkit: stderr: rdmsr to register 0x64d on vcpu 1

	I1003 17:53:04.244951    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:53:04 INFO : hyperkit: stderr: rdmsr to register 0x64e on vcpu 1

	I1003 17:53:04.244960    8925 main.go:141] libmachine: (embed-certs-266000) DBG | 2023/10/03 17:53:04 INFO : hyperkit: stderr: rdmsr to register 0x34 on vcpu 1

	I1003 17:53:05.177994    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:07.678238    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:12.019197    8925 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-266000

	I1003 17:53:12.019224    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:12.019383    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:12.019478    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:12.019552    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:12.019637    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:12.019771    8925 main.go:141] libmachine: Using SSH client type: native

	I1003 17:53:12.020030    8925 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.89.28 22 <nil> <nil>}

	I1003 17:53:12.020043    8925 main.go:141] libmachine: About to run SSH command:

			if ! grep -xq '.*\sembed-certs-266000' /etc/hosts; then

				if grep -xq '127.0.1.1\s.*' /etc/hosts; then

					sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 embed-certs-266000/g' /etc/hosts;

				else 

					echo '127.0.1.1 embed-certs-266000' | sudo tee -a /etc/hosts; 

				fi

			fi

	I1003 17:53:12.090792    8925 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	I1003 17:53:12.090811    8925 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/17345-936/.minikube CaCertPath:/Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/17345-936/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/17345-936/.minikube}

	I1003 17:53:12.090831    8925 buildroot.go:174] setting up certificates

	I1003 17:53:12.090842    8925 provision.go:83] configureAuth start

	I1003 17:53:12.090852    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetMachineName

	I1003 17:53:12.090994    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetIP

	I1003 17:53:12.091090    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:12.091173    8925 provision.go:138] copyHostCerts

	I1003 17:53:12.091256    8925 exec_runner.go:144] found /Users/jenkins/minikube-integration/17345-936/.minikube/ca.pem, removing ...

	I1003 17:53:12.091267    8925 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17345-936/.minikube/ca.pem

	I1003 17:53:12.091405    8925 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/17345-936/.minikube/ca.pem (1078 bytes)

	I1003 17:53:12.091638    8925 exec_runner.go:144] found /Users/jenkins/minikube-integration/17345-936/.minikube/cert.pem, removing ...

	I1003 17:53:12.091645    8925 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17345-936/.minikube/cert.pem

	I1003 17:53:12.091724    8925 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/17345-936/.minikube/cert.pem (1123 bytes)

	I1003 17:53:12.091884    8925 exec_runner.go:144] found /Users/jenkins/minikube-integration/17345-936/.minikube/key.pem, removing ...

	I1003 17:53:12.091896    8925 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/17345-936/.minikube/key.pem

	I1003 17:53:12.091971    8925 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/17345-936/.minikube/key.pem (1675 bytes)

	I1003 17:53:12.092114    8925 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/17345-936/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca-key.pem org=jenkins.embed-certs-266000 san=[192.168.89.28 192.168.89.28 localhost 127.0.0.1 minikube embed-certs-266000]

	I1003 17:53:12.170400    8925 provision.go:172] copyRemoteCerts

	I1003 17:53:12.170496    8925 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker

	I1003 17:53:12.170519    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:12.170711    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:12.170869    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:12.170980    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:12.171119    8925 sshutil.go:53] new ssh client: &{IP:192.168.89.28 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/id_rsa Username:docker}

	I1003 17:53:12.211813    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)

	I1003 17:53:12.227397    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)

	I1003 17:53:12.242956    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)

	I1003 17:53:12.258530    8925 provision.go:86] duration metric: configureAuth took 167.671868ms

	I1003 17:53:12.258543    8925 buildroot.go:189] setting minikube options for container-runtime

	I1003 17:53:12.258680    8925 config.go:182] Loaded profile config "embed-certs-266000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I1003 17:53:12.258695    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:53:12.258831    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:12.258925    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:12.259016    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:12.259087    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:12.259171    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:12.259288    8925 main.go:141] libmachine: Using SSH client type: native

	I1003 17:53:12.259542    8925 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.89.28 22 <nil> <nil>}

	I1003 17:53:12.259550    8925 main.go:141] libmachine: About to run SSH command:

	df --output=fstype / | tail -n 1

	I1003 17:53:12.327484    8925 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs

	I1003 17:53:12.327496    8925 buildroot.go:70] root file system type: tmpfs

	I1003 17:53:12.327575    8925 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...

	I1003 17:53:12.327590    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:12.327738    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:12.327836    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:12.327920    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:12.328015    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:12.328153    8925 main.go:141] libmachine: Using SSH client type: native

	I1003 17:53:12.328419    8925 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.89.28 22 <nil> <nil>}

	I1003 17:53:12.328468    8925 main.go:141] libmachine: About to run SSH command:

	sudo mkdir -p /lib/systemd/system && printf %!s(MISSING) "[Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP \$MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	" | sudo tee /lib/systemd/system/docker.service.new

	I1003 17:53:12.408203    8925 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]

	Description=Docker Application Container Engine

	Documentation=https://docs.docker.com

	After=network.target  minikube-automount.service docker.socket

	Requires= minikube-automount.service docker.socket 

	StartLimitBurst=3

	StartLimitIntervalSec=60

	[Service]

	Type=notify

	Restart=on-failure

	# This file is a systemd drop-in unit that inherits from the base dockerd configuration.

	# The base configuration already specifies an 'ExecStart=...' command. The first directive

	# here is to clear out that command inherited from the base configuration. Without this,

	# the command from the base configuration and the command specified here are treated as

	# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd

	# will catch this invalid input and refuse to start the service with an error like:

	#  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.

	# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other

	# container runtimes. If left unlimited, it may result in OOM issues with MySQL.

	ExecStart=

	ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12 

	ExecReload=/bin/kill -s HUP $MAINPID

	# Having non-zero Limit*s causes performance problems due to accounting overhead

	# in the kernel. We recommend using cgroups to do container-local accounting.

	LimitNOFILE=infinity

	LimitNPROC=infinity

	LimitCORE=infinity

	# Uncomment TasksMax if your systemd version supports it.

	# Only systemd 226 and above support this version.

	TasksMax=infinity

	TimeoutStartSec=0

	# set delegate yes so that systemd does not reset the cgroups of docker containers

	Delegate=yes

	# kill only the docker process, not all processes in the cgroup

	KillMode=process

	[Install]

	WantedBy=multi-user.target

	I1003 17:53:12.408228    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:12.408358    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:12.408460    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:12.408548    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:12.408643    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:12.408782    8925 main.go:141] libmachine: Using SSH client type: native

	I1003 17:53:12.409045    8925 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.89.28 22 <nil> <nil>}

	I1003 17:53:12.409059    8925 main.go:141] libmachine: About to run SSH command:

	sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }

	I1003 17:53:13.035110    8925 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory

	Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.

	I1003 17:53:13.035146    8925 machine.go:91] provisioned docker machine in 14.199042093s

	I1003 17:53:13.035178    8925 start.go:300] post-start starting for "embed-certs-266000" (driver="hyperkit")

	I1003 17:53:13.035214    8925 start.go:329] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]

	I1003 17:53:13.035224    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:53:13.035571    8925 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs

	I1003 17:53:13.035585    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:13.035768    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:13.035886    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:13.036028    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:13.036167    8925 sshutil.go:53] new ssh client: &{IP:192.168.89.28 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/id_rsa Username:docker}

	I1003 17:53:13.075274    8925 ssh_runner.go:195] Run: cat /etc/os-release

	I1003 17:53:13.077876    8925 info.go:137] Remote host: Buildroot 2021.02.12

	I1003 17:53:13.077889    8925 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17345-936/.minikube/addons for local assets ...

	I1003 17:53:13.077984    8925 filesync.go:126] Scanning /Users/jenkins/minikube-integration/17345-936/.minikube/files for local assets ...

	I1003 17:53:13.078151    8925 filesync.go:149] local asset: /Users/jenkins/minikube-integration/17345-936/.minikube/files/etc/ssl/certs/14342.pem -> 14342.pem in /etc/ssl/certs

	I1003 17:53:13.078329    8925 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs

	I1003 17:53:13.084794    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/files/etc/ssl/certs/14342.pem --> /etc/ssl/certs/14342.pem (1708 bytes)

	I1003 17:53:13.100111    8925 start.go:303] post-start completed in 64.917388ms

	I1003 17:53:13.100126    8925 fix.go:56] fixHost completed within 14.364592267s

	I1003 17:53:13.100143    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:13.100275    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:13.100360    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:13.100444    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:13.100537    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:13.100646    8925 main.go:141] libmachine: Using SSH client type: native

	I1003 17:53:13.100893    8925 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x13f3fc0] 0x13f6ca0 <nil>  [] 0s} 192.168.89.28 22 <nil> <nil>}

	I1003 17:53:13.100902    8925 main.go:141] libmachine: About to run SSH command:

	date +%!s(MISSING).%!N(MISSING)

	I1003 17:53:13.167389    8925 main.go:141] libmachine: SSH cmd err, output: <nil>: 1696380793.083787048

	I1003 17:53:13.167400    8925 fix.go:206] guest clock: 1696380793.083787048

	I1003 17:53:13.167405    8925 fix.go:219] Guest: 2023-10-03 17:53:13.083787048 -0700 PDT Remote: 2023-10-03 17:53:13.100131 -0700 PDT m=+14.846205768 (delta=-16.343952ms)

	I1003 17:53:13.167421    8925 fix.go:190] guest clock delta is within tolerance: -16.343952ms

	I1003 17:53:13.167425    8925 start.go:83] releasing machines lock for "embed-certs-266000", held for 14.43192049s

	I1003 17:53:13.167444    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:53:13.167571    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetIP

	I1003 17:53:13.167675    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:53:13.167975    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:53:13.168071    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:53:13.168169    8925 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/

	I1003 17:53:13.168200    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:13.168221    8925 ssh_runner.go:195] Run: cat /version.json

	I1003 17:53:13.168231    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:13.168331    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:13.168350    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:13.168473    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:13.168486    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:13.168572    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:13.168593    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:13.168673    8925 sshutil.go:53] new ssh client: &{IP:192.168.89.28 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/id_rsa Username:docker}

	I1003 17:53:13.168723    8925 sshutil.go:53] new ssh client: &{IP:192.168.89.28 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/id_rsa Username:docker}

	I1003 17:53:13.259614    8925 ssh_runner.go:195] Run: systemctl --version

	I1003 17:53:13.263830    8925 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"

	W1003 17:53:13.267223    8925 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found

	I1003 17:53:13.267269    8925 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;

	I1003 17:53:13.276563    8925 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)

	I1003 17:53:13.276573    8925 start.go:469] detecting cgroup driver to use...

	I1003 17:53:13.276677    8925 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock

	" | sudo tee /etc/crictl.yaml"

	I1003 17:53:09.679266    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:11.679679    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:13.288362    8925 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"

	I1003 17:53:13.319283    8925 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"

	I1003 17:53:13.325690    8925 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...

	I1003 17:53:13.325733    8925 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"

	I1003 17:53:13.332163    8925 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I1003 17:53:13.338567    8925 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"

	I1003 17:53:13.344933    8925 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"

	I1003 17:53:13.351296    8925 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"

	I1003 17:53:13.357818    8925 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"

	I1003 17:53:13.364231    8925 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables

	I1003 17:53:13.369977    8925 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

	I1003 17:53:13.375921    8925 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I1003 17:53:13.457351    8925 ssh_runner.go:195] Run: sudo systemctl restart containerd

	I1003 17:53:13.469009    8925 start.go:469] detecting cgroup driver to use...

	I1003 17:53:13.469079    8925 ssh_runner.go:195] Run: sudo systemctl cat docker.service

	I1003 17:53:13.479522    8925 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I1003 17:53:13.489534    8925 ssh_runner.go:195] Run: sudo systemctl stop -f containerd

	I1003 17:53:13.501344    8925 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I1003 17:53:13.509753    8925 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I1003 17:53:13.518150    8925 ssh_runner.go:195] Run: sudo systemctl stop -f crio

	I1003 17:53:13.543741    8925 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio

	I1003 17:53:13.552622    8925 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock

	" | sudo tee /etc/crictl.yaml"

	I1003 17:53:13.563917    8925 ssh_runner.go:195] Run: which cri-dockerd

	I1003 17:53:13.566308    8925 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d

	I1003 17:53:13.572541    8925 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)

	I1003 17:53:13.583633    8925 ssh_runner.go:195] Run: sudo systemctl unmask docker.service

	I1003 17:53:13.669185    8925 ssh_runner.go:195] Run: sudo systemctl enable docker.socket

	I1003 17:53:13.754264    8925 docker.go:554] configuring docker to use "cgroupfs" as cgroup driver...

	I1003 17:53:13.754437    8925 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)

	I1003 17:53:13.766215    8925 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I1003 17:53:13.851844    8925 ssh_runner.go:195] Run: sudo systemctl restart docker

	I1003 17:53:15.120737    8925 ssh_runner.go:235] Completed: sudo systemctl restart docker: (1.268847855s)

	I1003 17:53:15.120797    8925 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I1003 17:53:15.200674    8925 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket

	I1003 17:53:15.284633    8925 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket

	I1003 17:53:15.366708    8925 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I1003 17:53:15.463191    8925 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket

	I1003 17:53:15.475324    8925 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I1003 17:53:15.557799    8925 ssh_runner.go:195] Run: sudo systemctl restart cri-docker

	I1003 17:53:15.614303    8925 start.go:516] Will wait 60s for socket path /var/run/cri-dockerd.sock

	I1003 17:53:15.614392    8925 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock

	I1003 17:53:15.618825    8925 start.go:537] Will wait 60s for crictl version

	I1003 17:53:15.618870    8925 ssh_runner.go:195] Run: which crictl

	I1003 17:53:15.622128    8925 ssh_runner.go:195] Run: sudo /usr/bin/crictl version

	I1003 17:53:15.658045    8925 start.go:553] Version:  0.1.0

	RuntimeName:  docker

	RuntimeVersion:  24.0.6

	RuntimeApiVersion:  v1

	I1003 17:53:15.658123    8925 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I1003 17:53:15.675955    8925 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}

	I1003 17:53:15.717980    8925 out.go:204] * Preparing Kubernetes v1.28.2 on Docker 24.0.6 ...

	I1003 17:53:15.718030    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetIP

	I1003 17:53:15.718436    8925 ssh_runner.go:195] Run: grep 192.168.89.1	host.minikube.internal$ /etc/hosts

	I1003 17:53:15.722744    8925 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.89.1	host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I1003 17:53:15.730526    8925 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I1003 17:53:15.730586    8925 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I1003 17:53:15.743741    8925 docker.go:664] Got preloaded images: -- stdout --

	registry.k8s.io/kube-apiserver:v1.28.2

	registry.k8s.io/kube-controller-manager:v1.28.2

	registry.k8s.io/kube-scheduler:v1.28.2

	registry.k8s.io/kube-proxy:v1.28.2

	registry.k8s.io/etcd:3.5.9-0

	registry.k8s.io/coredns/coredns:v1.10.1

	registry.k8s.io/pause:3.9

	gcr.io/k8s-minikube/storage-provisioner:v5

	gcr.io/k8s-minikube/busybox:1.28.4-glibc

	-- /stdout --

	I1003 17:53:15.743765    8925 docker.go:594] Images already preloaded, skipping extraction

	I1003 17:53:15.743838    8925 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}

	I1003 17:53:15.756782    8925 docker.go:664] Got preloaded images: -- stdout --

	registry.k8s.io/kube-apiserver:v1.28.2

	registry.k8s.io/kube-controller-manager:v1.28.2

	registry.k8s.io/kube-scheduler:v1.28.2

	registry.k8s.io/kube-proxy:v1.28.2

	registry.k8s.io/etcd:3.5.9-0

	registry.k8s.io/coredns/coredns:v1.10.1

	registry.k8s.io/pause:3.9

	gcr.io/k8s-minikube/storage-provisioner:v5

	gcr.io/k8s-minikube/busybox:1.28.4-glibc

	-- /stdout --

	I1003 17:53:15.756804    8925 cache_images.go:84] Images are preloaded, skipping loading

	I1003 17:53:15.756885    8925 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}

	I1003 17:53:15.774029    8925 cni.go:84] Creating CNI manager for ""

	I1003 17:53:15.774044    8925 cni.go:158] "hyperkit" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I1003 17:53:15.774060    8925 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16

	I1003 17:53:15.774083    8925 kubeadm.go:176] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.89.28 APIServerPort:8443 KubernetesVersion:v1.28.2 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:embed-certs-266000 NodeName:embed-certs-266000 DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.89.28"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.89.28 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPa

th:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}

	I1003 17:53:15.774163    8925 kubeadm.go:181] kubeadm config:

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: InitConfiguration

	localAPIEndpoint:

	  advertiseAddress: 192.168.89.28

	  bindPort: 8443

	bootstrapTokens:

	  - groups:

	      - system:bootstrappers:kubeadm:default-node-token

	    ttl: 24h0m0s

	    usages:

	      - signing

	      - authentication

	nodeRegistration:

	  criSocket: unix:///var/run/cri-dockerd.sock

	  name: "embed-certs-266000"

	  kubeletExtraArgs:

	    node-ip: 192.168.89.28

	  taints: []

	---

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: ClusterConfiguration

	apiServer:

	  certSANs: ["127.0.0.1", "localhost", "192.168.89.28"]

	  extraArgs:

	    enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"

	controllerManager:

	  extraArgs:

	    allocate-node-cidrs: "true"

	    leader-elect: "false"

	scheduler:

	  extraArgs:

	    leader-elect: "false"

	certificatesDir: /var/lib/minikube/certs

	clusterName: mk

	controlPlaneEndpoint: control-plane.minikube.internal:8443

	etcd:

	  local:

	    dataDir: /var/lib/minikube/etcd

	    extraArgs:

	      proxy-refresh-interval: "70000"

	kubernetesVersion: v1.28.2

	networking:

	  dnsDomain: cluster.local

	  podSubnet: "10.244.0.0/16"

	  serviceSubnet: 10.96.0.0/12

	---

	apiVersion: kubelet.config.k8s.io/v1beta1

	kind: KubeletConfiguration

	authentication:

	  x509:

	    clientCAFile: /var/lib/minikube/certs/ca.crt

	cgroupDriver: cgroupfs

	hairpinMode: hairpin-veth

	runtimeRequestTimeout: 15m

	clusterDomain: "cluster.local"

	# disable disk resource management by default

	imageGCHighThresholdPercent: 100

	evictionHard:

	  nodefs.available: "0%!"(MISSING)

	  nodefs.inodesFree: "0%!"(MISSING)

	  imagefs.available: "0%!"(MISSING)

	failSwapOn: false

	staticPodPath: /etc/kubernetes/manifests

	---

	apiVersion: kubeproxy.config.k8s.io/v1alpha1

	kind: KubeProxyConfiguration

	clusterCIDR: "10.244.0.0/16"

	metricsBindAddress: 0.0.0.0:10249

	conntrack:

	  maxPerCore: 0

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"

	  tcpEstablishedTimeout: 0s

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"

	  tcpCloseWaitTimeout: 0s

	I1003 17:53:15.774225    8925 kubeadm.go:976] kubelet [Unit]

	Wants=docker.socket

	[Service]

	ExecStart=

	ExecStart=/var/lib/minikube/binaries/v1.28.2/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --hostname-override=embed-certs-266000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.89.28

	[Install]

	 config:

	{KubernetesVersion:v1.28.2 ClusterName:embed-certs-266000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}

	I1003 17:53:15.774280    8925 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.2

	I1003 17:53:15.781142    8925 binaries.go:44] Found k8s binaries, skipping transfer

	I1003 17:53:15.781196    8925 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube

	I1003 17:53:15.787678    8925 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (380 bytes)

	I1003 17:53:15.799149    8925 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)

	I1003 17:53:15.810323    8925 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2104 bytes)

	I1003 17:53:15.821772    8925 ssh_runner.go:195] Run: grep 192.168.89.28	control-plane.minikube.internal$ /etc/hosts

	I1003 17:53:15.824134    8925 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.89.28	control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I1003 17:53:15.833386    8925 certs.go:56] Setting up /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/embed-certs-266000 for IP: 192.168.89.28

	I1003 17:53:15.833409    8925 certs.go:190] acquiring lock for shared ca certs: {Name:mkbe2d32823f10149630205b70f0f5863fe62868 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I1003 17:53:15.833564    8925 certs.go:199] skipping minikubeCA CA generation: /Users/jenkins/minikube-integration/17345-936/.minikube/ca.key

	I1003 17:53:15.833625    8925 certs.go:199] skipping proxyClientCA CA generation: /Users/jenkins/minikube-integration/17345-936/.minikube/proxy-client-ca.key

	I1003 17:53:15.833719    8925 certs.go:315] skipping minikube-user signed cert generation: /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/embed-certs-266000/client.key

	I1003 17:53:15.833784    8925 certs.go:315] skipping minikube signed cert generation: /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/embed-certs-266000/apiserver.key.72417ce3

	I1003 17:53:15.833840    8925 certs.go:315] skipping aggregator signed cert generation: /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/embed-certs-266000/proxy-client.key

	I1003 17:53:15.834031    8925 certs.go:437] found cert: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/Users/jenkins/minikube-integration/17345-936/.minikube/certs/1434.pem (1338 bytes)

	W1003 17:53:15.834072    8925 certs.go:433] ignoring /Users/jenkins/minikube-integration/17345-936/.minikube/certs/Users/jenkins/minikube-integration/17345-936/.minikube/certs/1434_empty.pem, impossibly tiny 0 bytes

	I1003 17:53:15.834081    8925 certs.go:437] found cert: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca-key.pem (1675 bytes)

	I1003 17:53:15.834109    8925 certs.go:437] found cert: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/Users/jenkins/minikube-integration/17345-936/.minikube/certs/ca.pem (1078 bytes)

	I1003 17:53:15.834141    8925 certs.go:437] found cert: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/Users/jenkins/minikube-integration/17345-936/.minikube/certs/cert.pem (1123 bytes)

	I1003 17:53:15.834174    8925 certs.go:437] found cert: /Users/jenkins/minikube-integration/17345-936/.minikube/certs/Users/jenkins/minikube-integration/17345-936/.minikube/certs/key.pem (1675 bytes)

	I1003 17:53:15.834241    8925 certs.go:437] found cert: /Users/jenkins/minikube-integration/17345-936/.minikube/files/etc/ssl/certs/Users/jenkins/minikube-integration/17345-936/.minikube/files/etc/ssl/certs/14342.pem (1708 bytes)

	I1003 17:53:15.834758    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/embed-certs-266000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)

	I1003 17:53:15.851204    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/embed-certs-266000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)

	I1003 17:53:15.867728    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/embed-certs-266000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)

	I1003 17:53:15.884682    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/embed-certs-266000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)

	I1003 17:53:15.900862    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)

	I1003 17:53:15.917454    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)

	I1003 17:53:15.933565    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)

	I1003 17:53:15.949456    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)

	I1003 17:53:15.966351    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/certs/1434.pem --> /usr/share/ca-certificates/1434.pem (1338 bytes)

	I1003 17:53:15.982771    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/files/etc/ssl/certs/14342.pem --> /usr/share/ca-certificates/14342.pem (1708 bytes)

	I1003 17:53:15.998858    8925 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/17345-936/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)

	I1003 17:53:16.015002    8925 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)

	I1003 17:53:16.026386    8925 ssh_runner.go:195] Run: openssl version

	I1003 17:53:16.030058    8925 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1434.pem && ln -fs /usr/share/ca-certificates/1434.pem /etc/ssl/certs/1434.pem"

	I1003 17:53:16.036637    8925 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1434.pem

	I1003 17:53:16.039516    8925 certs.go:480] hashing: -rw-r--r-- 1 root root 1338 Oct  4 00:04 /usr/share/ca-certificates/1434.pem

	I1003 17:53:16.039558    8925 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1434.pem

	I1003 17:53:16.043239    8925 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1434.pem /etc/ssl/certs/51391683.0"

	I1003 17:53:16.050515    8925 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/14342.pem && ln -fs /usr/share/ca-certificates/14342.pem /etc/ssl/certs/14342.pem"

	I1003 17:53:16.058654    8925 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/14342.pem

	I1003 17:53:16.062253    8925 certs.go:480] hashing: -rw-r--r-- 1 root root 1708 Oct  4 00:04 /usr/share/ca-certificates/14342.pem

	I1003 17:53:16.062300    8925 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/14342.pem

	I1003 17:53:16.066937    8925 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/14342.pem /etc/ssl/certs/3ec20f2e.0"

	I1003 17:53:16.074200    8925 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"

	I1003 17:53:16.080867    8925 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem

	I1003 17:53:16.083971    8925 certs.go:480] hashing: -rw-r--r-- 1 root root 1111 Oct  4 00:04 /usr/share/ca-certificates/minikubeCA.pem

	I1003 17:53:16.084016    8925 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem

	I1003 17:53:16.087809    8925 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"

	I1003 17:53:16.094755    8925 ssh_runner.go:195] Run: ls /var/lib/minikube/certs/etcd

	I1003 17:53:16.097860    8925 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400

	I1003 17:53:16.101676    8925 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400

	I1003 17:53:16.105418    8925 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400

	I1003 17:53:16.109152    8925 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400

	I1003 17:53:16.112974    8925 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400

	I1003 17:53:16.117086    8925 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400

	I1003 17:53:16.121063    8925 kubeadm.go:404] StartCluster: {Name:embed-certs-266000 KeepContext:false EmbedCerts:true MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17240/minikube-v1.31.0-1695060926-17240-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1696360059-17345@sha256:76d99edd1576614d5c20a839dd16ae1d7c810f3b909a01797063d483159ea3ae Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVe

rsion:v1.28.2 ClusterName:embed-certs-266000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.89.28 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress:

Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I1003 17:53:16.121161    8925 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}

	I1003 17:53:16.133768    8925 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd

	I1003 17:53:16.139904    8925 kubeadm.go:419] found existing configuration files, will attempt cluster restart

	I1003 17:53:16.139923    8925 kubeadm.go:636] restartCluster start

	I1003 17:53:16.139984    8925 ssh_runner.go:195] Run: sudo test -d /data/minikube

	I1003 17:53:16.145895    8925 kubeadm.go:127] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:16.146281    8925 kubeconfig.go:135] verify returned: extract IP: "embed-certs-266000" does not appear in /Users/jenkins/minikube-integration/17345-936/kubeconfig

	I1003 17:53:16.146422    8925 kubeconfig.go:146] "embed-certs-266000" context is missing from /Users/jenkins/minikube-integration/17345-936/kubeconfig - will repair!

	I1003 17:53:16.146679    8925 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17345-936/kubeconfig: {Name:mk9e7f46373d8a996f8969d40b90073c6059a598 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I1003 17:53:16.148201    8925 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new

	I1003 17:53:16.153814    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:16.153860    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:16.161585    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:16.161593    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:16.161664    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:16.168927    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:16.669963    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:16.670120    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:16.679559    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:17.170111    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:17.170336    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:17.179886    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:17.669045    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:17.669149    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:17.676882    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:18.169175    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:18.169368    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:18.178902    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:14.181258    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:16.677348    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:18.669719    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:18.669913    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:18.679204    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:19.171112    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:19.171255    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:19.181008    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:19.671141    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:19.671284    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:19.681025    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:20.170577    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:20.170771    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:20.180073    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:20.669229    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:20.669353    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:20.677704    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:21.169953    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:21.170053    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:21.178775    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:21.670214    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:21.670430    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:21.679956    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:22.169920    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:22.170143    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:22.179461    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:22.669766    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:22.669966    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:22.679206    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:23.169911    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:23.170023    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:23.179531    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:18.678864    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:20.679188    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:23.178916    8573 pod_ready.go:102] pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:23.873467    8573 pod_ready.go:81] duration metric: took 4m0.002176177s waiting for pod "metrics-server-74d5856cc6-2hjjx" in "kube-system" namespace to be "Ready" ...

	E1003 17:53:23.873505    8573 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded

	I1003 17:53:23.873523    8573 pod_ready.go:38] duration metric: took 4m1.20010098s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I1003 17:53:23.873661    8573 kubeadm.go:640] restartCluster took 5m39.056953879s

	W1003 17:53:23.873724    8573 out.go:239] ! Unable to restart cluster, will reset it: extra: timed out waiting 4m0s for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready"

	I1003 17:53:23.873763    8573 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force"

	I1003 17:53:25.321910    8573 ssh_runner.go:235] Completed: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm reset --cri-socket /var/run/dockershim.sock --force": (1.448010936s)

	I1003 17:53:25.321981    8573 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I1003 17:53:25.330923    8573 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml

	I1003 17:53:25.337216    8573 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf

	I1003 17:53:25.343172    8573 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory

	I1003 17:53:25.343203    8573 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.16.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU"

	I1003 17:53:25.387660    8573 kubeadm.go:322] [init] Using Kubernetes version: v1.16.0

	I1003 17:53:25.387817    8573 kubeadm.go:322] [preflight] Running pre-flight checks

	I1003 17:53:25.537968    8573 kubeadm.go:322] [preflight] Pulling images required for setting up a Kubernetes cluster

	I1003 17:53:25.538060    8573 kubeadm.go:322] [preflight] This might take a minute or two, depending on the speed of your internet connection

	I1003 17:53:25.538138    8573 kubeadm.go:322] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	I1003 17:53:25.647337    8573 kubeadm.go:322] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	I1003 17:53:25.648168    8573 kubeadm.go:322] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	I1003 17:53:25.654000    8573 kubeadm.go:322] [kubelet-start] Activating the kubelet service

	I1003 17:53:25.738694    8573 kubeadm.go:322] [certs] Using certificateDir folder "/var/lib/minikube/certs"

	I1003 17:53:25.763062    8573 out.go:204]   - Generating certificates and keys ...

	I1003 17:53:25.763165    8573 kubeadm.go:322] [certs] Using existing ca certificate authority

	I1003 17:53:25.763237    8573 kubeadm.go:322] [certs] Using existing apiserver certificate and key on disk

	I1003 17:53:25.763298    8573 kubeadm.go:322] [certs] Using existing apiserver-kubelet-client certificate and key on disk

	I1003 17:53:25.763358    8573 kubeadm.go:322] [certs] Using existing front-proxy-ca certificate authority

	I1003 17:53:25.763422    8573 kubeadm.go:322] [certs] Using existing front-proxy-client certificate and key on disk

	I1003 17:53:25.763469    8573 kubeadm.go:322] [certs] Using existing etcd/ca certificate authority

	I1003 17:53:25.763535    8573 kubeadm.go:322] [certs] Using existing etcd/server certificate and key on disk

	I1003 17:53:25.763592    8573 kubeadm.go:322] [certs] Using existing etcd/peer certificate and key on disk

	I1003 17:53:25.763657    8573 kubeadm.go:322] [certs] Using existing etcd/healthcheck-client certificate and key on disk

	I1003 17:53:25.763717    8573 kubeadm.go:322] [certs] Using existing apiserver-etcd-client certificate and key on disk

	I1003 17:53:25.763756    8573 kubeadm.go:322] [certs] Using the existing "sa" key

	I1003 17:53:25.763806    8573 kubeadm.go:322] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	I1003 17:53:25.906488    8573 kubeadm.go:322] [kubeconfig] Writing "admin.conf" kubeconfig file

	I1003 17:53:26.025985    8573 kubeadm.go:322] [kubeconfig] Writing "kubelet.conf" kubeconfig file

	I1003 17:53:26.479308    8573 kubeadm.go:322] [kubeconfig] Writing "controller-manager.conf" kubeconfig file

	I1003 17:53:26.777629    8573 kubeadm.go:322] [kubeconfig] Writing "scheduler.conf" kubeconfig file

	I1003 17:53:26.778376    8573 kubeadm.go:322] [control-plane] Using manifest folder "/etc/kubernetes/manifests"

	I1003 17:53:23.669477    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:23.669579    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:23.679247    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:24.169208    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:24.169272    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:24.176957    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:24.669589    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:24.669655    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:24.677757    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:25.169268    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:25.169375    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:25.177225    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:25.669726    8925 api_server.go:166] Checking apiserver status ...

	I1003 17:53:25.669806    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	W1003 17:53:25.678029    8925 api_server.go:170] stopped: unable to get apiserver pid: sudo pgrep -xnf kube-apiserver.*minikube.*: Process exited with status 1

	stdout:

	stderr:

	I1003 17:53:26.154725    8925 kubeadm.go:611] needs reconfigure: apiserver error: context deadline exceeded

	I1003 17:53:26.154743    8925 kubeadm.go:1128] stopping kube-system containers ...

	I1003 17:53:26.154810    8925 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}

	I1003 17:53:26.169596    8925 docker.go:463] Stopping containers: [eb2a46e61397 abec2df4c2da f44c2bbc7476 543be7d04758 2b0a4e3d22e2 2f070e817ce5 4741c77d6f6a 8c93b310f8f9 0a7a77ab7096 e8f7fb5404ec 4f12854d854c f29b31d85147 b3da9674cc06 fa4e4c358136 2d6007f0ea49]

	I1003 17:53:26.169736    8925 ssh_runner.go:195] Run: docker stop eb2a46e61397 abec2df4c2da f44c2bbc7476 543be7d04758 2b0a4e3d22e2 2f070e817ce5 4741c77d6f6a 8c93b310f8f9 0a7a77ab7096 e8f7fb5404ec 4f12854d854c f29b31d85147 b3da9674cc06 fa4e4c358136 2d6007f0ea49

	I1003 17:53:26.183583    8925 ssh_runner.go:195] Run: sudo systemctl stop kubelet

	I1003 17:53:26.194073    8925 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf

	I1003 17:53:26.200126    8925 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory

	I1003 17:53:26.200167    8925 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml

	I1003 17:53:26.206050    8925 kubeadm.go:713] reconfiguring cluster from /var/tmp/minikube/kubeadm.yaml

	I1003 17:53:26.206059    8925 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.2:$PATH" kubeadm init phase certs all --config /var/tmp/minikube/kubeadm.yaml"

	I1003 17:53:26.280907    8925 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.2:$PATH" kubeadm init phase kubeconfig all --config /var/tmp/minikube/kubeadm.yaml"

	I1003 17:53:27.053825    8925 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.2:$PATH" kubeadm init phase kubelet-start --config /var/tmp/minikube/kubeadm.yaml"

	I1003 17:53:27.191934    8925 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.2:$PATH" kubeadm init phase control-plane all --config /var/tmp/minikube/kubeadm.yaml"

	I1003 17:53:27.251723    8925 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.2:$PATH" kubeadm init phase etcd local --config /var/tmp/minikube/kubeadm.yaml"

	I1003 17:53:27.296022    8925 api_server.go:52] waiting for apiserver process to appear ...

	I1003 17:53:27.296098    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I1003 17:53:27.307140    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I1003 17:53:27.817502    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I1003 17:53:26.799786    8573 out.go:204]   - Booting up control plane ...

	I1003 17:53:26.799879    8573 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-apiserver"

	I1003 17:53:26.799981    8573 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-controller-manager"

	I1003 17:53:26.800086    8573 kubeadm.go:322] [control-plane] Creating static Pod manifest for "kube-scheduler"

	I1003 17:53:26.800178    8573 kubeadm.go:322] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	I1003 17:53:26.800304    8573 kubeadm.go:322] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	I1003 17:53:28.317372    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I1003 17:53:28.817948    8925 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I1003 17:53:28.860791    8925 api_server.go:72] duration metric: took 1.564675204s to wait for apiserver process to appear ...

	I1003 17:53:28.860805    8925 api_server.go:88] waiting for apiserver healthz status ...

	I1003 17:53:28.860821    8925 api_server.go:253] Checking apiserver healthz at https://192.168.89.28:8443/healthz ...

	I1003 17:53:31.277841    8925 api_server.go:279] https://192.168.89.28:8443/healthz returned 403:

	{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"forbidden: User \"system:anonymous\" cannot get path \"/healthz\"","reason":"Forbidden","details":{},"code":403}

	W1003 17:53:31.277856    8925 api_server.go:103] status: https://192.168.89.28:8443/healthz returned error 403:

	{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"forbidden: User \"system:anonymous\" cannot get path \"/healthz\"","reason":"Forbidden","details":{},"code":403}

	I1003 17:53:31.277865    8925 api_server.go:253] Checking apiserver healthz at https://192.168.89.28:8443/healthz ...

	I1003 17:53:31.295898    8925 api_server.go:279] https://192.168.89.28:8443/healthz returned 403:

	{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"forbidden: User \"system:anonymous\" cannot get path \"/healthz\"","reason":"Forbidden","details":{},"code":403}

	W1003 17:53:31.295915    8925 api_server.go:103] status: https://192.168.89.28:8443/healthz returned error 403:

	{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"forbidden: User \"system:anonymous\" cannot get path \"/healthz\"","reason":"Forbidden","details":{},"code":403}

	I1003 17:53:31.797468    8925 api_server.go:253] Checking apiserver healthz at https://192.168.89.28:8443/healthz ...

	I1003 17:53:31.803326    8925 api_server.go:279] https://192.168.89.28:8443/healthz returned 500:

	[+]ping ok

	[+]log ok

	[+]etcd ok

	[+]poststarthook/start-kube-apiserver-admission-initializer ok

	[+]poststarthook/generic-apiserver-start-informers ok

	[+]poststarthook/priority-and-fairness-config-consumer ok

	[+]poststarthook/priority-and-fairness-filter ok

	[+]poststarthook/storage-object-count-tracker-hook ok

	[+]poststarthook/start-apiextensions-informers ok

	[+]poststarthook/start-apiextensions-controllers ok

	[+]poststarthook/crd-informer-synced ok

	[+]poststarthook/start-service-ip-repair-controllers ok

	[-]poststarthook/rbac/bootstrap-roles failed: reason withheld

	[-]poststarthook/scheduling/bootstrap-system-priority-classes failed: reason withheld

	[+]poststarthook/priority-and-fairness-config-producer ok

	[+]poststarthook/start-system-namespaces-controller ok

	[+]poststarthook/bootstrap-controller ok

	[+]poststarthook/start-cluster-authentication-info-controller ok

	[+]poststarthook/start-kube-apiserver-identity-lease-controller ok

	[+]poststarthook/start-deprecated-kube-apiserver-identity-lease-garbage-collector ok

	[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok

	[+]poststarthook/start-legacy-token-tracking-controller ok

	[+]poststarthook/aggregator-reload-proxy-client-cert ok

	[+]poststarthook/start-kube-aggregator-informers ok

	[+]poststarthook/apiservice-registration-controller ok

	[+]poststarthook/apiservice-status-available-controller ok

	[+]poststarthook/kube-apiserver-autoregistration ok

	[+]autoregister-completion ok

	[+]poststarthook/apiservice-openapi-controller ok

	[+]poststarthook/apiservice-openapiv3-controller ok

	[+]poststarthook/apiservice-discovery-controller ok

	healthz check failed

	W1003 17:53:31.803340    8925 api_server.go:103] status: https://192.168.89.28:8443/healthz returned error 500:

	[+]ping ok

	[+]log ok

	[+]etcd ok

	[+]poststarthook/start-kube-apiserver-admission-initializer ok

	[+]poststarthook/generic-apiserver-start-informers ok

	[+]poststarthook/priority-and-fairness-config-consumer ok

	[+]poststarthook/priority-and-fairness-filter ok

	[+]poststarthook/storage-object-count-tracker-hook ok

	[+]poststarthook/start-apiextensions-informers ok

	[+]poststarthook/start-apiextensions-controllers ok

	[+]poststarthook/crd-informer-synced ok

	[+]poststarthook/start-service-ip-repair-controllers ok

	[-]poststarthook/rbac/bootstrap-roles failed: reason withheld

	[-]poststarthook/scheduling/bootstrap-system-priority-classes failed: reason withheld

	[+]poststarthook/priority-and-fairness-config-producer ok

	[+]poststarthook/start-system-namespaces-controller ok

	[+]poststarthook/bootstrap-controller ok

	[+]poststarthook/start-cluster-authentication-info-controller ok

	[+]poststarthook/start-kube-apiserver-identity-lease-controller ok

	[+]poststarthook/start-deprecated-kube-apiserver-identity-lease-garbage-collector ok

	[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok

	[+]poststarthook/start-legacy-token-tracking-controller ok

	[+]poststarthook/aggregator-reload-proxy-client-cert ok

	[+]poststarthook/start-kube-aggregator-informers ok

	[+]poststarthook/apiservice-registration-controller ok

	[+]poststarthook/apiservice-status-available-controller ok

	[+]poststarthook/kube-apiserver-autoregistration ok

	[+]autoregister-completion ok

	[+]poststarthook/apiservice-openapi-controller ok

	[+]poststarthook/apiservice-openapiv3-controller ok

	[+]poststarthook/apiservice-discovery-controller ok

	healthz check failed

	I1003 17:53:32.296564    8925 api_server.go:253] Checking apiserver healthz at https://192.168.89.28:8443/healthz ...

	I1003 17:53:32.301082    8925 api_server.go:279] https://192.168.89.28:8443/healthz returned 500:

	[+]ping ok

	[+]log ok

	[+]etcd ok

	[+]poststarthook/start-kube-apiserver-admission-initializer ok

	[+]poststarthook/generic-apiserver-start-informers ok

	[+]poststarthook/priority-and-fairness-config-consumer ok

	[+]poststarthook/priority-and-fairness-filter ok

	[+]poststarthook/storage-object-count-tracker-hook ok

	[+]poststarthook/start-apiextensions-informers ok

	[+]poststarthook/start-apiextensions-controllers ok

	[+]poststarthook/crd-informer-synced ok

	[+]poststarthook/start-service-ip-repair-controllers ok

	[-]poststarthook/rbac/bootstrap-roles failed: reason withheld

	[+]poststarthook/scheduling/bootstrap-system-priority-classes ok

	[+]poststarthook/priority-and-fairness-config-producer ok

	[+]poststarthook/start-system-namespaces-controller ok

	[+]poststarthook/bootstrap-controller ok

	[+]poststarthook/start-cluster-authentication-info-controller ok

	[+]poststarthook/start-kube-apiserver-identity-lease-controller ok

	[+]poststarthook/start-deprecated-kube-apiserver-identity-lease-garbage-collector ok

	[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok

	[+]poststarthook/start-legacy-token-tracking-controller ok

	[+]poststarthook/aggregator-reload-proxy-client-cert ok

	[+]poststarthook/start-kube-aggregator-informers ok

	[+]poststarthook/apiservice-registration-controller ok

	[+]poststarthook/apiservice-status-available-controller ok

	[+]poststarthook/kube-apiserver-autoregistration ok

	[+]autoregister-completion ok

	[+]poststarthook/apiservice-openapi-controller ok

	[+]poststarthook/apiservice-openapiv3-controller ok

	[+]poststarthook/apiservice-discovery-controller ok

	healthz check failed

	W1003 17:53:32.301093    8925 api_server.go:103] status: https://192.168.89.28:8443/healthz returned error 500:

	[+]ping ok

	[+]log ok

	[+]etcd ok

	[+]poststarthook/start-kube-apiserver-admission-initializer ok

	[+]poststarthook/generic-apiserver-start-informers ok

	[+]poststarthook/priority-and-fairness-config-consumer ok

	[+]poststarthook/priority-and-fairness-filter ok

	[+]poststarthook/storage-object-count-tracker-hook ok

	[+]poststarthook/start-apiextensions-informers ok

	[+]poststarthook/start-apiextensions-controllers ok

	[+]poststarthook/crd-informer-synced ok

	[+]poststarthook/start-service-ip-repair-controllers ok

	[-]poststarthook/rbac/bootstrap-roles failed: reason withheld

	[+]poststarthook/scheduling/bootstrap-system-priority-classes ok

	[+]poststarthook/priority-and-fairness-config-producer ok

	[+]poststarthook/start-system-namespaces-controller ok

	[+]poststarthook/bootstrap-controller ok

	[+]poststarthook/start-cluster-authentication-info-controller ok

	[+]poststarthook/start-kube-apiserver-identity-lease-controller ok

	[+]poststarthook/start-deprecated-kube-apiserver-identity-lease-garbage-collector ok

	[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok

	[+]poststarthook/start-legacy-token-tracking-controller ok

	[+]poststarthook/aggregator-reload-proxy-client-cert ok

	[+]poststarthook/start-kube-aggregator-informers ok

	[+]poststarthook/apiservice-registration-controller ok

	[+]poststarthook/apiservice-status-available-controller ok

	[+]poststarthook/kube-apiserver-autoregistration ok

	[+]autoregister-completion ok

	[+]poststarthook/apiservice-openapi-controller ok

	[+]poststarthook/apiservice-openapiv3-controller ok

	[+]poststarthook/apiservice-discovery-controller ok

	healthz check failed

	I1003 17:53:32.796292    8925 api_server.go:253] Checking apiserver healthz at https://192.168.89.28:8443/healthz ...

	I1003 17:53:32.801313    8925 api_server.go:279] https://192.168.89.28:8443/healthz returned 200:

	ok

	I1003 17:53:32.806632    8925 api_server.go:141] control plane version: v1.28.2

	I1003 17:53:32.806643    8925 api_server.go:131] duration metric: took 3.945641876s to wait for apiserver health ...

	I1003 17:53:32.806649    8925 cni.go:84] Creating CNI manager for ""

	I1003 17:53:32.806658    8925 cni.go:158] "hyperkit" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I1003 17:53:32.830161    8925 out.go:177] * Configuring bridge CNI (Container Networking Interface) ...

	I1003 17:53:32.850963    8925 ssh_runner.go:195] Run: sudo mkdir -p /etc/cni/net.d

	I1003 17:53:32.858263    8925 ssh_runner.go:362] scp memory --> /etc/cni/net.d/1-k8s.conflist (457 bytes)

	I1003 17:53:32.870344    8925 system_pods.go:43] waiting for kube-system pods to appear ...

	I1003 17:53:32.876191    8925 system_pods.go:59] 8 kube-system pods found

	I1003 17:53:32.876206    8925 system_pods.go:61] "coredns-5dd5756b68-xvc7v" [f714c635-5187-4c8a-a49d-70510642b539] Running / Ready:ContainersNotReady (containers with unready status: [coredns]) / ContainersReady:ContainersNotReady (containers with unready status: [coredns])

	I1003 17:53:32.876214    8925 system_pods.go:61] "etcd-embed-certs-266000" [cb071e74-024f-480f-941c-c4ab4bef5dfd] Running / Ready:ContainersNotReady (containers with unready status: [etcd]) / ContainersReady:ContainersNotReady (containers with unready status: [etcd])

	I1003 17:53:32.876219    8925 system_pods.go:61] "kube-apiserver-embed-certs-266000" [70dd3031-9219-4495-9927-51457dd62a72] Running / Ready:ContainersNotReady (containers with unready status: [kube-apiserver]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-apiserver])

	I1003 17:53:32.876224    8925 system_pods.go:61] "kube-controller-manager-embed-certs-266000" [a460093d-b32f-4fe9-9d12-ad348683a754] Running / Ready:ContainersNotReady (containers with unready status: [kube-controller-manager]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-controller-manager])

	I1003 17:53:32.876228    8925 system_pods.go:61] "kube-proxy-qprsf" [5fe95721-0476-49ff-b774-688441c07b14] Running

	I1003 17:53:32.876232    8925 system_pods.go:61] "kube-scheduler-embed-certs-266000" [47575e8a-7e42-4c76-aeaa-5aa51e29d07f] Running / Ready:ContainersNotReady (containers with unready status: [kube-scheduler]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-scheduler])

	I1003 17:53:32.876236    8925 system_pods.go:61] "metrics-server-57f55c9bc5-hq7nv" [8d35bb67-7bdb-4480-9ed5-946341148887] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:53:32.876244    8925 system_pods.go:61] "storage-provisioner" [1a989265-c8ab-44cd-8241-21d25d990d12] Running / Ready:ContainersNotReady (containers with unready status: [storage-provisioner]) / ContainersReady:ContainersNotReady (containers with unready status: [storage-provisioner])

	I1003 17:53:32.876264    8925 system_pods.go:74] duration metric: took 5.908239ms to wait for pod list to return data ...

	I1003 17:53:32.876276    8925 node_conditions.go:102] verifying NodePressure condition ...

	I1003 17:53:32.878132    8925 node_conditions.go:122] node storage ephemeral capacity is 17784752Ki

	I1003 17:53:32.878149    8925 node_conditions.go:123] node cpu capacity is 2

	I1003 17:53:32.878159    8925 node_conditions.go:105] duration metric: took 1.878476ms to run NodePressure ...

	I1003 17:53:32.878170    8925 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.2:$PATH" kubeadm init phase addon all --config /var/tmp/minikube/kubeadm.yaml"

	I1003 17:53:33.021851    8925 kubeadm.go:772] waiting for restarted kubelet to initialise ...

	I1003 17:53:33.025213    8925 kubeadm.go:787] kubelet initialised

	I1003 17:53:33.025223    8925 kubeadm.go:788] duration metric: took 3.358411ms waiting for restarted kubelet to initialise ...

	I1003 17:53:33.025231    8925 pod_ready.go:35] extra waiting up to 4m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I1003 17:53:33.029643    8925 pod_ready.go:78] waiting up to 4m0s for pod "coredns-5dd5756b68-xvc7v" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:33.032806    8925 pod_ready.go:97] node "embed-certs-266000" hosting pod "coredns-5dd5756b68-xvc7v" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:33.032817    8925 pod_ready.go:81] duration metric: took 3.163881ms waiting for pod "coredns-5dd5756b68-xvc7v" in "kube-system" namespace to be "Ready" ...

	E1003 17:53:33.032823    8925 pod_ready.go:66] WaitExtra: waitPodCondition: node "embed-certs-266000" hosting pod "coredns-5dd5756b68-xvc7v" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:33.032829    8925 pod_ready.go:78] waiting up to 4m0s for pod "etcd-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:33.035709    8925 pod_ready.go:97] node "embed-certs-266000" hosting pod "etcd-embed-certs-266000" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:33.035718    8925 pod_ready.go:81] duration metric: took 2.884716ms waiting for pod "etcd-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	E1003 17:53:33.035724    8925 pod_ready.go:66] WaitExtra: waitPodCondition: node "embed-certs-266000" hosting pod "etcd-embed-certs-266000" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:33.035728    8925 pod_ready.go:78] waiting up to 4m0s for pod "kube-apiserver-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:33.038491    8925 pod_ready.go:97] node "embed-certs-266000" hosting pod "kube-apiserver-embed-certs-266000" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:33.038502    8925 pod_ready.go:81] duration metric: took 2.768457ms waiting for pod "kube-apiserver-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	E1003 17:53:33.038508    8925 pod_ready.go:66] WaitExtra: waitPodCondition: node "embed-certs-266000" hosting pod "kube-apiserver-embed-certs-266000" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:33.038513    8925 pod_ready.go:78] waiting up to 4m0s for pod "kube-controller-manager-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:33.273438    8925 pod_ready.go:97] node "embed-certs-266000" hosting pod "kube-controller-manager-embed-certs-266000" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:33.273458    8925 pod_ready.go:81] duration metric: took 234.928094ms waiting for pod "kube-controller-manager-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	E1003 17:53:33.273468    8925 pod_ready.go:66] WaitExtra: waitPodCondition: node "embed-certs-266000" hosting pod "kube-controller-manager-embed-certs-266000" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:33.273475    8925 pod_ready.go:78] waiting up to 4m0s for pod "kube-proxy-qprsf" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:33.672353    8925 pod_ready.go:97] node "embed-certs-266000" hosting pod "kube-proxy-qprsf" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:33.672369    8925 pod_ready.go:81] duration metric: took 398.870634ms waiting for pod "kube-proxy-qprsf" in "kube-system" namespace to be "Ready" ...

	E1003 17:53:33.672375    8925 pod_ready.go:66] WaitExtra: waitPodCondition: node "embed-certs-266000" hosting pod "kube-proxy-qprsf" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:33.672379    8925 pod_ready.go:78] waiting up to 4m0s for pod "kube-scheduler-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:34.073757    8925 pod_ready.go:97] node "embed-certs-266000" hosting pod "kube-scheduler-embed-certs-266000" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:34.073773    8925 pod_ready.go:81] duration metric: took 401.372924ms waiting for pod "kube-scheduler-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	E1003 17:53:34.073780    8925 pod_ready.go:66] WaitExtra: waitPodCondition: node "embed-certs-266000" hosting pod "kube-scheduler-embed-certs-266000" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:34.073796    8925 pod_ready.go:78] waiting up to 4m0s for pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:34.472686    8925 pod_ready.go:97] node "embed-certs-266000" hosting pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:34.472703    8925 pod_ready.go:81] duration metric: took 398.882097ms waiting for pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace to be "Ready" ...

	E1003 17:53:34.472710    8925 pod_ready.go:66] WaitExtra: waitPodCondition: node "embed-certs-266000" hosting pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace is currently not "Ready" (skipping!): node "embed-certs-266000" has status "Ready":"False"

	I1003 17:53:34.472716    8925 pod_ready.go:38] duration metric: took 1.447420195s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I1003 17:53:34.472734    8925 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"

	I1003 17:53:34.481423    8925 ops.go:34] apiserver oom_adj: -16

	I1003 17:53:34.481440    8925 kubeadm.go:640] restartCluster took 18.340785639s

	I1003 17:53:34.481445    8925 kubeadm.go:406] StartCluster complete in 18.359672003s

	I1003 17:53:34.481462    8925 settings.go:142] acquiring lock: {Name:mk44d7423fc554238d4da5b26088e530eff95e9f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I1003 17:53:34.481542    8925 settings.go:150] Updating kubeconfig:  /Users/jenkins/minikube-integration/17345-936/kubeconfig

	I1003 17:53:34.482292    8925 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17345-936/kubeconfig: {Name:mk9e7f46373d8a996f8969d40b90073c6059a598 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I1003 17:53:34.482695    8925 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"

	I1003 17:53:34.482753    8925 addons.go:499] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false]

	I1003 17:53:34.482796    8925 addons.go:69] Setting storage-provisioner=true in profile "embed-certs-266000"

	I1003 17:53:34.482809    8925 addons.go:231] Setting addon storage-provisioner=true in "embed-certs-266000"

	W1003 17:53:34.482816    8925 addons.go:240] addon storage-provisioner should already be in state true

	I1003 17:53:34.482808    8925 addons.go:69] Setting dashboard=true in profile "embed-certs-266000"

	I1003 17:53:34.482836    8925 addons.go:231] Setting addon dashboard=true in "embed-certs-266000"

	W1003 17:53:34.482844    8925 addons.go:240] addon dashboard should already be in state true

	I1003 17:53:34.482846    8925 addons.go:69] Setting metrics-server=true in profile "embed-certs-266000"

	I1003 17:53:34.482842    8925 addons.go:69] Setting default-storageclass=true in profile "embed-certs-266000"

	I1003 17:53:34.482868    8925 addons.go:231] Setting addon metrics-server=true in "embed-certs-266000"

	I1003 17:53:34.482869    8925 host.go:66] Checking if "embed-certs-266000" exists ...

	I1003 17:53:34.482870    8925 host.go:66] Checking if "embed-certs-266000" exists ...

	W1003 17:53:34.482876    8925 addons.go:240] addon metrics-server should already be in state true

	I1003 17:53:34.482879    8925 config.go:182] Loaded profile config "embed-certs-266000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I1003 17:53:34.482888    8925 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "embed-certs-266000"

	I1003 17:53:34.482927    8925 host.go:66] Checking if "embed-certs-266000" exists ...

	I1003 17:53:34.483145    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:53:34.483145    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:53:34.483167    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:53:34.483173    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:53:34.483182    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:53:34.483213    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:53:34.483236    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:53:34.483269    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:53:34.494694    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56202

	I1003 17:53:34.494739    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56204

	I1003 17:53:34.494695    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56203

	I1003 17:53:34.495236    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:53:34.495965    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:53:34.496123    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:53:34.496143    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:53:34.496370    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:53:34.496520    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:53:34.496536    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:53:34.496553    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:53:34.497159    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:53:34.497534    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:53:34.497554    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:53:34.497564    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetState

	I1003 17:53:34.497689    8925 main.go:141] libmachine: (embed-certs-266000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:53:34.497758    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:53:34.497781    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:53:34.497785    8925 main.go:141] libmachine: (embed-certs-266000) DBG | hyperkit pid from json: 8936

	I1003 17:53:34.497844    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:53:34.499102    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56208

	I1003 17:53:34.499680    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:53:34.499830    8925 kapi.go:248] "coredns" deployment in "kube-system" namespace and "embed-certs-266000" context rescaled to 1 replicas

	I1003 17:53:34.499839    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:53:34.499866    8925 start.go:223] Will wait 6m0s for node &{Name: IP:192.168.89.28 Port:8443 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}

	I1003 17:53:34.522011    8925 out.go:177] * Verifying Kubernetes components...

	I1003 17:53:34.502278    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:53:34.503879    8925 addons.go:231] Setting addon default-storageclass=true in "embed-certs-266000"

	I1003 17:53:34.508623    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56210

	I1003 17:53:34.509346    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56211

	W1003 17:53:34.522072    8925 addons.go:240] addon default-storageclass should already be in state true

	I1003 17:53:34.522526    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:53:34.563731    8925 host.go:66] Checking if "embed-certs-266000" exists ...

	I1003 17:53:34.522534    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:53:34.563739    8925 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I1003 17:53:34.522619    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:53:34.563803    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:53:34.564103    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:53:34.564165    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:53:34.564187    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:53:34.564190    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:53:34.564207    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:53:34.564199    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:53:34.564239    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:53:34.564895    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:53:34.564930    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:53:34.565451    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetState

	I1003 17:53:34.565451    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:53:34.565511    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:53:34.565623    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetState

	I1003 17:53:34.565673    8925 main.go:141] libmachine: (embed-certs-266000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:53:34.565711    8925 main.go:141] libmachine: (embed-certs-266000) DBG | hyperkit pid from json: 8936

	I1003 17:53:34.566810    8925 main.go:141] libmachine: (embed-certs-266000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:53:34.567275    8925 main.go:141] libmachine: (embed-certs-266000) DBG | hyperkit pid from json: 8936

	I1003 17:53:34.568437    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:53:34.568488    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:53:34.589776    8925 out.go:177]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5

	I1003 17:53:34.574187    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56214

	I1003 17:53:34.574786    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56215

	I1003 17:53:34.610830    8925 out.go:177]   - Using image docker.io/kubernetesui/dashboard:v2.7.0

	I1003 17:53:34.610960    8925 addons.go:423] installing /etc/kubernetes/addons/storage-provisioner.yaml

	I1003 17:53:34.670886    8925 out.go:177]   - Using image registry.k8s.io/echoserver:1.4

	I1003 17:53:34.633683    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)

	I1003 17:53:34.611305    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:53:34.611272    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:53:34.675074    8925 node_ready.go:35] waiting up to 6m0s for node "embed-certs-266000" to be "Ready" ...

	I1003 17:53:34.675263    8925 start.go:896] CoreDNS already contains "host.minikube.internal" host record, skipping...

	I1003 17:53:34.707899    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:34.707958    8925 addons.go:423] installing /etc/kubernetes/addons/dashboard-ns.yaml

	I1003 17:53:34.707977    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)

	I1003 17:53:34.708006    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:34.708230    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:34.708264    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:34.708521    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:34.708558    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:34.708732    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:53:34.708755    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:53:34.708778    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:53:34.708788    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:34.708801    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:53:34.708809    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:34.708998    8925 sshutil.go:53] new ssh client: &{IP:192.168.89.28 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/id_rsa Username:docker}

	I1003 17:53:34.709101    8925 sshutil.go:53] new ssh client: &{IP:192.168.89.28 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/id_rsa Username:docker}

	I1003 17:53:34.709257    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:53:34.709313    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:53:34.709512    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetState

	I1003 17:53:34.709689    8925 main.go:141] libmachine: (embed-certs-266000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:53:34.709819    8925 main.go:141] libmachine: (embed-certs-266000) DBG | hyperkit pid from json: 8936

	I1003 17:53:34.709911    8925 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:53:34.709948    8925 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:53:34.712506    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:53:34.749698    8925 out.go:177]   - Using image fake.domain/registry.k8s.io/echoserver:1.4

	I1003 17:53:34.718664    8925 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56220

	I1003 17:53:34.767953    8925 addons.go:423] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml

	I1003 17:53:34.771723    8925 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml

	I1003 17:53:34.787001    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)

	I1003 17:53:34.787117    8925 addons.go:423] installing /etc/kubernetes/addons/metrics-apiservice.yaml

	I1003 17:53:34.787143    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)

	I1003 17:53:34.787186    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:34.787585    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:34.787735    8925 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:53:34.787819    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:34.787991    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:34.788144    8925 sshutil.go:53] new ssh client: &{IP:192.168.89.28 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/id_rsa Username:docker}

	I1003 17:53:34.788357    8925 main.go:141] libmachine: Using API Version  1

	I1003 17:53:34.788375    8925 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:53:34.788716    8925 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:53:34.788857    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetState

	I1003 17:53:34.788980    8925 main.go:141] libmachine: (embed-certs-266000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:53:34.789080    8925 main.go:141] libmachine: (embed-certs-266000) DBG | hyperkit pid from json: 8936

	I1003 17:53:34.790303    8925 main.go:141] libmachine: (embed-certs-266000) Calling .DriverName

	I1003 17:53:34.790466    8925 addons.go:423] installing /etc/kubernetes/addons/storageclass.yaml

	I1003 17:53:34.790474    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)

	I1003 17:53:34.790484    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHHostname

	I1003 17:53:34.790572    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHPort

	I1003 17:53:34.790646    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHKeyPath

	I1003 17:53:34.790716    8925 main.go:141] libmachine: (embed-certs-266000) Calling .GetSSHUsername

	I1003 17:53:34.790817    8925 sshutil.go:53] new ssh client: &{IP:192.168.89.28 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/embed-certs-266000/id_rsa Username:docker}

	I1003 17:53:34.822330    8925 addons.go:423] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml

	I1003 17:53:34.822348    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)

	I1003 17:53:34.920505    8925 addons.go:423] installing /etc/kubernetes/addons/dashboard-configmap.yaml

	I1003 17:53:34.920521    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)

	I1003 17:53:34.954111    8925 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml

	I1003 17:53:34.977738    8925 addons.go:423] installing /etc/kubernetes/addons/dashboard-dp.yaml

	I1003 17:53:34.977750    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)

	I1003 17:53:35.006334    8925 addons.go:423] installing /etc/kubernetes/addons/metrics-server-deployment.yaml

	I1003 17:53:35.006346    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)

	I1003 17:53:35.022476    8925 addons.go:423] installing /etc/kubernetes/addons/dashboard-role.yaml

	I1003 17:53:35.022512    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)

	I1003 17:53:35.044632    8925 addons.go:423] installing /etc/kubernetes/addons/metrics-server-rbac.yaml

	I1003 17:53:35.044645    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)

	I1003 17:53:35.065207    8925 addons.go:423] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml

	I1003 17:53:35.065221    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)

	I1003 17:53:35.090646    8925 addons.go:423] installing /etc/kubernetes/addons/metrics-server-service.yaml

	I1003 17:53:35.090675    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)

	I1003 17:53:35.189646    8925 addons.go:423] installing /etc/kubernetes/addons/dashboard-sa.yaml

	I1003 17:53:35.189673    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)

	I1003 17:53:35.221797    8925 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml

	I1003 17:53:35.222834    8925 addons.go:423] installing /etc/kubernetes/addons/dashboard-secret.yaml

	I1003 17:53:35.222842    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)

	I1003 17:53:35.251859    8925 addons.go:423] installing /etc/kubernetes/addons/dashboard-svc.yaml

	I1003 17:53:35.251877    8925 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)

	I1003 17:53:35.296623    8925 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml

	I1003 17:53:35.875555    8925 node_ready.go:49] node "embed-certs-266000" has status "Ready":"True"

	I1003 17:53:35.875570    8925 node_ready.go:38] duration metric: took 1.167567662s waiting for node "embed-certs-266000" to be "Ready" ...

	I1003 17:53:35.875575    8925 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I1003 17:53:35.880105    8925 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5dd5756b68-xvc7v" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:36.177387    8925 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (1.390399446s)

	I1003 17:53:36.177430    8925 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: (1.223261376s)

	I1003 17:53:36.177444    8925 main.go:141] libmachine: Making call to close driver server

	I1003 17:53:36.177452    8925 main.go:141] libmachine: (embed-certs-266000) Calling .Close

	I1003 17:53:36.177452    8925 main.go:141] libmachine: Making call to close driver server

	I1003 17:53:36.177462    8925 main.go:141] libmachine: (embed-certs-266000) Calling .Close

	I1003 17:53:36.177724    8925 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:53:36.177731    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Closing plugin on server side

	I1003 17:53:36.177734    8925 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:53:36.177733    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Closing plugin on server side

	I1003 17:53:36.177737    8925 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:53:36.177745    8925 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:53:36.177753    8925 main.go:141] libmachine: Making call to close driver server

	I1003 17:53:36.177762    8925 main.go:141] libmachine: (embed-certs-266000) Calling .Close

	I1003 17:53:36.177754    8925 main.go:141] libmachine: Making call to close driver server

	I1003 17:53:36.177791    8925 main.go:141] libmachine: (embed-certs-266000) Calling .Close

	I1003 17:53:36.177917    8925 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:53:36.177938    8925 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:53:36.177960    8925 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:53:36.177969    8925 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:53:36.177969    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Closing plugin on server side

	I1003 17:53:36.182441    8925 main.go:141] libmachine: Making call to close driver server

	I1003 17:53:36.182467    8925 main.go:141] libmachine: (embed-certs-266000) Calling .Close

	I1003 17:53:36.182653    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Closing plugin on server side

	I1003 17:53:36.182682    8925 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:53:36.182692    8925 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:53:36.268821    8925 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (1.046948444s)

	I1003 17:53:36.268849    8925 main.go:141] libmachine: Making call to close driver server

	I1003 17:53:36.268855    8925 main.go:141] libmachine: (embed-certs-266000) Calling .Close

	I1003 17:53:36.269078    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Closing plugin on server side

	I1003 17:53:36.269107    8925 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:53:36.269116    8925 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:53:36.269124    8925 main.go:141] libmachine: Making call to close driver server

	I1003 17:53:36.269130    8925 main.go:141] libmachine: (embed-certs-266000) Calling .Close

	I1003 17:53:36.269336    8925 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:53:36.269337    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Closing plugin on server side

	I1003 17:53:36.269346    8925 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:53:36.269354    8925 addons.go:467] Verifying addon metrics-server=true in "embed-certs-266000"

	I1003 17:53:36.553459    8925 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.2/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (1.256747233s)

	I1003 17:53:36.553516    8925 main.go:141] libmachine: Making call to close driver server

	I1003 17:53:36.553528    8925 main.go:141] libmachine: (embed-certs-266000) Calling .Close

	I1003 17:53:36.553779    8925 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:53:36.553781    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Closing plugin on server side

	I1003 17:53:36.553816    8925 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:53:36.553822    8925 main.go:141] libmachine: Making call to close driver server

	I1003 17:53:36.553828    8925 main.go:141] libmachine: (embed-certs-266000) Calling .Close

	I1003 17:53:36.554024    8925 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:53:36.554032    8925 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:53:36.554047    8925 main.go:141] libmachine: (embed-certs-266000) DBG | Closing plugin on server side

	I1003 17:53:36.576347    8925 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:

		minikube -p embed-certs-266000 addons enable metrics-server	

	I1003 17:53:36.596631    8925 out.go:177] * Enabled addons: storage-provisioner, default-storageclass, metrics-server, dashboard

	I1003 17:53:36.618272    8925 addons.go:502] enable addons completed in 2.135448667s: enabled=[storage-provisioner default-storageclass metrics-server dashboard]

	I1003 17:53:38.278202    8925 pod_ready.go:102] pod "coredns-5dd5756b68-xvc7v" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:40.278354    8925 pod_ready.go:102] pod "coredns-5dd5756b68-xvc7v" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:40.777833    8925 pod_ready.go:92] pod "coredns-5dd5756b68-xvc7v" in "kube-system" namespace has status "Ready":"True"

	I1003 17:53:40.777847    8925 pod_ready.go:81] duration metric: took 4.897593407s waiting for pod "coredns-5dd5756b68-xvc7v" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:40.777853    8925 pod_ready.go:78] waiting up to 6m0s for pod "etcd-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:42.789962    8925 pod_ready.go:102] pod "etcd-embed-certs-266000" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:45.290174    8925 pod_ready.go:102] pod "etcd-embed-certs-266000" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:45.793828    8925 pod_ready.go:92] pod "etcd-embed-certs-266000" in "kube-system" namespace has status "Ready":"True"

	I1003 17:53:45.793844    8925 pod_ready.go:81] duration metric: took 5.015888731s waiting for pod "etcd-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:45.793852    8925 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:45.798509    8925 pod_ready.go:92] pod "kube-apiserver-embed-certs-266000" in "kube-system" namespace has status "Ready":"True"

	I1003 17:53:45.798520    8925 pod_ready.go:81] duration metric: took 4.647798ms waiting for pod "kube-apiserver-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:45.798526    8925 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:45.805001    8925 pod_ready.go:92] pod "kube-controller-manager-embed-certs-266000" in "kube-system" namespace has status "Ready":"True"

	I1003 17:53:45.805032    8925 pod_ready.go:81] duration metric: took 6.474179ms waiting for pod "kube-controller-manager-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:45.805042    8925 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-qprsf" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:45.873628    8925 pod_ready.go:92] pod "kube-proxy-qprsf" in "kube-system" namespace has status "Ready":"True"

	I1003 17:53:45.873650    8925 pod_ready.go:81] duration metric: took 68.594196ms waiting for pod "kube-proxy-qprsf" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:45.873658    8925 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:46.274774    8925 pod_ready.go:92] pod "kube-scheduler-embed-certs-266000" in "kube-system" namespace has status "Ready":"True"

	I1003 17:53:46.274786    8925 pod_ready.go:81] duration metric: took 401.11079ms waiting for pod "kube-scheduler-embed-certs-266000" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:46.274792    8925 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace to be "Ready" ...

	I1003 17:53:48.579505    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:50.582410    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:53.077957    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:56.296723    8573 kubeadm.go:322] [apiclient] All control plane components are healthy after 29.504105 seconds

	I1003 17:53:56.296857    8573 kubeadm.go:322] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace

	I1003 17:53:56.306365    8573 kubeadm.go:322] [kubelet] Creating a ConfigMap "kubelet-config-1.16" in namespace kube-system with the configuration for the kubelets in the cluster

	I1003 17:53:56.820031    8573 kubeadm.go:322] [upload-certs] Skipping phase. Please see --upload-certs

	I1003 17:53:56.820145    8573 kubeadm.go:322] [mark-control-plane] Marking the node old-k8s-version-999000 as control-plane by adding the label "node-role.kubernetes.io/master=''"

	I1003 17:53:57.327519    8573 kubeadm.go:322] [bootstrap-token] Using token: uco7j9.67cqqblyt4ng6tbd

	I1003 17:53:55.078611    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:57.079679    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:57.369549    8573 out.go:204]   - Configuring RBAC rules ...

	I1003 17:53:57.369645    8573 kubeadm.go:322] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles

	I1003 17:53:57.369796    8573 kubeadm.go:322] [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials

	I1003 17:53:57.371726    8573 kubeadm.go:322] [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token

	I1003 17:53:57.407004    8573 kubeadm.go:322] [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster

	I1003 17:53:57.409248    8573 kubeadm.go:322] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace

	I1003 17:53:57.456972    8573 kubeadm.go:322] [addons] Applied essential addon: CoreDNS

	I1003 17:53:57.739417    8573 kubeadm.go:322] [addons] Applied essential addon: kube-proxy

	I1003 17:53:57.740668    8573 kubeadm.go:322] 

	I1003 17:53:57.740729    8573 kubeadm.go:322] Your Kubernetes control-plane has initialized successfully!

	I1003 17:53:57.740739    8573 kubeadm.go:322] 

	I1003 17:53:57.740810    8573 kubeadm.go:322] To start using your cluster, you need to run the following as a regular user:

	I1003 17:53:57.740825    8573 kubeadm.go:322] 

	I1003 17:53:57.740846    8573 kubeadm.go:322]   mkdir -p $HOME/.kube

	I1003 17:53:57.740923    8573 kubeadm.go:322]   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

	I1003 17:53:57.740972    8573 kubeadm.go:322]   sudo chown $(id -u):$(id -g) $HOME/.kube/config

	I1003 17:53:57.740977    8573 kubeadm.go:322] 

	I1003 17:53:57.741036    8573 kubeadm.go:322] You should now deploy a pod network to the cluster.

	I1003 17:53:57.741099    8573 kubeadm.go:322] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

	I1003 17:53:57.741157    8573 kubeadm.go:322]   https://kubernetes.io/docs/concepts/cluster-administration/addons/

	I1003 17:53:57.741177    8573 kubeadm.go:322] 

	I1003 17:53:57.741251    8573 kubeadm.go:322] You can now join any number of control-plane nodes by copying certificate authorities 

	I1003 17:53:57.741323    8573 kubeadm.go:322] and service account keys on each node and then running the following as root:

	I1003 17:53:57.741342    8573 kubeadm.go:322] 

	I1003 17:53:57.741430    8573 kubeadm.go:322]   kubeadm join control-plane.minikube.internal:8443 --token uco7j9.67cqqblyt4ng6tbd \

	I1003 17:53:57.741524    8573 kubeadm.go:322]     --discovery-token-ca-cert-hash sha256:2b59974ad89b5ef813c69b1ce07fb90ab231e3e44062b0fdeb7dfddbad93b600 \

	I1003 17:53:57.741550    8573 kubeadm.go:322]     --control-plane 	  

	I1003 17:53:57.741561    8573 kubeadm.go:322] 

	I1003 17:53:57.741639    8573 kubeadm.go:322] Then you can join any number of worker nodes by running the following on each as root:

	I1003 17:53:57.741644    8573 kubeadm.go:322] 

	I1003 17:53:57.741705    8573 kubeadm.go:322] kubeadm join control-plane.minikube.internal:8443 --token uco7j9.67cqqblyt4ng6tbd \

	I1003 17:53:57.741792    8573 kubeadm.go:322]     --discovery-token-ca-cert-hash sha256:2b59974ad89b5ef813c69b1ce07fb90ab231e3e44062b0fdeb7dfddbad93b600 

	I1003 17:53:57.742682    8573 kubeadm.go:322] 	[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

	I1003 17:53:57.742822    8573 kubeadm.go:322] 	[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 24.0.6. Latest validated version: 18.09

	I1003 17:53:57.742902    8573 kubeadm.go:322] 	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	I1003 17:53:57.742921    8573 cni.go:84] Creating CNI manager for ""

	I1003 17:53:57.742932    8573 cni.go:162] CNI unnecessary in this configuration, recommending no CNI

	I1003 17:53:57.742951    8573 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"

	I1003 17:53:57.743022    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:53:57.743018    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl label nodes minikube.k8s.io/version=v1.31.2 minikube.k8s.io/commit=d9526fa6c1a1bb1b20f95e15606f1308e308d84a minikube.k8s.io/name=old-k8s-version-999000 minikube.k8s.io/updated_at=2023_10_03T17_53_57_0700 minikube.k8s.io/primary=true --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:53:57.756569    8573 ops.go:34] apiserver oom_adj: -16

	I1003 17:53:57.971360    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:53:58.053379    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:53:59.579482    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:01.580077    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:53:58.620581    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:53:59.120766    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:53:59.621152    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:00.120860    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:00.620656    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:01.120829    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:01.620459    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:02.122091    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:02.620783    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:03.120901    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:04.080776    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:06.579728    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:03.620964    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:04.121627    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:04.621225    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:05.121678    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:05.621997    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:06.120681    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:06.620714    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:07.120843    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:07.621605    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:08.122037    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:08.620879    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:09.121755    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:09.621839    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:10.121750    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:10.622282    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:11.121598    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:11.622319    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:12.121106    8573 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.16.0/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I1003 17:54:12.247235    8573 kubeadm.go:1081] duration metric: took 14.50369489s to wait for elevateKubeSystemPrivileges.

	I1003 17:54:12.247253    8573 kubeadm.go:406] StartCluster complete in 6m27.448446338s

	I1003 17:54:12.247274    8573 settings.go:142] acquiring lock: {Name:mk44d7423fc554238d4da5b26088e530eff95e9f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I1003 17:54:12.247348    8573 settings.go:150] Updating kubeconfig:  /Users/jenkins/minikube-integration/17345-936/kubeconfig

	I1003 17:54:12.248176    8573 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/17345-936/kubeconfig: {Name:mk9e7f46373d8a996f8969d40b90073c6059a598 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I1003 17:54:12.248428    8573 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"

	I1003 17:54:12.248454    8573 addons.go:499] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false]

	I1003 17:54:12.248508    8573 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-999000"

	I1003 17:54:12.248514    8573 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-999000"

	I1003 17:54:12.248521    8573 addons.go:69] Setting dashboard=true in profile "old-k8s-version-999000"

	I1003 17:54:12.248524    8573 addons.go:231] Setting addon storage-provisioner=true in "old-k8s-version-999000"

	W1003 17:54:12.248534    8573 addons.go:240] addon storage-provisioner should already be in state true

	I1003 17:54:12.248536    8573 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-999000"

	I1003 17:54:12.248533    8573 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-999000"

	I1003 17:54:12.248548    8573 addons.go:231] Setting addon dashboard=true in "old-k8s-version-999000"

	I1003 17:54:12.248550    8573 addons.go:231] Setting addon metrics-server=true in "old-k8s-version-999000"

	W1003 17:54:12.248556    8573 addons.go:240] addon dashboard should already be in state true

	W1003 17:54:12.248560    8573 addons.go:240] addon metrics-server should already be in state true

	I1003 17:54:12.248574    8573 host.go:66] Checking if "old-k8s-version-999000" exists ...

	I1003 17:54:12.248577    8573 host.go:66] Checking if "old-k8s-version-999000" exists ...

	I1003 17:54:12.248585    8573 host.go:66] Checking if "old-k8s-version-999000" exists ...

	I1003 17:54:12.248586    8573 config.go:182] Loaded profile config "old-k8s-version-999000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.16.0

	I1003 17:54:12.248830    8573 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:54:12.248847    8573 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:54:12.248851    8573 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:54:12.248865    8573 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:54:12.248874    8573 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:54:12.248896    8573 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:54:12.249005    8573 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:54:12.249023    8573 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:54:12.259830    8573 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56225

	I1003 17:54:12.260303    8573 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:54:12.260429    8573 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56227

	I1003 17:54:12.260870    8573 main.go:141] libmachine: Using API Version  1

	I1003 17:54:12.260902    8573 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:54:12.260969    8573 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:54:12.261197    8573 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:54:12.261328    8573 main.go:141] libmachine: Using API Version  1

	I1003 17:54:12.261339    8573 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:54:12.261590    8573 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:54:12.261794    8573 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:54:12.261831    8573 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:54:12.262775    8573 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:54:12.262978    8573 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:54:12.267326    8573 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56229

	I1003 17:54:12.267366    8573 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56230

	I1003 17:54:12.267953    8573 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:54:12.267998    8573 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:54:12.268420    8573 main.go:141] libmachine: Using API Version  1

	I1003 17:54:12.268465    8573 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:54:12.268592    8573 main.go:141] libmachine: Using API Version  1

	I1003 17:54:12.268617    8573 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:54:12.268789    8573 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:54:12.268970    8573 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:54:12.269159    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetState

	I1003 17:54:12.269297    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:54:12.269326    8573 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:54:12.269360    8573 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:54:12.269413    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | hyperkit pid from json: 8584

	I1003 17:54:12.273558    8573 addons.go:231] Setting addon default-storageclass=true in "old-k8s-version-999000"

	W1003 17:54:12.273582    8573 addons.go:240] addon default-storageclass should already be in state true

	I1003 17:54:12.273603    8573 host.go:66] Checking if "old-k8s-version-999000" exists ...

	I1003 17:54:12.273841    8573 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56233

	I1003 17:54:12.274014    8573 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:54:12.274048    8573 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:54:12.274825    8573 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:54:12.275608    8573 main.go:141] libmachine: Using API Version  1

	I1003 17:54:12.275637    8573 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:54:12.275971    8573 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:54:12.276791    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetState

	I1003 17:54:12.277505    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:54:12.277564    8573 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56235

	I1003 17:54:12.277598    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | hyperkit pid from json: 8584

	I1003 17:54:12.278075    8573 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:54:12.278384    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .DriverName

	I1003 17:54:12.278537    8573 main.go:141] libmachine: Using API Version  1

	I1003 17:54:12.278553    8573 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:54:12.318564    8573 out.go:177]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5

	I1003 17:54:12.278878    8573 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:54:12.279807    8573 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56237

	I1003 17:54:12.285106    8573 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56238

	I1003 17:54:12.355675    8573 addons.go:423] installing /etc/kubernetes/addons/storage-provisioner.yaml

	I1003 17:54:12.355689    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)

	I1003 17:54:12.355703    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHHostname

	I1003 17:54:12.355873    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetState

	I1003 17:54:12.355917    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHPort

	I1003 17:54:12.355989    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:54:12.356055    8573 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:54:12.356057    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHKeyPath

	I1003 17:54:12.356085    8573 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:54:12.356098    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | hyperkit pid from json: 8584

	I1003 17:54:12.356190    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHUsername

	I1003 17:54:12.356324    8573 sshutil.go:53] new ssh client: &{IP:192.168.89.26 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/old-k8s-version-999000/id_rsa Username:docker}

	I1003 17:54:12.356508    8573 main.go:141] libmachine: Using API Version  1

	I1003 17:54:12.356540    8573 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:54:12.356626    8573 main.go:141] libmachine: Using API Version  1

	I1003 17:54:12.356661    8573 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:54:12.356907    8573 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:54:12.356946    8573 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:54:12.357095    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetState

	I1003 17:54:12.357198    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:54:12.357301    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | hyperkit pid from json: 8584

	I1003 17:54:12.357452    8573 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:54:12.357478    8573 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:54:12.357533    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .DriverName

	I1003 17:54:12.379697    8573 out.go:177]   - Using image registry.k8s.io/echoserver:1.4

	I1003 17:54:12.358567    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .DriverName

	I1003 17:54:12.367143    8573 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:56242

	I1003 17:54:12.412238    8573 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.16.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml

	I1003 17:54:12.420190    8573 kapi.go:248] "coredns" deployment in "kube-system" namespace and "old-k8s-version-999000" context rescaled to 1 replicas

	I1003 17:54:12.453698    8573 out.go:177]   - Using image docker.io/kubernetesui/dashboard:v2.7.0

	I1003 17:54:12.454249    8573 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:54:12.490833    8573 out.go:177]   - Using image fake.domain/registry.k8s.io/echoserver:1.4

	I1003 17:54:12.490906    8573 start.go:223] Will wait 6m0s for node &{Name: IP:192.168.89.26 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}

	I1003 17:54:12.491615    8573 main.go:141] libmachine: Using API Version  1

	I1003 17:54:12.528605    8573 addons.go:423] installing /etc/kubernetes/addons/metrics-apiservice.yaml

	I1003 17:54:12.549754    8573 addons.go:423] installing /etc/kubernetes/addons/dashboard-ns.yaml

	I1003 17:54:12.570861    8573 out.go:177] * Verifying Kubernetes components...

	I1003 17:54:08.580442    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:11.080763    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:13.081049    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:12.570882    8573 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:54:12.570879    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)

	I1003 17:54:12.570883    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)

	I1003 17:54:12.591636    8573 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I1003 17:54:12.591644    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHHostname

	I1003 17:54:12.591652    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHHostname

	I1003 17:54:12.584448    8573 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.16.0/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^        forward . \/etc\/resolv.conf.*/i \        hosts {\n           192.168.89.1 host.minikube.internal\n           fallthrough\n        }' -e '/^        errors *$/i \        log' | sudo /var/lib/minikube/binaries/v1.16.0/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"

	I1003 17:54:12.591838    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHPort

	I1003 17:54:12.591860    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHPort

	I1003 17:54:12.591976    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHKeyPath

	I1003 17:54:12.592001    8573 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:54:12.592001    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHKeyPath

	I1003 17:54:12.592103    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHUsername

	I1003 17:54:12.592139    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHUsername

	I1003 17:54:12.592139    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetState

	I1003 17:54:12.592201    8573 sshutil.go:53] new ssh client: &{IP:192.168.89.26 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/old-k8s-version-999000/id_rsa Username:docker}

	I1003 17:54:12.592261    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0

	I1003 17:54:12.592270    8573 sshutil.go:53] new ssh client: &{IP:192.168.89.26 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/old-k8s-version-999000/id_rsa Username:docker}

	I1003 17:54:12.592360    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | hyperkit pid from json: 8584

	I1003 17:54:12.593381    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .DriverName

	I1003 17:54:12.593540    8573 addons.go:423] installing /etc/kubernetes/addons/storageclass.yaml

	I1003 17:54:12.593548    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)

	I1003 17:54:12.593556    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHHostname

	I1003 17:54:12.593666    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHPort

	I1003 17:54:12.593795    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHKeyPath

	I1003 17:54:12.593876    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .GetSSHUsername

	I1003 17:54:12.593973    8573 sshutil.go:53] new ssh client: &{IP:192.168.89.26 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/17345-936/.minikube/machines/old-k8s-version-999000/id_rsa Username:docker}

	I1003 17:54:12.817250    8573 addons.go:423] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml

	I1003 17:54:12.817263    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)

	I1003 17:54:12.891140    8573 addons.go:423] installing /etc/kubernetes/addons/metrics-server-deployment.yaml

	I1003 17:54:12.891152    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)

	I1003 17:54:12.907463    8573 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.16.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml

	I1003 17:54:12.931074    8573 addons.go:423] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml

	I1003 17:54:12.931087    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)

	I1003 17:54:13.049748    8573 addons.go:423] installing /etc/kubernetes/addons/metrics-server-rbac.yaml

	I1003 17:54:13.049761    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)

	I1003 17:54:13.068070    8573 addons.go:423] installing /etc/kubernetes/addons/dashboard-configmap.yaml

	I1003 17:54:13.068083    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)

	I1003 17:54:13.079832    8573 addons.go:423] installing /etc/kubernetes/addons/metrics-server-service.yaml

	I1003 17:54:13.079846    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)

	I1003 17:54:13.234616    8573 addons.go:423] installing /etc/kubernetes/addons/dashboard-dp.yaml

	I1003 17:54:13.234630    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)

	I1003 17:54:13.289845    8573 main.go:141] libmachine: Making call to close driver server

	I1003 17:54:13.289861    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .Close

	I1003 17:54:13.290025    8573 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:54:13.290038    8573 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:54:13.290060    8573 main.go:141] libmachine: Making call to close driver server

	I1003 17:54:13.290060    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | Closing plugin on server side

	I1003 17:54:13.290066    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .Close

	I1003 17:54:13.290204    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | Closing plugin on server side

	I1003 17:54:13.290246    8573 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:54:13.290252    8573 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:54:13.317252    8573 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.16.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml

	I1003 17:54:13.448179    8573 addons.go:423] installing /etc/kubernetes/addons/dashboard-role.yaml

	I1003 17:54:13.448191    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)

	I1003 17:54:13.509487    8573 addons.go:423] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml

	I1003 17:54:13.509502    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)

	I1003 17:54:13.540459    8573 addons.go:423] installing /etc/kubernetes/addons/dashboard-sa.yaml

	I1003 17:54:13.540485    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)

	I1003 17:54:13.558158    8573 start.go:923] {"host.minikube.internal": 192.168.89.1} host record injected into CoreDNS's ConfigMap

	I1003 17:54:13.558176    8573 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-999000" to be "Ready" ...

	I1003 17:54:13.558215    8573 main.go:141] libmachine: Making call to close driver server

	I1003 17:54:13.558226    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .Close

	I1003 17:54:13.558505    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | Closing plugin on server side

	I1003 17:54:13.558534    8573 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:54:13.558543    8573 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:54:13.558551    8573 main.go:141] libmachine: Making call to close driver server

	I1003 17:54:13.558560    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .Close

	I1003 17:54:13.558666    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | Closing plugin on server side

	I1003 17:54:13.558690    8573 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:54:13.558702    8573 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:54:13.562444    8573 main.go:141] libmachine: Making call to close driver server

	I1003 17:54:13.562455    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .Close

	I1003 17:54:13.562864    8573 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:54:13.562873    8573 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:54:13.563545    8573 node_ready.go:49] node "old-k8s-version-999000" has status "Ready":"True"

	I1003 17:54:13.563554    8573 node_ready.go:38] duration metric: took 5.364949ms waiting for node "old-k8s-version-999000" to be "Ready" ...

	I1003 17:54:13.563560    8573 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I1003 17:54:13.566447    8573 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5644d7b6d9-j8kjk" in "kube-system" namespace to be "Ready" ...

	I1003 17:54:13.584882    8573 addons.go:423] installing /etc/kubernetes/addons/dashboard-secret.yaml

	I1003 17:54:13.584908    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)

	I1003 17:54:13.654353    8573 addons.go:423] installing /etc/kubernetes/addons/dashboard-svc.yaml

	I1003 17:54:13.654367    8573 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)

	I1003 17:54:13.700288    8573 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.16.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml

	I1003 17:54:14.053899    8573 main.go:141] libmachine: Making call to close driver server

	I1003 17:54:14.053918    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .Close

	I1003 17:54:14.054140    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | Closing plugin on server side

	I1003 17:54:14.054140    8573 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:54:14.054160    8573 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:54:14.054174    8573 main.go:141] libmachine: Making call to close driver server

	I1003 17:54:14.054182    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .Close

	I1003 17:54:14.054318    8573 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:54:14.054328    8573 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:54:14.054339    8573 addons.go:467] Verifying addon metrics-server=true in "old-k8s-version-999000"

	I1003 17:54:14.054347    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | Closing plugin on server side

	I1003 17:54:14.311400    8573 main.go:141] libmachine: Making call to close driver server

	I1003 17:54:14.311415    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .Close

	I1003 17:54:14.311616    8573 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:54:14.311627    8573 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:54:14.311631    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | Closing plugin on server side

	I1003 17:54:14.311640    8573 main.go:141] libmachine: Making call to close driver server

	I1003 17:54:14.311645    8573 main.go:141] libmachine: (old-k8s-version-999000) Calling .Close

	I1003 17:54:14.311774    8573 main.go:141] libmachine: Successfully made call to close driver server

	I1003 17:54:14.311782    8573 main.go:141] libmachine: Making call to close connection to plugin binary

	I1003 17:54:14.311782    8573 main.go:141] libmachine: (old-k8s-version-999000) DBG | Closing plugin on server side

	I1003 17:54:14.350374    8573 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:

		minikube -p old-k8s-version-999000 addons enable metrics-server	

	I1003 17:54:14.408334    8573 out.go:177] * Enabled addons: storage-provisioner, default-storageclass, metrics-server, dashboard

	I1003 17:54:15.579787    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:18.079556    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:14.467049    8573 addons.go:502] enable addons completed in 2.218559282s: enabled=[storage-provisioner default-storageclass metrics-server dashboard]

	I1003 17:54:15.576167    8573 pod_ready.go:102] pod "coredns-5644d7b6d9-j8kjk" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:17.577279    8573 pod_ready.go:102] pod "coredns-5644d7b6d9-j8kjk" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:18.098063    8573 pod_ready.go:92] pod "coredns-5644d7b6d9-j8kjk" in "kube-system" namespace has status "Ready":"True"

	I1003 17:54:18.098075    8573 pod_ready.go:81] duration metric: took 4.531543868s waiting for pod "coredns-5644d7b6d9-j8kjk" in "kube-system" namespace to be "Ready" ...

	I1003 17:54:18.098082    8573 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5644d7b6d9-mmznc" in "kube-system" namespace to be "Ready" ...

	I1003 17:54:18.099554    8573 pod_ready.go:97] error getting pod "coredns-5644d7b6d9-mmznc" in "kube-system" namespace (skipping!): pods "coredns-5644d7b6d9-mmznc" not found

	I1003 17:54:18.099566    8573 pod_ready.go:81] duration metric: took 1.479205ms waiting for pod "coredns-5644d7b6d9-mmznc" in "kube-system" namespace to be "Ready" ...

	E1003 17:54:18.099573    8573 pod_ready.go:66] WaitExtra: waitPodCondition: error getting pod "coredns-5644d7b6d9-mmznc" in "kube-system" namespace (skipping!): pods "coredns-5644d7b6d9-mmznc" not found

	I1003 17:54:18.099578    8573 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-n5dz4" in "kube-system" namespace to be "Ready" ...

	I1003 17:54:18.103634    8573 pod_ready.go:92] pod "kube-proxy-n5dz4" in "kube-system" namespace has status "Ready":"True"

	I1003 17:54:18.103664    8573 pod_ready.go:81] duration metric: took 4.081766ms waiting for pod "kube-proxy-n5dz4" in "kube-system" namespace to be "Ready" ...

	I1003 17:54:18.103671    8573 pod_ready.go:38] duration metric: took 4.540028932s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I1003 17:54:18.103684    8573 api_server.go:52] waiting for apiserver process to appear ...

	I1003 17:54:18.103734    8573 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I1003 17:54:18.122570    8573 api_server.go:72] duration metric: took 5.572756214s to wait for apiserver process to appear ...

	I1003 17:54:18.122582    8573 api_server.go:88] waiting for apiserver healthz status ...

	I1003 17:54:18.122591    8573 api_server.go:253] Checking apiserver healthz at https://192.168.89.26:8443/healthz ...

	I1003 17:54:18.130223    8573 api_server.go:279] https://192.168.89.26:8443/healthz returned 200:

	ok

	I1003 17:54:18.130935    8573 api_server.go:141] control plane version: v1.16.0

	I1003 17:54:18.130945    8573 api_server.go:131] duration metric: took 8.358834ms to wait for apiserver health ...

	I1003 17:54:18.130950    8573 system_pods.go:43] waiting for kube-system pods to appear ...

	I1003 17:54:18.140130    8573 system_pods.go:59] 4 kube-system pods found

	I1003 17:54:18.140144    8573 system_pods.go:61] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:18.140148    8573 system_pods.go:61] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:18.140154    8573 system_pods.go:61] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:18.140159    8573 system_pods.go:61] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:18.140165    8573 system_pods.go:74] duration metric: took 9.209059ms to wait for pod list to return data ...

	I1003 17:54:18.140170    8573 default_sa.go:34] waiting for default service account to be created ...

	I1003 17:54:18.142989    8573 default_sa.go:45] found service account: "default"

	I1003 17:54:18.143001    8573 default_sa.go:55] duration metric: took 2.825797ms for default service account to be created ...

	I1003 17:54:18.143006    8573 system_pods.go:116] waiting for k8s-apps to be running ...

	I1003 17:54:18.153054    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:18.153068    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:18.153073    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:18.153078    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:18.153083    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:18.153094    8573 retry.go:31] will retry after 226.921371ms: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:20.579775    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:23.080230    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:18.383517    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:18.393588    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:18.393597    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:18.393603    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:18.393608    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:18.393618    8573 retry.go:31] will retry after 240.356122ms: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:18.637394    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:18.637408    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:18.637428    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:18.637435    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:18.637439    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:18.637463    8573 retry.go:31] will retry after 339.814107ms: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:18.981107    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:18.981121    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:18.981125    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:18.981131    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:18.981136    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:18.981145    8573 retry.go:31] will retry after 585.900315ms: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:19.571807    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:19.571821    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:19.571825    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:19.571830    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:19.571837    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:19.571846    8573 retry.go:31] will retry after 655.332794ms: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:20.231314    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:20.231328    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:20.231332    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:20.231337    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:20.231342    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:20.231352    8573 retry.go:31] will retry after 838.737678ms: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:21.074088    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:21.074102    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:21.074107    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:21.074112    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:21.074120    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:21.074133    8573 retry.go:31] will retry after 1.137926553s: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:22.216066    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:22.216081    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:22.216086    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:22.216091    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:22.216098    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:22.216108    8573 retry.go:31] will retry after 1.10722135s: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:23.327672    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:23.327685    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:23.327689    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:23.327696    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:23.327701    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:23.327731    8573 retry.go:31] will retry after 1.207694348s: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:25.580152    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:27.580468    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:24.546264    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:24.546278    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:24.546283    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:24.546288    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:24.546293    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:24.546302    8573 retry.go:31] will retry after 1.82803848s: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:26.378967    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:26.378984    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:26.378990    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:26.378999    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:26.379005    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:26.379016    8573 retry.go:31] will retry after 2.743419263s: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:29.581231    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:31.581826    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:29.127146    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:29.127159    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:29.127163    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:29.127168    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:29.127175    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:29.127185    8573 retry.go:31] will retry after 3.427074218s: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:32.558434    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:32.558446    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:32.558453    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:32.558457    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:32.558463    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:32.558471    8573 retry.go:31] will retry after 3.216901488s: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:34.081489    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:36.082366    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:35.779722    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:35.779743    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:35.779748    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:35.779753    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:35.779770    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:35.779800    8573 retry.go:31] will retry after 4.989520383s: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:38.580752    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:41.082054    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:40.772572    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:40.772586    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:40.772590    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:40.772595    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:40.772606    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:40.772616    8573 retry.go:31] will retry after 4.544055698s: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:43.582286    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:46.081515    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:45.320423    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:45.320437    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:45.320441    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:45.320446    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:45.320450    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:45.320459    8573 retry.go:31] will retry after 6.403426327s: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:48.582570    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:51.082030    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:51.727474    8573 system_pods.go:86] 4 kube-system pods found

	I1003 17:54:51.727487    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:54:51.727491    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:54:51.727496    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:54:51.727507    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:54:51.727519    8573 retry.go:31] will retry after 8.616718475s: missing components: etcd, kube-apiserver, kube-controller-manager, kube-scheduler

	I1003 17:54:53.582183    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:56.079678    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:54:58.079876    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:00.081639    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:02.582703    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:00.350005    8573 system_pods.go:86] 5 kube-system pods found

	I1003 17:55:00.350018    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:55:00.350023    8573 system_pods.go:89] "kube-apiserver-old-k8s-version-999000" [009b1577-5dd0-4650-804e-76e7e86627db] Running

	I1003 17:55:00.350027    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:55:00.350033    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:55:00.350037    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:55:00.350048    8573 retry.go:31] will retry after 9.704264403s: missing components: etcd, kube-controller-manager, kube-scheduler

	I1003 17:55:05.083588    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:07.582484    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:10.083062    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:12.582774    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:10.060562    8573 system_pods.go:86] 6 kube-system pods found

	I1003 17:55:10.060576    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:55:10.060580    8573 system_pods.go:89] "kube-apiserver-old-k8s-version-999000" [009b1577-5dd0-4650-804e-76e7e86627db] Running

	I1003 17:55:10.060584    8573 system_pods.go:89] "kube-controller-manager-old-k8s-version-999000" [865a0ba6-242c-4844-8a55-da672f8051e7] Running

	I1003 17:55:10.060587    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:55:10.060592    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:55:10.060597    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:55:10.060607    8573 retry.go:31] will retry after 10.937907543s: missing components: etcd, kube-scheduler

	I1003 17:55:15.082404    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:17.083839    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:19.582476    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:21.583322    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:21.002216    8573 system_pods.go:86] 8 kube-system pods found

	I1003 17:55:21.002231    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:55:21.002235    8573 system_pods.go:89] "etcd-old-k8s-version-999000" [04ac8db7-6ef7-4730-aa3e-b5d67434a9c1] Pending

	I1003 17:55:21.002239    8573 system_pods.go:89] "kube-apiserver-old-k8s-version-999000" [009b1577-5dd0-4650-804e-76e7e86627db] Running

	I1003 17:55:21.002243    8573 system_pods.go:89] "kube-controller-manager-old-k8s-version-999000" [865a0ba6-242c-4844-8a55-da672f8051e7] Running

	I1003 17:55:21.002246    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:55:21.002250    8573 system_pods.go:89] "kube-scheduler-old-k8s-version-999000" [e212b2b6-1299-4113-87e1-5962ff7727b0] Running

	I1003 17:55:21.002255    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:55:21.002260    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:55:21.002269    8573 retry.go:31] will retry after 18.162201783s: missing components: etcd

	I1003 17:55:24.082965    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:26.582005    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:29.083368    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:31.581918    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:33.583032    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:36.083127    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:38.085251    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:39.172149    8573 system_pods.go:86] 8 kube-system pods found

	I1003 17:55:39.172163    8573 system_pods.go:89] "coredns-5644d7b6d9-j8kjk" [2bd2b4ca-3cd4-4b75-9d9b-c32264783550] Running

	I1003 17:55:39.172167    8573 system_pods.go:89] "etcd-old-k8s-version-999000" [04ac8db7-6ef7-4730-aa3e-b5d67434a9c1] Running

	I1003 17:55:39.172171    8573 system_pods.go:89] "kube-apiserver-old-k8s-version-999000" [009b1577-5dd0-4650-804e-76e7e86627db] Running

	I1003 17:55:39.172174    8573 system_pods.go:89] "kube-controller-manager-old-k8s-version-999000" [865a0ba6-242c-4844-8a55-da672f8051e7] Running

	I1003 17:55:39.172177    8573 system_pods.go:89] "kube-proxy-n5dz4" [4a3d0e69-ef75-49b8-9c56-f25ed5abae9f] Running

	I1003 17:55:39.172181    8573 system_pods.go:89] "kube-scheduler-old-k8s-version-999000" [e212b2b6-1299-4113-87e1-5962ff7727b0] Running

	I1003 17:55:39.172201    8573 system_pods.go:89] "metrics-server-74d5856cc6-d65gh" [f88b6c39-6469-4f73-b621-ed79a8d324e3] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])

	I1003 17:55:39.172209    8573 system_pods.go:89] "storage-provisioner" [8cc8064c-9411-4a9f-a78a-4f024c7c4ba5] Running

	I1003 17:55:39.172215    8573 system_pods.go:126] duration metric: took 1m21.026542249s to wait for k8s-apps to be running ...

	I1003 17:55:39.172221    8573 system_svc.go:44] waiting for kubelet service to be running ....

	I1003 17:55:39.172277    8573 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I1003 17:55:39.180707    8573 system_svc.go:56] duration metric: took 8.481644ms WaitForService to wait for kubelet.

	I1003 17:55:39.180718    8573 kubeadm.go:581] duration metric: took 1m26.628244573s to wait for : map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] ...

	I1003 17:55:39.180730    8573 node_conditions.go:102] verifying NodePressure condition ...

	I1003 17:55:39.182993    8573 node_conditions.go:122] node storage ephemeral capacity is 17784752Ki

	I1003 17:55:39.183004    8573 node_conditions.go:123] node cpu capacity is 2

	I1003 17:55:39.183013    8573 node_conditions.go:105] duration metric: took 2.277045ms to run NodePressure ...

	I1003 17:55:39.183021    8573 start.go:228] waiting for startup goroutines ...

	I1003 17:55:39.183026    8573 start.go:233] waiting for cluster config update ...

	I1003 17:55:39.183033    8573 start.go:242] writing updated cluster config ...

	I1003 17:55:39.183349    8573 ssh_runner.go:195] Run: rm -f paused

	I1003 17:55:39.220899    8573 start.go:600] kubectl: 1.27.2, cluster: 1.16.0 (minor skew: 11)

	I1003 17:55:39.242842    8573 out.go:177] 

	W1003 17:55:39.263630    8573 out.go:239] ! /usr/local/bin/kubectl is version 1.27.2, which may have incompatibilities with Kubernetes 1.16.0.

	I1003 17:55:39.284643    8573 out.go:177]   - Want kubectl v1.16.0? Try 'minikube kubectl -- get pods -A'

	I1003 17:55:39.326677    8573 out.go:177] * Done! kubectl is now configured to use "old-k8s-version-999000" cluster and "default" namespace by default

	I1003 17:55:40.583721    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:43.084062    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:45.582730    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	I1003 17:55:47.584488    8925 pod_ready.go:102] pod "metrics-server-57f55c9bc5-hq7nv" in "kube-system" namespace has status "Ready":"False"

	* 

	* ==> Docker <==

	* -- Journal begins at Wed 2023-10-04 00:47:26 UTC, ends at Wed 2023-10-04 00:55:50 UTC. --

	Oct 04 00:54:25 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:54:25.551612975Z" level=warning msg="cleaning up after shim disconnected" id=ffac3ac9f6029878508f1f826fbb093c552d7c5526416ba17c1cf074dcd3a009 namespace=moby

	Oct 04 00:54:25 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:54:25.551621086Z" level=info msg="cleaning up dead shim" namespace=moby

	Oct 04 00:54:36 old-k8s-version-999000 dockerd[1174]: time="2023-10-04T00:54:36.468161252Z" level=warning msg="Error getting v2 registry: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host"

	Oct 04 00:54:36 old-k8s-version-999000 dockerd[1174]: time="2023-10-04T00:54:36.468180608Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host"

	Oct 04 00:54:36 old-k8s-version-999000 dockerd[1174]: time="2023-10-04T00:54:36.469276074Z" level=error msg="Handler for POST /images/create returned error: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host"

	Oct 04 00:54:47 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:54:47.533861302Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Oct 04 00:54:47 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:54:47.534715229Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Oct 04 00:54:47 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:54:47.534752910Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Oct 04 00:54:47 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:54:47.534762555Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Oct 04 00:54:47 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:54:47.821323266Z" level=info msg="shim disconnected" id=49393a7eb9c5e553fe76d8641e5a00217fe810e33f0981dea1810ce6cf3aaeb6 namespace=moby

	Oct 04 00:54:47 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:54:47.821360750Z" level=warning msg="cleaning up after shim disconnected" id=49393a7eb9c5e553fe76d8641e5a00217fe810e33f0981dea1810ce6cf3aaeb6 namespace=moby

	Oct 04 00:54:47 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:54:47.821367781Z" level=info msg="cleaning up dead shim" namespace=moby

	Oct 04 00:54:47 old-k8s-version-999000 dockerd[1174]: time="2023-10-04T00:54:47.822124470Z" level=info msg="ignoring event" container=49393a7eb9c5e553fe76d8641e5a00217fe810e33f0981dea1810ce6cf3aaeb6 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Oct 04 00:54:47 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:54:47.834354040Z" level=warning msg="cleanup warnings time=\"2023-10-04T00:54:47Z\" level=warning msg=\"failed to remove runc container\" error=\"runc did not terminate successfully: exit status 255: \" runtime=io.containerd.runc.v2\n" namespace=moby

	Oct 04 00:54:59 old-k8s-version-999000 dockerd[1174]: time="2023-10-04T00:54:59.477661390Z" level=warning msg="Error getting v2 registry: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host"

	Oct 04 00:54:59 old-k8s-version-999000 dockerd[1174]: time="2023-10-04T00:54:59.477679863Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host"

	Oct 04 00:54:59 old-k8s-version-999000 dockerd[1174]: time="2023-10-04T00:54:59.478771066Z" level=error msg="Handler for POST /images/create returned error: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host"

	Oct 04 00:55:18 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:55:18.541937974Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1

	Oct 04 00:55:18 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:55:18.541985944Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Oct 04 00:55:18 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:55:18.542007488Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1

	Oct 04 00:55:18 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:55:18.542046535Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

	Oct 04 00:55:18 old-k8s-version-999000 dockerd[1174]: time="2023-10-04T00:55:18.823144566Z" level=info msg="ignoring event" container=cb3c8e6b26b7502234e3561f4f201174fcf79d93c7a32a3282d8b8979425d10a module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

	Oct 04 00:55:18 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:55:18.823823990Z" level=info msg="shim disconnected" id=cb3c8e6b26b7502234e3561f4f201174fcf79d93c7a32a3282d8b8979425d10a namespace=moby

	Oct 04 00:55:18 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:55:18.824073913Z" level=warning msg="cleaning up after shim disconnected" id=cb3c8e6b26b7502234e3561f4f201174fcf79d93c7a32a3282d8b8979425d10a namespace=moby

	Oct 04 00:55:18 old-k8s-version-999000 dockerd[1180]: time="2023-10-04T00:55:18.824114100Z" level=info msg="cleaning up dead shim" namespace=moby

	* 

	* ==> container status <==

	* CONTAINER ID   IMAGE                    COMMAND                  CREATED              STATUS                      PORTS     NAMES

	cb3c8e6b26b7   a90209bb39e3             "nginx -g 'daemon of…"   32 seconds ago       Exited (1) 31 seconds ago             k8s_dashboard-metrics-scraper_dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard_6822b5ec-4e79-434e-9476-780d39223604_3

	3483d762d651   kubernetesui/dashboard   "/dashboard --insecu…"   About a minute ago   Up About a minute                     k8s_kubernetes-dashboard_kubernetes-dashboard-84b68f675b-r7twb_kubernetes-dashboard_690221d1-6163-48d2-a73b-b254e3fde483_0

	a5181ccc884f   k8s.gcr.io/pause:3.1     "/pause"                 About a minute ago   Up About a minute                     k8s_POD_metrics-server-74d5856cc6-d65gh_kube-system_f88b6c39-6469-4f73-b621-ed79a8d324e3_0

	151b4ebba63c   k8s.gcr.io/pause:3.1     "/pause"                 About a minute ago   Up About a minute                     k8s_POD_dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard_6822b5ec-4e79-434e-9476-780d39223604_0

	64401d4f85b4   k8s.gcr.io/pause:3.1     "/pause"                 About a minute ago   Up About a minute                     k8s_POD_kubernetes-dashboard-84b68f675b-r7twb_kubernetes-dashboard_690221d1-6163-48d2-a73b-b254e3fde483_0

	c9f45072dd2f   6e38f40d628d             "/storage-provisioner"   About a minute ago   Up About a minute                     k8s_storage-provisioner_storage-provisioner_kube-system_8cc8064c-9411-4a9f-a78a-4f024c7c4ba5_0

	bba95c09f09b   k8s.gcr.io/pause:3.1     "/pause"                 About a minute ago   Up About a minute                     k8s_POD_storage-provisioner_kube-system_8cc8064c-9411-4a9f-a78a-4f024c7c4ba5_0

	09395282d81f   bf261d157914             "/coredns -conf /etc…"   About a minute ago   Up About a minute                     k8s_coredns_coredns-5644d7b6d9-j8kjk_kube-system_2bd2b4ca-3cd4-4b75-9d9b-c32264783550_0

	cae666a84d48   c21b0c7400f9             "/usr/local/bin/kube…"   About a minute ago   Up About a minute                     k8s_kube-proxy_kube-proxy-n5dz4_kube-system_4a3d0e69-ef75-49b8-9c56-f25ed5abae9f_0

	4cb190773e37   k8s.gcr.io/pause:3.1     "/pause"                 About a minute ago   Up About a minute                     k8s_POD_coredns-5644d7b6d9-j8kjk_kube-system_2bd2b4ca-3cd4-4b75-9d9b-c32264783550_0

	c0b09dece5d2   k8s.gcr.io/pause:3.1     "/pause"                 About a minute ago   Up About a minute                     k8s_POD_kube-proxy-n5dz4_kube-system_4a3d0e69-ef75-49b8-9c56-f25ed5abae9f_0

	d24b09cd034f   06a629a7e51c             "kube-controller-man…"   2 minutes ago        Up 2 minutes                          k8s_kube-controller-manager_kube-controller-manager-old-k8s-version-999000_kube-system_7376ddb4f190a0ded9394063437bcb4e_0

	760e2fef4c00   b305571ca60a             "kube-apiserver --ad…"   2 minutes ago        Up 2 minutes                          k8s_kube-apiserver_kube-apiserver-old-k8s-version-999000_kube-system_ac7e9adb92d45d87846ed91a24f161f0_0

	edb06b1a9f79   b2756210eeab             "etcd --advertise-cl…"   2 minutes ago        Up 2 minutes                          k8s_etcd_etcd-old-k8s-version-999000_kube-system_2fe1ecf8263b33a12f43c2331d720c77_0

	709bba9aba99   301ddc62b80b             "kube-scheduler --au…"   2 minutes ago        Up 2 minutes                          k8s_kube-scheduler_kube-scheduler-old-k8s-version-999000_kube-system_b3d303074fe0ca1d42a8bd9ed248df09_0

	0369fc6031a6   k8s.gcr.io/pause:3.1     "/pause"                 2 minutes ago        Up 2 minutes                          k8s_POD_kube-controller-manager-old-k8s-version-999000_kube-system_7376ddb4f190a0ded9394063437bcb4e_0

	98c09d7e51c6   k8s.gcr.io/pause:3.1     "/pause"                 2 minutes ago        Up 2 minutes                          k8s_POD_kube-apiserver-old-k8s-version-999000_kube-system_ac7e9adb92d45d87846ed91a24f161f0_0

	7e96eae037d5   k8s.gcr.io/pause:3.1     "/pause"                 2 minutes ago        Up 2 minutes                          k8s_POD_etcd-old-k8s-version-999000_kube-system_2fe1ecf8263b33a12f43c2331d720c77_0

	006e25dc5a18   k8s.gcr.io/pause:3.1     "/pause"                 2 minutes ago        Up 2 minutes                          k8s_POD_kube-scheduler-old-k8s-version-999000_kube-system_b3d303074fe0ca1d42a8bd9ed248df09_0

	time="2023-10-04T00:55:50Z" level=fatal msg="validate service connection: validate CRI v1 runtime API for endpoint \"unix:///var/run/dockershim.sock\": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService"

	* 

	* ==> coredns [09395282d81f] <==

	* .:53

	2023-10-04T00:54:13.576Z [INFO] plugin/reload: Running configuration MD5 = f64cb9b977c7dfca58c4fab108535a76

	2023-10-04T00:54:13.576Z [INFO] CoreDNS-1.6.2

	2023-10-04T00:54:13.576Z [INFO] linux/amd64, go1.12.8, 795a3eb

	CoreDNS-1.6.2

	linux/amd64, go1.12.8, 795a3eb

	[INFO] Reloading

	2023-10-04T00:54:43.203Z [INFO] plugin/reload: Running configuration MD5 = 8f1c6898b10c905bc8ba86d9d1ac791d

	[INFO] Reloading complete

	2023-10-04T00:54:43.207Z [INFO] 127.0.0.1:53559 - 20724 "HINFO IN 4676725647575672703.8872169051848729423. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.004206028s

	* 

	* ==> describe nodes <==

	* Name:               old-k8s-version-999000

	Roles:              master

	Labels:             beta.kubernetes.io/arch=amd64

	                    beta.kubernetes.io/os=linux

	                    kubernetes.io/arch=amd64

	                    kubernetes.io/hostname=old-k8s-version-999000

	                    kubernetes.io/os=linux

	                    minikube.k8s.io/commit=d9526fa6c1a1bb1b20f95e15606f1308e308d84a

	                    minikube.k8s.io/name=old-k8s-version-999000

	                    minikube.k8s.io/primary=true

	                    minikube.k8s.io/updated_at=2023_10_03T17_53_57_0700

	                    minikube.k8s.io/version=v1.31.2

	                    node-role.kubernetes.io/master=

	Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock

	                    node.alpha.kubernetes.io/ttl: 0

	                    volumes.kubernetes.io/controller-managed-attach-detach: true

	CreationTimestamp:  Wed, 04 Oct 2023 00:53:52 +0000

	Taints:             <none>

	Unschedulable:      false

	Conditions:

	  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message

	  ----             ------  -----------------                 ------------------                ------                       -------

	  MemoryPressure   False   Wed, 04 Oct 2023 00:54:53 +0000   Wed, 04 Oct 2023 00:53:49 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available

	  DiskPressure     False   Wed, 04 Oct 2023 00:54:53 +0000   Wed, 04 Oct 2023 00:53:49 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure

	  PIDPressure      False   Wed, 04 Oct 2023 00:54:53 +0000   Wed, 04 Oct 2023 00:53:49 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available

	  Ready            True    Wed, 04 Oct 2023 00:54:53 +0000   Wed, 04 Oct 2023 00:53:49 +0000   KubeletReady                 kubelet is posting ready status

	Addresses:

	  InternalIP:  192.168.89.26

	  Hostname:    old-k8s-version-999000

	Capacity:

	 cpu:                2

	 ephemeral-storage:  17784752Ki

	 hugepages-2Mi:      0

	 memory:             2166052Ki

	 pods:               110

	Allocatable:

	 cpu:                2

	 ephemeral-storage:  17784752Ki

	 hugepages-2Mi:      0

	 memory:             2166052Ki

	 pods:               110

	System Info:

	 Machine ID:                 310ca2150ba84e9dbe853e7e3bf12bd0

	 System UUID:                2f4011ee-0000-0000-9654-149d997cd0f1

	 Boot ID:                    f9383ec7-c92a-4730-ba4b-271960898723

	 Kernel Version:             5.10.57

	 OS Image:                   Buildroot 2021.02.12

	 Operating System:           linux

	 Architecture:               amd64

	 Container Runtime Version:  docker://24.0.6

	 Kubelet Version:            v1.16.0

	 Kube-Proxy Version:         v1.16.0

	PodCIDR:                     10.244.0.0/24

	PodCIDRs:                    10.244.0.0/24

	Non-terminated Pods:         (10 in total)

	  Namespace                  Name                                              CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE

	  ---------                  ----                                              ------------  ----------  ---------------  -------------  ---

	  kube-system                coredns-5644d7b6d9-j8kjk                          100m (5%!)(MISSING)     0 (0%!)(MISSING)      70Mi (3%!)(MISSING)        170Mi (8%!)(MISSING)     98s

	  kube-system                etcd-old-k8s-version-999000                       0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         30s

	  kube-system                kube-apiserver-old-k8s-version-999000             250m (12%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         54s

	  kube-system                kube-controller-manager-old-k8s-version-999000    200m (10%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         46s

	  kube-system                kube-proxy-n5dz4                                  0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         98s

	  kube-system                kube-scheduler-old-k8s-version-999000             100m (5%!)(MISSING)     0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         39s

	  kube-system                metrics-server-74d5856cc6-d65gh                   100m (5%!)(MISSING)     0 (0%!)(MISSING)      200Mi (9%!)(MISSING)       0 (0%!)(MISSING)         96s

	  kube-system                storage-provisioner                               0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         97s

	  kubernetes-dashboard       dashboard-metrics-scraper-d6b4b5544-xrh8x         0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         96s

	  kubernetes-dashboard       kubernetes-dashboard-84b68f675b-r7twb             0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         96s

	Allocated resources:

	  (Total limits may be over 100 percent, i.e., overcommitted.)

	  Resource           Requests     Limits

	  --------           --------     ------

	  cpu                750m (37%!)(MISSING)   0 (0%!)(MISSING)

	  memory             270Mi (12%!)(MISSING)  170Mi (8%!)(MISSING)

	  ephemeral-storage  0 (0%!)(MISSING)       0 (0%!)(MISSING)

	Events:

	  Type    Reason                   Age                  From                                Message

	  ----    ------                   ----                 ----                                -------

	  Normal  NodeHasSufficientMemory  2m3s (x8 over 2m3s)  kubelet, old-k8s-version-999000     Node old-k8s-version-999000 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    2m3s (x8 over 2m3s)  kubelet, old-k8s-version-999000     Node old-k8s-version-999000 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     2m3s (x7 over 2m3s)  kubelet, old-k8s-version-999000     Node old-k8s-version-999000 status is now: NodeHasSufficientPID

	  Normal  Starting                 97s                  kube-proxy, old-k8s-version-999000  Starting kube-proxy.

	* 

	* ==> dmesg <==

	* [  +5.190529] ACPI Error: Could not enable RealTimeClock event (20200925/evxfevnt-182)

	[  +0.000000] ACPI Warning: Could not enable fixed event - RealTimeClock (4) (20200925/evxface-618)

	[  +0.007918] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2

	[  +2.320502] systemd-fstab-generator[125]: Ignoring "noauto" for root device

	[  +0.045331] systemd[1]: systemd-journald.service: unit configures an IP firewall, but the local system does not support BPF/cgroup firewalling.

	[  +0.000002] systemd[1]: (This warning is only shown for the first unit using IP firewalling.)

	[  +1.966509] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory

	[  +0.000005] NFSD: unable to find recovery directory /var/lib/nfs/v4recovery

	[  +0.000001] NFSD: Unable to initialize client recovery tracking! (-2)

	[  +3.774359] systemd-fstab-generator[523]: Ignoring "noauto" for root device

	[  +0.093349] systemd-fstab-generator[534]: Ignoring "noauto" for root device

	[  +0.825187] systemd-fstab-generator[787]: Ignoring "noauto" for root device

	[  +0.220022] systemd-fstab-generator[824]: Ignoring "noauto" for root device

	[  +0.092233] systemd-fstab-generator[835]: Ignoring "noauto" for root device

	[  +0.098491] systemd-fstab-generator[848]: Ignoring "noauto" for root device

	[  +1.216465] kauditd_printk_skb: 30 callbacks suppressed

	[  +5.031315] systemd-fstab-generator[1146]: Ignoring "noauto" for root device

	[  +1.275146] kauditd_printk_skb: 29 callbacks suppressed

	[ +14.443982] systemd-fstab-generator[1621]: Ignoring "noauto" for root device

	[Oct 4 00:48] kauditd_printk_skb: 29 callbacks suppressed

	[  +0.089480] kmem.limit_in_bytes is deprecated and will be removed. Please report your usecase to linux-mm@kvack.org if you depend on this functionality.

	[ +22.716749] kauditd_printk_skb: 5 callbacks suppressed

	[Oct 4 00:53] systemd-fstab-generator[7030]: Ignoring "noauto" for root device

	[Oct 4 00:54] kauditd_printk_skb: 4 callbacks suppressed

	[  +2.550556] TCP: eth0: Driver has suspect GRO implementation, TCP performance may be compromised.

	* 

	* ==> etcd [edb06b1a9f79] <==

	* 2023-10-04 00:53:49.373645 I | raft: newRaft 8fabf80c96c2efcd [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]

	2023-10-04 00:53:49.373655 I | raft: 8fabf80c96c2efcd became follower at term 1

	2023-10-04 00:53:49.376619 W | auth: simple token is not cryptographically signed

	2023-10-04 00:53:49.381016 I | etcdserver: starting server... [version: 3.3.15, cluster version: to_be_decided]

	2023-10-04 00:53:49.381533 I | etcdserver: 8fabf80c96c2efcd as single-node; fast-forwarding 9 ticks (election ticks 10)

	2023-10-04 00:53:49.381911 I | etcdserver/membership: added member 8fabf80c96c2efcd [https://192.168.89.26:2380] to cluster a292607cda8d3231

	2023-10-04 00:53:49.383651 I | embed: ClientTLS: cert = /var/lib/minikube/certs/etcd/server.crt, key = /var/lib/minikube/certs/etcd/server.key, ca = , trusted-ca = /var/lib/minikube/certs/etcd/ca.crt, client-cert-auth = true, crl-file = 

	2023-10-04 00:53:49.383893 I | embed: listening for metrics on http://192.168.89.26:2381

	2023-10-04 00:53:49.384094 I | embed: listening for metrics on http://127.0.0.1:2381

	2023-10-04 00:53:49.476426 I | raft: 8fabf80c96c2efcd is starting a new election at term 1

	2023-10-04 00:53:49.476462 I | raft: 8fabf80c96c2efcd became candidate at term 2

	2023-10-04 00:53:49.476471 I | raft: 8fabf80c96c2efcd received MsgVoteResp from 8fabf80c96c2efcd at term 2

	2023-10-04 00:53:49.476477 I | raft: 8fabf80c96c2efcd became leader at term 2

	2023-10-04 00:53:49.476481 I | raft: raft.node: 8fabf80c96c2efcd elected leader 8fabf80c96c2efcd at term 2

	2023-10-04 00:53:49.476760 I | etcdserver: setting up the initial cluster version to 3.3

	2023-10-04 00:53:49.477282 N | etcdserver/membership: set the initial cluster version to 3.3

	2023-10-04 00:53:49.477439 I | etcdserver/api: enabled capabilities for version 3.3

	2023-10-04 00:53:49.477527 I | etcdserver: published {Name:old-k8s-version-999000 ClientURLs:[https://192.168.89.26:2379]} to cluster a292607cda8d3231

	2023-10-04 00:53:49.481905 I | embed: ready to serve client requests

	2023-10-04 00:53:49.482705 I | embed: serving client requests on 127.0.0.1:2379

	2023-10-04 00:53:49.482781 I | embed: ready to serve client requests

	2023-10-04 00:53:49.483443 I | embed: serving client requests on 192.168.89.26:2379

	2023-10-04 00:54:12.682619 W | etcdserver: read-only range request "key:\"/registry/configmaps/kube-system/coredns\" " with result "range_response_count:1 size:466" took too long (185.642327ms) to execute

	2023-10-04 00:54:12.690677 W | etcdserver: read-only range request "key:\"/registry/pods/kube-system/coredns-5644d7b6d9-mmznc\" " with result "range_response_count:1 size:1694" took too long (160.055274ms) to execute

	2023-10-04 00:54:12.690801 W | etcdserver: read-only range request "key:\"/registry/pods/kube-system/coredns-5644d7b6d9-j8kjk\" " with result "range_response_count:1 size:1367" took too long (152.662738ms) to execute

	* 

	* ==> kernel <==

	*  00:55:51 up 8 min,  0 users,  load average: 0.15, 0.17, 0.08

	Linux old-k8s-version-999000 5.10.57 #1 SMP Mon Sep 18 23:12:38 UTC 2023 x86_64 GNU/Linux

	PRETTY_NAME="Buildroot 2021.02.12"

	* 

	* ==> kube-apiserver [760e2fef4c00] <==

	* I1004 00:53:53.829137       1 controller.go:130] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).

	I1004 00:53:53.843858       1 storage_scheduling.go:139] created PriorityClass system-node-critical with value 2000001000

	I1004 00:53:53.847839       1 storage_scheduling.go:139] created PriorityClass system-cluster-critical with value 2000000000

	I1004 00:53:53.847869       1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.

	I1004 00:53:54.573165       1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io

	I1004 00:53:55.618209       1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io

	I1004 00:53:55.897895       1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io

	W1004 00:53:56.237808       1 lease.go:222] Resetting endpoints for master service "kubernetes" to [192.168.89.26]

	I1004 00:53:56.238497       1 controller.go:606] quota admission added evaluator for: endpoints

	I1004 00:53:57.103824       1 controller.go:606] quota admission added evaluator for: serviceaccounts

	I1004 00:53:57.551315       1 controller.go:606] quota admission added evaluator for: deployments.apps

	I1004 00:53:57.842856       1 controller.go:606] quota admission added evaluator for: daemonsets.apps

	I1004 00:54:12.258179       1 controller.go:606] quota admission added evaluator for: replicasets.apps

	I1004 00:54:12.277442       1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps

	I1004 00:54:12.335767       1 controller.go:606] quota admission added evaluator for: events.events.k8s.io

	I1004 00:54:15.485045       1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io

	W1004 00:54:15.485129       1 handler_proxy.go:99] no RequestInfo found in the context

	E1004 00:54:15.485178       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable

	, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]

	I1004 00:54:15.485185       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.

	I1004 00:55:15.486187       1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io

	W1004 00:55:15.486274       1 handler_proxy.go:99] no RequestInfo found in the context

	E1004 00:55:15.486364       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable

	, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]

	I1004 00:55:15.486399       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.

	* 

	* ==> kube-controller-manager [d24b09cd034f] <==

	* I1004 00:54:14.219503       1 event.go:255] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kubernetes-dashboard", Name:"kubernetes-dashboard-84b68f675b", UID:"78f4d5bc-059d-433a-b102-fb4ccc16178e", APIVersion:"apps/v1", ResourceVersion:"416", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "kubernetes-dashboard-84b68f675b-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	E1004 00:54:14.227474       1 replica_set.go:450] Sync "kubernetes-dashboard/kubernetes-dashboard-84b68f675b" failed with pods "kubernetes-dashboard-84b68f675b-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	E1004 00:54:14.231534       1 replica_set.go:450] Sync "kubernetes-dashboard/dashboard-metrics-scraper-d6b4b5544" failed with pods "dashboard-metrics-scraper-d6b4b5544-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I1004 00:54:14.232309       1 event.go:255] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kubernetes-dashboard", Name:"dashboard-metrics-scraper-d6b4b5544", UID:"a507cef1-35b8-4673-a79c-cd37a0c2a97d", APIVersion:"apps/v1", ResourceVersion:"418", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "dashboard-metrics-scraper-d6b4b5544-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	E1004 00:54:14.243737       1 replica_set.go:450] Sync "kubernetes-dashboard/kubernetes-dashboard-84b68f675b" failed with pods "kubernetes-dashboard-84b68f675b-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I1004 00:54:14.244259       1 event.go:255] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kubernetes-dashboard", Name:"kubernetes-dashboard-84b68f675b", UID:"78f4d5bc-059d-433a-b102-fb4ccc16178e", APIVersion:"apps/v1", ResourceVersion:"422", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "kubernetes-dashboard-84b68f675b-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	E1004 00:54:14.247224       1 replica_set.go:450] Sync "kubernetes-dashboard/dashboard-metrics-scraper-d6b4b5544" failed with pods "dashboard-metrics-scraper-d6b4b5544-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I1004 00:54:14.247381       1 event.go:255] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kubernetes-dashboard", Name:"dashboard-metrics-scraper-d6b4b5544", UID:"a507cef1-35b8-4673-a79c-cd37a0c2a97d", APIVersion:"apps/v1", ResourceVersion:"418", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "dashboard-metrics-scraper-d6b4b5544-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	E1004 00:54:14.253635       1 replica_set.go:450] Sync "kubernetes-dashboard/kubernetes-dashboard-84b68f675b" failed with pods "kubernetes-dashboard-84b68f675b-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I1004 00:54:14.254059       1 event.go:255] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kubernetes-dashboard", Name:"kubernetes-dashboard-84b68f675b", UID:"78f4d5bc-059d-433a-b102-fb4ccc16178e", APIVersion:"apps/v1", ResourceVersion:"422", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "kubernetes-dashboard-84b68f675b-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	E1004 00:54:14.265054       1 replica_set.go:450] Sync "kubernetes-dashboard/dashboard-metrics-scraper-d6b4b5544" failed with pods "dashboard-metrics-scraper-d6b4b5544-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I1004 00:54:14.265210       1 event.go:255] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kubernetes-dashboard", Name:"dashboard-metrics-scraper-d6b4b5544", UID:"a507cef1-35b8-4673-a79c-cd37a0c2a97d", APIVersion:"apps/v1", ResourceVersion:"418", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "dashboard-metrics-scraper-d6b4b5544-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	E1004 00:54:14.272287       1 replica_set.go:450] Sync "kubernetes-dashboard/kubernetes-dashboard-84b68f675b" failed with pods "kubernetes-dashboard-84b68f675b-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I1004 00:54:14.272487       1 event.go:255] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kubernetes-dashboard", Name:"kubernetes-dashboard-84b68f675b", UID:"78f4d5bc-059d-433a-b102-fb4ccc16178e", APIVersion:"apps/v1", ResourceVersion:"422", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "kubernetes-dashboard-84b68f675b-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	E1004 00:54:14.279080       1 replica_set.go:450] Sync "kubernetes-dashboard/dashboard-metrics-scraper-d6b4b5544" failed with pods "dashboard-metrics-scraper-d6b4b5544-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I1004 00:54:14.279126       1 event.go:255] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kubernetes-dashboard", Name:"dashboard-metrics-scraper-d6b4b5544", UID:"a507cef1-35b8-4673-a79c-cd37a0c2a97d", APIVersion:"apps/v1", ResourceVersion:"418", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "dashboard-metrics-scraper-d6b4b5544-" is forbidden: error looking up service account kubernetes-dashboard/kubernetes-dashboard: serviceaccount "kubernetes-dashboard" not found

	I1004 00:54:14.318684       1 event.go:255] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kubernetes-dashboard", Name:"kubernetes-dashboard-84b68f675b", UID:"78f4d5bc-059d-433a-b102-fb4ccc16178e", APIVersion:"apps/v1", ResourceVersion:"422", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: kubernetes-dashboard-84b68f675b-r7twb

	I1004 00:54:14.366224       1 event.go:255] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kubernetes-dashboard", Name:"dashboard-metrics-scraper-d6b4b5544", UID:"a507cef1-35b8-4673-a79c-cd37a0c2a97d", APIVersion:"apps/v1", ResourceVersion:"418", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: dashboard-metrics-scraper-d6b4b5544-xrh8x

	I1004 00:54:14.980686       1 event.go:255] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kube-system", Name:"metrics-server-74d5856cc6", UID:"e7ee2a31-ecd2-465a-b24d-01951656f5e0", APIVersion:"apps/v1", ResourceVersion:"381", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: metrics-server-74d5856cc6-d65gh

	E1004 00:54:43.059175       1 resource_quota_controller.go:407] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request

	W1004 00:54:44.816247       1 garbagecollector.go:640] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]

	E1004 00:55:13.311591       1 resource_quota_controller.go:407] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request

	W1004 00:55:16.819549       1 garbagecollector.go:640] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]

	E1004 00:55:43.564265       1 resource_quota_controller.go:407] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request

	W1004 00:55:48.823254       1 garbagecollector.go:640] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]

	* 

	* ==> kube-proxy [cae666a84d48] <==

	* W1004 00:54:13.398535       1 server_others.go:329] Flag proxy-mode="" unknown, assuming iptables proxy

	I1004 00:54:13.405139       1 node.go:135] Successfully retrieved node IP: 192.168.89.26

	I1004 00:54:13.405179       1 server_others.go:149] Using iptables Proxier.

	I1004 00:54:13.405433       1 server.go:529] Version: v1.16.0

	I1004 00:54:13.407643       1 config.go:313] Starting service config controller

	I1004 00:54:13.407655       1 shared_informer.go:197] Waiting for caches to sync for service config

	I1004 00:54:13.407665       1 config.go:131] Starting endpoints config controller

	I1004 00:54:13.407671       1 shared_informer.go:197] Waiting for caches to sync for endpoints config

	I1004 00:54:13.507850       1 shared_informer.go:204] Caches are synced for endpoints config 

	I1004 00:54:13.507867       1 shared_informer.go:204] Caches are synced for service config 

	* 

	* ==> kube-scheduler [709bba9aba99] <==

	* W1004 00:53:52.891075       1 authentication.go:79] Authentication is disabled

	I1004 00:53:52.891199       1 deprecated_insecure_serving.go:51] Serving healthz insecurely on [::]:10251

	I1004 00:53:52.891720       1 secure_serving.go:123] Serving securely on 127.0.0.1:10259

	E1004 00:53:52.949476       1 reflector.go:123] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:236: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	E1004 00:53:52.953583       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope

	E1004 00:53:52.953704       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	E1004 00:53:52.953843       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	E1004 00:53:52.953941       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope

	E1004 00:53:52.954081       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	E1004 00:53:52.954187       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	E1004 00:53:52.954333       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	E1004 00:53:52.954446       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope

	E1004 00:53:52.954617       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	E1004 00:53:52.963895       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	E1004 00:53:53.950931       1 reflector.go:123] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:236: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	E1004 00:53:53.954630       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope

	E1004 00:53:53.956550       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	E1004 00:53:53.958876       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	E1004 00:53:53.959807       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope

	E1004 00:53:53.960650       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	E1004 00:53:53.964128       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	E1004 00:53:53.964191       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	E1004 00:53:53.967589       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope

	E1004 00:53:53.968264       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	E1004 00:53:53.970319       1 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	* 

	* ==> kubelet <==

	* -- Journal begins at Wed 2023-10-04 00:47:26 UTC, ends at Wed 2023-10-04 00:55:51 UTC. --

	Oct 04 00:54:27 old-k8s-version-999000 kubelet[7048]: E1004 00:54:27.245345    7048 pod_workers.go:191] Error syncing pod 6822b5ec-4e79-434e-9476-780d39223604 ("dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"

	Oct 04 00:54:32 old-k8s-version-999000 kubelet[7048]: E1004 00:54:32.920475    7048 pod_workers.go:191] Error syncing pod 6822b5ec-4e79-434e-9476-780d39223604 ("dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"

	Oct 04 00:54:36 old-k8s-version-999000 kubelet[7048]: E1004 00:54:36.469590    7048 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host

	Oct 04 00:54:36 old-k8s-version-999000 kubelet[7048]: E1004 00:54:36.469632    7048 kuberuntime_image.go:50] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host

	Oct 04 00:54:36 old-k8s-version-999000 kubelet[7048]: E1004 00:54:36.469661    7048 kuberuntime_manager.go:783] container start failed: ErrImagePull: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host

	Oct 04 00:54:36 old-k8s-version-999000 kubelet[7048]: E1004 00:54:36.469679    7048 pod_workers.go:191] Error syncing pod f88b6c39-6469-4f73-b621-ed79a8d324e3 ("metrics-server-74d5856cc6-d65gh_kube-system(f88b6c39-6469-4f73-b621-ed79a8d324e3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host"

	Oct 04 00:54:47 old-k8s-version-999000 kubelet[7048]: E1004 00:54:47.474053    7048 pod_workers.go:191] Error syncing pod f88b6c39-6469-4f73-b621-ed79a8d324e3 ("metrics-server-74d5856cc6-d65gh_kube-system(f88b6c39-6469-4f73-b621-ed79a8d324e3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""

	Oct 04 00:54:48 old-k8s-version-999000 kubelet[7048]: W1004 00:54:48.388851    7048 docker_sandbox.go:394] failed to read pod IP from plugin/docker: Couldn't find network status for kubernetes-dashboard/dashboard-metrics-scraper-d6b4b5544-xrh8x through plugin: invalid network status for

	Oct 04 00:54:48 old-k8s-version-999000 kubelet[7048]: E1004 00:54:48.392258    7048 pod_workers.go:191] Error syncing pod 6822b5ec-4e79-434e-9476-780d39223604 ("dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"

	Oct 04 00:54:49 old-k8s-version-999000 kubelet[7048]: W1004 00:54:49.398999    7048 docker_sandbox.go:394] failed to read pod IP from plugin/docker: Couldn't find network status for kubernetes-dashboard/dashboard-metrics-scraper-d6b4b5544-xrh8x through plugin: invalid network status for

	Oct 04 00:54:52 old-k8s-version-999000 kubelet[7048]: E1004 00:54:52.920753    7048 pod_workers.go:191] Error syncing pod 6822b5ec-4e79-434e-9476-780d39223604 ("dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"

	Oct 04 00:54:59 old-k8s-version-999000 kubelet[7048]: E1004 00:54:59.479091    7048 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host

	Oct 04 00:54:59 old-k8s-version-999000 kubelet[7048]: E1004 00:54:59.479135    7048 kuberuntime_image.go:50] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host

	Oct 04 00:54:59 old-k8s-version-999000 kubelet[7048]: E1004 00:54:59.479166    7048 kuberuntime_manager.go:783] container start failed: ErrImagePull: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host

	Oct 04 00:54:59 old-k8s-version-999000 kubelet[7048]: E1004 00:54:59.479185    7048 pod_workers.go:191] Error syncing pod f88b6c39-6469-4f73-b621-ed79a8d324e3 ("metrics-server-74d5856cc6-d65gh_kube-system(f88b6c39-6469-4f73-b621-ed79a8d324e3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.89.1:53: no such host"

	Oct 04 00:55:04 old-k8s-version-999000 kubelet[7048]: E1004 00:55:04.462714    7048 pod_workers.go:191] Error syncing pod 6822b5ec-4e79-434e-9476-780d39223604 ("dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"

	Oct 04 00:55:13 old-k8s-version-999000 kubelet[7048]: E1004 00:55:13.465629    7048 pod_workers.go:191] Error syncing pod f88b6c39-6469-4f73-b621-ed79a8d324e3 ("metrics-server-74d5856cc6-d65gh_kube-system(f88b6c39-6469-4f73-b621-ed79a8d324e3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""

	Oct 04 00:55:18 old-k8s-version-999000 kubelet[7048]: W1004 00:55:18.573989    7048 docker_sandbox.go:394] failed to read pod IP from plugin/docker: Couldn't find network status for kubernetes-dashboard/dashboard-metrics-scraper-d6b4b5544-xrh8x through plugin: invalid network status for

	Oct 04 00:55:19 old-k8s-version-999000 kubelet[7048]: W1004 00:55:19.816267    7048 docker_sandbox.go:394] failed to read pod IP from plugin/docker: Couldn't find network status for kubernetes-dashboard/dashboard-metrics-scraper-d6b4b5544-xrh8x through plugin: invalid network status for

	Oct 04 00:55:19 old-k8s-version-999000 kubelet[7048]: E1004 00:55:19.820802    7048 pod_workers.go:191] Error syncing pod 6822b5ec-4e79-434e-9476-780d39223604 ("dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"

	Oct 04 00:55:20 old-k8s-version-999000 kubelet[7048]: W1004 00:55:20.827908    7048 docker_sandbox.go:394] failed to read pod IP from plugin/docker: Couldn't find network status for kubernetes-dashboard/dashboard-metrics-scraper-d6b4b5544-xrh8x through plugin: invalid network status for

	Oct 04 00:55:22 old-k8s-version-999000 kubelet[7048]: E1004 00:55:22.921259    7048 pod_workers.go:191] Error syncing pod 6822b5ec-4e79-434e-9476-780d39223604 ("dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"

	Oct 04 00:55:24 old-k8s-version-999000 kubelet[7048]: E1004 00:55:24.463664    7048 pod_workers.go:191] Error syncing pod f88b6c39-6469-4f73-b621-ed79a8d324e3 ("metrics-server-74d5856cc6-d65gh_kube-system(f88b6c39-6469-4f73-b621-ed79a8d324e3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""

	Oct 04 00:55:38 old-k8s-version-999000 kubelet[7048]: E1004 00:55:38.464041    7048 pod_workers.go:191] Error syncing pod 6822b5ec-4e79-434e-9476-780d39223604 ("dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-d6b4b5544-xrh8x_kubernetes-dashboard(6822b5ec-4e79-434e-9476-780d39223604)"

	Oct 04 00:55:38 old-k8s-version-999000 kubelet[7048]: E1004 00:55:38.465433    7048 pod_workers.go:191] Error syncing pod f88b6c39-6469-4f73-b621-ed79a8d324e3 ("metrics-server-74d5856cc6-d65gh_kube-system(f88b6c39-6469-4f73-b621-ed79a8d324e3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""

	* 

	* ==> kubernetes-dashboard [3483d762d651] <==

	* 2023/10/04 00:54:20 Using namespace: kubernetes-dashboard

	2023/10/04 00:54:20 Using in-cluster config to connect to apiserver

	2023/10/04 00:54:20 Using secret token for csrf signing

	2023/10/04 00:54:20 Initializing csrf token from kubernetes-dashboard-csrf secret

	2023/10/04 00:54:20 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf

	2023/10/04 00:54:20 Successful initial request to the apiserver, version: v1.16.0

	2023/10/04 00:54:20 Generating JWE encryption key

	2023/10/04 00:54:20 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting

	2023/10/04 00:54:20 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard

	2023/10/04 00:54:20 Initializing JWE encryption key from synchronized object

	2023/10/04 00:54:20 Creating in-cluster Sidecar client

	2023/10/04 00:54:20 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.

	2023/10/04 00:54:20 Serving insecurely on HTTP port: 9090

	2023/10/04 00:54:50 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.

	2023/10/04 00:55:20 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.

	2023/10/04 00:55:50 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.

	2023/10/04 00:54:20 Starting overwatch

	* 

	* ==> storage-provisioner [c9f45072dd2f] <==

	* I1004 00:54:14.120617       1 storage_provisioner.go:116] Initializing the minikube storage provisioner...

	I1004 00:54:14.234076       1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!

	I1004 00:54:14.234407       1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...

	I1004 00:54:14.257076       1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath

	I1004 00:54:14.259466       1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-999000_6bcae9f0-0fb4-4105-ab41-e0502e883f4e!

	I1004 00:54:14.268482       1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"ebe4c117-6978-4283-949d-b6b3ee957c60", APIVersion:"v1", ResourceVersion:"428", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-999000_6bcae9f0-0fb4-4105-ab41-e0502e883f4e became leader

	I1004 00:54:14.362077       1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-999000_6bcae9f0-0fb4-4105-ab41-e0502e883f4e!

-- /stdout --

helpers_test.go:254: (dbg) Run:  out/minikube-darwin-amd64 status --format={{.APIServer}} -p old-k8s-version-999000 -n old-k8s-version-999000

helpers_test.go:261: (dbg) Run:  kubectl --context old-k8s-version-999000 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running

helpers_test.go:272: non-running pods: metrics-server-74d5856cc6-d65gh

helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/VerifyKubernetesImages]: describe non-running pods <======

helpers_test.go:277: (dbg) Run:  kubectl --context old-k8s-version-999000 describe pod metrics-server-74d5856cc6-d65gh

helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-999000 describe pod metrics-server-74d5856cc6-d65gh: exit status 1 (52.711035ms)

** stderr ** 

	Error from server (NotFound): pods "metrics-server-74d5856cc6-d65gh" not found

** /stderr **

helpers_test.go:279: kubectl --context old-k8s-version-999000 describe pod metrics-server-74d5856cc6-d65gh: exit status 1

--- FAIL: TestStartStop/group/old-k8s-version/serial/VerifyKubernetesImages (2.73s)

=== RUN   TestDownloadOnly/v1.16.0/json-events

aaa_download_only_test.go:69: (dbg) Run:  out/minikube-darwin-amd64 start -o=json --download-only -p download-only-433000 --force --alsologtostderr --kubernetes-version=v1.16.0 --container-runtime=docker --driver=hyperkit 

aaa_download_only_test.go:69: (dbg) Done: out/minikube-darwin-amd64 start -o=json --download-only -p download-only-433000 --force --alsologtostderr --kubernetes-version=v1.16.0 --container-runtime=docker --driver=hyperkit : (8.584660049s)

--- PASS: TestDownloadOnly/v1.16.0/json-events (8.58s)

=== RUN   TestDownloadOnly/v1.16.0/preload-exists

--- PASS: TestDownloadOnly/v1.16.0/preload-exists (0.00s)

=== RUN   TestDownloadOnly/v1.16.0/kubectl

--- PASS: TestDownloadOnly/v1.16.0/kubectl (0.00s)

=== RUN   TestDownloadOnly/v1.16.0/LogsDuration

aaa_download_only_test.go:169: (dbg) Run:  out/minikube-darwin-amd64 logs -p download-only-433000

aaa_download_only_test.go:169: (dbg) Non-zero exit: out/minikube-darwin-amd64 logs -p download-only-433000: exit status 85 (296.947784ms)

-- stdout --

	* 

	* ==> Audit <==

	* |---------|--------------------------------|----------------------|---------|---------|---------------------|----------|

	| Command |              Args              |       Profile        |  User   | Version |     Start Time      | End Time |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|----------|

	| start   | -o=json --download-only        | download-only-433000 | jenkins | v1.31.2 | 03 Oct 23 17:03 PDT |          |

	|         | -p download-only-433000        |                      |         |         |                     |          |

	|         | --force --alsologtostderr      |                      |         |         |                     |          |

	|         | --kubernetes-version=v1.16.0   |                      |         |         |                     |          |

	|         | --container-runtime=docker     |                      |         |         |                     |          |

	|         | --driver=hyperkit              |                      |         |         |                     |          |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|----------|

	* 

	* ==> Last Start <==

	* Log file created at: 2023/10/03 17:03:24

	Running on machine: MacOS-Agent-1

	Binary: Built with gc go1.21.1 for darwin/amd64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I1003 17:03:24.213841    1436 out.go:296] Setting OutFile to fd 1 ...

	I1003 17:03:24.214136    1436 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:03:24.214141    1436 out.go:309] Setting ErrFile to fd 2...

	I1003 17:03:24.214145    1436 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:03:24.214318    1436 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17345-936/.minikube/bin

	W1003 17:03:24.214421    1436 root.go:314] Error reading config file at /Users/jenkins/minikube-integration/17345-936/.minikube/config/config.json: open /Users/jenkins/minikube-integration/17345-936/.minikube/config/config.json: no such file or directory

	I1003 17:03:24.215990    1436 out.go:303] Setting JSON to true

	I1003 17:03:24.239898    1436 start.go:128] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":172,"bootTime":1696377632,"procs":418,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.0","kernelVersion":"23.0.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}

	W1003 17:03:24.239994    1436 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I1003 17:03:24.261401    1436 out.go:97] [download-only-433000] minikube v1.31.2 on Darwin 14.0

	I1003 17:03:24.284243    1436 out.go:169] MINIKUBE_LOCATION=17345

	I1003 17:03:24.261622    1436 notify.go:220] Checking for updates...

	W1003 17:03:24.261677    1436 preload.go:295] Failed to list preload files: open /Users/jenkins/minikube-integration/17345-936/.minikube/cache/preloaded-tarball: no such file or directory

	I1003 17:03:24.331159    1436 out.go:169] KUBECONFIG=/Users/jenkins/minikube-integration/17345-936/kubeconfig

	I1003 17:03:24.352945    1436 out.go:169] MINIKUBE_BIN=out/minikube-darwin-amd64

	I1003 17:03:24.374321    1436 out.go:169] MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I1003 17:03:24.396479    1436 out.go:169] MINIKUBE_HOME=/Users/jenkins/minikube-integration/17345-936/.minikube

	W1003 17:03:24.440229    1436 out.go:272] minikube skips various validations when --force is supplied; this may lead to unexpected behavior

	I1003 17:03:24.440745    1436 driver.go:373] Setting default libvirt URI to qemu:///system

	I1003 17:03:24.542111    1436 out.go:97] Using the hyperkit driver based on user configuration

	I1003 17:03:24.542140    1436 start.go:298] selected driver: hyperkit

	I1003 17:03:24.542148    1436 start.go:902] validating driver "hyperkit" against <nil>

	I1003 17:03:24.542272    1436 install.go:52] acquiring lock: {Name:mk4023283b30b374c3f04c8805d539e68824c0b8 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I1003 17:03:24.542455    1436 install.go:117] Validating docker-machine-driver-hyperkit, PATH=/Users/jenkins/minikube-integration/17345-936/.minikube/bin:/Users/jenkins/workspace/out/:/usr/bin:/bin:/usr/sbin:/sbin:/Users/jenkins/google-cloud-sdk/bin:/usr/local/bin/:/usr/local/go/bin/:/Users/jenkins/go/bin

	I1003 17:03:24.677362    1436 install.go:137] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit version is 1.31.2

	I1003 17:03:24.681753    1436 install.go:79] stdout: /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:03:24.681773    1436 install.go:81] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit looks good

	I1003 17:03:24.681809    1436 start_flags.go:307] no existing cluster config was found, will generate one from the flags 

	I1003 17:03:24.686681    1436 start_flags.go:384] Using suggested 6000MB memory alloc based on sys=32768MB, container=0MB

	I1003 17:03:24.686845    1436 start_flags.go:905] Wait components to verify : map[apiserver:true system_pods:true]

	I1003 17:03:24.686875    1436 cni.go:84] Creating CNI manager for ""

	I1003 17:03:24.686888    1436 cni.go:162] CNI unnecessary in this configuration, recommending no CNI

	I1003 17:03:24.686895    1436 start_flags.go:321] config:

	{Name:download-only-433000 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1696360059-17345@sha256:76d99edd1576614d5c20a839dd16ae1d7c810f3b909a01797063d483159ea3ae Memory:6000 CPUs:2 DiskSize:20000 VMDriver: Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:download-only-433000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Container

Runtime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I1003 17:03:24.687150    1436 iso.go:125] acquiring lock: {Name:mk09a17bc6b7923952debc9d2a1712efcb7ec756 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I1003 17:03:24.708248    1436 out.go:97] Downloading VM boot image ...

	I1003 17:03:24.708332    1436 download.go:107] Downloading: https://storage.googleapis.com/minikube-builds/iso/17240/minikube-v1.31.0-1695060926-17240-amd64.iso?checksum=file:https://storage.googleapis.com/minikube-builds/iso/17240/minikube-v1.31.0-1695060926-17240-amd64.iso.sha256 -> /Users/jenkins/minikube-integration/17345-936/.minikube/cache/iso/amd64/minikube-v1.31.0-1695060926-17240-amd64.iso

	I1003 17:03:27.915835    1436 out.go:97] Starting control plane node download-only-433000 in cluster download-only-433000

	I1003 17:03:27.915872    1436 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker

	I1003 17:03:27.971038    1436 preload.go:119] Found remote preload: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.16.0/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4

	I1003 17:03:27.971089    1436 cache.go:57] Caching tarball of preloaded images

	I1003 17:03:27.971457    1436 preload.go:132] Checking if preload exists for k8s version v1.16.0 and runtime docker

	I1003 17:03:27.992873    1436 out.go:97] Downloading Kubernetes v1.16.0 preload ...

	I1003 17:03:27.992899    1436 preload.go:238] getting checksum for preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4 ...

	I1003 17:03:28.074663    1436 download.go:107] Downloading: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.16.0/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4?checksum=md5:326f3ce331abb64565b50b8c9e791244 -> /Users/jenkins/minikube-integration/17345-936/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.16.0-docker-overlay2-amd64.tar.lz4

	* 

	* The control plane node "" does not exist.

	  To start a cluster, run: "minikube start -p download-only-433000"

-- /stdout --

aaa_download_only_test.go:170: minikube logs failed with error: exit status 85

--- PASS: TestDownloadOnly/v1.16.0/LogsDuration (0.30s)

=== RUN   TestDownloadOnly/v1.28.2/json-events

aaa_download_only_test.go:69: (dbg) Run:  out/minikube-darwin-amd64 start -o=json --download-only -p download-only-433000 --force --alsologtostderr --kubernetes-version=v1.28.2 --container-runtime=docker --driver=hyperkit 

aaa_download_only_test.go:69: (dbg) Done: out/minikube-darwin-amd64 start -o=json --download-only -p download-only-433000 --force --alsologtostderr --kubernetes-version=v1.28.2 --container-runtime=docker --driver=hyperkit : (5.476177679s)

--- PASS: TestDownloadOnly/v1.28.2/json-events (5.48s)

=== RUN   TestDownloadOnly/v1.28.2/preload-exists

--- PASS: TestDownloadOnly/v1.28.2/preload-exists (0.00s)

=== RUN   TestDownloadOnly/v1.28.2/kubectl

--- PASS: TestDownloadOnly/v1.28.2/kubectl (0.00s)

=== RUN   TestDownloadOnly/v1.28.2/LogsDuration

aaa_download_only_test.go:169: (dbg) Run:  out/minikube-darwin-amd64 logs -p download-only-433000

aaa_download_only_test.go:169: (dbg) Non-zero exit: out/minikube-darwin-amd64 logs -p download-only-433000: exit status 85 (319.012485ms)

-- stdout --

	* 

	* ==> Audit <==

	* |---------|--------------------------------|----------------------|---------|---------|---------------------|----------|

	| Command |              Args              |       Profile        |  User   | Version |     Start Time      | End Time |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|----------|

	| start   | -o=json --download-only        | download-only-433000 | jenkins | v1.31.2 | 03 Oct 23 17:03 PDT |          |

	|         | -p download-only-433000        |                      |         |         |                     |          |

	|         | --force --alsologtostderr      |                      |         |         |                     |          |

	|         | --kubernetes-version=v1.16.0   |                      |         |         |                     |          |

	|         | --container-runtime=docker     |                      |         |         |                     |          |

	|         | --driver=hyperkit              |                      |         |         |                     |          |

	| start   | -o=json --download-only        | download-only-433000 | jenkins | v1.31.2 | 03 Oct 23 17:03 PDT |          |

	|         | -p download-only-433000        |                      |         |         |                     |          |

	|         | --force --alsologtostderr      |                      |         |         |                     |          |

	|         | --kubernetes-version=v1.28.2   |                      |         |         |                     |          |

	|         | --container-runtime=docker     |                      |         |         |                     |          |

	|         | --driver=hyperkit              |                      |         |         |                     |          |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|----------|

	* 

	* ==> Last Start <==

	* Log file created at: 2023/10/03 17:03:33

	Running on machine: MacOS-Agent-1

	Binary: Built with gc go1.21.1 for darwin/amd64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I1003 17:03:33.098316    1449 out.go:296] Setting OutFile to fd 1 ...

	I1003 17:03:33.098591    1449 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:03:33.098597    1449 out.go:309] Setting ErrFile to fd 2...

	I1003 17:03:33.098601    1449 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:03:33.098775    1449 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17345-936/.minikube/bin

	W1003 17:03:33.098868    1449 root.go:314] Error reading config file at /Users/jenkins/minikube-integration/17345-936/.minikube/config/config.json: open /Users/jenkins/minikube-integration/17345-936/.minikube/config/config.json: no such file or directory

	I1003 17:03:33.100043    1449 out.go:303] Setting JSON to true

	I1003 17:03:33.121438    1449 start.go:128] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":181,"bootTime":1696377632,"procs":408,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.0","kernelVersion":"23.0.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}

	W1003 17:03:33.121544    1449 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I1003 17:03:33.142586    1449 out.go:97] [download-only-433000] minikube v1.31.2 on Darwin 14.0

	I1003 17:03:33.179815    1449 out.go:169] MINIKUBE_LOCATION=17345

	I1003 17:03:33.142694    1449 notify.go:220] Checking for updates...

	I1003 17:03:33.239631    1449 out.go:169] KUBECONFIG=/Users/jenkins/minikube-integration/17345-936/kubeconfig

	I1003 17:03:33.299899    1449 out.go:169] MINIKUBE_BIN=out/minikube-darwin-amd64

	I1003 17:03:33.342741    1449 out.go:169] MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I1003 17:03:33.386690    1449 out.go:169] MINIKUBE_HOME=/Users/jenkins/minikube-integration/17345-936/.minikube

	W1003 17:03:33.434822    1449 out.go:272] minikube skips various validations when --force is supplied; this may lead to unexpected behavior

	I1003 17:03:33.435544    1449 config.go:182] Loaded profile config "download-only-433000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.16.0

	W1003 17:03:33.435628    1449 start.go:810] api.Load failed for download-only-433000: filestore "download-only-433000": Docker machine "download-only-433000" does not exist. Use "docker-machine ls" to list machines. Use "docker-machine create" to add a new one.

	I1003 17:03:33.435804    1449 driver.go:373] Setting default libvirt URI to qemu:///system

	W1003 17:03:33.435843    1449 start.go:810] api.Load failed for download-only-433000: filestore "download-only-433000": Docker machine "download-only-433000" does not exist. Use "docker-machine ls" to list machines. Use "docker-machine create" to add a new one.

	I1003 17:03:33.466806    1449 out.go:97] Using the hyperkit driver based on existing profile

	I1003 17:03:33.466852    1449 start.go:298] selected driver: hyperkit

	I1003 17:03:33.466864    1449 start.go:902] validating driver "hyperkit" against &{Name:download-only-433000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17240/minikube-v1.31.0-1695060926-17240-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1696360059-17345@sha256:76d99edd1576614d5c20a839dd16ae1d7c810f3b909a01797063d483159ea3ae Memory:6000 CPUs:2 DiskSize:20000 VMDriver: Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kuber

netesConfig:{KubernetesVersion:v1.16.0 ClusterName:download-only-433000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirro

r: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I1003 17:03:33.467206    1449 install.go:52] acquiring lock: {Name:mk4023283b30b374c3f04c8805d539e68824c0b8 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I1003 17:03:33.467397    1449 install.go:117] Validating docker-machine-driver-hyperkit, PATH=/Users/jenkins/minikube-integration/17345-936/.minikube/bin:/Users/jenkins/workspace/out/:/usr/bin:/bin:/usr/sbin:/sbin:/Users/jenkins/google-cloud-sdk/bin:/usr/local/bin/:/usr/local/go/bin/:/Users/jenkins/go/bin

	I1003 17:03:33.476313    1449 install.go:137] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit version is 1.31.2

	I1003 17:03:33.479929    1449 install.go:79] stdout: /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:03:33.479947    1449 install.go:81] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit looks good

	I1003 17:03:33.482561    1449 cni.go:84] Creating CNI manager for ""

	I1003 17:03:33.482580    1449 cni.go:158] "hyperkit" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge

	I1003 17:03:33.482596    1449 start_flags.go:321] config:

	{Name:download-only-433000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17240/minikube-v1.31.0-1695060926-17240-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1696360059-17345@sha256:76d99edd1576614d5c20a839dd16ae1d7c810f3b909a01797063d483159ea3ae Memory:6000 CPUs:2 DiskSize:20000 VMDriver: Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.2 ClusterName:download-only-433000 Namespace:

default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketV

MnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I1003 17:03:33.482723    1449 iso.go:125] acquiring lock: {Name:mk09a17bc6b7923952debc9d2a1712efcb7ec756 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I1003 17:03:33.503747    1449 out.go:97] Starting control plane node download-only-433000 in cluster download-only-433000

	I1003 17:03:33.503780    1449 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I1003 17:03:33.561298    1449 preload.go:119] Found remote preload: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.28.2/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-amd64.tar.lz4

	I1003 17:03:33.561329    1449 cache.go:57] Caching tarball of preloaded images

	I1003 17:03:33.561687    1449 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I1003 17:03:33.583322    1449 out.go:97] Downloading Kubernetes v1.28.2 preload ...

	I1003 17:03:33.583365    1449 preload.go:238] getting checksum for preloaded-images-k8s-v18-v1.28.2-docker-overlay2-amd64.tar.lz4 ...

	I1003 17:03:33.668802    1449 download.go:107] Downloading: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.28.2/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-amd64.tar.lz4?checksum=md5:30a5cb95ef165c1e9196502a3ab2be2b -> /Users/jenkins/minikube-integration/17345-936/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-amd64.tar.lz4

	I1003 17:03:36.935188    1449 preload.go:249] saving checksum for preloaded-images-k8s-v18-v1.28.2-docker-overlay2-amd64.tar.lz4 ...

	I1003 17:03:36.935398    1449 preload.go:256] verifying checksum of /Users/jenkins/minikube-integration/17345-936/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.2-docker-overlay2-amd64.tar.lz4 ...

	I1003 17:03:37.553260    1449 cache.go:60] Finished verifying existence of preloaded tar for  v1.28.2 on docker

	I1003 17:03:37.553334    1449 profile.go:148] Saving config to /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/download-only-433000/config.json ...

	I1003 17:03:37.553751    1449 preload.go:132] Checking if preload exists for k8s version v1.28.2 and runtime docker

	I1003 17:03:37.553997    1449 download.go:107] Downloading: https://dl.k8s.io/release/v1.28.2/bin/darwin/amd64/kubectl?checksum=file:https://dl.k8s.io/release/v1.28.2/bin/darwin/amd64/kubectl.sha256 -> /Users/jenkins/minikube-integration/17345-936/.minikube/cache/darwin/amd64/v1.28.2/kubectl

	* 

	* The control plane node "" does not exist.

	  To start a cluster, run: "minikube start -p download-only-433000"

-- /stdout --

aaa_download_only_test.go:170: minikube logs failed with error: exit status 85

--- PASS: TestDownloadOnly/v1.28.2/LogsDuration (0.32s)

=== RUN   TestDownloadOnly/DeleteAll

aaa_download_only_test.go:187: (dbg) Run:  out/minikube-darwin-amd64 delete --all

--- PASS: TestDownloadOnly/DeleteAll (0.38s)

=== RUN   TestDownloadOnly/DeleteAlwaysSucceeds

aaa_download_only_test.go:199: (dbg) Run:  out/minikube-darwin-amd64 delete -p download-only-433000

--- PASS: TestDownloadOnly/DeleteAlwaysSucceeds (0.36s)

=== RUN   TestBinaryMirror

aaa_download_only_test.go:304: (dbg) Run:  out/minikube-darwin-amd64 start --download-only -p binary-mirror-526000 --alsologtostderr --binary-mirror http://127.0.0.1:49339 --driver=hyperkit 

helpers_test.go:175: Cleaning up "binary-mirror-526000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-amd64 delete -p binary-mirror-526000

--- PASS: TestBinaryMirror (1.01s)

=== RUN   TestOffline

=== PAUSE TestOffline

=== CONT  TestOffline

aab_offline_test.go:55: (dbg) Run:  out/minikube-darwin-amd64 start -p offline-docker-530000 --alsologtostderr -v=1 --memory=2048 --wait=true --driver=hyperkit 

aab_offline_test.go:55: (dbg) Done: out/minikube-darwin-amd64 start -p offline-docker-530000 --alsologtostderr -v=1 --memory=2048 --wait=true --driver=hyperkit : (50.181028442s)

helpers_test.go:175: Cleaning up "offline-docker-530000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-amd64 delete -p offline-docker-530000

helpers_test.go:178: (dbg) Done: out/minikube-darwin-amd64 delete -p offline-docker-530000: (5.280810268s)

--- PASS: TestOffline (55.46s)

=== RUN   TestCertOptions

=== PAUSE TestCertOptions

=== CONT  TestCertOptions

cert_options_test.go:49: (dbg) Run:  out/minikube-darwin-amd64 start -p cert-options-984000 --memory=2048 --apiserver-ips=127.0.0.1 --apiserver-ips=192.168.15.15 --apiserver-names=localhost --apiserver-names=www.google.com --apiserver-port=8555 --driver=hyperkit 

E1003 17:30:48.103473    1434 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/17345-936/.minikube/profiles/ingress-addon-legacy-746000/client.crt: no such file or directory

cert_options_test.go:49: (dbg) Done: out/minikube-darwin-amd64 start -p cert-options-984000 --memory=2048 --apiserver-ips=127.0.0.1 --apiserver-ips=192.168.15.15 --apiserver-names=localhost --apiserver-names=www.google.com --apiserver-port=8555 --driver=hyperkit : (36.538388635s)

cert_options_test.go:60: (dbg) Run:  out/minikube-darwin-amd64 -p cert-options-984000 ssh "openssl x509 -text -noout -in /var/lib/minikube/certs/apiserver.crt"

cert_options_test.go:88: (dbg) Run:  kubectl --context cert-options-984000 config view

cert_options_test.go:100: (dbg) Run:  out/minikube-darwin-amd64 ssh -p cert-options-984000 -- "sudo cat /etc/kubernetes/admin.conf"

helpers_test.go:175: Cleaning up "cert-options-984000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-amd64 delete -p cert-options-984000

helpers_test.go:178: (dbg) Done: out/minikube-darwin-amd64 delete -p cert-options-984000: (5.268822896s)

--- PASS: TestCertOptions (42.15s)

=== RUN   TestCertExpiration

=== PAUSE TestCertExpiration

=== CONT  TestCertExpiration

cert_options_test.go:123: (dbg) Run:  out/minikube-darwin-amd64 start -p cert-expiration-777000 --memory=2048 --cert-expiration=3m --driver=hyperkit 

cert_options_test.go:123: (dbg) Done: out/minikube-darwin-amd64 start -p cert-expiration-777000 --memory=2048 --cert-expiration=3m --driver=hyperkit : (34.474455071s)

cert_options_test.go:131: (dbg) Run:  out/minikube-darwin-amd64 start -p cert-expiration-777000 --memory=2048 --cert-expiration=8760h --driver=hyperkit 

cert_options_test.go:131: (dbg) Done: out/minikube-darwin-amd64 start -p cert-expiration-777000 --memory=2048 --cert-expiration=8760h --driver=hyperkit : (22.209471716s)

helpers_test.go:175: Cleaning up "cert-expiration-777000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-amd64 delete -p cert-expiration-777000

helpers_test.go:178: (dbg) Done: out/minikube-darwin-amd64 delete -p cert-expiration-777000: (5.271085793s)

--- PASS: TestCertExpiration (241.96s)

=== RUN   TestDockerFlags

=== PAUSE TestDockerFlags

=== CONT  TestDockerFlags

docker_test.go:51: (dbg) Run:  out/minikube-darwin-amd64 start -p docker-flags-975000 --cache-images=false --memory=2048 --install-addons=false --wait=false --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug --docker-opt=icc=true --alsologtostderr -v=5 --driver=hyperkit 

docker_test.go:51: (dbg) Done: out/minikube-darwin-amd64 start -p docker-flags-975000 --cache-images=false --memory=2048 --install-addons=false --wait=false --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug --docker-opt=icc=true --alsologtostderr -v=5 --driver=hyperkit : (43.961188455s)

docker_test.go:56: (dbg) Run:  out/minikube-darwin-amd64 -p docker-flags-975000 ssh "sudo systemctl show docker --property=Environment --no-pager"

docker_test.go:67: (dbg) Run:  out/minikube-darwin-amd64 -p docker-flags-975000 ssh "sudo systemctl show docker --property=ExecStart --no-pager"

helpers_test.go:175: Cleaning up "docker-flags-975000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-amd64 delete -p docker-flags-975000

helpers_test.go:178: (dbg) Done: out/minikube-darwin-amd64 delete -p docker-flags-975000: (3.461157019s)

--- PASS: TestDockerFlags (47.75s)

=== RUN   TestForceSystemdFlag

=== PAUSE TestForceSystemdFlag

=== CONT  TestForceSystemdFlag

docker_test.go:91: (dbg) Run:  out/minikube-darwin-amd64 start -p force-systemd-flag-204000 --memory=2048 --force-systemd --alsologtostderr -v=5 --driver=hyperkit 

docker_test.go:91: (dbg) Done: out/minikube-darwin-amd64 start -p force-systemd-flag-204000 --memory=2048 --force-systemd --alsologtostderr -v=5 --driver=hyperkit : (34.966779766s)

docker_test.go:110: (dbg) Run:  out/minikube-darwin-amd64 -p force-systemd-flag-204000 ssh "docker info --format {{.CgroupDriver}}"

helpers_test.go:175: Cleaning up "force-systemd-flag-204000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-amd64 delete -p force-systemd-flag-204000

helpers_test.go:178: (dbg) Done: out/minikube-darwin-amd64 delete -p force-systemd-flag-204000: (3.426305157s)

--- PASS: TestForceSystemdFlag (38.56s)

=== RUN   TestForceSystemdEnv

=== PAUSE TestForceSystemdEnv

=== CONT  TestForceSystemdEnv

docker_test.go:155: (dbg) Run:  out/minikube-darwin-amd64 start -p force-systemd-env-746000 --memory=2048 --alsologtostderr -v=5 --driver=hyperkit 

docker_test.go:155: (dbg) Done: out/minikube-darwin-amd64 start -p force-systemd-env-746000 --memory=2048 --alsologtostderr -v=5 --driver=hyperkit : (34.33628953s)

docker_test.go:110: (dbg) Run:  out/minikube-darwin-amd64 -p force-systemd-env-746000 ssh "docker info --format {{.CgroupDriver}}"

helpers_test.go:175: Cleaning up "force-systemd-env-746000" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-darwin-amd64 delete -p force-systemd-env-746000

helpers_test.go:178: (dbg) Done: out/minikube-darwin-amd64 delete -p force-systemd-env-746000: (5.265315636s)

--- PASS: TestForceSystemdEnv (39.75s)

=== RUN   TestHyperKitDriverInstallOrUpdate

=== PAUSE TestHyperKitDriverInstallOrUpdate

=== CONT  TestHyperKitDriverInstallOrUpdate

--- PASS: TestHyperKitDriverInstallOrUpdate (5.34s)

=== RUN   TestErrorSpam/setup

error_spam_test.go:81: (dbg) Run:  out/minikube-darwin-amd64 start -p nospam-908000 -n=1 --memory=2250 --wait=false --log_dir=/var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 --driver=hyperkit 

error_spam_test.go:81: (dbg) Done: out/minikube-darwin-amd64 start -p nospam-908000 -n=1 --memory=2250 --wait=false --log_dir=/var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 --driver=hyperkit : (34.143412604s)

--- PASS: TestErrorSpam/setup (34.14s)

=== RUN   TestErrorSpam/start

error_spam_test.go:216: Cleaning up 1 logfile(s) ...

error_spam_test.go:159: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 start --dry-run

error_spam_test.go:159: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 start --dry-run

error_spam_test.go:182: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 start --dry-run

--- PASS: TestErrorSpam/start (1.55s)

=== RUN   TestErrorSpam/status

error_spam_test.go:216: Cleaning up 0 logfile(s) ...

error_spam_test.go:159: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 status

error_spam_test.go:159: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 status

error_spam_test.go:182: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 status

--- PASS: TestErrorSpam/status (0.45s)

=== RUN   TestErrorSpam/pause

error_spam_test.go:216: Cleaning up 0 logfile(s) ...

error_spam_test.go:159: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 pause

error_spam_test.go:159: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 pause

error_spam_test.go:182: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 pause

--- PASS: TestErrorSpam/pause (1.21s)

=== RUN   TestErrorSpam/unpause

error_spam_test.go:216: Cleaning up 0 logfile(s) ...

error_spam_test.go:159: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 unpause

error_spam_test.go:159: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 unpause

error_spam_test.go:182: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 unpause

--- PASS: TestErrorSpam/unpause (1.28s)

=== RUN   TestErrorSpam/stop

error_spam_test.go:216: Cleaning up 0 logfile(s) ...

error_spam_test.go:159: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 stop

error_spam_test.go:159: (dbg) Done: out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 stop: (5.198811915s)

error_spam_test.go:159: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 stop

error_spam_test.go:182: (dbg) Run:  out/minikube-darwin-amd64 -p nospam-908000 --log_dir /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/nospam-908000 stop

--- PASS: TestErrorSpam/stop (5.62s)

=== RUN   TestFunctional/serial/CopySyncFile

functional_test.go:1851: local sync path: /Users/jenkins/minikube-integration/17345-936/.minikube/files/etc/test/nested/copy/1434/hosts

--- PASS: TestFunctional/serial/CopySyncFile (0.00s)

=== RUN   TestFunctional/serial/StartWithProxy

functional_test.go:2230: (dbg) Run:  out/minikube-darwin-amd64 start -p functional-157000 --memory=4000 --apiserver-port=8441 --wait=all --driver=hyperkit 

functional_test.go:2230: (dbg) Done: out/minikube-darwin-amd64 start -p functional-157000 --memory=4000 --apiserver-port=8441 --wait=all --driver=hyperkit : (50.956345967s)

--- PASS: TestFunctional/serial/StartWithProxy (50.96s)

=== RUN   TestFunctional/serial/AuditLog

--- PASS: TestFunctional/serial/AuditLog (0.00s)

=== RUN   TestFunctional/serial/SoftStart

functional_test.go:655: (dbg) Run:  out/minikube-darwin-amd64 start -p functional-157000 --alsologtostderr -v=8

functional_test.go:655: (dbg) Done: out/minikube-darwin-amd64 start -p functional-157000 --alsologtostderr -v=8: (39.507710785s)

functional_test.go:659: soft start took 39.508302689s for "functional-157000" cluster.

--- PASS: TestFunctional/serial/SoftStart (39.51s)

=== RUN   TestFunctional/serial/KubeContext

functional_test.go:677: (dbg) Run:  kubectl config current-context

--- PASS: TestFunctional/serial/KubeContext (0.04s)

=== RUN   TestFunctional/serial/KubectlGetPods

functional_test.go:692: (dbg) Run:  kubectl --context functional-157000 get po -A

--- PASS: TestFunctional/serial/KubectlGetPods (0.08s)

=== RUN   TestFunctional/serial/CacheCmd/cache/add_remote

functional_test.go:1045: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 cache add registry.k8s.io/pause:3.1

functional_test.go:1045: (dbg) Done: out/minikube-darwin-amd64 -p functional-157000 cache add registry.k8s.io/pause:3.1: (1.522170781s)

functional_test.go:1045: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 cache add registry.k8s.io/pause:3.3

functional_test.go:1045: (dbg) Done: out/minikube-darwin-amd64 -p functional-157000 cache add registry.k8s.io/pause:3.3: (1.503384876s)

functional_test.go:1045: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 cache add registry.k8s.io/pause:latest

functional_test.go:1045: (dbg) Done: out/minikube-darwin-amd64 -p functional-157000 cache add registry.k8s.io/pause:latest: (1.39829089s)

--- PASS: TestFunctional/serial/CacheCmd/cache/add_remote (4.42s)

=== RUN   TestFunctional/serial/CacheCmd/cache/add_local

functional_test.go:1073: (dbg) Run:  docker build -t minikube-local-cache-test:functional-157000 /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalserialCacheCmdcacheadd_local4183888949/001

functional_test.go:1085: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 cache add minikube-local-cache-test:functional-157000

functional_test.go:1090: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 cache delete minikube-local-cache-test:functional-157000

functional_test.go:1079: (dbg) Run:  docker rmi minikube-local-cache-test:functional-157000

--- PASS: TestFunctional/serial/CacheCmd/cache/add_local (1.57s)

=== RUN   TestFunctional/serial/CacheCmd/cache/CacheDelete

functional_test.go:1098: (dbg) Run:  out/minikube-darwin-amd64 cache delete registry.k8s.io/pause:3.3

--- PASS: TestFunctional/serial/CacheCmd/cache/CacheDelete (0.06s)

=== RUN   TestFunctional/serial/CacheCmd/cache/list

functional_test.go:1106: (dbg) Run:  out/minikube-darwin-amd64 cache list

--- PASS: TestFunctional/serial/CacheCmd/cache/list (0.06s)

=== RUN   TestFunctional/serial/CacheCmd/cache/verify_cache_inside_node

functional_test.go:1120: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh sudo crictl images

--- PASS: TestFunctional/serial/CacheCmd/cache/verify_cache_inside_node (0.17s)

=== RUN   TestFunctional/serial/CacheCmd/cache/cache_reload

functional_test.go:1143: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh sudo docker rmi registry.k8s.io/pause:latest

functional_test.go:1149: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh sudo crictl inspecti registry.k8s.io/pause:latest

functional_test.go:1149: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p functional-157000 ssh sudo crictl inspecti registry.k8s.io/pause:latest: exit status 1 (139.611345ms)

-- stdout --

	FATA[0000] no such image "registry.k8s.io/pause:latest" present 

-- /stdout --

** stderr ** 

	ssh: Process exited with status 1

** /stderr **

functional_test.go:1154: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 cache reload

functional_test.go:1159: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh sudo crictl inspecti registry.k8s.io/pause:latest

--- PASS: TestFunctional/serial/CacheCmd/cache/cache_reload (1.41s)

=== RUN   TestFunctional/serial/CacheCmd/cache/delete

functional_test.go:1168: (dbg) Run:  out/minikube-darwin-amd64 cache delete registry.k8s.io/pause:3.1

functional_test.go:1168: (dbg) Run:  out/minikube-darwin-amd64 cache delete registry.k8s.io/pause:latest

--- PASS: TestFunctional/serial/CacheCmd/cache/delete (0.13s)

=== RUN   TestFunctional/serial/MinikubeKubectlCmd

functional_test.go:712: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 kubectl -- --context functional-157000 get pods

--- PASS: TestFunctional/serial/MinikubeKubectlCmd (0.52s)

=== RUN   TestFunctional/serial/MinikubeKubectlCmdDirectly

functional_test.go:737: (dbg) Run:  out/kubectl --context functional-157000 get pods

--- PASS: TestFunctional/serial/MinikubeKubectlCmdDirectly (0.71s)

=== RUN   TestFunctional/serial/ExtraConfig

functional_test.go:753: (dbg) Run:  out/minikube-darwin-amd64 start -p functional-157000 --extra-config=apiserver.enable-admission-plugins=NamespaceAutoProvision --wait=all

functional_test.go:753: (dbg) Done: out/minikube-darwin-amd64 start -p functional-157000 --extra-config=apiserver.enable-admission-plugins=NamespaceAutoProvision --wait=all: (38.61536619s)

functional_test.go:757: restart took 38.615536951s for "functional-157000" cluster.

--- PASS: TestFunctional/serial/ExtraConfig (38.62s)

=== RUN   TestFunctional/serial/ComponentHealth

functional_test.go:806: (dbg) Run:  kubectl --context functional-157000 get po -l tier=control-plane -n kube-system -o=json

functional_test.go:821: etcd phase: Running

functional_test.go:831: etcd status: Ready

functional_test.go:821: kube-apiserver phase: Running

functional_test.go:831: kube-apiserver status: Ready

functional_test.go:821: kube-controller-manager phase: Running

functional_test.go:831: kube-controller-manager status: Ready

functional_test.go:821: kube-scheduler phase: Running

functional_test.go:831: kube-scheduler status: Ready

--- PASS: TestFunctional/serial/ComponentHealth (0.05s)

=== RUN   TestFunctional/serial/LogsCmd

functional_test.go:1232: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 logs

functional_test.go:1232: (dbg) Done: out/minikube-darwin-amd64 -p functional-157000 logs: (2.867646511s)

--- PASS: TestFunctional/serial/LogsCmd (2.87s)

=== RUN   TestFunctional/serial/LogsFileCmd

functional_test.go:1246: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 logs --file /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalserialLogsFileCmd2560905699/001/logs.txt

functional_test.go:1246: (dbg) Done: out/minikube-darwin-amd64 -p functional-157000 logs --file /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalserialLogsFileCmd2560905699/001/logs.txt: (2.792416841s)

--- PASS: TestFunctional/serial/LogsFileCmd (2.79s)

=== RUN   TestFunctional/serial/InvalidService

functional_test.go:2317: (dbg) Run:  kubectl --context functional-157000 apply -f testdata/invalidsvc.yaml

functional_test.go:2331: (dbg) Run:  out/minikube-darwin-amd64 service invalid-svc -p functional-157000

functional_test.go:2331: (dbg) Non-zero exit: out/minikube-darwin-amd64 service invalid-svc -p functional-157000: exit status 115 (254.004717ms)

-- stdout --

	|-----------|-------------|-------------|---------------------------|

	| NAMESPACE |    NAME     | TARGET PORT |            URL            |

	|-----------|-------------|-------------|---------------------------|

	| default   | invalid-svc |          80 | http://192.168.87.4:32445 |

	|-----------|-------------|-------------|---------------------------|

-- /stdout --

** stderr ** 

	X Exiting due to SVC_UNREACHABLE: service not available: no running pod for service invalid-svc found

	* 

	╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮

	│                                                                                                                            │

	│    * If the above advice does not help, please let us know:                                                                │

	│      https://github.com/kubernetes/minikube/issues/new/choose                                                              │

	│                                                                                                                            │

	│    * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.                                   │

	│    * Please also attach the following file to the GitHub issue:                                                            │

	│    * - /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/minikube_service_96b204199e3191fa1740d4430b018a3c8028d52d_0.log    │

	│                                                                                                                            │

	╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

** /stderr **

functional_test.go:2323: (dbg) Run:  kubectl --context functional-157000 delete -f testdata/invalidsvc.yaml

--- PASS: TestFunctional/serial/InvalidService (4.24s)

=== RUN   TestFunctional/parallel/ConfigCmd

=== PAUSE TestFunctional/parallel/ConfigCmd

=== CONT  TestFunctional/parallel/ConfigCmd

functional_test.go:1195: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 config unset cpus

functional_test.go:1195: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 config get cpus

functional_test.go:1195: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p functional-157000 config get cpus: exit status 14 (41.546656ms)

** stderr ** 

	Error: specified key could not be found in config

** /stderr **

functional_test.go:1195: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 config set cpus 2

functional_test.go:1195: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 config get cpus

functional_test.go:1195: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 config unset cpus

functional_test.go:1195: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 config get cpus

functional_test.go:1195: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p functional-157000 config get cpus: exit status 14 (40.80786ms)

** stderr ** 

	Error: specified key could not be found in config

** /stderr **

--- PASS: TestFunctional/parallel/ConfigCmd (0.39s)

=== RUN   TestFunctional/parallel/DashboardCmd

=== PAUSE TestFunctional/parallel/DashboardCmd

=== CONT  TestFunctional/parallel/DashboardCmd

functional_test.go:901: (dbg) daemon: [out/minikube-darwin-amd64 dashboard --url --port 36195 -p functional-157000 --alsologtostderr -v=1]

functional_test.go:906: (dbg) stopping [out/minikube-darwin-amd64 dashboard --url --port 36195 -p functional-157000 --alsologtostderr -v=1] ...

helpers_test.go:508: unable to kill pid 2219: os: process already finished

--- PASS: TestFunctional/parallel/DashboardCmd (14.65s)

=== RUN   TestFunctional/parallel/DryRun

=== PAUSE TestFunctional/parallel/DryRun

=== CONT  TestFunctional/parallel/DryRun

functional_test.go:970: (dbg) Run:  out/minikube-darwin-amd64 start -p functional-157000 --dry-run --memory 250MB --alsologtostderr --driver=hyperkit 

functional_test.go:970: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p functional-157000 --dry-run --memory 250MB --alsologtostderr --driver=hyperkit : exit status 23 (572.851224ms)

-- stdout --

	* [functional-157000] minikube v1.31.2 on Darwin 14.0

	  - MINIKUBE_LOCATION=17345

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17345-936/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17345-936/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Using the hyperkit driver based on existing profile

-- /stdout --

** stderr ** 

	I1003 17:07:47.832626    2193 out.go:296] Setting OutFile to fd 1 ...

	I1003 17:07:47.832909    2193 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:07:47.832914    2193 out.go:309] Setting ErrFile to fd 2...

	I1003 17:07:47.832918    2193 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:07:47.833104    2193 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17345-936/.minikube/bin

	I1003 17:07:47.834544    2193 out.go:303] Setting JSON to false

	I1003 17:07:47.856564    2193 start.go:128] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":435,"bootTime":1696377632,"procs":472,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.0","kernelVersion":"23.0.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}

	W1003 17:07:47.856671    2193 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I1003 17:07:47.878712    2193 out.go:177] * [functional-157000] minikube v1.31.2 on Darwin 14.0

	I1003 17:07:47.957746    2193 out.go:177]   - MINIKUBE_LOCATION=17345

	I1003 17:07:47.920833    2193 notify.go:220] Checking for updates...

	I1003 17:07:47.999647    2193 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17345-936/kubeconfig

	I1003 17:07:48.020733    2193 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-amd64

	I1003 17:07:48.062814    2193 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I1003 17:07:48.120520    2193 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17345-936/.minikube

	I1003 17:07:48.141995    2193 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I1003 17:07:48.164653    2193 config.go:182] Loaded profile config "functional-157000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I1003 17:07:48.165367    2193 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:07:48.165445    2193 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:07:48.174269    2193 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:49953

	I1003 17:07:48.174634    2193 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:07:48.175075    2193 main.go:141] libmachine: Using API Version  1

	I1003 17:07:48.175088    2193 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:07:48.175339    2193 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:07:48.175437    2193 main.go:141] libmachine: (functional-157000) Calling .DriverName

	I1003 17:07:48.175619    2193 driver.go:373] Setting default libvirt URI to qemu:///system

	I1003 17:07:48.175842    2193 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:07:48.175868    2193 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:07:48.183310    2193 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:49955

	I1003 17:07:48.183654    2193 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:07:48.184009    2193 main.go:141] libmachine: Using API Version  1

	I1003 17:07:48.184026    2193 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:07:48.184243    2193 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:07:48.184343    2193 main.go:141] libmachine: (functional-157000) Calling .DriverName

	I1003 17:07:48.212575    2193 out.go:177] * Using the hyperkit driver based on existing profile

	I1003 17:07:48.255848    2193 start.go:298] selected driver: hyperkit

	I1003 17:07:48.255875    2193 start.go:902] validating driver "hyperkit" against &{Name:functional-157000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17240/minikube-v1.31.0-1695060926-17240-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1696360059-17345@sha256:76d99edd1576614d5c20a839dd16ae1d7c810f3b909a01797063d483159ea3ae Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kubernet

esConfig:{KubernetesVersion:v1.28.2 ClusterName:functional-157000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:apiserver Key:enable-admission-plugins Value:NamespaceAutoProvision}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8441 NodeName:} Nodes:[{Name: IP:192.168.87.4 Port:8441 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDis

ks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I1003 17:07:48.256071    2193 start.go:913] status for hyperkit: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I1003 17:07:48.280777    2193 out.go:177] 

	W1003 17:07:48.302702    2193 out.go:239] X Exiting due to RSRC_INSUFFICIENT_REQ_MEMORY: Requested memory allocation 250MiB is less than the usable minimum of 1800MB

	X Exiting due to RSRC_INSUFFICIENT_REQ_MEMORY: Requested memory allocation 250MiB is less than the usable minimum of 1800MB

	I1003 17:07:48.323798    2193 out.go:177] 

** /stderr **

functional_test.go:987: (dbg) Run:  out/minikube-darwin-amd64 start -p functional-157000 --dry-run --alsologtostderr -v=1 --driver=hyperkit 

--- PASS: TestFunctional/parallel/DryRun (1.12s)

=== RUN   TestFunctional/parallel/InternationalLanguage

=== PAUSE TestFunctional/parallel/InternationalLanguage

=== CONT  TestFunctional/parallel/InternationalLanguage

functional_test.go:1016: (dbg) Run:  out/minikube-darwin-amd64 start -p functional-157000 --dry-run --memory 250MB --alsologtostderr --driver=hyperkit 

functional_test.go:1016: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p functional-157000 --dry-run --memory 250MB --alsologtostderr --driver=hyperkit : exit status 23 (448.170676ms)

-- stdout --

	* [functional-157000] minikube v1.31.2 sur Darwin 14.0

	  - MINIKUBE_LOCATION=17345

	  - KUBECONFIG=/Users/jenkins/minikube-integration/17345-936/kubeconfig

	  - MINIKUBE_BIN=out/minikube-darwin-amd64

	  - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	  - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17345-936/.minikube

	  - MINIKUBE_FORCE_SYSTEMD=

	* Utilisation du pilote hyperkit basé sur le profil existant

-- /stdout --

** stderr ** 

	I1003 17:07:47.378025    2186 out.go:296] Setting OutFile to fd 1 ...

	I1003 17:07:47.378247    2186 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:07:47.378252    2186 out.go:309] Setting ErrFile to fd 2...

	I1003 17:07:47.378256    2186 out.go:343] TERM=,COLORTERM=, which probably does not support color

	I1003 17:07:47.378462    2186 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/17345-936/.minikube/bin

	I1003 17:07:47.380044    2186 out.go:303] Setting JSON to false

	I1003 17:07:47.402155    2186 start.go:128] hostinfo: {"hostname":"MacOS-Agent-1.local","uptime":435,"bootTime":1696377632,"procs":470,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.0","kernelVersion":"23.0.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"b7610dcb-1435-5842-8d5a-b2388403fea3"}

	W1003 17:07:47.402254    2186 start.go:136] gopshost.Virtualization returned error: not implemented yet

	I1003 17:07:47.425857    2186 out.go:177] * [functional-157000] minikube v1.31.2 sur Darwin 14.0

	I1003 17:07:47.467677    2186 out.go:177]   - MINIKUBE_LOCATION=17345

	I1003 17:07:47.467711    2186 notify.go:220] Checking for updates...

	I1003 17:07:47.511485    2186 out.go:177]   - KUBECONFIG=/Users/jenkins/minikube-integration/17345-936/kubeconfig

	I1003 17:07:47.532652    2186 out.go:177]   - MINIKUBE_BIN=out/minikube-darwin-amd64

	I1003 17:07:47.553717    2186 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I1003 17:07:47.574506    2186 out.go:177]   - MINIKUBE_HOME=/Users/jenkins/minikube-integration/17345-936/.minikube

	I1003 17:07:47.595664    2186 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I1003 17:07:47.617078    2186 config.go:182] Loaded profile config "functional-157000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.28.2

	I1003 17:07:47.617416    2186 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:07:47.617463    2186 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:07:47.625721    2186 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:49948

	I1003 17:07:47.626089    2186 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:07:47.626505    2186 main.go:141] libmachine: Using API Version  1

	I1003 17:07:47.626532    2186 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:07:47.626761    2186 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:07:47.626882    2186 main.go:141] libmachine: (functional-157000) Calling .DriverName

	I1003 17:07:47.627067    2186 driver.go:373] Setting default libvirt URI to qemu:///system

	I1003 17:07:47.627310    2186 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit

	I1003 17:07:47.627334    2186 main.go:141] libmachine: Launching plugin server for driver hyperkit

	I1003 17:07:47.634888    2186 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:49950

	I1003 17:07:47.635220    2186 main.go:141] libmachine: () Calling .GetVersion

	I1003 17:07:47.635570    2186 main.go:141] libmachine: Using API Version  1

	I1003 17:07:47.635593    2186 main.go:141] libmachine: () Calling .SetConfigRaw

	I1003 17:07:47.635782    2186 main.go:141] libmachine: () Calling .GetMachineName

	I1003 17:07:47.635876    2186 main.go:141] libmachine: (functional-157000) Calling .DriverName

	I1003 17:07:47.664611    2186 out.go:177] * Utilisation du pilote hyperkit basé sur le profil existant

	I1003 17:07:47.685645    2186 start.go:298] selected driver: hyperkit

	I1003 17:07:47.685667    2186 start.go:902] validating driver "hyperkit" against &{Name:functional-157000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/17240/minikube-v1.31.0-1695060926-17240-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.40-1696360059-17345@sha256:76d99edd1576614d5c20a839dd16ae1d7c810f3b909a01797063d483159ea3ae Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 Kubernet

esConfig:{KubernetesVersion:v1.28.2 ClusterName:functional-157000 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:apiserver Key:enable-admission-plugins Value:NamespaceAutoProvision}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8441 NodeName:} Nodes:[{Name: IP:192.168.87.4 Port:8441 KubernetesVersion:v1.28.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDis

ks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 AutoPauseInterval:1m0s}

	I1003 17:07:47.685812    2186 start.go:913] status for hyperkit: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I1003 17:07:47.709670    2186 out.go:177] 

	W1003 17:07:47.730640    2186 out.go:239] X Fermeture en raison de RSRC_INSUFFICIENT_REQ_MEMORY : L'allocation de mémoire demandée 250 Mio est inférieure au minimum utilisable de 1800 Mo

	X Fermeture en raison de RSRC_INSUFFICIENT_REQ_MEMORY : L'allocation de mémoire demandée 250 Mio est inférieure au minimum utilisable de 1800 Mo

	I1003 17:07:47.751747    2186 out.go:177] 

** /stderr **

--- PASS: TestFunctional/parallel/InternationalLanguage (0.45s)

=== RUN   TestFunctional/parallel/StatusCmd

=== PAUSE TestFunctional/parallel/StatusCmd

=== CONT  TestFunctional/parallel/StatusCmd

functional_test.go:850: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 status

functional_test.go:856: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 status -f host:{{.Host}},kublet:{{.Kubelet}},apiserver:{{.APIServer}},kubeconfig:{{.Kubeconfig}}

functional_test.go:868: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 status -o json

--- PASS: TestFunctional/parallel/StatusCmd (0.47s)

=== RUN   TestFunctional/parallel/ServiceCmdConnect

=== PAUSE TestFunctional/parallel/ServiceCmdConnect

=== CONT  TestFunctional/parallel/ServiceCmdConnect

functional_test.go:1628: (dbg) Run:  kubectl --context functional-157000 create deployment hello-node-connect --image=registry.k8s.io/echoserver:1.8

functional_test.go:1634: (dbg) Run:  kubectl --context functional-157000 expose deployment hello-node-connect --type=NodePort --port=8080

functional_test.go:1639: (dbg) TestFunctional/parallel/ServiceCmdConnect: waiting 10m0s for pods matching "app=hello-node-connect" in namespace "default" ...

helpers_test.go:344: "hello-node-connect-55497b8b78-mt5lj" [214f54ee-82d9-4152-a5f4-8e2f50a82623] Pending / Ready:ContainersNotReady (containers with unready status: [echoserver]) / ContainersReady:ContainersNotReady (containers with unready status: [echoserver])

helpers_test.go:344: "hello-node-connect-55497b8b78-mt5lj" [214f54ee-82d9-4152-a5f4-8e2f50a82623] Running

functional_test.go:1639: (dbg) TestFunctional/parallel/ServiceCmdConnect: app=hello-node-connect healthy within 11.011282969s

functional_test.go:1648: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 service hello-node-connect --url

functional_test.go:1654: found endpoint for hello-node-connect: http://192.168.87.4:30436

functional_test.go:1674: http://192.168.87.4:30436: success! body:

Hostname: hello-node-connect-55497b8b78-mt5lj

Pod Information:

	-no pod information available-

Server values:

	server_version=nginx: 1.13.3 - lua: 10008

Request Information:

	client_address=10.244.0.1

	method=GET

	real path=/

	query=

	request_version=1.1

	request_uri=http://192.168.87.4:8080/

Request Headers:

	accept-encoding=gzip

	host=192.168.87.4:30436

	user-agent=Go-http-client/1.1

Request Body:

	-no body in request-

--- PASS: TestFunctional/parallel/ServiceCmdConnect (11.37s)

=== RUN   TestFunctional/parallel/AddonsCmd

=== PAUSE TestFunctional/parallel/AddonsCmd

=== CONT  TestFunctional/parallel/AddonsCmd

functional_test.go:1689: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 addons list

functional_test.go:1701: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 addons list -o json

--- PASS: TestFunctional/parallel/AddonsCmd (0.23s)

=== RUN   TestFunctional/parallel/PersistentVolumeClaim

=== PAUSE TestFunctional/parallel/PersistentVolumeClaim

=== CONT  TestFunctional/parallel/PersistentVolumeClaim

functional_test_pvc_test.go:44: (dbg) TestFunctional/parallel/PersistentVolumeClaim: waiting 4m0s for pods matching "integration-test=storage-provisioner" in namespace "kube-system" ...

helpers_test.go:344: "storage-provisioner" [9ed72367-3def-4208-b029-b9d9b950b056] Running

functional_test_pvc_test.go:44: (dbg) TestFunctional/parallel/PersistentVolumeClaim: integration-test=storage-provisioner healthy within 5.013698579s

functional_test_pvc_test.go:49: (dbg) Run:  kubectl --context functional-157000 get storageclass -o=json

functional_test_pvc_test.go:69: (dbg) Run:  kubectl --context functional-157000 apply -f testdata/storage-provisioner/pvc.yaml

functional_test_pvc_test.go:76: (dbg) Run:  kubectl --context functional-157000 get pvc myclaim -o=json

functional_test_pvc_test.go:125: (dbg) Run:  kubectl --context functional-157000 apply -f testdata/storage-provisioner/pod.yaml

functional_test_pvc_test.go:130: (dbg) TestFunctional/parallel/PersistentVolumeClaim: waiting 3m0s for pods matching "test=storage-provisioner" in namespace "default" ...

helpers_test.go:344: "sp-pod" [1f6b2c2d-c871-4869-a45d-aac6e6c5a310] Pending

helpers_test.go:344: "sp-pod" [1f6b2c2d-c871-4869-a45d-aac6e6c5a310] Pending / Ready:ContainersNotReady (containers with unready status: [myfrontend]) / ContainersReady:ContainersNotReady (containers with unready status: [myfrontend])

helpers_test.go:344: "sp-pod" [1f6b2c2d-c871-4869-a45d-aac6e6c5a310] Running

functional_test_pvc_test.go:130: (dbg) TestFunctional/parallel/PersistentVolumeClaim: test=storage-provisioner healthy within 13.016458213s

functional_test_pvc_test.go:100: (dbg) Run:  kubectl --context functional-157000 exec sp-pod -- touch /tmp/mount/foo

functional_test_pvc_test.go:106: (dbg) Run:  kubectl --context functional-157000 delete -f testdata/storage-provisioner/pod.yaml

functional_test_pvc_test.go:106: (dbg) Done: kubectl --context functional-157000 delete -f testdata/storage-provisioner/pod.yaml: (1.06674203s)

functional_test_pvc_test.go:125: (dbg) Run:  kubectl --context functional-157000 apply -f testdata/storage-provisioner/pod.yaml

functional_test_pvc_test.go:130: (dbg) TestFunctional/parallel/PersistentVolumeClaim: waiting 3m0s for pods matching "test=storage-provisioner" in namespace "default" ...

helpers_test.go:344: "sp-pod" [0855b3be-ed95-492a-95d8-01b0a8404e96] Pending

helpers_test.go:344: "sp-pod" [0855b3be-ed95-492a-95d8-01b0a8404e96] Pending / Ready:ContainersNotReady (containers with unready status: [myfrontend]) / ContainersReady:ContainersNotReady (containers with unready status: [myfrontend])

helpers_test.go:344: "sp-pod" [0855b3be-ed95-492a-95d8-01b0a8404e96] Running

functional_test_pvc_test.go:130: (dbg) TestFunctional/parallel/PersistentVolumeClaim: test=storage-provisioner healthy within 8.010609263s

functional_test_pvc_test.go:114: (dbg) Run:  kubectl --context functional-157000 exec sp-pod -- ls /tmp/mount

--- PASS: TestFunctional/parallel/PersistentVolumeClaim (27.78s)

=== RUN   TestFunctional/parallel/SSHCmd

=== PAUSE TestFunctional/parallel/SSHCmd

=== CONT  TestFunctional/parallel/SSHCmd

functional_test.go:1724: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "echo hello"

functional_test.go:1741: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "cat /etc/hostname"

--- PASS: TestFunctional/parallel/SSHCmd (0.30s)

=== RUN   TestFunctional/parallel/CpCmd

=== PAUSE TestFunctional/parallel/CpCmd

=== CONT  TestFunctional/parallel/CpCmd

helpers_test.go:556: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 cp testdata/cp-test.txt /home/docker/cp-test.txt

helpers_test.go:534: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh -n functional-157000 "sudo cat /home/docker/cp-test.txt"

helpers_test.go:556: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 cp functional-157000:/home/docker/cp-test.txt /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalparallelCpCmd2654809619/001/cp-test.txt

helpers_test.go:534: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh -n functional-157000 "sudo cat /home/docker/cp-test.txt"

--- PASS: TestFunctional/parallel/CpCmd (0.58s)

=== RUN   TestFunctional/parallel/MySQL

=== PAUSE TestFunctional/parallel/MySQL

=== CONT  TestFunctional/parallel/MySQL

functional_test.go:1789: (dbg) Run:  kubectl --context functional-157000 replace --force -f testdata/mysql.yaml

functional_test.go:1795: (dbg) TestFunctional/parallel/MySQL: waiting 10m0s for pods matching "app=mysql" in namespace "default" ...

helpers_test.go:344: "mysql-859648c796-qlmvn" [22b18ef0-02e9-41ef-b77b-04822dcc16e1] Pending / Ready:ContainersNotReady (containers with unready status: [mysql]) / ContainersReady:ContainersNotReady (containers with unready status: [mysql])

helpers_test.go:344: "mysql-859648c796-qlmvn" [22b18ef0-02e9-41ef-b77b-04822dcc16e1] Running

functional_test.go:1795: (dbg) TestFunctional/parallel/MySQL: app=mysql healthy within 23.016119347s

functional_test.go:1803: (dbg) Run:  kubectl --context functional-157000 exec mysql-859648c796-qlmvn -- mysql -ppassword -e "show databases;"

functional_test.go:1803: (dbg) Non-zero exit: kubectl --context functional-157000 exec mysql-859648c796-qlmvn -- mysql -ppassword -e "show databases;": exit status 1 (115.744846ms)

** stderr ** 

	mysql: [Warning] Using a password on the command line interface can be insecure.

	ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)

	command terminated with exit code 1

** /stderr **

functional_test.go:1803: (dbg) Run:  kubectl --context functional-157000 exec mysql-859648c796-qlmvn -- mysql -ppassword -e "show databases;"

--- PASS: TestFunctional/parallel/MySQL (24.76s)

=== RUN   TestFunctional/parallel/FileSync

=== PAUSE TestFunctional/parallel/FileSync

=== CONT  TestFunctional/parallel/FileSync

functional_test.go:1925: Checking for existence of /etc/test/nested/copy/1434/hosts within VM

functional_test.go:1927: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "sudo cat /etc/test/nested/copy/1434/hosts"

functional_test.go:1932: file sync test content: Test file for checking file sync process

--- PASS: TestFunctional/parallel/FileSync (0.16s)

=== RUN   TestFunctional/parallel/CertSync

=== PAUSE TestFunctional/parallel/CertSync

=== CONT  TestFunctional/parallel/CertSync

functional_test.go:1968: Checking for existence of /etc/ssl/certs/1434.pem within VM

functional_test.go:1969: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "sudo cat /etc/ssl/certs/1434.pem"

functional_test.go:1968: Checking for existence of /usr/share/ca-certificates/1434.pem within VM

functional_test.go:1969: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "sudo cat /usr/share/ca-certificates/1434.pem"

functional_test.go:1968: Checking for existence of /etc/ssl/certs/51391683.0 within VM

functional_test.go:1969: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "sudo cat /etc/ssl/certs/51391683.0"

functional_test.go:1995: Checking for existence of /etc/ssl/certs/14342.pem within VM

functional_test.go:1996: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "sudo cat /etc/ssl/certs/14342.pem"

functional_test.go:1995: Checking for existence of /usr/share/ca-certificates/14342.pem within VM

functional_test.go:1996: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "sudo cat /usr/share/ca-certificates/14342.pem"

functional_test.go:1995: Checking for existence of /etc/ssl/certs/3ec20f2e.0 within VM

functional_test.go:1996: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "sudo cat /etc/ssl/certs/3ec20f2e.0"

--- PASS: TestFunctional/parallel/CertSync (0.93s)

=== RUN   TestFunctional/parallel/NodeLabels

=== PAUSE TestFunctional/parallel/NodeLabels

=== CONT  TestFunctional/parallel/NodeLabels

functional_test.go:218: (dbg) Run:  kubectl --context functional-157000 get nodes --output=go-template "--template='{{range $k, $v := (index .items 0).metadata.labels}}{{$k}} {{end}}'"

--- PASS: TestFunctional/parallel/NodeLabels (0.05s)

=== RUN   TestFunctional/parallel/NonActiveRuntimeDisabled

=== PAUSE TestFunctional/parallel/NonActiveRuntimeDisabled

=== CONT  TestFunctional/parallel/NonActiveRuntimeDisabled

functional_test.go:2023: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "sudo systemctl is-active crio"

functional_test.go:2023: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p functional-157000 ssh "sudo systemctl is-active crio": exit status 1 (161.661783ms)

-- stdout --

	inactive

-- /stdout --

** stderr ** 

	ssh: Process exited with status 3

** /stderr **

--- PASS: TestFunctional/parallel/NonActiveRuntimeDisabled (0.16s)

=== RUN   TestFunctional/parallel/License

=== PAUSE TestFunctional/parallel/License

=== CONT  TestFunctional/parallel/License

functional_test.go:2284: (dbg) Run:  out/minikube-darwin-amd64 license

--- PASS: TestFunctional/parallel/License (0.39s)

=== RUN   TestFunctional/parallel/TunnelCmd/serial/RunSecondTunnel

functional_test_tunnel_test.go:154: (dbg) daemon: [out/minikube-darwin-amd64 -p functional-157000 tunnel --alsologtostderr]

functional_test_tunnel_test.go:154: (dbg) daemon: [out/minikube-darwin-amd64 -p functional-157000 tunnel --alsologtostderr]

functional_test_tunnel_test.go:194: (dbg) stopping [out/minikube-darwin-amd64 -p functional-157000 tunnel --alsologtostderr] ...

helpers_test.go:490: unable to find parent, assuming dead: process does not exist

functional_test_tunnel_test.go:194: (dbg) stopping [out/minikube-darwin-amd64 -p functional-157000 tunnel --alsologtostderr] ...

helpers_test.go:508: unable to kill pid 2012: os: process already finished

--- PASS: TestFunctional/parallel/TunnelCmd/serial/RunSecondTunnel (0.36s)

=== RUN   TestFunctional/parallel/TunnelCmd/serial/StartTunnel

functional_test_tunnel_test.go:129: (dbg) daemon: [out/minikube-darwin-amd64 -p functional-157000 tunnel --alsologtostderr]

--- PASS: TestFunctional/parallel/TunnelCmd/serial/StartTunnel (0.02s)

=== RUN   TestFunctional/parallel/TunnelCmd/serial/WaitService/Setup

functional_test_tunnel_test.go:212: (dbg) Run:  kubectl --context functional-157000 apply -f testdata/testsvc.yaml

functional_test_tunnel_test.go:216: (dbg) TestFunctional/parallel/TunnelCmd/serial/WaitService/Setup: waiting 4m0s for pods matching "run=nginx-svc" in namespace "default" ...

helpers_test.go:344: "nginx-svc" [11d17719-bb65-455f-a5ec-cc48e495c6b9] Pending / Ready:ContainersNotReady (containers with unready status: [nginx]) / ContainersReady:ContainersNotReady (containers with unready status: [nginx])

helpers_test.go:344: "nginx-svc" [11d17719-bb65-455f-a5ec-cc48e495c6b9] Running

functional_test_tunnel_test.go:216: (dbg) TestFunctional/parallel/TunnelCmd/serial/WaitService/Setup: run=nginx-svc healthy within 10.008237078s

--- PASS: TestFunctional/parallel/TunnelCmd/serial/WaitService/Setup (10.17s)

=== RUN   TestFunctional/parallel/TunnelCmd/serial/WaitService/IngressIP

functional_test_tunnel_test.go:234: (dbg) Run:  kubectl --context functional-157000 get svc nginx-svc -o jsonpath={.status.loadBalancer.ingress[0].ip}

--- PASS: TestFunctional/parallel/TunnelCmd/serial/WaitService/IngressIP (0.05s)

=== RUN   TestFunctional/parallel/TunnelCmd/serial/AccessDirect

functional_test_tunnel_test.go:299: tunnel at http://10.104.252.233 is working!

--- PASS: TestFunctional/parallel/TunnelCmd/serial/AccessDirect (0.02s)

=== RUN   TestFunctional/parallel/TunnelCmd/serial/DNSResolutionByDig

functional_test_tunnel_test.go:319: (dbg) Run:  dig +time=5 +tries=3 @10.96.0.10 nginx-svc.default.svc.cluster.local. A

functional_test_tunnel_test.go:327: DNS resolution by dig for nginx-svc.default.svc.cluster.local. is working!

--- PASS: TestFunctional/parallel/TunnelCmd/serial/DNSResolutionByDig (0.04s)

=== RUN   TestFunctional/parallel/TunnelCmd/serial/DNSResolutionByDscacheutil

functional_test_tunnel_test.go:351: (dbg) Run:  dscacheutil -q host -a name nginx-svc.default.svc.cluster.local.

functional_test_tunnel_test.go:359: DNS resolution by dscacheutil for nginx-svc.default.svc.cluster.local. is working!

--- PASS: TestFunctional/parallel/TunnelCmd/serial/DNSResolutionByDscacheutil (0.03s)

=== RUN   TestFunctional/parallel/TunnelCmd/serial/AccessThroughDNS

functional_test_tunnel_test.go:424: tunnel at http://nginx-svc.default.svc.cluster.local. is working!

--- PASS: TestFunctional/parallel/TunnelCmd/serial/AccessThroughDNS (0.02s)

=== RUN   TestFunctional/parallel/TunnelCmd/serial/DeleteTunnel

functional_test_tunnel_test.go:434: (dbg) stopping [out/minikube-darwin-amd64 -p functional-157000 tunnel --alsologtostderr] ...

--- PASS: TestFunctional/parallel/TunnelCmd/serial/DeleteTunnel (0.13s)

=== RUN   TestFunctional/parallel/ServiceCmd/DeployApp

functional_test.go:1438: (dbg) Run:  kubectl --context functional-157000 create deployment hello-node --image=registry.k8s.io/echoserver:1.8

functional_test.go:1444: (dbg) Run:  kubectl --context functional-157000 expose deployment hello-node --type=NodePort --port=8080

functional_test.go:1449: (dbg) TestFunctional/parallel/ServiceCmd/DeployApp: waiting 10m0s for pods matching "app=hello-node" in namespace "default" ...

helpers_test.go:344: "hello-node-d7447cc7f-g9q6h" [149820f0-b67c-4e98-aa02-d056343c2415] Pending / Ready:ContainersNotReady (containers with unready status: [echoserver]) / ContainersReady:ContainersNotReady (containers with unready status: [echoserver])

helpers_test.go:344: "hello-node-d7447cc7f-g9q6h" [149820f0-b67c-4e98-aa02-d056343c2415] Running

functional_test.go:1449: (dbg) TestFunctional/parallel/ServiceCmd/DeployApp: app=hello-node healthy within 8.009691105s

--- PASS: TestFunctional/parallel/ServiceCmd/DeployApp (8.12s)

=== RUN   TestFunctional/parallel/ProfileCmd/profile_not_create

functional_test.go:1269: (dbg) Run:  out/minikube-darwin-amd64 profile lis

functional_test.go:1274: (dbg) Run:  out/minikube-darwin-amd64 profile list --output json

--- PASS: TestFunctional/parallel/ProfileCmd/profile_not_create (0.32s)

=== RUN   TestFunctional/parallel/ProfileCmd/profile_list

functional_test.go:1309: (dbg) Run:  out/minikube-darwin-amd64 profile list

functional_test.go:1314: Took "188.158774ms" to run "out/minikube-darwin-amd64 profile list"

functional_test.go:1323: (dbg) Run:  out/minikube-darwin-amd64 profile list -l

functional_test.go:1328: Took "64.111677ms" to run "out/minikube-darwin-amd64 profile list -l"

--- PASS: TestFunctional/parallel/ProfileCmd/profile_list (0.25s)

=== RUN   TestFunctional/parallel/ProfileCmd/profile_json_output

functional_test.go:1360: (dbg) Run:  out/minikube-darwin-amd64 profile list -o json

functional_test.go:1365: Took "188.150453ms" to run "out/minikube-darwin-amd64 profile list -o json"

functional_test.go:1373: (dbg) Run:  out/minikube-darwin-amd64 profile list -o json --light

functional_test.go:1378: Took "62.96182ms" to run "out/minikube-darwin-amd64 profile list -o json --light"

--- PASS: TestFunctional/parallel/ProfileCmd/profile_json_output (0.25s)

=== RUN   TestFunctional/parallel/MountCmd/any-port

functional_test_mount_test.go:73: (dbg) daemon: [out/minikube-darwin-amd64 mount -p functional-157000 /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalparallelMountCmdany-port2959614672/001:/mount-9p --alsologtostderr -v=1]

functional_test_mount_test.go:107: wrote "test-1696378063505070000" to /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalparallelMountCmdany-port2959614672/001/created-by-test

functional_test_mount_test.go:107: wrote "test-1696378063505070000" to /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalparallelMountCmdany-port2959614672/001/created-by-test-removed-by-pod

functional_test_mount_test.go:107: wrote "test-1696378063505070000" to /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalparallelMountCmdany-port2959614672/001/test-1696378063505070000

functional_test_mount_test.go:115: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "findmnt -T /mount-9p | grep 9p"

functional_test_mount_test.go:115: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p functional-157000 ssh "findmnt -T /mount-9p | grep 9p": exit status 1 (141.889987ms)

** stderr ** 

	ssh: Process exited with status 1

** /stderr **

functional_test_mount_test.go:115: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "findmnt -T /mount-9p | grep 9p"

functional_test_mount_test.go:129: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh -- ls -la /mount-9p

functional_test_mount_test.go:133: guest mount directory contents

total 2

-rw-r--r-- 1 docker docker 24 Oct  4 00:07 created-by-test

-rw-r--r-- 1 docker docker 24 Oct  4 00:07 created-by-test-removed-by-pod

-rw-r--r-- 1 docker docker 24 Oct  4 00:07 test-1696378063505070000

functional_test_mount_test.go:137: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh cat /mount-9p/test-1696378063505070000

functional_test_mount_test.go:148: (dbg) Run:  kubectl --context functional-157000 replace --force -f testdata/busybox-mount-test.yaml

functional_test_mount_test.go:153: (dbg) TestFunctional/parallel/MountCmd/any-port: waiting 4m0s for pods matching "integration-test=busybox-mount" in namespace "default" ...

helpers_test.go:344: "busybox-mount" [d20b8d46-0049-4bd2-bf77-c947dc7dfaf1] Pending

helpers_test.go:344: "busybox-mount" [d20b8d46-0049-4bd2-bf77-c947dc7dfaf1] Pending / Ready:ContainersNotReady (containers with unready status: [mount-munger]) / ContainersReady:ContainersNotReady (containers with unready status: [mount-munger])

helpers_test.go:344: "busybox-mount" [d20b8d46-0049-4bd2-bf77-c947dc7dfaf1] Pending: Initialized:PodCompleted / Ready:PodCompleted / ContainersReady:PodCompleted

helpers_test.go:344: "busybox-mount" [d20b8d46-0049-4bd2-bf77-c947dc7dfaf1] Succeeded: Initialized:PodCompleted / Ready:PodCompleted / ContainersReady:PodCompleted

functional_test_mount_test.go:153: (dbg) TestFunctional/parallel/MountCmd/any-port: integration-test=busybox-mount healthy within 5.01250957s

functional_test_mount_test.go:169: (dbg) Run:  kubectl --context functional-157000 logs busybox-mount

functional_test_mount_test.go:181: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh stat /mount-9p/created-by-test

functional_test_mount_test.go:181: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh stat /mount-9p/created-by-pod

functional_test_mount_test.go:90: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "sudo umount -f /mount-9p"

functional_test_mount_test.go:94: (dbg) stopping [out/minikube-darwin-amd64 mount -p functional-157000 /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalparallelMountCmdany-port2959614672/001:/mount-9p --alsologtostderr -v=1] ...

--- PASS: TestFunctional/parallel/MountCmd/any-port (7.29s)

=== RUN   TestFunctional/parallel/ServiceCmd/List

functional_test.go:1458: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 service list

--- PASS: TestFunctional/parallel/ServiceCmd/List (0.37s)

=== RUN   TestFunctional/parallel/ServiceCmd/JSONOutput

functional_test.go:1488: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 service list -o json

functional_test.go:1493: Took "367.218902ms" to run "out/minikube-darwin-amd64 -p functional-157000 service list -o json"

--- PASS: TestFunctional/parallel/ServiceCmd/JSONOutput (0.37s)

=== RUN   TestFunctional/parallel/ServiceCmd/HTTPS

functional_test.go:1508: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 service --namespace=default --https --url hello-node

functional_test.go:1521: found endpoint: https://192.168.87.4:31072

--- PASS: TestFunctional/parallel/ServiceCmd/HTTPS (0.24s)

=== RUN   TestFunctional/parallel/ServiceCmd/Format

functional_test.go:1539: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 service hello-node --url --format={{.IP}}

--- PASS: TestFunctional/parallel/ServiceCmd/Format (0.24s)

=== RUN   TestFunctional/parallel/ServiceCmd/URL

functional_test.go:1558: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 service hello-node --url

functional_test.go:1564: found endpoint for hello-node: http://192.168.87.4:31072

--- PASS: TestFunctional/parallel/ServiceCmd/URL (0.28s)

=== RUN   TestFunctional/parallel/MountCmd/specific-port

functional_test_mount_test.go:213: (dbg) daemon: [out/minikube-darwin-amd64 mount -p functional-157000 /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalparallelMountCmdspecific-port3159435151/001:/mount-9p --alsologtostderr -v=1 --port 46464]

functional_test_mount_test.go:243: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "findmnt -T /mount-9p | grep 9p"

functional_test_mount_test.go:243: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p functional-157000 ssh "findmnt -T /mount-9p | grep 9p": exit status 1 (145.852541ms)

** stderr ** 

	ssh: Process exited with status 1

** /stderr **

functional_test_mount_test.go:243: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "findmnt -T /mount-9p | grep 9p"

functional_test_mount_test.go:257: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh -- ls -la /mount-9p

functional_test_mount_test.go:261: guest mount directory contents

total 0

functional_test_mount_test.go:263: (dbg) stopping [out/minikube-darwin-amd64 mount -p functional-157000 /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalparallelMountCmdspecific-port3159435151/001:/mount-9p --alsologtostderr -v=1 --port 46464] ...

functional_test_mount_test.go:264: reading mount text

functional_test_mount_test.go:278: done reading mount text

functional_test_mount_test.go:230: (dbg) Run:  out/minikube-darwin-amd64 -p functional-157000 ssh "sudo umount -f /mount-9p"

functional_test_mount_test.go:230: (dbg) Non-zero exit: out/minikube-darwin-amd64 -p functional-157000 ssh "sudo umount -f /mount-9p": exit status 1 (114.883561ms)

-- stdout --

	umount: /mount-9p: not mounted.

-- /stdout --

** stderr ** 

	ssh: Process exited with status 32

** /stderr **

functional_test_mount_test.go:232: "out/minikube-darwin-amd64 -p functional-157000 ssh \"sudo umount -f /mount-9p\"": exit status 1

functional_test_mount_test.go:234: (dbg) stopping [out/minikube-darwin-amd64 mount -p functional-157000 /var/folders/52/zh_qmlrn1f36yr6lgs7nxtym0000gp/T/TestFunctionalparallelMountCmdspecific-port3159435151/001:/mount-9p --alsologtostderr -v=1 --port 46464] ...

--- PASS: TestFunctional/parallel/MountCmd/specific-port (1.56s)