=== RUN TestMultiControlPlane/serial/DegradedAfterClusterRestart
ha_test.go:390: (dbg) Run: out/minikube-darwin-amd64 profile list --output json
ha_test.go:413: expected profile "ha-857000" in json of 'profile list' to have "Degraded" status but have "HAppy" status. got *"{\"invalid\":[],\"valid\":[{\"Name\":\"ha-857000\",\"Status\":\"HAppy\",\"Config\":{\"Name\":\"ha-857000\",\"KeepContext\":false,\"EmbedCerts\":false,\"MinikubeISO\":\"https://storage.googleapis.com/minikube-builds/iso/19646/minikube-v1.34.0-1726415472-19646-amd64.iso\",\"KicBaseImage\":\"gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726358845-19644@sha256:4c67a32a16c2d4f824f00267c172fd225757ca75441e363d925dc9583137f0b0\",\"Memory\":2200,\"CPUs\":2,\"DiskSize\":20000,\"Driver\":\"hyperkit\",\"HyperkitVpnKitSock\":\"\",\"HyperkitVSockPorts\":[],\"DockerEnv\":null,\"ContainerVolumeMounts\":null,\"InsecureRegistry\":null,\"RegistryMirror\":[],\"HostOnlyCIDR\":\"192.168.59.1/24\",\"HypervVirtualSwitch\":\"\",\"HypervUseExternalSwitch\":false,\"HypervExternalAdapter\":\"\",\"KVMNetwork\":\"default\",\"KVMQemuURI\":\"qemu:///system\",\"KVMGPU\":false,\"KVMHidden\":false,\"KVMNUMACount\":
1,\"APIServerPort\":8443,\"DockerOpt\":null,\"DisableDriverMounts\":false,\"NFSShare\":[],\"NFSSharesRoot\":\"/nfsshares\",\"UUID\":\"\",\"NoVTXCheck\":false,\"DNSProxy\":false,\"HostDNSResolver\":true,\"HostOnlyNicType\":\"virtio\",\"NatNicType\":\"virtio\",\"SSHIPAddress\":\"\",\"SSHUser\":\"root\",\"SSHKey\":\"\",\"SSHPort\":22,\"KubernetesConfig\":{\"KubernetesVersion\":\"v1.31.1\",\"ClusterName\":\"ha-857000\",\"Namespace\":\"default\",\"APIServerHAVIP\":\"192.169.0.254\",\"APIServerName\":\"minikubeCA\",\"APIServerNames\":null,\"APIServerIPs\":null,\"DNSDomain\":\"cluster.local\",\"ContainerRuntime\":\"docker\",\"CRISocket\":\"\",\"NetworkPlugin\":\"cni\",\"FeatureGates\":\"\",\"ServiceCIDR\":\"10.96.0.0/12\",\"ImageRepository\":\"\",\"LoadBalancerStartIP\":\"\",\"LoadBalancerEndIP\":\"\",\"CustomIngressCert\":\"\",\"RegistryAliases\":\"\",\"ExtraOptions\":null,\"ShouldLoadCachedImages\":true,\"EnableDefaultCNI\":false,\"CNI\":\"\"},\"Nodes\":[{\"Name\":\"\",\"IP\":\"192.169.0.5\",\"Port\":8443,\"Kubern
etesVersion\":\"v1.31.1\",\"ContainerRuntime\":\"docker\",\"ControlPlane\":true,\"Worker\":true},{\"Name\":\"m02\",\"IP\":\"192.169.0.6\",\"Port\":8443,\"KubernetesVersion\":\"v1.31.1\",\"ContainerRuntime\":\"docker\",\"ControlPlane\":true,\"Worker\":true},{\"Name\":\"m03\",\"IP\":\"192.169.0.7\",\"Port\":8443,\"KubernetesVersion\":\"v1.31.1\",\"ContainerRuntime\":\"docker\",\"ControlPlane\":true,\"Worker\":true},{\"Name\":\"m04\",\"IP\":\"192.169.0.8\",\"Port\":0,\"KubernetesVersion\":\"v1.31.1\",\"ContainerRuntime\":\"docker\",\"ControlPlane\":false,\"Worker\":true}],\"Addons\":{\"ambassador\":false,\"auto-pause\":false,\"cloud-spanner\":false,\"csi-hostpath-driver\":false,\"dashboard\":false,\"default-storageclass\":false,\"efk\":false,\"freshpod\":false,\"gcp-auth\":false,\"gvisor\":false,\"headlamp\":false,\"helm-tiller\":false,\"inaccel\":false,\"ingress\":false,\"ingress-dns\":false,\"inspektor-gadget\":false,\"istio\":false,\"istio-provisioner\":false,\"kong\":false,\"kubeflow\":false,\"kubevirt\":fal
se,\"logviewer\":false,\"metallb\":false,\"metrics-server\":false,\"nvidia-device-plugin\":false,\"nvidia-driver-installer\":false,\"nvidia-gpu-device-plugin\":false,\"olm\":false,\"pod-security-policy\":false,\"portainer\":false,\"registry\":false,\"registry-aliases\":false,\"registry-creds\":false,\"storage-provisioner\":false,\"storage-provisioner-gluster\":false,\"storage-provisioner-rancher\":false,\"volcano\":false,\"volumesnapshots\":false,\"yakd\":false},\"CustomAddonImages\":null,\"CustomAddonRegistries\":null,\"VerifyComponents\":{\"apiserver\":true,\"apps_running\":true,\"default_sa\":true,\"extra\":true,\"kubelet\":true,\"node_ready\":true,\"system_pods\":true},\"StartHostTimeout\":360000000000,\"ScheduledStop\":null,\"ExposedPorts\":[],\"ListenAddress\":\"\",\"Network\":\"\",\"Subnet\":\"\",\"MultiNodeRequested\":true,\"ExtraDisks\":0,\"CertExpiration\":94608000000000000,\"Mount\":false,\"MountString\":\"/Users:/minikube-host\",\"Mount9PVersion\":\"9p2000.L\",\"MountGID\":\"docker\",\"MountIP\":\
"\",\"MountMSize\":262144,\"MountOptions\":[],\"MountPort\":0,\"MountType\":\"9p\",\"MountUID\":\"docker\",\"BinaryMirror\":\"\",\"DisableOptimizations\":false,\"DisableMetrics\":false,\"CustomQemuFirmwarePath\":\"\",\"SocketVMnetClientPath\":\"\",\"SocketVMnetPath\":\"\",\"StaticIP\":\"\",\"SSHAuthSock\":\"\",\"SSHAgentPID\":0,\"GPUs\":\"\",\"AutoPauseInterval\":60000000000},\"Active\":false,\"ActiveKubeContext\":true}]}"*. args: "out/minikube-darwin-amd64 profile list --output json"
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-darwin-amd64 status --format={{.Host}} -p ha-857000 -n ha-857000
helpers_test.go:244: <<< TestMultiControlPlane/serial/DegradedAfterClusterRestart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestMultiControlPlane/serial/DegradedAfterClusterRestart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-darwin-amd64 -p ha-857000 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-darwin-amd64 -p ha-857000 logs -n 25: (3.583705356s)
helpers_test.go:252: TestMultiControlPlane/serial/DegradedAfterClusterRestart logs:
-- stdout --
==> Audit <==
|---------|------------------------------------------------------------------------------------------------------------------------------|-----------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|------------------------------------------------------------------------------------------------------------------------------|-----------|---------|---------|---------------------|---------------------|
| cp | ha-857000 cp ha-857000-m03:/home/docker/cp-test.txt | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | ha-857000-m04:/home/docker/cp-test_ha-857000-m03_ha-857000-m04.txt | | | | | |
| ssh | ha-857000 ssh -n | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | ha-857000-m03 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| ssh | ha-857000 ssh -n ha-857000-m04 sudo cat | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | /home/docker/cp-test_ha-857000-m03_ha-857000-m04.txt | | | | | |
| cp | ha-857000 cp testdata/cp-test.txt | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | ha-857000-m04:/home/docker/cp-test.txt | | | | | |
| ssh | ha-857000 ssh -n | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | ha-857000-m04 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| cp | ha-857000 cp ha-857000-m04:/home/docker/cp-test.txt | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | /var/folders/vq/yhv778t970xgml0dzm5fdwlr0000gp/T/TestMultiControlPlaneserialCopyFile3490570692/001/cp-test_ha-857000-m04.txt | | | | | |
| ssh | ha-857000 ssh -n | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | ha-857000-m04 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| cp | ha-857000 cp ha-857000-m04:/home/docker/cp-test.txt | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | ha-857000:/home/docker/cp-test_ha-857000-m04_ha-857000.txt | | | | | |
| ssh | ha-857000 ssh -n | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | ha-857000-m04 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| ssh | ha-857000 ssh -n ha-857000 sudo cat | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | /home/docker/cp-test_ha-857000-m04_ha-857000.txt | | | | | |
| cp | ha-857000 cp ha-857000-m04:/home/docker/cp-test.txt | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | ha-857000-m02:/home/docker/cp-test_ha-857000-m04_ha-857000-m02.txt | | | | | |
| ssh | ha-857000 ssh -n | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | ha-857000-m04 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| ssh | ha-857000 ssh -n ha-857000-m02 sudo cat | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | /home/docker/cp-test_ha-857000-m04_ha-857000-m02.txt | | | | | |
| cp | ha-857000 cp ha-857000-m04:/home/docker/cp-test.txt | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | ha-857000-m03:/home/docker/cp-test_ha-857000-m04_ha-857000-m03.txt | | | | | |
| ssh | ha-857000 ssh -n | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | ha-857000-m04 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| ssh | ha-857000 ssh -n ha-857000-m03 sudo cat | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | /home/docker/cp-test_ha-857000-m04_ha-857000-m03.txt | | | | | |
| node | ha-857000 node stop m02 -v=7 | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:04 PDT |
| | --alsologtostderr | | | | | |
| node | ha-857000 node start m02 -v=7 | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:04 PDT | 17 Sep 24 02:05 PDT |
| | --alsologtostderr | | | | | |
| node | list -p ha-857000 -v=7 | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:05 PDT | |
| | --alsologtostderr | | | | | |
| stop | -p ha-857000 -v=7 | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:05 PDT | 17 Sep 24 02:06 PDT |
| | --alsologtostderr | | | | | |
| start | -p ha-857000 --wait=true -v=7 | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:06 PDT | |
| | --alsologtostderr | | | | | |
| node | list -p ha-857000 | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:07 PDT | |
| node | ha-857000 node delete m03 -v=7 | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:07 PDT | |
| | --alsologtostderr | | | | | |
| stop | ha-857000 stop -v=7 | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:07 PDT | 17 Sep 24 02:10 PDT |
| | --alsologtostderr | | | | | |
| start | -p ha-857000 --wait=true | ha-857000 | jenkins | v1.34.0 | 17 Sep 24 02:10 PDT | 17 Sep 24 02:13 PDT |
| | -v=7 --alsologtostderr | | | | | |
| | --driver=hyperkit | | | | | |
|---------|------------------------------------------------------------------------------------------------------------------------------|-----------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/09/17 02:10:27
Running on machine: MacOS-Agent-2
Binary: Built with gc go1.23.0 for darwin/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0917 02:10:27.105477 4110 out.go:345] Setting OutFile to fd 1 ...
I0917 02:10:27.105665 4110 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0917 02:10:27.105670 4110 out.go:358] Setting ErrFile to fd 2...
I0917 02:10:27.105674 4110 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0917 02:10:27.105845 4110 root.go:338] Updating PATH: /Users/jenkins/minikube-integration/19648-1025/.minikube/bin
I0917 02:10:27.107332 4110 out.go:352] Setting JSON to false
I0917 02:10:27.130053 4110 start.go:129] hostinfo: {"hostname":"MacOS-Agent-2.local","uptime":2397,"bootTime":1726561830,"procs":437,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"14.6.1","kernelVersion":"23.6.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"2965c349-98a5-5970-aaa9-9eedd3ae5959"}
W0917 02:10:27.130205 4110 start.go:137] gopshost.Virtualization returned error: not implemented yet
I0917 02:10:27.152188 4110 out.go:177] * [ha-857000] minikube v1.34.0 on Darwin 14.6.1
I0917 02:10:27.194040 4110 out.go:177] - MINIKUBE_LOCATION=19648
I0917 02:10:27.194117 4110 notify.go:220] Checking for updates...
I0917 02:10:27.238575 4110 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/19648-1025/kubeconfig
I0917 02:10:27.259736 4110 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0917 02:10:27.280930 4110 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0917 02:10:27.301762 4110 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/19648-1025/.minikube
I0917 02:10:27.322633 4110 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0917 02:10:27.344421 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:10:27.344920 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:10:27.344973 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:10:27.354413 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52088
I0917 02:10:27.354771 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:10:27.355142 4110 main.go:141] libmachine: Using API Version 1
I0917 02:10:27.355153 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:10:27.355356 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:10:27.355460 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:10:27.355684 4110 driver.go:394] Setting default libvirt URI to qemu:///system
I0917 02:10:27.355976 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:10:27.356005 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:10:27.364420 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52090
I0917 02:10:27.364811 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:10:27.365167 4110 main.go:141] libmachine: Using API Version 1
I0917 02:10:27.365180 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:10:27.365391 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:10:27.365504 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:10:27.393706 4110 out.go:177] * Using the hyperkit driver based on existing profile
I0917 02:10:27.435894 4110 start.go:297] selected driver: hyperkit
I0917 02:10:27.435922 4110 start.go:901] validating driver "hyperkit" against &{Name:ha-857000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19646/minikube-v1.34.0-1726415472-19646-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726358845-19644@sha256:4c67a32a16c2d4f824f00267c172fd225757ca75441e363d925dc9583137f0b0 Memory:2200 CPUs:2 DiskSize:20000 Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{Kuber
netesVersion:v1.31.1 ClusterName:ha-857000 Namespace:default APIServerHAVIP:192.169.0.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.169.0.5 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m02 IP:192.169.0.6 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m03 IP:192.169.0.7 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m04 IP:192.169.0.8 Port:0 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:false Worker:true}] Addons:map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclas
s:false efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:false storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersio
n:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0917 02:10:27.436195 4110 start.go:912] status for hyperkit: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0917 02:10:27.436329 4110 install.go:52] acquiring lock: {Name:mk4023283b30b374c3f04c8805d539e68824c0b8 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0917 02:10:27.436542 4110 install.go:117] Validating docker-machine-driver-hyperkit, PATH=/Users/jenkins/minikube-integration/19648-1025/.minikube/bin:/Users/jenkins/workspace/out/:/usr/bin:/bin:/usr/sbin:/sbin:/Users/jenkins/google-cloud-sdk/bin:/usr/local/bin/:/usr/local/go/bin/:/Users/jenkins/go/bin
I0917 02:10:27.445831 4110 install.go:137] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit version is 1.34.0
I0917 02:10:27.449537 4110 install.go:79] stdout: /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:10:27.449556 4110 install.go:81] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit looks good
I0917 02:10:27.452252 4110 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0917 02:10:27.452291 4110 cni.go:84] Creating CNI manager for ""
I0917 02:10:27.452327 4110 cni.go:136] multinode detected (4 nodes found), recommending kindnet
I0917 02:10:27.452403 4110 start.go:340] cluster config:
{Name:ha-857000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19646/minikube-v1.34.0-1726415472-19646-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726358845-19644@sha256:4c67a32a16c2d4f824f00267c172fd225757ca75441e363d925dc9583137f0b0 Memory:2200 CPUs:2 DiskSize:20000 Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:ha-857000 Namespace:default APIServerHAVIP:192.16
9.0.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.169.0.5 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m02 IP:192.169.0.6 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m03 IP:192.169.0.7 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m04 IP:192.169.0.8 Port:0 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:false Worker:true}] Addons:map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:false efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-t
iller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:false storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0
MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0917 02:10:27.452523 4110 iso.go:125] acquiring lock: {Name:mkc407d80a0f8e78f0e24d63c464bb315c80139b Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0917 02:10:27.494874 4110 out.go:177] * Starting "ha-857000" primary control-plane node in "ha-857000" cluster
I0917 02:10:27.515806 4110 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime docker
I0917 02:10:27.515897 4110 preload.go:146] Found local preload: /Users/jenkins/minikube-integration/19648-1025/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-docker-overlay2-amd64.tar.lz4
I0917 02:10:27.515918 4110 cache.go:56] Caching tarball of preloaded images
I0917 02:10:27.516138 4110 preload.go:172] Found /Users/jenkins/minikube-integration/19648-1025/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0917 02:10:27.516158 4110 cache.go:59] Finished verifying existence of preloaded tar for v1.31.1 on docker
I0917 02:10:27.516383 4110 profile.go:143] Saving config to /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/config.json ...
I0917 02:10:27.517269 4110 start.go:360] acquireMachinesLock for ha-857000: {Name:mkcb27accfc361dc9c175eeb8bb69dc9967aa557 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0917 02:10:27.517388 4110 start.go:364] duration metric: took 96.177µs to acquireMachinesLock for "ha-857000"
I0917 02:10:27.517441 4110 start.go:96] Skipping create...Using existing machine configuration
I0917 02:10:27.517460 4110 fix.go:54] fixHost starting:
I0917 02:10:27.517898 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:10:27.517930 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:10:27.526784 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52092
I0917 02:10:27.527129 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:10:27.527462 4110 main.go:141] libmachine: Using API Version 1
I0917 02:10:27.527473 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:10:27.527739 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:10:27.527880 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:10:27.527995 4110 main.go:141] libmachine: (ha-857000) Calling .GetState
I0917 02:10:27.528094 4110 main.go:141] libmachine: (ha-857000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:10:27.528210 4110 main.go:141] libmachine: (ha-857000) DBG | hyperkit pid from json: 3964
I0917 02:10:27.529100 4110 main.go:141] libmachine: (ha-857000) DBG | hyperkit pid 3964 missing from process table
I0917 02:10:27.529122 4110 fix.go:112] recreateIfNeeded on ha-857000: state=Stopped err=<nil>
I0917 02:10:27.529141 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
W0917 02:10:27.529225 4110 fix.go:138] unexpected machine state, will restart: <nil>
I0917 02:10:27.570570 4110 out.go:177] * Restarting existing hyperkit VM for "ha-857000" ...
I0917 02:10:27.591801 4110 main.go:141] libmachine: (ha-857000) Calling .Start
I0917 02:10:27.592089 4110 main.go:141] libmachine: (ha-857000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:10:27.592131 4110 main.go:141] libmachine: (ha-857000) DBG | clean start, hyperkit pid file doesn't exist: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/hyperkit.pid
I0917 02:10:27.592193 4110 main.go:141] libmachine: (ha-857000) DBG | Using UUID f6fb2ac0-c2e1-40b6-91c0-dbf4ea1b682c
I0917 02:10:27.699994 4110 main.go:141] libmachine: (ha-857000) DBG | Generated MAC c2:63:2b:63:80:76
I0917 02:10:27.700019 4110 main.go:141] libmachine: (ha-857000) DBG | Starting with cmdline: loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000
I0917 02:10:27.700136 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 DEBUG: hyperkit: Start &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"f6fb2ac0-c2e1-40b6-91c0-dbf4ea1b682c", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc00029b6e0)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/bzimage", Initrd:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os.Process)(nil)}
I0917 02:10:27.700165 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 DEBUG: hyperkit: check &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"f6fb2ac0-c2e1-40b6-91c0-dbf4ea1b682c", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc00029b6e0)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/bzimage", Initrd:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os.Process)(nil)}
I0917 02:10:27.700210 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 DEBUG: hyperkit: Arguments: []string{"-A", "-u", "-F", "/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/hyperkit.pid", "-c", "2", "-m", "2200M", "-s", "0:0,hostbridge", "-s", "31,lpc", "-s", "1:0,virtio-net", "-U", "f6fb2ac0-c2e1-40b6-91c0-dbf4ea1b682c", "-s", "2:0,virtio-blk,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/ha-857000.rawdisk", "-s", "3,ahci-cd,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/boot2docker.iso", "-s", "4,virtio-rnd", "-l", "com1,autopty=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/tty,log=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/console-ring", "-f", "kexec,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/bzimage,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/initrd,earlyprintk=s
erial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000"}
I0917 02:10:27.700256 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 DEBUG: hyperkit: CmdLine: "/usr/local/bin/hyperkit -A -u -F /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/hyperkit.pid -c 2 -m 2200M -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-net -U f6fb2ac0-c2e1-40b6-91c0-dbf4ea1b682c -s 2:0,virtio-blk,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/ha-857000.rawdisk -s 3,ahci-cd,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/boot2docker.iso -s 4,virtio-rnd -l com1,autopty=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/tty,log=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/console-ring -f kexec,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/bzimage,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/initrd,earlyprintk=serial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset
norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000"
I0917 02:10:27.700270 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 DEBUG: hyperkit: Redirecting stdout/stderr to logger
I0917 02:10:27.701709 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 DEBUG: hyperkit: Pid is 4124
I0917 02:10:27.702059 4110 main.go:141] libmachine: (ha-857000) DBG | Attempt 0
I0917 02:10:27.702070 4110 main.go:141] libmachine: (ha-857000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:10:27.702132 4110 main.go:141] libmachine: (ha-857000) DBG | hyperkit pid from json: 4124
I0917 02:10:27.703343 4110 main.go:141] libmachine: (ha-857000) DBG | Searching for c2:63:2b:63:80:76 in /var/db/dhcpd_leases ...
I0917 02:10:27.703398 4110 main.go:141] libmachine: (ha-857000) DBG | Found 7 entries in /var/db/dhcpd_leases!
I0917 02:10:27.703416 4110 main.go:141] libmachine: (ha-857000) DBG | dhcp entry: {Name:minikube IPAddress:192.169.0.5 HWAddress:c2:63:2b:63:80:76 ID:1,c2:63:2b:63:80:76 Lease:0x66e94781}
I0917 02:10:27.703422 4110 main.go:141] libmachine: (ha-857000) DBG | Found match: c2:63:2b:63:80:76
I0917 02:10:27.703434 4110 main.go:141] libmachine: (ha-857000) DBG | IP: 192.169.0.5
I0917 02:10:27.703500 4110 main.go:141] libmachine: (ha-857000) Calling .GetConfigRaw
I0917 02:10:27.704135 4110 main.go:141] libmachine: (ha-857000) Calling .GetIP
I0917 02:10:27.704313 4110 profile.go:143] Saving config to /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/config.json ...
I0917 02:10:27.704745 4110 machine.go:93] provisionDockerMachine start ...
I0917 02:10:27.704755 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:10:27.704862 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:27.704967 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:27.705062 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:27.705172 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:27.705289 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:27.705426 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:10:27.705645 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.5 22 <nil> <nil>}
I0917 02:10:27.705655 4110 main.go:141] libmachine: About to run SSH command:
hostname
I0917 02:10:27.709824 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 INFO : hyperkit: stderr: Using fd 5 for I/O notifications
I0917 02:10:27.761328 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 INFO : hyperkit: stderr: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/boot2docker.iso: fcntl(F_PUNCHHOLE) Operation not permitted: block device will not support TRIM/DISCARD
I0917 02:10:27.762023 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0
I0917 02:10:27.762037 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0
I0917 02:10:27.762058 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0
I0917 02:10:27.762068 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:27 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0
I0917 02:10:28.142704 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:28 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 0
I0917 02:10:28.142720 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:28 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 0
I0917 02:10:28.257454 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:28 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0
I0917 02:10:28.257477 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:28 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0
I0917 02:10:28.257500 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:28 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0
I0917 02:10:28.257510 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:28 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0
I0917 02:10:28.258332 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:28 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 1
I0917 02:10:28.258356 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:28 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 1
I0917 02:10:33.845455 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:33 INFO : hyperkit: stderr: rdmsr to register 0x64d on vcpu 1
I0917 02:10:33.845506 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:33 INFO : hyperkit: stderr: rdmsr to register 0x64e on vcpu 1
I0917 02:10:33.845516 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:33 INFO : hyperkit: stderr: rdmsr to register 0x34 on vcpu 1
I0917 02:10:33.869458 4110 main.go:141] libmachine: (ha-857000) DBG | 2024/09/17 02:10:33 INFO : hyperkit: stderr: rdmsr to register 0xc0011029 on vcpu 1
I0917 02:10:38.774269 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: minikube
I0917 02:10:38.774287 4110 main.go:141] libmachine: (ha-857000) Calling .GetMachineName
I0917 02:10:38.774460 4110 buildroot.go:166] provisioning hostname "ha-857000"
I0917 02:10:38.774470 4110 main.go:141] libmachine: (ha-857000) Calling .GetMachineName
I0917 02:10:38.774556 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:38.774689 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:38.774787 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:38.774865 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:38.774959 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:38.775097 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:10:38.775254 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.5 22 <nil> <nil>}
I0917 02:10:38.775262 4110 main.go:141] libmachine: About to run SSH command:
sudo hostname ha-857000 && echo "ha-857000" | sudo tee /etc/hostname
I0917 02:10:38.842954 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: ha-857000
I0917 02:10:38.842972 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:38.843114 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:38.843224 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:38.843309 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:38.843398 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:38.843557 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:10:38.843701 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.5 22 <nil> <nil>}
I0917 02:10:38.843712 4110 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sha-857000' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 ha-857000/g' /etc/hosts;
else
echo '127.0.1.1 ha-857000' | sudo tee -a /etc/hosts;
fi
fi
I0917 02:10:38.908790 4110 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0917 02:10:38.908811 4110 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/19648-1025/.minikube CaCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/19648-1025/.minikube}
I0917 02:10:38.908824 4110 buildroot.go:174] setting up certificates
I0917 02:10:38.908830 4110 provision.go:84] configureAuth start
I0917 02:10:38.908845 4110 main.go:141] libmachine: (ha-857000) Calling .GetMachineName
I0917 02:10:38.908979 4110 main.go:141] libmachine: (ha-857000) Calling .GetIP
I0917 02:10:38.909073 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:38.909177 4110 provision.go:143] copyHostCerts
I0917 02:10:38.909208 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem
I0917 02:10:38.909278 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem, removing ...
I0917 02:10:38.909287 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem
I0917 02:10:38.909606 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem (1123 bytes)
I0917 02:10:38.909812 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem
I0917 02:10:38.909853 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem, removing ...
I0917 02:10:38.909857 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem
I0917 02:10:38.909935 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem (1675 bytes)
I0917 02:10:38.910085 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem
I0917 02:10:38.910127 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem, removing ...
I0917 02:10:38.910132 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem
I0917 02:10:38.910214 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem (1078 bytes)
I0917 02:10:38.910362 4110 provision.go:117] generating server cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem org=jenkins.ha-857000 san=[127.0.0.1 192.169.0.5 ha-857000 localhost minikube]
I0917 02:10:38.962566 4110 provision.go:177] copyRemoteCerts
I0917 02:10:38.962618 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0917 02:10:38.962632 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:38.962737 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:38.962836 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:38.962932 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:38.963020 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/id_rsa Username:docker}
I0917 02:10:38.998776 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem -> /etc/docker/ca.pem
I0917 02:10:38.998851 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0917 02:10:39.018683 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem -> /etc/docker/server.pem
I0917 02:10:39.018741 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem --> /etc/docker/server.pem (1196 bytes)
I0917 02:10:39.038754 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem -> /etc/docker/server-key.pem
I0917 02:10:39.038814 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0917 02:10:39.058064 4110 provision.go:87] duration metric: took 149.217348ms to configureAuth
I0917 02:10:39.058076 4110 buildroot.go:189] setting minikube options for container-runtime
I0917 02:10:39.058257 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:10:39.058270 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:10:39.058416 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:39.058513 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:39.058598 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:39.058689 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:39.058780 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:39.058915 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:10:39.059035 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.5 22 <nil> <nil>}
I0917 02:10:39.059042 4110 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0917 02:10:39.117847 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs
I0917 02:10:39.117859 4110 buildroot.go:70] root file system type: tmpfs
I0917 02:10:39.117937 4110 provision.go:314] Updating docker unit: /lib/systemd/system/docker.service ...
I0917 02:10:39.117952 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:39.118078 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:39.118171 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:39.118258 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:39.118338 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:39.118469 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:10:39.118616 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.5 22 <nil> <nil>}
I0917 02:10:39.118663 4110 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0917 02:10:39.186097 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0917 02:10:39.186120 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:39.186247 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:39.186347 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:39.186426 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:39.186527 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:39.186659 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:10:39.186806 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.5 22 <nil> <nil>}
I0917 02:10:39.186817 4110 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0917 02:10:40.814202 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
I0917 02:10:40.814217 4110 machine.go:96] duration metric: took 13.109237782s to provisionDockerMachine
I0917 02:10:40.814229 4110 start.go:293] postStartSetup for "ha-857000" (driver="hyperkit")
I0917 02:10:40.814236 4110 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0917 02:10:40.814246 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:10:40.814438 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0917 02:10:40.814456 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:40.814571 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:40.814667 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:40.814762 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:40.814848 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/id_rsa Username:docker}
I0917 02:10:40.854204 4110 ssh_runner.go:195] Run: cat /etc/os-release
I0917 02:10:40.857656 4110 info.go:137] Remote host: Buildroot 2023.02.9
I0917 02:10:40.857668 4110 filesync.go:126] Scanning /Users/jenkins/minikube-integration/19648-1025/.minikube/addons for local assets ...
I0917 02:10:40.857773 4110 filesync.go:126] Scanning /Users/jenkins/minikube-integration/19648-1025/.minikube/files for local assets ...
I0917 02:10:40.857955 4110 filesync.go:149] local asset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> 15602.pem in /etc/ssl/certs
I0917 02:10:40.857962 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> /etc/ssl/certs/15602.pem
I0917 02:10:40.858166 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0917 02:10:40.867201 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem --> /etc/ssl/certs/15602.pem (1708 bytes)
I0917 02:10:40.895727 4110 start.go:296] duration metric: took 81.487995ms for postStartSetup
I0917 02:10:40.895754 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:10:40.895937 4110 ssh_runner.go:195] Run: sudo ls --almost-all -1 /var/lib/minikube/backup
I0917 02:10:40.895964 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:40.896062 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:40.896140 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:40.896211 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:40.896292 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/id_rsa Username:docker}
I0917 02:10:40.931812 4110 machine.go:197] restoring vm config from /var/lib/minikube/backup: [etc]
I0917 02:10:40.931872 4110 ssh_runner.go:195] Run: sudo rsync --archive --update /var/lib/minikube/backup/etc /
I0917 02:10:40.965671 4110 fix.go:56] duration metric: took 13.447980679s for fixHost
I0917 02:10:40.965693 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:40.965831 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:40.965924 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:40.966013 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:40.966122 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:40.966261 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:10:40.966403 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.5 22 <nil> <nil>}
I0917 02:10:40.966410 4110 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0917 02:10:41.023835 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: 1726564240.935930388
I0917 02:10:41.023847 4110 fix.go:216] guest clock: 1726564240.935930388
I0917 02:10:41.023853 4110 fix.go:229] Guest: 2024-09-17 02:10:40.935930388 -0700 PDT Remote: 2024-09-17 02:10:40.965683 -0700 PDT m=+13.896006994 (delta=-29.752612ms)
I0917 02:10:41.023870 4110 fix.go:200] guest clock delta is within tolerance: -29.752612ms
I0917 02:10:41.023873 4110 start.go:83] releasing machines lock for "ha-857000", held for 13.506240986s
I0917 02:10:41.023893 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:10:41.024017 4110 main.go:141] libmachine: (ha-857000) Calling .GetIP
I0917 02:10:41.024124 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:10:41.024416 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:10:41.024496 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:10:41.024577 4110 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0917 02:10:41.024607 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:41.024622 4110 ssh_runner.go:195] Run: cat /version.json
I0917 02:10:41.024633 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:10:41.024692 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:41.024731 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:10:41.024799 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:41.024812 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:10:41.024882 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:41.024908 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:10:41.025002 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/id_rsa Username:docker}
I0917 02:10:41.025031 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/id_rsa Username:docker}
I0917 02:10:41.057444 4110 ssh_runner.go:195] Run: systemctl --version
I0917 02:10:41.119261 4110 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0917 02:10:41.123760 4110 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0917 02:10:41.123809 4110 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0917 02:10:41.136297 4110 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0917 02:10:41.136307 4110 start.go:495] detecting cgroup driver to use...
I0917 02:10:41.136412 4110 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0917 02:10:41.153182 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0917 02:10:41.162387 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0917 02:10:41.171363 4110 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0917 02:10:41.171411 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0917 02:10:41.180339 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0917 02:10:41.189205 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0917 02:10:41.198331 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0917 02:10:41.207214 4110 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0917 02:10:41.216288 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0917 02:10:41.225185 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0917 02:10:41.234170 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0917 02:10:41.243192 4110 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0917 02:10:41.251363 4110 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0917 02:10:41.259648 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:10:41.359254 4110 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0917 02:10:41.378053 4110 start.go:495] detecting cgroup driver to use...
I0917 02:10:41.378144 4110 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0917 02:10:41.391608 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0917 02:10:41.406431 4110 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0917 02:10:41.426598 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0917 02:10:41.437654 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0917 02:10:41.448507 4110 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0917 02:10:41.470118 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0917 02:10:41.481632 4110 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/cri-dockerd.sock
" | sudo tee /etc/crictl.yaml"
I0917 02:10:41.496609 4110 ssh_runner.go:195] Run: which cri-dockerd
I0917 02:10:41.499690 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I0917 02:10:41.507723 4110 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (190 bytes)
I0917 02:10:41.520894 4110 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0917 02:10:41.633690 4110 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0917 02:10:41.735063 4110 docker.go:574] configuring docker to use "cgroupfs" as cgroup driver...
I0917 02:10:41.735129 4110 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)
I0917 02:10:41.749181 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:10:41.842846 4110 ssh_runner.go:195] Run: sudo systemctl restart docker
I0917 02:10:44.137188 4110 ssh_runner.go:235] Completed: sudo systemctl restart docker: (2.294283491s)
I0917 02:10:44.137256 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.socket
I0917 02:10:44.147554 4110 ssh_runner.go:195] Run: sudo systemctl stop cri-docker.socket
I0917 02:10:44.160480 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0917 02:10:44.170998 4110 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket
I0917 02:10:44.262329 4110 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket
I0917 02:10:44.355414 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:10:44.456404 4110 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket
I0917 02:10:44.470268 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0917 02:10:44.481488 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:10:44.585298 4110 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.service
I0917 02:10:44.651024 4110 start.go:542] Will wait 60s for socket path /var/run/cri-dockerd.sock
I0917 02:10:44.651127 4110 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock
I0917 02:10:44.655468 4110 start.go:563] Will wait 60s for crictl version
I0917 02:10:44.655523 4110 ssh_runner.go:195] Run: which crictl
I0917 02:10:44.660816 4110 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0917 02:10:44.685805 4110 start.go:579] Version: 0.1.0
RuntimeName: docker
RuntimeVersion: 27.2.1
RuntimeApiVersion: v1
I0917 02:10:44.685900 4110 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0917 02:10:44.701620 4110 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0917 02:10:44.762577 4110 out.go:235] * Preparing Kubernetes v1.31.1 on Docker 27.2.1 ...
I0917 02:10:44.762643 4110 main.go:141] libmachine: (ha-857000) Calling .GetIP
I0917 02:10:44.763055 4110 ssh_runner.go:195] Run: grep 192.169.0.1 host.minikube.internal$ /etc/hosts
I0917 02:10:44.767764 4110 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.169.0.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0917 02:10:44.778676 4110 kubeadm.go:883] updating cluster {Name:ha-857000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19646/minikube-v1.34.0-1726415472-19646-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726358845-19644@sha256:4c67a32a16c2d4f824f00267c172fd225757ca75441e363d925dc9583137f0b0 Memory:2200 CPUs:2 DiskSize:20000 Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.
1 ClusterName:ha-857000 Namespace:default APIServerHAVIP:192.169.0.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.169.0.5 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m02 IP:192.169.0.6 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m03 IP:192.169.0.7 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m04 IP:192.169.0.8 Port:0 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:false Worker:true}] Addons:map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:false efk:false f
reshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:false storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID
:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0917 02:10:44.778770 4110 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime docker
I0917 02:10:44.778845 4110 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0917 02:10:44.792490 4110 docker.go:685] Got preloaded images: -- stdout --
registry.k8s.io/kube-apiserver:v1.31.1
registry.k8s.io/kube-controller-manager:v1.31.1
registry.k8s.io/kube-scheduler:v1.31.1
registry.k8s.io/kube-proxy:v1.31.1
kindest/kindnetd:v20240813-c6f155d6
registry.k8s.io/coredns/coredns:v1.11.3
registry.k8s.io/etcd:3.5.15-0
registry.k8s.io/pause:3.10
ghcr.io/kube-vip/kube-vip:v0.8.0
gcr.io/k8s-minikube/storage-provisioner:v5
gcr.io/k8s-minikube/busybox:1.28
-- /stdout --
I0917 02:10:44.792502 4110 docker.go:615] Images already preloaded, skipping extraction
I0917 02:10:44.792587 4110 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0917 02:10:44.806122 4110 docker.go:685] Got preloaded images: -- stdout --
registry.k8s.io/kube-apiserver:v1.31.1
registry.k8s.io/kube-scheduler:v1.31.1
registry.k8s.io/kube-controller-manager:v1.31.1
registry.k8s.io/kube-proxy:v1.31.1
kindest/kindnetd:v20240813-c6f155d6
registry.k8s.io/coredns/coredns:v1.11.3
registry.k8s.io/etcd:3.5.15-0
registry.k8s.io/pause:3.10
ghcr.io/kube-vip/kube-vip:v0.8.0
gcr.io/k8s-minikube/storage-provisioner:v5
gcr.io/k8s-minikube/busybox:1.28
-- /stdout --
I0917 02:10:44.806141 4110 cache_images.go:84] Images are preloaded, skipping loading
I0917 02:10:44.806152 4110 kubeadm.go:934] updating node { 192.169.0.5 8443 v1.31.1 docker true true} ...
I0917 02:10:44.806226 4110 kubeadm.go:946] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=ha-857000 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.169.0.5
[Install]
config:
{KubernetesVersion:v1.31.1 ClusterName:ha-857000 Namespace:default APIServerHAVIP:192.169.0.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0917 02:10:44.806308 4110 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0917 02:10:44.838425 4110 cni.go:84] Creating CNI manager for ""
I0917 02:10:44.838438 4110 cni.go:136] multinode detected (4 nodes found), recommending kindnet
I0917 02:10:44.838451 4110 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0917 02:10:44.838467 4110 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.169.0.5 APIServerPort:8443 KubernetesVersion:v1.31.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:ha-857000 NodeName:ha-857000 DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.169.0.5"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.169.0.5 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manif
ests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///var/run/cri-dockerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0917 02:10:44.838548 4110 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.169.0.5
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///var/run/cri-dockerd.sock
name: "ha-857000"
kubeletExtraArgs:
node-ip: 192.169.0.5
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.169.0.5"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.31.1
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///var/run/cri-dockerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0917 02:10:44.838565 4110 kube-vip.go:115] generating kube-vip config ...
I0917 02:10:44.838624 4110 ssh_runner.go:195] Run: sudo sh -c "modprobe --all ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack"
I0917 02:10:44.852006 4110 kube-vip.go:167] auto-enabling control-plane load-balancing in kube-vip
I0917 02:10:44.852072 4110 kube-vip.go:137] kube-vip config:
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
name: kube-vip
namespace: kube-system
spec:
containers:
- args:
- manager
env:
- name: vip_arp
value: "true"
- name: port
value: "8443"
- name: vip_nodename
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: vip_interface
value: eth0
- name: vip_cidr
value: "32"
- name: dns_mode
value: first
- name: cp_enable
value: "true"
- name: cp_namespace
value: kube-system
- name: vip_leaderelection
value: "true"
- name: vip_leasename
value: plndr-cp-lock
- name: vip_leaseduration
value: "5"
- name: vip_renewdeadline
value: "3"
- name: vip_retryperiod
value: "1"
- name: address
value: 192.169.0.254
- name: prometheus_server
value: :2112
- name : lb_enable
value: "true"
- name: lb_port
value: "8443"
image: ghcr.io/kube-vip/kube-vip:v0.8.0
imagePullPolicy: IfNotPresent
name: kube-vip
resources: {}
securityContext:
capabilities:
add:
- NET_ADMIN
- NET_RAW
volumeMounts:
- mountPath: /etc/kubernetes/admin.conf
name: kubeconfig
hostAliases:
- hostnames:
- kubernetes
ip: 127.0.0.1
hostNetwork: true
volumes:
- hostPath:
path: "/etc/kubernetes/admin.conf"
name: kubeconfig
status: {}
I0917 02:10:44.852126 4110 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.1
I0917 02:10:44.861875 4110 binaries.go:44] Found k8s binaries, skipping transfer
I0917 02:10:44.861926 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube /etc/kubernetes/manifests
I0917 02:10:44.870065 4110 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (307 bytes)
I0917 02:10:44.883323 4110 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0917 02:10:44.896671 4110 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2148 bytes)
I0917 02:10:44.910190 4110 ssh_runner.go:362] scp memory --> /etc/kubernetes/manifests/kube-vip.yaml (1440 bytes)
I0917 02:10:44.923776 4110 ssh_runner.go:195] Run: grep 192.169.0.254 control-plane.minikube.internal$ /etc/hosts
I0917 02:10:44.926683 4110 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.169.0.254 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0917 02:10:44.936751 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:10:45.031050 4110 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0917 02:10:45.045803 4110 certs.go:68] Setting up /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000 for IP: 192.169.0.5
I0917 02:10:45.045815 4110 certs.go:194] generating shared ca certs ...
I0917 02:10:45.045826 4110 certs.go:226] acquiring lock for ca certs: {Name:mkbc37b68ace578bc430db43facbca466a5a1602 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0917 02:10:45.046013 4110 certs.go:235] skipping valid "minikubeCA" ca cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key
I0917 02:10:45.046090 4110 certs.go:235] skipping valid "proxyClientCA" ca cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key
I0917 02:10:45.046101 4110 certs.go:256] generating profile certs ...
I0917 02:10:45.046208 4110 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/client.key
I0917 02:10:45.046290 4110 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.key.34d356ea
I0917 02:10:45.046357 4110 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.key
I0917 02:10:45.046364 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt -> /var/lib/minikube/certs/ca.crt
I0917 02:10:45.046385 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key -> /var/lib/minikube/certs/ca.key
I0917 02:10:45.046406 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.crt -> /var/lib/minikube/certs/proxy-client-ca.crt
I0917 02:10:45.046424 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key -> /var/lib/minikube/certs/proxy-client-ca.key
I0917 02:10:45.046442 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.crt -> /var/lib/minikube/certs/apiserver.crt
I0917 02:10:45.046474 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.key -> /var/lib/minikube/certs/apiserver.key
I0917 02:10:45.046503 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.crt -> /var/lib/minikube/certs/proxy-client.crt
I0917 02:10:45.046520 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.key -> /var/lib/minikube/certs/proxy-client.key
I0917 02:10:45.046624 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem (1338 bytes)
W0917 02:10:45.046679 4110 certs.go:480] ignoring /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560_empty.pem, impossibly tiny 0 bytes
I0917 02:10:45.046688 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem (1679 bytes)
I0917 02:10:45.046749 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem (1078 bytes)
I0917 02:10:45.046790 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem (1123 bytes)
I0917 02:10:45.046829 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem (1675 bytes)
I0917 02:10:45.046908 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem (1708 bytes)
I0917 02:10:45.046945 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> /usr/share/ca-certificates/15602.pem
I0917 02:10:45.046966 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt -> /usr/share/ca-certificates/minikubeCA.pem
I0917 02:10:45.046984 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem -> /usr/share/ca-certificates/1560.pem
I0917 02:10:45.047483 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0917 02:10:45.080356 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0917 02:10:45.112920 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0917 02:10:45.138450 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0917 02:10:45.175252 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1440 bytes)
I0917 02:10:45.218044 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0917 02:10:45.251977 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0917 02:10:45.309085 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0917 02:10:45.353596 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem --> /usr/share/ca-certificates/15602.pem (1708 bytes)
I0917 02:10:45.384476 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0917 02:10:45.404778 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem --> /usr/share/ca-certificates/1560.pem (1338 bytes)
I0917 02:10:45.423525 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0917 02:10:45.437207 4110 ssh_runner.go:195] Run: openssl version
I0917 02:10:45.441704 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/15602.pem && ln -fs /usr/share/ca-certificates/15602.pem /etc/ssl/certs/15602.pem"
I0917 02:10:45.450346 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/15602.pem
I0917 02:10:45.453899 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 17 08:55 /usr/share/ca-certificates/15602.pem
I0917 02:10:45.453945 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/15602.pem
I0917 02:10:45.458361 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/15602.pem /etc/ssl/certs/3ec20f2e.0"
I0917 02:10:45.466854 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0917 02:10:45.475379 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0917 02:10:45.478924 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 17 08:38 /usr/share/ca-certificates/minikubeCA.pem
I0917 02:10:45.478963 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0917 02:10:45.483279 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0917 02:10:45.491638 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1560.pem && ln -fs /usr/share/ca-certificates/1560.pem /etc/ssl/certs/1560.pem"
I0917 02:10:45.500375 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1560.pem
I0917 02:10:45.504070 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 17 08:55 /usr/share/ca-certificates/1560.pem
I0917 02:10:45.504128 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1560.pem
I0917 02:10:45.508583 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1560.pem /etc/ssl/certs/51391683.0"
I0917 02:10:45.516977 4110 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0917 02:10:45.520582 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0917 02:10:45.524889 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0917 02:10:45.529282 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0917 02:10:45.533668 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0917 02:10:45.538022 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0917 02:10:45.542262 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0917 02:10:45.546447 4110 kubeadm.go:392] StartCluster: {Name:ha-857000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19646/minikube-v1.34.0-1726415472-19646-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726358845-19644@sha256:4c67a32a16c2d4f824f00267c172fd225757ca75441e363d925dc9583137f0b0 Memory:2200 CPUs:2 DiskSize:20000 Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 C
lusterName:ha-857000 Namespace:default APIServerHAVIP:192.169.0.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.169.0.5 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m02 IP:192.169.0.6 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m03 IP:192.169.0.7 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true} {Name:m04 IP:192.169.0.8 Port:0 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:false Worker:true}] Addons:map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:false efk:false fres
hpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:false storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:do
cker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0917 02:10:45.546579 4110 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0917 02:10:45.558935 4110 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0917 02:10:45.566714 4110 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0917 02:10:45.566724 4110 kubeadm.go:593] restartPrimaryControlPlane start ...
I0917 02:10:45.566760 4110 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0917 02:10:45.574257 4110 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0917 02:10:45.574553 4110 kubeconfig.go:47] verify endpoint returned: get endpoint: "ha-857000" does not appear in /Users/jenkins/minikube-integration/19648-1025/kubeconfig
I0917 02:10:45.574638 4110 kubeconfig.go:62] /Users/jenkins/minikube-integration/19648-1025/kubeconfig needs updating (will repair): [kubeconfig missing "ha-857000" cluster setting kubeconfig missing "ha-857000" context setting]
I0917 02:10:45.574818 4110 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/19648-1025/kubeconfig: {Name:mk89516924578dcccd5bbd950d94f0ba54499729 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0917 02:10:45.575437 4110 loader.go:395] Config loaded from file: /Users/jenkins/minikube-integration/19648-1025/kubeconfig
I0917 02:10:45.575640 4110 kapi.go:59] client config for ha-857000: &rest.Config{Host:"https://192.169.0.5:8443", APIPath:"", ContentConfig:rest.ContentConfig{AcceptContentTypes:"", ContentType:"", GroupVersion:(*schema.GroupVersion)(nil), NegotiatedSerializer:runtime.NegotiatedSerializer(nil)}, Username:"", Password:"", BearerToken:"", BearerTokenFile:"", Impersonate:rest.ImpersonationConfig{UserName:"", UID:"", Groups:[]string(nil), Extra:map[string][]string(nil)}, AuthProvider:<nil>, AuthConfigPersister:rest.AuthProviderConfigPersister(nil), ExecProvider:<nil>, TLSClientConfig:rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/client.crt", KeyFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/client.key", CAFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt", CertData:[]uint8(nil), KeyData:[]uint8(nil), CAData:[]uint8(nil), NextProtos:[]string(nil)},
UserAgent:"", DisableCompression:false, Transport:http.RoundTripper(nil), WrapTransport:(transport.WrapperFunc)(0x673a720), QPS:0, Burst:0, RateLimiter:flowcontrol.RateLimiter(nil), WarningHandler:rest.WarningHandler(nil), Timeout:0, Dial:(func(context.Context, string, string) (net.Conn, error))(nil), Proxy:(func(*http.Request) (*url.URL, error))(nil)}
I0917 02:10:45.575954 4110 cert_rotation.go:140] Starting client certificate rotation controller
I0917 02:10:45.576155 4110 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0917 02:10:45.583535 4110 kubeadm.go:630] The running cluster does not require reconfiguration: 192.169.0.5
I0917 02:10:45.583548 4110 kubeadm.go:597] duration metric: took 16.820219ms to restartPrimaryControlPlane
I0917 02:10:45.583553 4110 kubeadm.go:394] duration metric: took 37.114772ms to StartCluster
I0917 02:10:45.583562 4110 settings.go:142] acquiring lock: {Name:mk5de70c670a23cf49fdbd19ddb5c1d2a9de9a04 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0917 02:10:45.583637 4110 settings.go:150] Updating kubeconfig: /Users/jenkins/minikube-integration/19648-1025/kubeconfig
I0917 02:10:45.584029 4110 lock.go:35] WriteFile acquiring /Users/jenkins/minikube-integration/19648-1025/kubeconfig: {Name:mk89516924578dcccd5bbd950d94f0ba54499729 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0917 02:10:45.584244 4110 start.go:233] HA (multi-control plane) cluster: will skip waiting for primary control-plane node &{Name: IP:192.169.0.5 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true}
I0917 02:10:45.584257 4110 start.go:241] waiting for startup goroutines ...
I0917 02:10:45.584290 4110 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:false efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:false storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0917 02:10:45.584399 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:10:45.629290 4110 out.go:177] * Enabled addons:
I0917 02:10:45.650483 4110 addons.go:510] duration metric: took 66.114939ms for enable addons: enabled=[]
I0917 02:10:45.650526 4110 start.go:246] waiting for cluster config update ...
I0917 02:10:45.650541 4110 start.go:255] writing updated cluster config ...
I0917 02:10:45.672110 4110 out.go:201]
I0917 02:10:45.693671 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:10:45.693812 4110 profile.go:143] Saving config to /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/config.json ...
I0917 02:10:45.716376 4110 out.go:177] * Starting "ha-857000-m02" control-plane node in "ha-857000" cluster
I0917 02:10:45.758138 4110 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime docker
I0917 02:10:45.758205 4110 cache.go:56] Caching tarball of preloaded images
I0917 02:10:45.758422 4110 preload.go:172] Found /Users/jenkins/minikube-integration/19648-1025/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0917 02:10:45.758440 4110 cache.go:59] Finished verifying existence of preloaded tar for v1.31.1 on docker
I0917 02:10:45.758566 4110 profile.go:143] Saving config to /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/config.json ...
I0917 02:10:45.759523 4110 start.go:360] acquireMachinesLock for ha-857000-m02: {Name:mkcb27accfc361dc9c175eeb8bb69dc9967aa557 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0917 02:10:45.759643 4110 start.go:364] duration metric: took 94.526µs to acquireMachinesLock for "ha-857000-m02"
I0917 02:10:45.759684 4110 start.go:96] Skipping create...Using existing machine configuration
I0917 02:10:45.759694 4110 fix.go:54] fixHost starting: m02
I0917 02:10:45.760135 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:10:45.760170 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:10:45.769422 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52114
I0917 02:10:45.769778 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:10:45.770120 4110 main.go:141] libmachine: Using API Version 1
I0917 02:10:45.770130 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:10:45.770332 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:10:45.770446 4110 main.go:141] libmachine: (ha-857000-m02) Calling .DriverName
I0917 02:10:45.770540 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetState
I0917 02:10:45.770620 4110 main.go:141] libmachine: (ha-857000-m02) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:10:45.770696 4110 main.go:141] libmachine: (ha-857000-m02) DBG | hyperkit pid from json: 3976
I0917 02:10:45.771617 4110 main.go:141] libmachine: (ha-857000-m02) DBG | hyperkit pid 3976 missing from process table
I0917 02:10:45.771641 4110 fix.go:112] recreateIfNeeded on ha-857000-m02: state=Stopped err=<nil>
I0917 02:10:45.771648 4110 main.go:141] libmachine: (ha-857000-m02) Calling .DriverName
W0917 02:10:45.771734 4110 fix.go:138] unexpected machine state, will restart: <nil>
I0917 02:10:45.793214 4110 out.go:177] * Restarting existing hyperkit VM for "ha-857000-m02" ...
I0917 02:10:45.835194 4110 main.go:141] libmachine: (ha-857000-m02) Calling .Start
I0917 02:10:45.835422 4110 main.go:141] libmachine: (ha-857000-m02) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:10:45.835478 4110 main.go:141] libmachine: (ha-857000-m02) minikube might have been shutdown in an unclean way, the hyperkit pid file still exists: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/hyperkit.pid
I0917 02:10:45.836481 4110 main.go:141] libmachine: (ha-857000-m02) DBG | hyperkit pid 3976 missing from process table
I0917 02:10:45.836493 4110 main.go:141] libmachine: (ha-857000-m02) DBG | pid 3976 is in state "Stopped"
I0917 02:10:45.836506 4110 main.go:141] libmachine: (ha-857000-m02) DBG | Removing stale pid file /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/hyperkit.pid...
I0917 02:10:45.836730 4110 main.go:141] libmachine: (ha-857000-m02) DBG | Using UUID 1940a045-0b1b-4b28-842d-d4858a62cbd3
I0917 02:10:45.862461 4110 main.go:141] libmachine: (ha-857000-m02) DBG | Generated MAC 9a:95:4e:4b:65:fe
I0917 02:10:45.862487 4110 main.go:141] libmachine: (ha-857000-m02) DBG | Starting with cmdline: loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000
I0917 02:10:45.862599 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 DEBUG: hyperkit: Start &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"1940a045-0b1b-4b28-842d-d4858a62cbd3", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc0003beae0)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/bzimage", Initrd:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os.Process)(nil)}
I0917 02:10:45.862645 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 DEBUG: hyperkit: check &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"1940a045-0b1b-4b28-842d-d4858a62cbd3", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc0003beae0)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/bzimage", Initrd:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os.Process)(nil)}
I0917 02:10:45.862683 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 DEBUG: hyperkit: Arguments: []string{"-A", "-u", "-F", "/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/hyperkit.pid", "-c", "2", "-m", "2200M", "-s", "0:0,hostbridge", "-s", "31,lpc", "-s", "1:0,virtio-net", "-U", "1940a045-0b1b-4b28-842d-d4858a62cbd3", "-s", "2:0,virtio-blk,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/ha-857000-m02.rawdisk", "-s", "3,ahci-cd,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/boot2docker.iso", "-s", "4,virtio-rnd", "-l", "com1,autopty=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/tty,log=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/console-ring", "-f", "kexec,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/bzimage,/Users/jenkins/minikube-integration/19648-1025/.minikube/machine
s/ha-857000-m02/initrd,earlyprintk=serial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000"}
I0917 02:10:45.862720 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 DEBUG: hyperkit: CmdLine: "/usr/local/bin/hyperkit -A -u -F /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/hyperkit.pid -c 2 -m 2200M -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-net -U 1940a045-0b1b-4b28-842d-d4858a62cbd3 -s 2:0,virtio-blk,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/ha-857000-m02.rawdisk -s 3,ahci-cd,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/boot2docker.iso -s 4,virtio-rnd -l com1,autopty=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/tty,log=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/console-ring -f kexec,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/bzimage,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/initrd,earlyprintk=serial loglevel=3 console=t
tyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000"
I0917 02:10:45.862741 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 DEBUG: hyperkit: Redirecting stdout/stderr to logger
I0917 02:10:45.864138 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 DEBUG: hyperkit: Pid is 4131
I0917 02:10:45.864563 4110 main.go:141] libmachine: (ha-857000-m02) DBG | Attempt 0
I0917 02:10:45.864573 4110 main.go:141] libmachine: (ha-857000-m02) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:10:45.864635 4110 main.go:141] libmachine: (ha-857000-m02) DBG | hyperkit pid from json: 4131
I0917 02:10:45.866426 4110 main.go:141] libmachine: (ha-857000-m02) DBG | Searching for 9a:95:4e:4b:65:fe in /var/db/dhcpd_leases ...
I0917 02:10:45.866511 4110 main.go:141] libmachine: (ha-857000-m02) DBG | Found 7 entries in /var/db/dhcpd_leases!
I0917 02:10:45.866527 4110 main.go:141] libmachine: (ha-857000-m02) DBG | dhcp entry: {Name:minikube IPAddress:192.169.0.5 HWAddress:c2:63:2b:63:80:76 ID:1,c2:63:2b:63:80:76 Lease:0x66ea990c}
I0917 02:10:45.866546 4110 main.go:141] libmachine: (ha-857000-m02) DBG | dhcp entry: {Name:minikube IPAddress:192.169.0.6 HWAddress:9a:95:4e:4b:65:fe ID:1,9a:95:4e:4b:65:fe Lease:0x66ea9817}
I0917 02:10:45.866556 4110 main.go:141] libmachine: (ha-857000-m02) DBG | Found match: 9a:95:4e:4b:65:fe
I0917 02:10:45.866585 4110 main.go:141] libmachine: (ha-857000-m02) DBG | IP: 192.169.0.6
I0917 02:10:45.866617 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetConfigRaw
I0917 02:10:45.867379 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetIP
I0917 02:10:45.867624 4110 profile.go:143] Saving config to /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/config.json ...
I0917 02:10:45.868172 4110 machine.go:93] provisionDockerMachine start ...
I0917 02:10:45.868192 4110 main.go:141] libmachine: (ha-857000-m02) Calling .DriverName
I0917 02:10:45.868319 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:10:45.868433 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:10:45.868540 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:10:45.868629 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:10:45.868743 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:10:45.868892 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:10:45.869038 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.6 22 <nil> <nil>}
I0917 02:10:45.869047 4110 main.go:141] libmachine: About to run SSH command:
hostname
I0917 02:10:45.871979 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 INFO : hyperkit: stderr: Using fd 5 for I/O notifications
I0917 02:10:45.880237 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 INFO : hyperkit: stderr: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/boot2docker.iso: fcntl(F_PUNCHHOLE) Operation not permitted: block device will not support TRIM/DISCARD
I0917 02:10:45.881261 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0
I0917 02:10:45.881280 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0
I0917 02:10:45.881317 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0
I0917 02:10:45.881331 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:45 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0
I0917 02:10:46.263104 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:46 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 0
I0917 02:10:46.263119 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:46 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 0
I0917 02:10:46.377844 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:46 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0
I0917 02:10:46.377864 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:46 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0
I0917 02:10:46.377874 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:46 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0
I0917 02:10:46.377890 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:46 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0
I0917 02:10:46.378727 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:46 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 1
I0917 02:10:46.378736 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:46 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 1
I0917 02:10:51.977750 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:51 INFO : hyperkit: stderr: rdmsr to register 0x64d on vcpu 0
I0917 02:10:51.977833 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:51 INFO : hyperkit: stderr: rdmsr to register 0x64e on vcpu 0
I0917 02:10:51.977841 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:51 INFO : hyperkit: stderr: rdmsr to register 0x34 on vcpu 0
I0917 02:10:52.002295 4110 main.go:141] libmachine: (ha-857000-m02) DBG | 2024/09/17 02:10:52 INFO : hyperkit: stderr: rdmsr to register 0xc0011029 on vcpu 0
I0917 02:11:20.931384 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: minikube
I0917 02:11:20.931398 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetMachineName
I0917 02:11:20.931549 4110 buildroot.go:166] provisioning hostname "ha-857000-m02"
I0917 02:11:20.931560 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetMachineName
I0917 02:11:20.931664 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:20.931762 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:11:20.931855 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:20.931937 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:20.932033 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:11:20.932169 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:20.932351 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.6 22 <nil> <nil>}
I0917 02:11:20.932359 4110 main.go:141] libmachine: About to run SSH command:
sudo hostname ha-857000-m02 && echo "ha-857000-m02" | sudo tee /etc/hostname
I0917 02:11:20.993183 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: ha-857000-m02
I0917 02:11:20.993198 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:20.993326 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:11:20.993440 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:20.993526 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:20.993618 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:11:20.993763 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:20.993914 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.6 22 <nil> <nil>}
I0917 02:11:20.993925 4110 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sha-857000-m02' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 ha-857000-m02/g' /etc/hosts;
else
echo '127.0.1.1 ha-857000-m02' | sudo tee -a /etc/hosts;
fi
fi
I0917 02:11:21.050925 4110 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0917 02:11:21.050951 4110 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/19648-1025/.minikube CaCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/19648-1025/.minikube}
I0917 02:11:21.050960 4110 buildroot.go:174] setting up certificates
I0917 02:11:21.050966 4110 provision.go:84] configureAuth start
I0917 02:11:21.050972 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetMachineName
I0917 02:11:21.051109 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetIP
I0917 02:11:21.051192 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:21.051304 4110 provision.go:143] copyHostCerts
I0917 02:11:21.051330 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem
I0917 02:11:21.051388 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem, removing ...
I0917 02:11:21.051394 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem
I0917 02:11:21.051551 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem (1078 bytes)
I0917 02:11:21.051732 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem
I0917 02:11:21.051778 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem, removing ...
I0917 02:11:21.051784 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem
I0917 02:11:21.051862 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem (1123 bytes)
I0917 02:11:21.051999 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem
I0917 02:11:21.052037 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem, removing ...
I0917 02:11:21.052041 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem
I0917 02:11:21.052127 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem (1675 bytes)
I0917 02:11:21.052261 4110 provision.go:117] generating server cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem org=jenkins.ha-857000-m02 san=[127.0.0.1 192.169.0.6 ha-857000-m02 localhost minikube]
I0917 02:11:21.131473 4110 provision.go:177] copyRemoteCerts
I0917 02:11:21.131534 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0917 02:11:21.131551 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:21.131683 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:11:21.131772 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:21.131866 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:11:21.131988 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.6 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/id_rsa Username:docker}
I0917 02:11:21.165457 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem -> /etc/docker/ca.pem
I0917 02:11:21.165530 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0917 02:11:21.185353 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem -> /etc/docker/server.pem
I0917 02:11:21.185424 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes)
I0917 02:11:21.204885 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem -> /etc/docker/server-key.pem
I0917 02:11:21.204944 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0917 02:11:21.224555 4110 provision.go:87] duration metric: took 173.578725ms to configureAuth
I0917 02:11:21.224572 4110 buildroot.go:189] setting minikube options for container-runtime
I0917 02:11:21.224752 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:11:21.224765 4110 main.go:141] libmachine: (ha-857000-m02) Calling .DriverName
I0917 02:11:21.224898 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:21.224985 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:11:21.225071 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:21.225151 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:21.225226 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:11:21.225334 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:21.225453 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.6 22 <nil> <nil>}
I0917 02:11:21.225471 4110 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0917 02:11:21.276594 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs
I0917 02:11:21.276610 4110 buildroot.go:70] root file system type: tmpfs
I0917 02:11:21.276682 4110 provision.go:314] Updating docker unit: /lib/systemd/system/docker.service ...
I0917 02:11:21.276692 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:21.276824 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:11:21.276911 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:21.276982 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:21.277068 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:11:21.277206 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:21.277343 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.6 22 <nil> <nil>}
I0917 02:11:21.277390 4110 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
Environment="NO_PROXY=192.169.0.5"
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0917 02:11:21.338440 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
Environment=NO_PROXY=192.169.0.5
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0917 02:11:21.338457 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:21.338602 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:11:21.338693 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:21.338786 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:21.338878 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:11:21.339018 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:21.339165 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.6 22 <nil> <nil>}
I0917 02:11:21.339180 4110 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0917 02:11:23.000541 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
I0917 02:11:23.000557 4110 machine.go:96] duration metric: took 37.131734761s to provisionDockerMachine
I0917 02:11:23.000565 4110 start.go:293] postStartSetup for "ha-857000-m02" (driver="hyperkit")
I0917 02:11:23.000572 4110 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0917 02:11:23.000581 4110 main.go:141] libmachine: (ha-857000-m02) Calling .DriverName
I0917 02:11:23.000771 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0917 02:11:23.000784 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:23.000877 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:11:23.000970 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:23.001060 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:11:23.001151 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.6 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/id_rsa Username:docker}
I0917 02:11:23.034070 4110 ssh_runner.go:195] Run: cat /etc/os-release
I0917 02:11:23.037044 4110 info.go:137] Remote host: Buildroot 2023.02.9
I0917 02:11:23.037054 4110 filesync.go:126] Scanning /Users/jenkins/minikube-integration/19648-1025/.minikube/addons for local assets ...
I0917 02:11:23.037149 4110 filesync.go:126] Scanning /Users/jenkins/minikube-integration/19648-1025/.minikube/files for local assets ...
I0917 02:11:23.037326 4110 filesync.go:149] local asset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> 15602.pem in /etc/ssl/certs
I0917 02:11:23.037333 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> /etc/ssl/certs/15602.pem
I0917 02:11:23.037542 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0917 02:11:23.045540 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem --> /etc/ssl/certs/15602.pem (1708 bytes)
I0917 02:11:23.064134 4110 start.go:296] duration metric: took 63.560241ms for postStartSetup
I0917 02:11:23.064153 4110 main.go:141] libmachine: (ha-857000-m02) Calling .DriverName
I0917 02:11:23.064355 4110 ssh_runner.go:195] Run: sudo ls --almost-all -1 /var/lib/minikube/backup
I0917 02:11:23.064367 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:23.064443 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:11:23.064537 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:23.064625 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:11:23.064699 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.6 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/id_rsa Username:docker}
I0917 02:11:23.096648 4110 machine.go:197] restoring vm config from /var/lib/minikube/backup: [etc]
I0917 02:11:23.096719 4110 ssh_runner.go:195] Run: sudo rsync --archive --update /var/lib/minikube/backup/etc /
I0917 02:11:23.150750 4110 fix.go:56] duration metric: took 37.39040777s for fixHost
I0917 02:11:23.150781 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:23.150933 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:11:23.151043 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:23.151139 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:23.151225 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:11:23.151344 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:23.151480 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.6 22 <nil> <nil>}
I0917 02:11:23.151487 4110 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0917 02:11:23.205108 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: 1726564282.931256187
I0917 02:11:23.205121 4110 fix.go:216] guest clock: 1726564282.931256187
I0917 02:11:23.205126 4110 fix.go:229] Guest: 2024-09-17 02:11:22.931256187 -0700 PDT Remote: 2024-09-17 02:11:23.150765 -0700 PDT m=+56.080359699 (delta=-219.508813ms)
I0917 02:11:23.205134 4110 fix.go:200] guest clock delta is within tolerance: -219.508813ms
I0917 02:11:23.205138 4110 start.go:83] releasing machines lock for "ha-857000-m02", held for 37.444836088s
I0917 02:11:23.205153 4110 main.go:141] libmachine: (ha-857000-m02) Calling .DriverName
I0917 02:11:23.205283 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetIP
I0917 02:11:23.226836 4110 out.go:177] * Found network options:
I0917 02:11:23.247780 4110 out.go:177] - NO_PROXY=192.169.0.5
W0917 02:11:23.268466 4110 proxy.go:119] fail to check proxy env: Error ip not in block
I0917 02:11:23.268508 4110 main.go:141] libmachine: (ha-857000-m02) Calling .DriverName
I0917 02:11:23.269341 4110 main.go:141] libmachine: (ha-857000-m02) Calling .DriverName
I0917 02:11:23.269597 4110 main.go:141] libmachine: (ha-857000-m02) Calling .DriverName
I0917 02:11:23.269778 4110 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
W0917 02:11:23.269794 4110 proxy.go:119] fail to check proxy env: Error ip not in block
I0917 02:11:23.269828 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:23.269896 4110 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0917 02:11:23.269915 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHHostname
I0917 02:11:23.270129 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:11:23.270153 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHPort
I0917 02:11:23.270351 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:23.270407 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHKeyPath
I0917 02:11:23.270526 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:11:23.270571 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetSSHUsername
I0917 02:11:23.270741 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.6 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/id_rsa Username:docker}
I0917 02:11:23.270760 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.6 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m02/id_rsa Username:docker}
W0917 02:11:23.355936 4110 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0917 02:11:23.356046 4110 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0917 02:11:23.371785 4110 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0917 02:11:23.371805 4110 start.go:495] detecting cgroup driver to use...
I0917 02:11:23.371897 4110 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0917 02:11:23.389343 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0917 02:11:23.397507 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0917 02:11:23.405706 4110 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0917 02:11:23.405760 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0917 02:11:23.413954 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0917 02:11:23.422064 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0917 02:11:23.430077 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0917 02:11:23.438247 4110 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0917 02:11:23.446615 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0917 02:11:23.455025 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0917 02:11:23.463904 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0917 02:11:23.472877 4110 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0917 02:11:23.480886 4110 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0917 02:11:23.488979 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:11:23.586431 4110 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0917 02:11:23.605512 4110 start.go:495] detecting cgroup driver to use...
I0917 02:11:23.605590 4110 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0917 02:11:23.619031 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0917 02:11:23.632481 4110 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0917 02:11:23.650301 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0917 02:11:23.661034 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0917 02:11:23.671499 4110 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0917 02:11:23.693809 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0917 02:11:23.704324 4110 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/cri-dockerd.sock
" | sudo tee /etc/crictl.yaml"
I0917 02:11:23.719425 4110 ssh_runner.go:195] Run: which cri-dockerd
I0917 02:11:23.722279 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I0917 02:11:23.729409 4110 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (190 bytes)
I0917 02:11:23.743121 4110 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0917 02:11:23.848749 4110 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0917 02:11:23.947630 4110 docker.go:574] configuring docker to use "cgroupfs" as cgroup driver...
I0917 02:11:23.947661 4110 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)
I0917 02:11:23.965207 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:11:24.060164 4110 ssh_runner.go:195] Run: sudo systemctl restart docker
I0917 02:11:26.333778 4110 ssh_runner.go:235] Completed: sudo systemctl restart docker: (2.273556023s)
I0917 02:11:26.333847 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.socket
I0917 02:11:26.345198 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0917 02:11:26.355965 4110 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket
I0917 02:11:26.461793 4110 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket
I0917 02:11:26.556361 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:11:26.674366 4110 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket
I0917 02:11:26.687753 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0917 02:11:26.697698 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:11:26.797118 4110 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.service
I0917 02:11:26.861306 4110 start.go:542] Will wait 60s for socket path /var/run/cri-dockerd.sock
I0917 02:11:26.861392 4110 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock
I0917 02:11:26.865857 4110 start.go:563] Will wait 60s for crictl version
I0917 02:11:26.865915 4110 ssh_runner.go:195] Run: which crictl
I0917 02:11:26.869732 4110 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0917 02:11:26.894886 4110 start.go:579] Version: 0.1.0
RuntimeName: docker
RuntimeVersion: 27.2.1
RuntimeApiVersion: v1
I0917 02:11:26.894999 4110 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0917 02:11:26.911893 4110 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0917 02:11:26.950833 4110 out.go:235] * Preparing Kubernetes v1.31.1 on Docker 27.2.1 ...
I0917 02:11:26.972458 4110 out.go:177] - env NO_PROXY=192.169.0.5
I0917 02:11:26.993284 4110 main.go:141] libmachine: (ha-857000-m02) Calling .GetIP
I0917 02:11:26.993711 4110 ssh_runner.go:195] Run: grep 192.169.0.1 host.minikube.internal$ /etc/hosts
I0917 02:11:26.998329 4110 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.169.0.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0917 02:11:27.008512 4110 mustload.go:65] Loading cluster: ha-857000
I0917 02:11:27.008684 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:11:27.008920 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:11:27.008943 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:11:27.017607 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52136
I0917 02:11:27.017941 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:11:27.018292 4110 main.go:141] libmachine: Using API Version 1
I0917 02:11:27.018310 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:11:27.018503 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:11:27.018620 4110 main.go:141] libmachine: (ha-857000) Calling .GetState
I0917 02:11:27.018699 4110 main.go:141] libmachine: (ha-857000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:11:27.018771 4110 main.go:141] libmachine: (ha-857000) DBG | hyperkit pid from json: 4124
I0917 02:11:27.019715 4110 host.go:66] Checking if "ha-857000" exists ...
I0917 02:11:27.019989 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:11:27.020015 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:11:27.028562 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52138
I0917 02:11:27.028902 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:11:27.029241 4110 main.go:141] libmachine: Using API Version 1
I0917 02:11:27.029257 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:11:27.029461 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:11:27.029566 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:11:27.029665 4110 certs.go:68] Setting up /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000 for IP: 192.169.0.6
I0917 02:11:27.029672 4110 certs.go:194] generating shared ca certs ...
I0917 02:11:27.029680 4110 certs.go:226] acquiring lock for ca certs: {Name:mkbc37b68ace578bc430db43facbca466a5a1602 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0917 02:11:27.029857 4110 certs.go:235] skipping valid "minikubeCA" ca cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key
I0917 02:11:27.029930 4110 certs.go:235] skipping valid "proxyClientCA" ca cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key
I0917 02:11:27.029938 4110 certs.go:256] generating profile certs ...
I0917 02:11:27.030058 4110 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/client.key
I0917 02:11:27.030140 4110 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.key.d3e75930
I0917 02:11:27.030214 4110 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.key
I0917 02:11:27.030221 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt -> /var/lib/minikube/certs/ca.crt
I0917 02:11:27.030242 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key -> /var/lib/minikube/certs/ca.key
I0917 02:11:27.030266 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.crt -> /var/lib/minikube/certs/proxy-client-ca.crt
I0917 02:11:27.030285 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key -> /var/lib/minikube/certs/proxy-client-ca.key
I0917 02:11:27.030303 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.crt -> /var/lib/minikube/certs/apiserver.crt
I0917 02:11:27.030337 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.key -> /var/lib/minikube/certs/apiserver.key
I0917 02:11:27.030366 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.crt -> /var/lib/minikube/certs/proxy-client.crt
I0917 02:11:27.030389 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.key -> /var/lib/minikube/certs/proxy-client.key
I0917 02:11:27.030486 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem (1338 bytes)
W0917 02:11:27.030540 4110 certs.go:480] ignoring /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560_empty.pem, impossibly tiny 0 bytes
I0917 02:11:27.030549 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem (1679 bytes)
I0917 02:11:27.030587 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem (1078 bytes)
I0917 02:11:27.030621 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem (1123 bytes)
I0917 02:11:27.030651 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem (1675 bytes)
I0917 02:11:27.030716 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem (1708 bytes)
I0917 02:11:27.030753 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem -> /usr/share/ca-certificates/1560.pem
I0917 02:11:27.030774 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> /usr/share/ca-certificates/15602.pem
I0917 02:11:27.030792 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt -> /usr/share/ca-certificates/minikubeCA.pem
I0917 02:11:27.030816 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:11:27.030911 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:11:27.031000 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:11:27.031078 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:11:27.031162 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/id_rsa Username:docker}
I0917 02:11:27.058778 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/sa.pub
I0917 02:11:27.062313 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/sa.pub --> memory (451 bytes)
I0917 02:11:27.070939 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/sa.key
I0917 02:11:27.074280 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/sa.key --> memory (1675 bytes)
I0917 02:11:27.083003 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/front-proxy-ca.crt
I0917 02:11:27.086057 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/front-proxy-ca.crt --> memory (1123 bytes)
I0917 02:11:27.094554 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/front-proxy-ca.key
I0917 02:11:27.097659 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/front-proxy-ca.key --> memory (1679 bytes)
I0917 02:11:27.106657 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/etcd/ca.crt
I0917 02:11:27.109894 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/etcd/ca.crt --> memory (1094 bytes)
I0917 02:11:27.118370 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/etcd/ca.key
I0917 02:11:27.121478 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/etcd/ca.key --> memory (1679 bytes)
I0917 02:11:27.130386 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0917 02:11:27.150256 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0917 02:11:27.169526 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0917 02:11:27.188769 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0917 02:11:27.207966 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1440 bytes)
I0917 02:11:27.227067 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0917 02:11:27.246289 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0917 02:11:27.265271 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0917 02:11:27.284669 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem --> /usr/share/ca-certificates/1560.pem (1338 bytes)
I0917 02:11:27.303761 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem --> /usr/share/ca-certificates/15602.pem (1708 bytes)
I0917 02:11:27.323113 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0917 02:11:27.342331 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/sa.pub (451 bytes)
I0917 02:11:27.355765 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/sa.key (1675 bytes)
I0917 02:11:27.369277 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/front-proxy-ca.crt (1123 bytes)
I0917 02:11:27.382837 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/front-proxy-ca.key (1679 bytes)
I0917 02:11:27.396474 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/etcd/ca.crt (1094 bytes)
I0917 02:11:27.410313 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/etcd/ca.key (1679 bytes)
I0917 02:11:27.423731 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (744 bytes)
I0917 02:11:27.437366 4110 ssh_runner.go:195] Run: openssl version
I0917 02:11:27.441447 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1560.pem && ln -fs /usr/share/ca-certificates/1560.pem /etc/ssl/certs/1560.pem"
I0917 02:11:27.450619 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1560.pem
I0917 02:11:27.453941 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 17 08:55 /usr/share/ca-certificates/1560.pem
I0917 02:11:27.453997 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1560.pem
I0917 02:11:27.458171 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1560.pem /etc/ssl/certs/51391683.0"
I0917 02:11:27.467199 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/15602.pem && ln -fs /usr/share/ca-certificates/15602.pem /etc/ssl/certs/15602.pem"
I0917 02:11:27.476144 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/15602.pem
I0917 02:11:27.479431 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 17 08:55 /usr/share/ca-certificates/15602.pem
I0917 02:11:27.479473 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/15602.pem
I0917 02:11:27.483603 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/15602.pem /etc/ssl/certs/3ec20f2e.0"
I0917 02:11:27.492580 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0917 02:11:27.501517 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0917 02:11:27.504871 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 17 08:38 /usr/share/ca-certificates/minikubeCA.pem
I0917 02:11:27.504915 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0917 02:11:27.509027 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0917 02:11:27.517892 4110 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0917 02:11:27.521155 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0917 02:11:27.525378 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0917 02:11:27.529633 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0917 02:11:27.533810 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0917 02:11:27.538003 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0917 02:11:27.542137 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0917 02:11:27.546288 4110 kubeadm.go:934] updating node {m02 192.169.0.6 8443 v1.31.1 docker true true} ...
I0917 02:11:27.546336 4110 kubeadm.go:946] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=ha-857000-m02 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.169.0.6
[Install]
config:
{KubernetesVersion:v1.31.1 ClusterName:ha-857000 Namespace:default APIServerHAVIP:192.169.0.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0917 02:11:27.546350 4110 kube-vip.go:115] generating kube-vip config ...
I0917 02:11:27.546384 4110 ssh_runner.go:195] Run: sudo sh -c "modprobe --all ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack"
I0917 02:11:27.558948 4110 kube-vip.go:167] auto-enabling control-plane load-balancing in kube-vip
I0917 02:11:27.558990 4110 kube-vip.go:137] kube-vip config:
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
name: kube-vip
namespace: kube-system
spec:
containers:
- args:
- manager
env:
- name: vip_arp
value: "true"
- name: port
value: "8443"
- name: vip_nodename
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: vip_interface
value: eth0
- name: vip_cidr
value: "32"
- name: dns_mode
value: first
- name: cp_enable
value: "true"
- name: cp_namespace
value: kube-system
- name: vip_leaderelection
value: "true"
- name: vip_leasename
value: plndr-cp-lock
- name: vip_leaseduration
value: "5"
- name: vip_renewdeadline
value: "3"
- name: vip_retryperiod
value: "1"
- name: address
value: 192.169.0.254
- name: prometheus_server
value: :2112
- name : lb_enable
value: "true"
- name: lb_port
value: "8443"
image: ghcr.io/kube-vip/kube-vip:v0.8.0
imagePullPolicy: IfNotPresent
name: kube-vip
resources: {}
securityContext:
capabilities:
add:
- NET_ADMIN
- NET_RAW
volumeMounts:
- mountPath: /etc/kubernetes/admin.conf
name: kubeconfig
hostAliases:
- hostnames:
- kubernetes
ip: 127.0.0.1
hostNetwork: true
volumes:
- hostPath:
path: "/etc/kubernetes/admin.conf"
name: kubeconfig
status: {}
I0917 02:11:27.559048 4110 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.1
I0917 02:11:27.568292 4110 binaries.go:44] Found k8s binaries, skipping transfer
I0917 02:11:27.568351 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /etc/kubernetes/manifests
I0917 02:11:27.577686 4110 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (311 bytes)
I0917 02:11:27.591394 4110 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0917 02:11:27.604835 4110 ssh_runner.go:362] scp memory --> /etc/kubernetes/manifests/kube-vip.yaml (1440 bytes)
I0917 02:11:27.618390 4110 ssh_runner.go:195] Run: grep 192.169.0.254 control-plane.minikube.internal$ /etc/hosts
I0917 02:11:27.621271 4110 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.169.0.254 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0917 02:11:27.630851 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:11:27.729065 4110 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0917 02:11:27.743762 4110 start.go:235] Will wait 6m0s for node &{Name:m02 IP:192.169.0.6 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true}
I0917 02:11:27.743972 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:11:27.765105 4110 out.go:177] * Verifying Kubernetes components...
I0917 02:11:27.805899 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:11:27.933521 4110 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0917 02:11:27.948089 4110 loader.go:395] Config loaded from file: /Users/jenkins/minikube-integration/19648-1025/kubeconfig
I0917 02:11:27.948282 4110 kapi.go:59] client config for ha-857000: &rest.Config{Host:"https://192.169.0.254:8443", APIPath:"", ContentConfig:rest.ContentConfig{AcceptContentTypes:"", ContentType:"", GroupVersion:(*schema.GroupVersion)(nil), NegotiatedSerializer:runtime.NegotiatedSerializer(nil)}, Username:"", Password:"", BearerToken:"", BearerTokenFile:"", Impersonate:rest.ImpersonationConfig{UserName:"", UID:"", Groups:[]string(nil), Extra:map[string][]string(nil)}, AuthProvider:<nil>, AuthConfigPersister:rest.AuthProviderConfigPersister(nil), ExecProvider:<nil>, TLSClientConfig:rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/client.crt", KeyFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/client.key", CAFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt", CertData:[]uint8(nil), KeyData:[]uint8(nil), CAData:[]uint8(nil), NextProtos:[]string(nil)}
, UserAgent:"", DisableCompression:false, Transport:http.RoundTripper(nil), WrapTransport:(transport.WrapperFunc)(0x673a720), QPS:0, Burst:0, RateLimiter:flowcontrol.RateLimiter(nil), WarningHandler:rest.WarningHandler(nil), Timeout:0, Dial:(func(context.Context, string, string) (net.Conn, error))(nil), Proxy:(func(*http.Request) (*url.URL, error))(nil)}
W0917 02:11:27.948321 4110 kubeadm.go:483] Overriding stale ClientConfig host https://192.169.0.254:8443 with https://192.169.0.5:8443
I0917 02:11:27.948495 4110 node_ready.go:35] waiting up to 6m0s for node "ha-857000-m02" to be "Ready" ...
I0917 02:11:27.948579 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:11:27.948584 4110 round_trippers.go:469] Request Headers:
I0917 02:11:27.948591 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:27.948595 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:28.948736 4110 round_trippers.go:574] Response Status: in 1000 milliseconds
I0917 02:11:28.948861 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:11:28.948870 4110 round_trippers.go:469] Request Headers:
I0917 02:11:28.948878 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:28.948882 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.256443 4110 round_trippers.go:574] Response Status: 200 OK in 7307 milliseconds
I0917 02:11:36.257038 4110 node_ready.go:49] node "ha-857000-m02" has status "Ready":"True"
I0917 02:11:36.257051 4110 node_ready.go:38] duration metric: took 8.308394835s for node "ha-857000-m02" to be "Ready" ...
I0917 02:11:36.257061 4110 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0917 02:11:36.257098 4110 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false
I0917 02:11:36.257107 4110 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false
I0917 02:11:36.257147 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods
I0917 02:11:36.257152 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.257158 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.257164 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.271996 4110 round_trippers.go:574] Response Status: 200 OK in 14 milliseconds
I0917 02:11:36.280676 4110 pod_ready.go:79] waiting up to 6m0s for pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.280736 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:11:36.280742 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.280752 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.280756 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.307985 4110 round_trippers.go:574] Response Status: 200 OK in 27 milliseconds
I0917 02:11:36.308476 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:11:36.308484 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.308491 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.308501 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.312984 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:11:36.313392 4110 pod_ready.go:93] pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:36.313402 4110 pod_ready.go:82] duration metric: took 32.709315ms for pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.313409 4110 pod_ready.go:79] waiting up to 6m0s for pod "coredns-7c65d6cfc9-nl5j5" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.313452 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-nl5j5
I0917 02:11:36.313457 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.313463 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.313468 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.319771 4110 round_trippers.go:574] Response Status: 200 OK in 6 milliseconds
I0917 02:11:36.320384 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:11:36.320393 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.320400 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.320403 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.322816 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:36.323378 4110 pod_ready.go:93] pod "coredns-7c65d6cfc9-nl5j5" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:36.323388 4110 pod_ready.go:82] duration metric: took 9.97387ms for pod "coredns-7c65d6cfc9-nl5j5" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.323395 4110 pod_ready.go:79] waiting up to 6m0s for pod "etcd-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.323435 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/etcd-ha-857000
I0917 02:11:36.323440 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.323446 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.323450 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.327486 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:11:36.328047 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:11:36.328054 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.328060 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.328063 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.331571 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:36.332110 4110 pod_ready.go:93] pod "etcd-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:36.332121 4110 pod_ready.go:82] duration metric: took 8.720083ms for pod "etcd-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.332128 4110 pod_ready.go:79] waiting up to 6m0s for pod "etcd-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.332168 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/etcd-ha-857000-m02
I0917 02:11:36.332173 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.332179 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.332184 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.336324 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:11:36.336846 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:11:36.336854 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.336860 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.336864 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.340608 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:36.341048 4110 pod_ready.go:93] pod "etcd-ha-857000-m02" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:36.341057 4110 pod_ready.go:82] duration metric: took 8.92351ms for pod "etcd-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.341064 4110 pod_ready.go:79] waiting up to 6m0s for pod "etcd-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.341104 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/etcd-ha-857000-m03
I0917 02:11:36.341110 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.341116 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.341121 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.343462 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:36.458248 4110 request.go:632] Waited for 114.333049ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:11:36.458307 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:11:36.458312 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.458318 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.458326 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.466021 4110 round_trippers.go:574] Response Status: 200 OK in 7 milliseconds
I0917 02:11:36.466526 4110 pod_ready.go:93] pod "etcd-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:36.466536 4110 pod_ready.go:82] duration metric: took 125.46489ms for pod "etcd-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.466548 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.657514 4110 request.go:632] Waited for 190.921312ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000
I0917 02:11:36.657567 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000
I0917 02:11:36.657574 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.657579 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.657584 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.659804 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:36.857671 4110 request.go:632] Waited for 197.395211ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:11:36.857701 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:11:36.857705 4110 round_trippers.go:469] Request Headers:
I0917 02:11:36.857711 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:36.857715 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:36.861065 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:36.861653 4110 pod_ready.go:93] pod "kube-apiserver-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:36.861669 4110 pod_ready.go:82] duration metric: took 395.104039ms for pod "kube-apiserver-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:11:36.861677 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:11:37.057332 4110 request.go:632] Waited for 195.603008ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m02
I0917 02:11:37.057382 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m02
I0917 02:11:37.057387 4110 round_trippers.go:469] Request Headers:
I0917 02:11:37.057393 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:37.057398 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:37.060216 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:37.258671 4110 request.go:632] Waited for 197.954534ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:11:37.258706 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:11:37.258713 4110 round_trippers.go:469] Request Headers:
I0917 02:11:37.258721 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:37.258727 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:37.267718 4110 round_trippers.go:574] Response Status: 200 OK in 8 milliseconds
I0917 02:11:37.268069 4110 pod_ready.go:93] pod "kube-apiserver-ha-857000-m02" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:37.268082 4110 pod_ready.go:82] duration metric: took 406.392892ms for pod "kube-apiserver-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:11:37.268090 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:11:37.457925 4110 request.go:632] Waited for 189.791882ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:11:37.457975 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:11:37.457980 4110 round_trippers.go:469] Request Headers:
I0917 02:11:37.457987 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:37.457992 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:37.461663 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:37.658806 4110 request.go:632] Waited for 196.487027ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:11:37.658861 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:11:37.658867 4110 round_trippers.go:469] Request Headers:
I0917 02:11:37.658874 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:37.658878 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:37.661429 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:37.661888 4110 pod_ready.go:93] pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:37.661897 4110 pod_ready.go:82] duration metric: took 393.794602ms for pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:11:37.661905 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:11:37.857414 4110 request.go:632] Waited for 195.469923ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000
I0917 02:11:37.857468 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000
I0917 02:11:37.857474 4110 round_trippers.go:469] Request Headers:
I0917 02:11:37.857481 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:37.857486 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:37.860019 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:38.057880 4110 request.go:632] Waited for 197.333642ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:11:38.057910 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:11:38.057915 4110 round_trippers.go:469] Request Headers:
I0917 02:11:38.057922 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:38.057927 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:38.060540 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:38.061091 4110 pod_ready.go:93] pod "kube-controller-manager-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:38.061101 4110 pod_ready.go:82] duration metric: took 399.184022ms for pod "kube-controller-manager-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:11:38.061109 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:11:38.257757 4110 request.go:632] Waited for 196.608954ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m02
I0917 02:11:38.257857 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m02
I0917 02:11:38.257871 4110 round_trippers.go:469] Request Headers:
I0917 02:11:38.257877 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:38.257882 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:38.259904 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:38.458082 4110 request.go:632] Waited for 197.709678ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:11:38.458138 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:11:38.458147 4110 round_trippers.go:469] Request Headers:
I0917 02:11:38.458154 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:38.458158 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:38.460347 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:38.460715 4110 pod_ready.go:98] node "ha-857000-m02" hosting pod "kube-controller-manager-ha-857000-m02" in "kube-system" namespace is currently not "Ready" (skipping!): node "ha-857000-m02" has status "Ready":"False"
I0917 02:11:38.460726 4110 pod_ready.go:82] duration metric: took 399.604676ms for pod "kube-controller-manager-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
E0917 02:11:38.460732 4110 pod_ready.go:67] WaitExtra: waitPodCondition: node "ha-857000-m02" hosting pod "kube-controller-manager-ha-857000-m02" in "kube-system" namespace is currently not "Ready" (skipping!): node "ha-857000-m02" has status "Ready":"False"
I0917 02:11:38.460739 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:11:38.658188 4110 request.go:632] Waited for 197.403717ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:11:38.658255 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:11:38.658261 4110 round_trippers.go:469] Request Headers:
I0917 02:11:38.658267 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:38.658271 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:38.660934 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:38.857786 4110 request.go:632] Waited for 196.168284ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:11:38.857841 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:11:38.857851 4110 round_trippers.go:469] Request Headers:
I0917 02:11:38.857863 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:38.857873 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:38.861470 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:38.861751 4110 pod_ready.go:93] pod "kube-controller-manager-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:38.861759 4110 pod_ready.go:82] duration metric: took 401.003253ms for pod "kube-controller-manager-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:11:38.861766 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-528ht" in "kube-system" namespace to be "Ready" ...
I0917 02:11:39.057800 4110 request.go:632] Waited for 195.986319ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-528ht
I0917 02:11:39.057882 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-528ht
I0917 02:11:39.057893 4110 round_trippers.go:469] Request Headers:
I0917 02:11:39.057904 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:39.057912 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:39.061639 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:39.257697 4110 request.go:632] Waited for 195.312452ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m04
I0917 02:11:39.257726 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m04
I0917 02:11:39.257731 4110 round_trippers.go:469] Request Headers:
I0917 02:11:39.257737 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:39.257741 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:39.260209 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:39.260462 4110 pod_ready.go:93] pod "kube-proxy-528ht" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:39.260471 4110 pod_ready.go:82] duration metric: took 398.692905ms for pod "kube-proxy-528ht" in "kube-system" namespace to be "Ready" ...
I0917 02:11:39.260478 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-g9wxm" in "kube-system" namespace to be "Ready" ...
I0917 02:11:39.459321 4110 request.go:632] Waited for 198.788481ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-g9wxm
I0917 02:11:39.459387 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-g9wxm
I0917 02:11:39.459394 4110 round_trippers.go:469] Request Headers:
I0917 02:11:39.459411 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:39.459422 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:39.461885 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:39.657441 4110 request.go:632] Waited for 195.121107ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:11:39.657541 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:11:39.657551 4110 round_trippers.go:469] Request Headers:
I0917 02:11:39.657579 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:39.657585 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:39.661441 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:39.661929 4110 pod_ready.go:93] pod "kube-proxy-g9wxm" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:39.661942 4110 pod_ready.go:82] duration metric: took 401.451734ms for pod "kube-proxy-g9wxm" in "kube-system" namespace to be "Ready" ...
I0917 02:11:39.661951 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-vskbj" in "kube-system" namespace to be "Ready" ...
I0917 02:11:39.857721 4110 request.go:632] Waited for 195.727193ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-vskbj
I0917 02:11:39.857785 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-vskbj
I0917 02:11:39.857791 4110 round_trippers.go:469] Request Headers:
I0917 02:11:39.857797 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:39.857802 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:39.859663 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:11:40.058574 4110 request.go:632] Waited for 198.443343ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:11:40.058668 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:11:40.058679 4110 round_trippers.go:469] Request Headers:
I0917 02:11:40.058690 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:40.058699 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:40.062499 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:40.063124 4110 pod_ready.go:93] pod "kube-proxy-vskbj" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:40.063133 4110 pod_ready.go:82] duration metric: took 401.170349ms for pod "kube-proxy-vskbj" in "kube-system" namespace to be "Ready" ...
I0917 02:11:40.063140 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-zrqvr" in "kube-system" namespace to be "Ready" ...
I0917 02:11:40.257873 4110 request.go:632] Waited for 194.653928ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-zrqvr
I0917 02:11:40.257927 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-zrqvr
I0917 02:11:40.257937 4110 round_trippers.go:469] Request Headers:
I0917 02:11:40.257948 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:40.257956 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:40.262255 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:11:40.458287 4110 request.go:632] Waited for 195.380222ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:11:40.458411 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:11:40.458421 4110 round_trippers.go:469] Request Headers:
I0917 02:11:40.458432 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:40.458443 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:40.462171 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:40.462629 4110 pod_ready.go:98] node "ha-857000-m02" hosting pod "kube-proxy-zrqvr" in "kube-system" namespace is currently not "Ready" (skipping!): node "ha-857000-m02" has status "Ready":"False"
I0917 02:11:40.462643 4110 pod_ready.go:82] duration metric: took 399.490798ms for pod "kube-proxy-zrqvr" in "kube-system" namespace to be "Ready" ...
E0917 02:11:40.462673 4110 pod_ready.go:67] WaitExtra: waitPodCondition: node "ha-857000-m02" hosting pod "kube-proxy-zrqvr" in "kube-system" namespace is currently not "Ready" (skipping!): node "ha-857000-m02" has status "Ready":"False"
I0917 02:11:40.462687 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:11:40.658101 4110 request.go:632] Waited for 195.359912ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000
I0917 02:11:40.658147 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000
I0917 02:11:40.658152 4110 round_trippers.go:469] Request Headers:
I0917 02:11:40.658159 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:40.658164 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:40.660407 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:40.858455 4110 request.go:632] Waited for 197.559018ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:11:40.858564 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:11:40.858583 4110 round_trippers.go:469] Request Headers:
I0917 02:11:40.858595 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:40.858601 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:40.861876 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:40.862327 4110 pod_ready.go:93] pod "kube-scheduler-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:40.862336 4110 pod_ready.go:82] duration metric: took 399.635382ms for pod "kube-scheduler-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:11:40.862343 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:11:41.057949 4110 request.go:632] Waited for 195.512959ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m02
I0917 02:11:41.058021 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m02
I0917 02:11:41.058032 4110 round_trippers.go:469] Request Headers:
I0917 02:11:41.058044 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:41.058051 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:41.061708 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:41.257802 4110 request.go:632] Waited for 195.475163ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:11:41.257884 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:11:41.257895 4110 round_trippers.go:469] Request Headers:
I0917 02:11:41.257906 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:41.257913 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:41.261190 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:41.261502 4110 pod_ready.go:98] node "ha-857000-m02" hosting pod "kube-scheduler-ha-857000-m02" in "kube-system" namespace is currently not "Ready" (skipping!): node "ha-857000-m02" has status "Ready":"False"
I0917 02:11:41.261513 4110 pod_ready.go:82] duration metric: took 399.156939ms for pod "kube-scheduler-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
E0917 02:11:41.261527 4110 pod_ready.go:67] WaitExtra: waitPodCondition: node "ha-857000-m02" hosting pod "kube-scheduler-ha-857000-m02" in "kube-system" namespace is currently not "Ready" (skipping!): node "ha-857000-m02" has status "Ready":"False"
I0917 02:11:41.261532 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:11:41.458981 4110 request.go:632] Waited for 197.407496ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m03
I0917 02:11:41.459061 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m03
I0917 02:11:41.459070 4110 round_trippers.go:469] Request Headers:
I0917 02:11:41.459078 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:41.459084 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:41.461880 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:41.657846 4110 request.go:632] Waited for 195.542216ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:11:41.657906 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:11:41.657913 4110 round_trippers.go:469] Request Headers:
I0917 02:11:41.657921 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:41.657934 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:41.660204 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:41.660601 4110 pod_ready.go:93] pod "kube-scheduler-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:11:41.660610 4110 pod_ready.go:82] duration metric: took 399.066544ms for pod "kube-scheduler-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:11:41.660617 4110 pod_ready.go:39] duration metric: took 5.403454072s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0917 02:11:41.660636 4110 api_server.go:52] waiting for apiserver process to appear ...
I0917 02:11:41.660697 4110 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0917 02:11:41.672821 4110 api_server.go:72] duration metric: took 13.928795458s to wait for apiserver process to appear ...
I0917 02:11:41.672831 4110 api_server.go:88] waiting for apiserver healthz status ...
I0917 02:11:41.672845 4110 api_server.go:253] Checking apiserver healthz at https://192.169.0.5:8443/healthz ...
I0917 02:11:41.683603 4110 api_server.go:279] https://192.169.0.5:8443/healthz returned 200:
ok
I0917 02:11:41.683654 4110 round_trippers.go:463] GET https://192.169.0.5:8443/version
I0917 02:11:41.683660 4110 round_trippers.go:469] Request Headers:
I0917 02:11:41.683666 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:41.683670 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:41.684276 4110 round_trippers.go:574] Response Status: 200 OK in 0 milliseconds
I0917 02:11:41.684340 4110 api_server.go:141] control plane version: v1.31.1
I0917 02:11:41.684350 4110 api_server.go:131] duration metric: took 11.515194ms to wait for apiserver health ...
I0917 02:11:41.684356 4110 system_pods.go:43] waiting for kube-system pods to appear ...
I0917 02:11:41.857675 4110 request.go:632] Waited for 173.274042ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods
I0917 02:11:41.857793 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods
I0917 02:11:41.857803 4110 round_trippers.go:469] Request Headers:
I0917 02:11:41.857823 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:41.857833 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:41.863157 4110 round_trippers.go:574] Response Status: 200 OK in 5 milliseconds
I0917 02:11:41.868330 4110 system_pods.go:59] 26 kube-system pods found
I0917 02:11:41.868348 4110 system_pods.go:61] "coredns-7c65d6cfc9-fg65r" [1690cf49-3cd5-45ba-bcff-6c2947fb1bef] Running
I0917 02:11:41.868352 4110 system_pods.go:61] "coredns-7c65d6cfc9-nl5j5" [dad0f1c6-0feb-4024-b0ee-95776c68bae8] Running
I0917 02:11:41.868360 4110 system_pods.go:61] "etcd-ha-857000" [e9da37c5-ad0f-47e0-b65f-361d2cb9f204] Running
I0917 02:11:41.868366 4110 system_pods.go:61] "etcd-ha-857000-m02" [f540f85a-9556-44c0-a560-188721a58bd5] Running / Ready:ContainersNotReady (containers with unready status: [etcd]) / ContainersReady:ContainersNotReady (containers with unready status: [etcd])
I0917 02:11:41.868371 4110 system_pods.go:61] "etcd-ha-857000-m03" [46cb6c7e-73e2-49b0-986f-c63a23ffa29d] Running
I0917 02:11:41.868377 4110 system_pods.go:61] "kindnet-4jk9v" [24a018c6-9cbb-4d17-a295-8fef456534a0] Running
I0917 02:11:41.868392 4110 system_pods.go:61] "kindnet-7pf7v" [eecd1421-3a2f-4e48-b2b2-abcbef7869e7] Running
I0917 02:11:41.868398 4110 system_pods.go:61] "kindnet-vc6z5" [a92e41cb-74d1-4bd4-bffd-b807c859efd3] Running
I0917 02:11:41.868402 4110 system_pods.go:61] "kindnet-vh2h2" [00d83b6a-e87e-4c36-b073-36e67c76d67d] Running / Ready:ContainersNotReady (containers with unready status: [kindnet-cni]) / ContainersReady:ContainersNotReady (containers with unready status: [kindnet-cni])
I0917 02:11:41.868406 4110 system_pods.go:61] "kube-apiserver-ha-857000" [b5451c9b-3be2-4454-bed6-fdc48031180e] Running
I0917 02:11:41.868424 4110 system_pods.go:61] "kube-apiserver-ha-857000-m02" [d043a0e6-6ab2-47b6-bb82-ceff496f8336] Running / Ready:ContainersNotReady (containers with unready status: [kube-apiserver]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-apiserver])
I0917 02:11:41.868430 4110 system_pods.go:61] "kube-apiserver-ha-857000-m03" [b3d91c89-8830-4dd9-8c20-4d6f821a3d88] Running
I0917 02:11:41.868434 4110 system_pods.go:61] "kube-controller-manager-ha-857000" [f4b0f56c-8d7d-4567-a4eb-088805c36c54] Running
I0917 02:11:41.868438 4110 system_pods.go:61] "kube-controller-manager-ha-857000-m02" [16670a48-0dc2-4665-9b1d-5fc25337ec58] Running / Ready:ContainersNotReady (containers with unready status: [kube-controller-manager]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-controller-manager])
I0917 02:11:41.868442 4110 system_pods.go:61] "kube-controller-manager-ha-857000-m03" [754fdff4-0e27-4f32-ab12-3a6695924396] Running
I0917 02:11:41.868445 4110 system_pods.go:61] "kube-proxy-528ht" [18b29dac-e4bf-4f26-988f-b1ba4019f9bc] Running
I0917 02:11:41.868448 4110 system_pods.go:61] "kube-proxy-g9wxm" [a5b974f1-ed28-4f42-8c86-6fed0bf32317] Running
I0917 02:11:41.868450 4110 system_pods.go:61] "kube-proxy-vskbj" [7a396757-8954-48d2-b708-dcdfbab21dc7] Running
I0917 02:11:41.868454 4110 system_pods.go:61] "kube-proxy-zrqvr" [bb98af15-e15a-4904-b68d-2de3c6e5864c] Running / Ready:ContainersNotReady (containers with unready status: [kube-proxy]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-proxy])
I0917 02:11:41.868456 4110 system_pods.go:61] "kube-scheduler-ha-857000" [fdce329c-0340-4e0a-8bc1-295e4970708b] Running
I0917 02:11:41.868468 4110 system_pods.go:61] "kube-scheduler-ha-857000-m02" [274256f9-cd98-4dd3-befc-35443dcca033] Running / Ready:ContainersNotReady (containers with unready status: [kube-scheduler]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-scheduler])
I0917 02:11:41.868473 4110 system_pods.go:61] "kube-scheduler-ha-857000-m03" [74d4633a-1c51-478d-9c68-d05c964089d9] Running
I0917 02:11:41.868484 4110 system_pods.go:61] "kube-vip-ha-857000" [84b805d8-9a8f-4c6f-b18f-76c98ca4776c] Running
I0917 02:11:41.868488 4110 system_pods.go:61] "kube-vip-ha-857000-m02" [b194d3e4-3b4d-4075-a6b1-f05d219393a0] Running
I0917 02:11:41.868490 4110 system_pods.go:61] "kube-vip-ha-857000-m03" [29be8efa-f99f-460a-80bd-ccc25f608e48] Running
I0917 02:11:41.868493 4110 system_pods.go:61] "storage-provisioner" [d81e7b55-a14e-4dc7-9193-ebe6914cdacf] Running
I0917 02:11:41.868498 4110 system_pods.go:74] duration metric: took 184.134673ms to wait for pod list to return data ...
I0917 02:11:41.868509 4110 default_sa.go:34] waiting for default service account to be created ...
I0917 02:11:42.057457 4110 request.go:632] Waited for 188.887232ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/default/serviceaccounts
I0917 02:11:42.057501 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/default/serviceaccounts
I0917 02:11:42.057507 4110 round_trippers.go:469] Request Headers:
I0917 02:11:42.057512 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:42.057516 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:42.060122 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:42.060299 4110 default_sa.go:45] found service account: "default"
I0917 02:11:42.060314 4110 default_sa.go:55] duration metric: took 191.792113ms for default service account to be created ...
I0917 02:11:42.060320 4110 system_pods.go:116] waiting for k8s-apps to be running ...
I0917 02:11:42.257458 4110 request.go:632] Waited for 197.098839ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods
I0917 02:11:42.257490 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods
I0917 02:11:42.257495 4110 round_trippers.go:469] Request Headers:
I0917 02:11:42.257501 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:42.257506 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:42.261392 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:11:42.267316 4110 system_pods.go:86] 26 kube-system pods found
I0917 02:11:42.267336 4110 system_pods.go:89] "coredns-7c65d6cfc9-fg65r" [1690cf49-3cd5-45ba-bcff-6c2947fb1bef] Running
I0917 02:11:42.267340 4110 system_pods.go:89] "coredns-7c65d6cfc9-nl5j5" [dad0f1c6-0feb-4024-b0ee-95776c68bae8] Running
I0917 02:11:42.267343 4110 system_pods.go:89] "etcd-ha-857000" [e9da37c5-ad0f-47e0-b65f-361d2cb9f204] Running
I0917 02:11:42.267356 4110 system_pods.go:89] "etcd-ha-857000-m02" [f540f85a-9556-44c0-a560-188721a58bd5] Running / Ready:ContainersNotReady (containers with unready status: [etcd]) / ContainersReady:ContainersNotReady (containers with unready status: [etcd])
I0917 02:11:42.267362 4110 system_pods.go:89] "etcd-ha-857000-m03" [46cb6c7e-73e2-49b0-986f-c63a23ffa29d] Running
I0917 02:11:42.267366 4110 system_pods.go:89] "kindnet-4jk9v" [24a018c6-9cbb-4d17-a295-8fef456534a0] Running
I0917 02:11:42.267369 4110 system_pods.go:89] "kindnet-7pf7v" [eecd1421-3a2f-4e48-b2b2-abcbef7869e7] Running
I0917 02:11:42.267372 4110 system_pods.go:89] "kindnet-vc6z5" [a92e41cb-74d1-4bd4-bffd-b807c859efd3] Running
I0917 02:11:42.267377 4110 system_pods.go:89] "kindnet-vh2h2" [00d83b6a-e87e-4c36-b073-36e67c76d67d] Running / Ready:ContainersNotReady (containers with unready status: [kindnet-cni]) / ContainersReady:ContainersNotReady (containers with unready status: [kindnet-cni])
I0917 02:11:42.267380 4110 system_pods.go:89] "kube-apiserver-ha-857000" [b5451c9b-3be2-4454-bed6-fdc48031180e] Running
I0917 02:11:42.267385 4110 system_pods.go:89] "kube-apiserver-ha-857000-m02" [d043a0e6-6ab2-47b6-bb82-ceff496f8336] Running / Ready:ContainersNotReady (containers with unready status: [kube-apiserver]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-apiserver])
I0917 02:11:42.267389 4110 system_pods.go:89] "kube-apiserver-ha-857000-m03" [b3d91c89-8830-4dd9-8c20-4d6f821a3d88] Running
I0917 02:11:42.267392 4110 system_pods.go:89] "kube-controller-manager-ha-857000" [f4b0f56c-8d7d-4567-a4eb-088805c36c54] Running
I0917 02:11:42.267398 4110 system_pods.go:89] "kube-controller-manager-ha-857000-m02" [16670a48-0dc2-4665-9b1d-5fc25337ec58] Running / Ready:ContainersNotReady (containers with unready status: [kube-controller-manager]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-controller-manager])
I0917 02:11:42.267402 4110 system_pods.go:89] "kube-controller-manager-ha-857000-m03" [754fdff4-0e27-4f32-ab12-3a6695924396] Running
I0917 02:11:42.267405 4110 system_pods.go:89] "kube-proxy-528ht" [18b29dac-e4bf-4f26-988f-b1ba4019f9bc] Running
I0917 02:11:42.267408 4110 system_pods.go:89] "kube-proxy-g9wxm" [a5b974f1-ed28-4f42-8c86-6fed0bf32317] Running
I0917 02:11:42.267411 4110 system_pods.go:89] "kube-proxy-vskbj" [7a396757-8954-48d2-b708-dcdfbab21dc7] Running
I0917 02:11:42.267415 4110 system_pods.go:89] "kube-proxy-zrqvr" [bb98af15-e15a-4904-b68d-2de3c6e5864c] Running / Ready:ContainersNotReady (containers with unready status: [kube-proxy]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-proxy])
I0917 02:11:42.267419 4110 system_pods.go:89] "kube-scheduler-ha-857000" [fdce329c-0340-4e0a-8bc1-295e4970708b] Running
I0917 02:11:42.267423 4110 system_pods.go:89] "kube-scheduler-ha-857000-m02" [274256f9-cd98-4dd3-befc-35443dcca033] Running / Ready:ContainersNotReady (containers with unready status: [kube-scheduler]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-scheduler])
I0917 02:11:42.267427 4110 system_pods.go:89] "kube-scheduler-ha-857000-m03" [74d4633a-1c51-478d-9c68-d05c964089d9] Running
I0917 02:11:42.267436 4110 system_pods.go:89] "kube-vip-ha-857000" [84b805d8-9a8f-4c6f-b18f-76c98ca4776c] Running
I0917 02:11:42.267438 4110 system_pods.go:89] "kube-vip-ha-857000-m02" [b194d3e4-3b4d-4075-a6b1-f05d219393a0] Running
I0917 02:11:42.267441 4110 system_pods.go:89] "kube-vip-ha-857000-m03" [29be8efa-f99f-460a-80bd-ccc25f608e48] Running
I0917 02:11:42.267444 4110 system_pods.go:89] "storage-provisioner" [d81e7b55-a14e-4dc7-9193-ebe6914cdacf] Running
I0917 02:11:42.267448 4110 system_pods.go:126] duration metric: took 207.120728ms to wait for k8s-apps to be running ...
I0917 02:11:42.267459 4110 system_svc.go:44] waiting for kubelet service to be running ....
I0917 02:11:42.267525 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0917 02:11:42.280323 4110 system_svc.go:56] duration metric: took 12.855514ms WaitForService to wait for kubelet
I0917 02:11:42.280342 4110 kubeadm.go:582] duration metric: took 14.536306226s to wait for: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0917 02:11:42.280356 4110 node_conditions.go:102] verifying NodePressure condition ...
I0917 02:11:42.458901 4110 request.go:632] Waited for 178.497588ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes
I0917 02:11:42.458965 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes
I0917 02:11:42.458970 4110 round_trippers.go:469] Request Headers:
I0917 02:11:42.458975 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:11:42.458980 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:11:42.461607 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:11:42.462345 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:11:42.462358 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:11:42.462367 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:11:42.462370 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:11:42.462374 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:11:42.462377 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:11:42.462380 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:11:42.462383 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:11:42.462386 4110 node_conditions.go:105] duration metric: took 182.022805ms to run NodePressure ...
I0917 02:11:42.462394 4110 start.go:241] waiting for startup goroutines ...
I0917 02:11:42.462412 4110 start.go:255] writing updated cluster config ...
I0917 02:11:42.484336 4110 out.go:201]
I0917 02:11:42.505774 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:11:42.505869 4110 profile.go:143] Saving config to /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/config.json ...
I0917 02:11:42.527331 4110 out.go:177] * Starting "ha-857000-m03" control-plane node in "ha-857000" cluster
I0917 02:11:42.569515 4110 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime docker
I0917 02:11:42.569551 4110 cache.go:56] Caching tarball of preloaded images
I0917 02:11:42.569751 4110 preload.go:172] Found /Users/jenkins/minikube-integration/19648-1025/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0917 02:11:42.569769 4110 cache.go:59] Finished verifying existence of preloaded tar for v1.31.1 on docker
I0917 02:11:42.569891 4110 profile.go:143] Saving config to /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/config.json ...
I0917 02:11:42.570622 4110 start.go:360] acquireMachinesLock for ha-857000-m03: {Name:mkcb27accfc361dc9c175eeb8bb69dc9967aa557 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0917 02:11:42.570733 4110 start.go:364] duration metric: took 89.66µs to acquireMachinesLock for "ha-857000-m03"
I0917 02:11:42.570758 4110 start.go:96] Skipping create...Using existing machine configuration
I0917 02:11:42.570766 4110 fix.go:54] fixHost starting: m03
I0917 02:11:42.571203 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:11:42.571238 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:11:42.581037 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52144
I0917 02:11:42.581469 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:11:42.581811 4110 main.go:141] libmachine: Using API Version 1
I0917 02:11:42.581822 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:11:42.582051 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:11:42.582209 4110 main.go:141] libmachine: (ha-857000-m03) Calling .DriverName
I0917 02:11:42.582294 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetState
I0917 02:11:42.582428 4110 main.go:141] libmachine: (ha-857000-m03) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:11:42.582545 4110 main.go:141] libmachine: (ha-857000-m03) DBG | hyperkit pid from json: 3442
I0917 02:11:42.583498 4110 main.go:141] libmachine: (ha-857000-m03) DBG | hyperkit pid 3442 missing from process table
I0917 02:11:42.583556 4110 fix.go:112] recreateIfNeeded on ha-857000-m03: state=Stopped err=<nil>
I0917 02:11:42.583568 4110 main.go:141] libmachine: (ha-857000-m03) Calling .DriverName
W0917 02:11:42.583655 4110 fix.go:138] unexpected machine state, will restart: <nil>
I0917 02:11:42.604438 4110 out.go:177] * Restarting existing hyperkit VM for "ha-857000-m03" ...
I0917 02:11:42.678579 4110 main.go:141] libmachine: (ha-857000-m03) Calling .Start
I0917 02:11:42.678864 4110 main.go:141] libmachine: (ha-857000-m03) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:11:42.678945 4110 main.go:141] libmachine: (ha-857000-m03) minikube might have been shutdown in an unclean way, the hyperkit pid file still exists: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/hyperkit.pid
I0917 02:11:42.680796 4110 main.go:141] libmachine: (ha-857000-m03) DBG | hyperkit pid 3442 missing from process table
I0917 02:11:42.680811 4110 main.go:141] libmachine: (ha-857000-m03) DBG | pid 3442 is in state "Stopped"
I0917 02:11:42.680856 4110 main.go:141] libmachine: (ha-857000-m03) DBG | Removing stale pid file /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/hyperkit.pid...
I0917 02:11:42.681059 4110 main.go:141] libmachine: (ha-857000-m03) DBG | Using UUID 3d8fdba9-dbf7-47ea-a80b-a24a99cad96e
I0917 02:11:42.708058 4110 main.go:141] libmachine: (ha-857000-m03) DBG | Generated MAC 16:4d:1d:5e:91:c8
I0917 02:11:42.708080 4110 main.go:141] libmachine: (ha-857000-m03) DBG | Starting with cmdline: loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000
I0917 02:11:42.708229 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 DEBUG: hyperkit: Start &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"3d8fdba9-dbf7-47ea-a80b-a24a99cad96e", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc000398c90)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/bzimage", Initrd:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os.Process)(nil)}
I0917 02:11:42.708256 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 DEBUG: hyperkit: check &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"3d8fdba9-dbf7-47ea-a80b-a24a99cad96e", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc000398c90)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/bzimage", Initrd:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os.Process)(nil)}
I0917 02:11:42.708317 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 DEBUG: hyperkit: Arguments: []string{"-A", "-u", "-F", "/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/hyperkit.pid", "-c", "2", "-m", "2200M", "-s", "0:0,hostbridge", "-s", "31,lpc", "-s", "1:0,virtio-net", "-U", "3d8fdba9-dbf7-47ea-a80b-a24a99cad96e", "-s", "2:0,virtio-blk,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/ha-857000-m03.rawdisk", "-s", "3,ahci-cd,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/boot2docker.iso", "-s", "4,virtio-rnd", "-l", "com1,autopty=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/tty,log=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/console-ring", "-f", "kexec,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/bzimage,/Users/jenkins/minikube-integration/19648-1025/.minikube/machine
s/ha-857000-m03/initrd,earlyprintk=serial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000"}
I0917 02:11:42.708369 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 DEBUG: hyperkit: CmdLine: "/usr/local/bin/hyperkit -A -u -F /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/hyperkit.pid -c 2 -m 2200M -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-net -U 3d8fdba9-dbf7-47ea-a80b-a24a99cad96e -s 2:0,virtio-blk,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/ha-857000-m03.rawdisk -s 3,ahci-cd,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/boot2docker.iso -s 4,virtio-rnd -l com1,autopty=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/tty,log=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/console-ring -f kexec,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/bzimage,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/initrd,earlyprintk=serial loglevel=3 console=t
tyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000"
I0917 02:11:42.708386 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 DEBUG: hyperkit: Redirecting stdout/stderr to logger
I0917 02:11:42.710198 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 DEBUG: hyperkit: Pid is 4146
I0917 02:11:42.710768 4110 main.go:141] libmachine: (ha-857000-m03) DBG | Attempt 0
I0917 02:11:42.710795 4110 main.go:141] libmachine: (ha-857000-m03) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:11:42.710847 4110 main.go:141] libmachine: (ha-857000-m03) DBG | hyperkit pid from json: 4146
I0917 02:11:42.712907 4110 main.go:141] libmachine: (ha-857000-m03) DBG | Searching for 16:4d:1d:5e:91:c8 in /var/db/dhcpd_leases ...
I0917 02:11:42.712978 4110 main.go:141] libmachine: (ha-857000-m03) DBG | Found 7 entries in /var/db/dhcpd_leases!
I0917 02:11:42.713009 4110 main.go:141] libmachine: (ha-857000-m03) DBG | dhcp entry: {Name:minikube IPAddress:192.169.0.6 HWAddress:9a:95:4e:4b:65:fe ID:1,9a:95:4e:4b:65:fe Lease:0x66ea991f}
I0917 02:11:42.713035 4110 main.go:141] libmachine: (ha-857000-m03) DBG | dhcp entry: {Name:minikube IPAddress:192.169.0.5 HWAddress:c2:63:2b:63:80:76 ID:1,c2:63:2b:63:80:76 Lease:0x66ea990c}
I0917 02:11:42.713060 4110 main.go:141] libmachine: (ha-857000-m03) DBG | dhcp entry: {Name:minikube IPAddress:192.169.0.8 HWAddress:a:b6:8:34:25:a6 ID:1,a:b6:8:34:25:a6 Lease:0x66e94661}
I0917 02:11:42.713079 4110 main.go:141] libmachine: (ha-857000-m03) DBG | dhcp entry: {Name:minikube IPAddress:192.169.0.7 HWAddress:16:4d:1d:5e:91:c8 ID:1,16:4d:1d:5e:91:c8 Lease:0x66ea9724}
I0917 02:11:42.713098 4110 main.go:141] libmachine: (ha-857000-m03) DBG | Found match: 16:4d:1d:5e:91:c8
I0917 02:11:42.713110 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetConfigRaw
I0917 02:11:42.713129 4110 main.go:141] libmachine: (ha-857000-m03) DBG | IP: 192.169.0.7
I0917 02:11:42.713812 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetIP
I0917 02:11:42.714067 4110 profile.go:143] Saving config to /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/config.json ...
I0917 02:11:42.714634 4110 machine.go:93] provisionDockerMachine start ...
I0917 02:11:42.714648 4110 main.go:141] libmachine: (ha-857000-m03) Calling .DriverName
I0917 02:11:42.714804 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:42.714912 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:42.715030 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:42.715172 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:42.715275 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:42.715462 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:42.715719 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.7 22 <nil> <nil>}
I0917 02:11:42.715729 4110 main.go:141] libmachine: About to run SSH command:
hostname
I0917 02:11:42.719370 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 INFO : hyperkit: stderr: Using fd 5 for I/O notifications
I0917 02:11:42.729567 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 INFO : hyperkit: stderr: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/boot2docker.iso: fcntl(F_PUNCHHOLE) Operation not permitted: block device will not support TRIM/DISCARD
I0917 02:11:42.730522 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0
I0917 02:11:42.730552 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0
I0917 02:11:42.730564 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0
I0917 02:11:42.730573 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:42 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0
I0917 02:11:43.130217 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:43 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 0
I0917 02:11:43.130237 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:43 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 0
I0917 02:11:43.246057 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:43 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0
I0917 02:11:43.246080 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:43 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0
I0917 02:11:43.246089 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:43 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0
I0917 02:11:43.246096 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:43 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0
I0917 02:11:43.246900 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:43 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 1
I0917 02:11:43.246909 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:43 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 1
I0917 02:11:48.954281 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:48 INFO : hyperkit: stderr: rdmsr to register 0x64d on vcpu 1
I0917 02:11:48.954379 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:48 INFO : hyperkit: stderr: rdmsr to register 0x64e on vcpu 1
I0917 02:11:48.954390 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:48 INFO : hyperkit: stderr: rdmsr to register 0x34 on vcpu 1
I0917 02:11:48.977816 4110 main.go:141] libmachine: (ha-857000-m03) DBG | 2024/09/17 02:11:48 INFO : hyperkit: stderr: rdmsr to register 0xc0011029 on vcpu 1
I0917 02:11:53.786367 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: minikube
I0917 02:11:53.786383 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetMachineName
I0917 02:11:53.786507 4110 buildroot.go:166] provisioning hostname "ha-857000-m03"
I0917 02:11:53.786518 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetMachineName
I0917 02:11:53.786619 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:53.786716 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:53.786814 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:53.786901 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:53.786991 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:53.787125 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:53.787256 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.7 22 <nil> <nil>}
I0917 02:11:53.787264 4110 main.go:141] libmachine: About to run SSH command:
sudo hostname ha-857000-m03 && echo "ha-857000-m03" | sudo tee /etc/hostname
I0917 02:11:53.860809 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: ha-857000-m03
I0917 02:11:53.860831 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:53.860995 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:53.861092 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:53.861199 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:53.861302 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:53.861448 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:53.861610 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.7 22 <nil> <nil>}
I0917 02:11:53.861621 4110 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sha-857000-m03' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 ha-857000-m03/g' /etc/hosts;
else
echo '127.0.1.1 ha-857000-m03' | sudo tee -a /etc/hosts;
fi
fi
I0917 02:11:53.932575 4110 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0917 02:11:53.932592 4110 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/19648-1025/.minikube CaCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/19648-1025/.minikube}
I0917 02:11:53.932604 4110 buildroot.go:174] setting up certificates
I0917 02:11:53.932611 4110 provision.go:84] configureAuth start
I0917 02:11:53.932618 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetMachineName
I0917 02:11:53.932757 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetIP
I0917 02:11:53.932853 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:53.932933 4110 provision.go:143] copyHostCerts
I0917 02:11:53.932962 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem
I0917 02:11:53.933012 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem, removing ...
I0917 02:11:53.933018 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem
I0917 02:11:53.933153 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem (1078 bytes)
I0917 02:11:53.933356 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem
I0917 02:11:53.933385 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem, removing ...
I0917 02:11:53.933389 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem
I0917 02:11:53.933461 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem (1123 bytes)
I0917 02:11:53.933602 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem
I0917 02:11:53.933640 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem, removing ...
I0917 02:11:53.933645 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem
I0917 02:11:53.933711 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem (1675 bytes)
I0917 02:11:53.933855 4110 provision.go:117] generating server cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem org=jenkins.ha-857000-m03 san=[127.0.0.1 192.169.0.7 ha-857000-m03 localhost minikube]
I0917 02:11:54.077333 4110 provision.go:177] copyRemoteCerts
I0917 02:11:54.077392 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0917 02:11:54.077407 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:54.077544 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:54.077643 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:54.077738 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:54.077820 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.7 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/id_rsa Username:docker}
I0917 02:11:54.116797 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem -> /etc/docker/server.pem
I0917 02:11:54.116876 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes)
I0917 02:11:54.136202 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem -> /etc/docker/server-key.pem
I0917 02:11:54.136278 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0917 02:11:54.156340 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem -> /etc/docker/ca.pem
I0917 02:11:54.156419 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0917 02:11:54.175630 4110 provision.go:87] duration metric: took 243.006586ms to configureAuth
I0917 02:11:54.175645 4110 buildroot.go:189] setting minikube options for container-runtime
I0917 02:11:54.175825 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:11:54.175845 4110 main.go:141] libmachine: (ha-857000-m03) Calling .DriverName
I0917 02:11:54.175978 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:54.176072 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:54.176183 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:54.176286 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:54.176390 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:54.176544 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:54.176682 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.7 22 <nil> <nil>}
I0917 02:11:54.176690 4110 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0917 02:11:54.238979 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs
I0917 02:11:54.238993 4110 buildroot.go:70] root file system type: tmpfs
I0917 02:11:54.239102 4110 provision.go:314] Updating docker unit: /lib/systemd/system/docker.service ...
I0917 02:11:54.239114 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:54.239249 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:54.239359 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:54.239453 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:54.239547 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:54.239702 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:54.239844 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.7 22 <nil> <nil>}
I0917 02:11:54.239889 4110 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
Environment="NO_PROXY=192.169.0.5"
Environment="NO_PROXY=192.169.0.5,192.169.0.6"
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0917 02:11:54.314599 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
Environment=NO_PROXY=192.169.0.5
Environment=NO_PROXY=192.169.0.5,192.169.0.6
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0917 02:11:54.314621 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:54.314767 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:54.314854 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:54.314947 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:54.315024 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:54.315150 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:54.315292 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.7 22 <nil> <nil>}
I0917 02:11:54.315304 4110 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0917 02:11:55.935197 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
I0917 02:11:55.935211 4110 machine.go:96] duration metric: took 13.220338614s to provisionDockerMachine
I0917 02:11:55.935219 4110 start.go:293] postStartSetup for "ha-857000-m03" (driver="hyperkit")
I0917 02:11:55.935226 4110 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0917 02:11:55.935240 4110 main.go:141] libmachine: (ha-857000-m03) Calling .DriverName
I0917 02:11:55.935436 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0917 02:11:55.935456 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:55.935555 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:55.935640 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:55.935720 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:55.935796 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.7 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/id_rsa Username:docker}
I0917 02:11:55.975655 4110 ssh_runner.go:195] Run: cat /etc/os-release
I0917 02:11:55.982326 4110 info.go:137] Remote host: Buildroot 2023.02.9
I0917 02:11:55.982340 4110 filesync.go:126] Scanning /Users/jenkins/minikube-integration/19648-1025/.minikube/addons for local assets ...
I0917 02:11:55.982439 4110 filesync.go:126] Scanning /Users/jenkins/minikube-integration/19648-1025/.minikube/files for local assets ...
I0917 02:11:55.982583 4110 filesync.go:149] local asset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> 15602.pem in /etc/ssl/certs
I0917 02:11:55.982589 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> /etc/ssl/certs/15602.pem
I0917 02:11:55.982752 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0917 02:11:55.995355 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem --> /etc/ssl/certs/15602.pem (1708 bytes)
I0917 02:11:56.016063 4110 start.go:296] duration metric: took 80.833975ms for postStartSetup
I0917 02:11:56.016085 4110 main.go:141] libmachine: (ha-857000-m03) Calling .DriverName
I0917 02:11:56.016278 4110 ssh_runner.go:195] Run: sudo ls --almost-all -1 /var/lib/minikube/backup
I0917 02:11:56.016292 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:56.016390 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:56.016474 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:56.016549 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:56.016621 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.7 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/id_rsa Username:docker}
I0917 02:11:56.056575 4110 machine.go:197] restoring vm config from /var/lib/minikube/backup: [etc]
I0917 02:11:56.056644 4110 ssh_runner.go:195] Run: sudo rsync --archive --update /var/lib/minikube/backup/etc /
I0917 02:11:56.090435 4110 fix.go:56] duration metric: took 13.519431085s for fixHost
I0917 02:11:56.090460 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:56.090600 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:56.090686 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:56.090776 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:56.090860 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:56.090993 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:11:56.091136 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.7 22 <nil> <nil>}
I0917 02:11:56.091142 4110 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0917 02:11:56.155623 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: 1726564316.081021180
I0917 02:11:56.155639 4110 fix.go:216] guest clock: 1726564316.081021180
I0917 02:11:56.155645 4110 fix.go:229] Guest: 2024-09-17 02:11:56.08102118 -0700 PDT Remote: 2024-09-17 02:11:56.09045 -0700 PDT m=+89.019475712 (delta=-9.42882ms)
I0917 02:11:56.155656 4110 fix.go:200] guest clock delta is within tolerance: -9.42882ms
I0917 02:11:56.155660 4110 start.go:83] releasing machines lock for "ha-857000-m03", held for 13.584681554s
I0917 02:11:56.155677 4110 main.go:141] libmachine: (ha-857000-m03) Calling .DriverName
I0917 02:11:56.155816 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetIP
I0917 02:11:56.177120 4110 out.go:177] * Found network options:
I0917 02:11:56.197056 4110 out.go:177] - NO_PROXY=192.169.0.5,192.169.0.6
W0917 02:11:56.217835 4110 proxy.go:119] fail to check proxy env: Error ip not in block
W0917 02:11:56.217862 4110 proxy.go:119] fail to check proxy env: Error ip not in block
I0917 02:11:56.217881 4110 main.go:141] libmachine: (ha-857000-m03) Calling .DriverName
I0917 02:11:56.218511 4110 main.go:141] libmachine: (ha-857000-m03) Calling .DriverName
I0917 02:11:56.218685 4110 main.go:141] libmachine: (ha-857000-m03) Calling .DriverName
I0917 02:11:56.218846 4110 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
W0917 02:11:56.218876 4110 proxy.go:119] fail to check proxy env: Error ip not in block
I0917 02:11:56.218892 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
W0917 02:11:56.218898 4110 proxy.go:119] fail to check proxy env: Error ip not in block
I0917 02:11:56.219005 4110 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0917 02:11:56.219024 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHHostname
I0917 02:11:56.219078 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:56.219246 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:56.219309 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHPort
I0917 02:11:56.219439 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:56.219492 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHKeyPath
I0917 02:11:56.219585 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.7 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/id_rsa Username:docker}
I0917 02:11:56.219614 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetSSHUsername
I0917 02:11:56.219751 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.7 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m03/id_rsa Username:docker}
W0917 02:11:56.256644 4110 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0917 02:11:56.256720 4110 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0917 02:11:56.309886 4110 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0917 02:11:56.309904 4110 start.go:495] detecting cgroup driver to use...
I0917 02:11:56.309980 4110 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0917 02:11:56.326165 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0917 02:11:56.334717 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0917 02:11:56.343026 4110 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0917 02:11:56.343079 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0917 02:11:56.351351 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0917 02:11:56.359978 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0917 02:11:56.368445 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0917 02:11:56.376813 4110 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0917 02:11:56.385309 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0917 02:11:56.393895 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0917 02:11:56.402441 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0917 02:11:56.410891 4110 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0917 02:11:56.418564 4110 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0917 02:11:56.426298 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:11:56.529182 4110 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0917 02:11:56.548629 4110 start.go:495] detecting cgroup driver to use...
I0917 02:11:56.548711 4110 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0917 02:11:56.564564 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0917 02:11:56.575668 4110 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0917 02:11:56.592483 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0917 02:11:56.605747 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0917 02:11:56.616286 4110 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0917 02:11:56.636099 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0917 02:11:56.646661 4110 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/cri-dockerd.sock
" | sudo tee /etc/crictl.yaml"
I0917 02:11:56.662025 4110 ssh_runner.go:195] Run: which cri-dockerd
I0917 02:11:56.665163 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I0917 02:11:56.672775 4110 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (190 bytes)
I0917 02:11:56.686783 4110 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0917 02:11:56.787618 4110 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0917 02:11:56.902014 4110 docker.go:574] configuring docker to use "cgroupfs" as cgroup driver...
I0917 02:11:56.902043 4110 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)
I0917 02:11:56.916683 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:11:57.010321 4110 ssh_runner.go:195] Run: sudo systemctl restart docker
I0917 02:11:59.292351 4110 ssh_runner.go:235] Completed: sudo systemctl restart docker: (2.28197073s)
I0917 02:11:59.292423 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.socket
I0917 02:11:59.302881 4110 ssh_runner.go:195] Run: sudo systemctl stop cri-docker.socket
I0917 02:11:59.315909 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0917 02:11:59.326097 4110 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket
I0917 02:11:59.423622 4110 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket
I0917 02:11:59.534194 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:11:59.650222 4110 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket
I0917 02:11:59.664197 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0917 02:11:59.675195 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:11:59.768785 4110 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.service
I0917 02:11:59.834137 4110 start.go:542] Will wait 60s for socket path /var/run/cri-dockerd.sock
I0917 02:11:59.834234 4110 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock
I0917 02:11:59.838654 4110 start.go:563] Will wait 60s for crictl version
I0917 02:11:59.838726 4110 ssh_runner.go:195] Run: which crictl
I0917 02:11:59.844060 4110 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0917 02:11:59.874850 4110 start.go:579] Version: 0.1.0
RuntimeName: docker
RuntimeVersion: 27.2.1
RuntimeApiVersion: v1
I0917 02:11:59.874944 4110 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0917 02:11:59.893142 4110 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0917 02:11:59.934010 4110 out.go:235] * Preparing Kubernetes v1.31.1 on Docker 27.2.1 ...
I0917 02:11:59.974908 4110 out.go:177] - env NO_PROXY=192.169.0.5
I0917 02:11:59.996010 4110 out.go:177] - env NO_PROXY=192.169.0.5,192.169.0.6
I0917 02:12:00.016678 4110 main.go:141] libmachine: (ha-857000-m03) Calling .GetIP
I0917 02:12:00.016979 4110 ssh_runner.go:195] Run: grep 192.169.0.1 host.minikube.internal$ /etc/hosts
I0917 02:12:00.020450 4110 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.169.0.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0917 02:12:00.029942 4110 mustload.go:65] Loading cluster: ha-857000
I0917 02:12:00.030121 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:12:00.030345 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:12:00.030368 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:12:00.039149 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52286
I0917 02:12:00.039489 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:12:00.039838 4110 main.go:141] libmachine: Using API Version 1
I0917 02:12:00.039856 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:12:00.040084 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:12:00.040206 4110 main.go:141] libmachine: (ha-857000) Calling .GetState
I0917 02:12:00.040304 4110 main.go:141] libmachine: (ha-857000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:12:00.040367 4110 main.go:141] libmachine: (ha-857000) DBG | hyperkit pid from json: 4124
I0917 02:12:00.041347 4110 host.go:66] Checking if "ha-857000" exists ...
I0917 02:12:00.041604 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:12:00.041629 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:12:00.050248 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52288
I0917 02:12:00.050590 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:12:00.050943 4110 main.go:141] libmachine: Using API Version 1
I0917 02:12:00.050963 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:12:00.051142 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:12:00.051249 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:12:00.051358 4110 certs.go:68] Setting up /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000 for IP: 192.169.0.7
I0917 02:12:00.051364 4110 certs.go:194] generating shared ca certs ...
I0917 02:12:00.051373 4110 certs.go:226] acquiring lock for ca certs: {Name:mkbc37b68ace578bc430db43facbca466a5a1602 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0917 02:12:00.051518 4110 certs.go:235] skipping valid "minikubeCA" ca cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key
I0917 02:12:00.051569 4110 certs.go:235] skipping valid "proxyClientCA" ca cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key
I0917 02:12:00.051578 4110 certs.go:256] generating profile certs ...
I0917 02:12:00.051672 4110 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/client.key
I0917 02:12:00.051762 4110 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.key.daf177bc
I0917 02:12:00.051812 4110 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.key
I0917 02:12:00.051819 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt -> /var/lib/minikube/certs/ca.crt
I0917 02:12:00.051841 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key -> /var/lib/minikube/certs/ca.key
I0917 02:12:00.051859 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.crt -> /var/lib/minikube/certs/proxy-client-ca.crt
I0917 02:12:00.051878 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key -> /var/lib/minikube/certs/proxy-client-ca.key
I0917 02:12:00.051895 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.crt -> /var/lib/minikube/certs/apiserver.crt
I0917 02:12:00.051919 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.key -> /var/lib/minikube/certs/apiserver.key
I0917 02:12:00.051943 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.crt -> /var/lib/minikube/certs/proxy-client.crt
I0917 02:12:00.051962 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.key -> /var/lib/minikube/certs/proxy-client.key
I0917 02:12:00.052037 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem (1338 bytes)
W0917 02:12:00.052085 4110 certs.go:480] ignoring /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560_empty.pem, impossibly tiny 0 bytes
I0917 02:12:00.052093 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem (1679 bytes)
I0917 02:12:00.052128 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem (1078 bytes)
I0917 02:12:00.052160 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem (1123 bytes)
I0917 02:12:00.052188 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem (1675 bytes)
I0917 02:12:00.052263 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem (1708 bytes)
I0917 02:12:00.052296 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> /usr/share/ca-certificates/15602.pem
I0917 02:12:00.052317 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt -> /usr/share/ca-certificates/minikubeCA.pem
I0917 02:12:00.052334 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem -> /usr/share/ca-certificates/1560.pem
I0917 02:12:00.052362 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHHostname
I0917 02:12:00.052450 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHPort
I0917 02:12:00.052535 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHKeyPath
I0917 02:12:00.052624 4110 main.go:141] libmachine: (ha-857000) Calling .GetSSHUsername
I0917 02:12:00.052722 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.5 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000/id_rsa Username:docker}
I0917 02:12:00.080096 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/sa.pub
I0917 02:12:00.083244 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/sa.pub --> memory (451 bytes)
I0917 02:12:00.090969 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/sa.key
I0917 02:12:00.094112 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/sa.key --> memory (1675 bytes)
I0917 02:12:00.101834 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/front-proxy-ca.crt
I0917 02:12:00.104986 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/front-proxy-ca.crt --> memory (1123 bytes)
I0917 02:12:00.113430 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/front-proxy-ca.key
I0917 02:12:00.116712 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/front-proxy-ca.key --> memory (1679 bytes)
I0917 02:12:00.124546 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/etcd/ca.crt
I0917 02:12:00.127709 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/etcd/ca.crt --> memory (1094 bytes)
I0917 02:12:00.135587 4110 ssh_runner.go:195] Run: stat -c %s /var/lib/minikube/certs/etcd/ca.key
I0917 02:12:00.138750 4110 ssh_runner.go:447] scp /var/lib/minikube/certs/etcd/ca.key --> memory (1679 bytes)
I0917 02:12:00.147884 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0917 02:12:00.168533 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0917 02:12:00.188900 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0917 02:12:00.208781 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0917 02:12:00.229275 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1440 bytes)
I0917 02:12:00.248994 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0917 02:12:00.269569 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0917 02:12:00.289646 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0917 02:12:00.309509 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem --> /usr/share/ca-certificates/15602.pem (1708 bytes)
I0917 02:12:00.329488 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0917 02:12:00.349487 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem --> /usr/share/ca-certificates/1560.pem (1338 bytes)
I0917 02:12:00.369414 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/sa.pub (451 bytes)
I0917 02:12:00.383327 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/sa.key (1675 bytes)
I0917 02:12:00.396803 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/front-proxy-ca.crt (1123 bytes)
I0917 02:12:00.410693 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/front-proxy-ca.key (1679 bytes)
I0917 02:12:00.424533 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/etcd/ca.crt (1094 bytes)
I0917 02:12:00.438144 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/certs/etcd/ca.key (1679 bytes)
I0917 02:12:00.451710 4110 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (744 bytes)
I0917 02:12:00.465698 4110 ssh_runner.go:195] Run: openssl version
I0917 02:12:00.470190 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/15602.pem && ln -fs /usr/share/ca-certificates/15602.pem /etc/ssl/certs/15602.pem"
I0917 02:12:00.478670 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/15602.pem
I0917 02:12:00.482005 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 17 08:55 /usr/share/ca-certificates/15602.pem
I0917 02:12:00.482051 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/15602.pem
I0917 02:12:00.486183 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/15602.pem /etc/ssl/certs/3ec20f2e.0"
I0917 02:12:00.494427 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0917 02:12:00.503098 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0917 02:12:00.506593 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 17 08:38 /usr/share/ca-certificates/minikubeCA.pem
I0917 02:12:00.506643 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0917 02:12:00.510950 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0917 02:12:00.519387 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1560.pem && ln -fs /usr/share/ca-certificates/1560.pem /etc/ssl/certs/1560.pem"
I0917 02:12:00.527796 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1560.pem
I0917 02:12:00.531174 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 17 08:55 /usr/share/ca-certificates/1560.pem
I0917 02:12:00.531231 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1560.pem
I0917 02:12:00.535528 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1560.pem /etc/ssl/certs/51391683.0"
I0917 02:12:00.543734 4110 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0917 02:12:00.547058 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0917 02:12:00.551336 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0917 02:12:00.555666 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0917 02:12:00.560095 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0917 02:12:00.564671 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0917 02:12:00.568907 4110 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0917 02:12:00.573116 4110 kubeadm.go:934] updating node {m03 192.169.0.7 8443 v1.31.1 docker true true} ...
I0917 02:12:00.573181 4110 kubeadm.go:946] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=ha-857000-m03 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.169.0.7
[Install]
config:
{KubernetesVersion:v1.31.1 ClusterName:ha-857000 Namespace:default APIServerHAVIP:192.169.0.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0917 02:12:00.573213 4110 kube-vip.go:115] generating kube-vip config ...
I0917 02:12:00.573252 4110 ssh_runner.go:195] Run: sudo sh -c "modprobe --all ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack"
I0917 02:12:00.585709 4110 kube-vip.go:167] auto-enabling control-plane load-balancing in kube-vip
I0917 02:12:00.585750 4110 kube-vip.go:137] kube-vip config:
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
name: kube-vip
namespace: kube-system
spec:
containers:
- args:
- manager
env:
- name: vip_arp
value: "true"
- name: port
value: "8443"
- name: vip_nodename
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: vip_interface
value: eth0
- name: vip_cidr
value: "32"
- name: dns_mode
value: first
- name: cp_enable
value: "true"
- name: cp_namespace
value: kube-system
- name: vip_leaderelection
value: "true"
- name: vip_leasename
value: plndr-cp-lock
- name: vip_leaseduration
value: "5"
- name: vip_renewdeadline
value: "3"
- name: vip_retryperiod
value: "1"
- name: address
value: 192.169.0.254
- name: prometheus_server
value: :2112
- name : lb_enable
value: "true"
- name: lb_port
value: "8443"
image: ghcr.io/kube-vip/kube-vip:v0.8.0
imagePullPolicy: IfNotPresent
name: kube-vip
resources: {}
securityContext:
capabilities:
add:
- NET_ADMIN
- NET_RAW
volumeMounts:
- mountPath: /etc/kubernetes/admin.conf
name: kubeconfig
hostAliases:
- hostnames:
- kubernetes
ip: 127.0.0.1
hostNetwork: true
volumes:
- hostPath:
path: "/etc/kubernetes/admin.conf"
name: kubeconfig
status: {}
I0917 02:12:00.585815 4110 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.1
I0917 02:12:00.593621 4110 binaries.go:44] Found k8s binaries, skipping transfer
I0917 02:12:00.593672 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /etc/kubernetes/manifests
I0917 02:12:00.600967 4110 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (311 bytes)
I0917 02:12:00.614925 4110 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0917 02:12:00.628761 4110 ssh_runner.go:362] scp memory --> /etc/kubernetes/manifests/kube-vip.yaml (1440 bytes)
I0917 02:12:00.642265 4110 ssh_runner.go:195] Run: grep 192.169.0.254 control-plane.minikube.internal$ /etc/hosts
I0917 02:12:00.645102 4110 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.169.0.254 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0917 02:12:00.654336 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:12:00.752482 4110 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0917 02:12:00.767122 4110 start.go:235] Will wait 6m0s for node &{Name:m03 IP:192.169.0.7 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:true Worker:true}
I0917 02:12:00.767316 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:12:00.788252 4110 out.go:177] * Verifying Kubernetes components...
I0917 02:12:00.808843 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:12:00.927434 4110 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0917 02:12:00.944321 4110 loader.go:395] Config loaded from file: /Users/jenkins/minikube-integration/19648-1025/kubeconfig
I0917 02:12:00.944565 4110 kapi.go:59] client config for ha-857000: &rest.Config{Host:"https://192.169.0.254:8443", APIPath:"", ContentConfig:rest.ContentConfig{AcceptContentTypes:"", ContentType:"", GroupVersion:(*schema.GroupVersion)(nil), NegotiatedSerializer:runtime.NegotiatedSerializer(nil)}, Username:"", Password:"", BearerToken:"", BearerTokenFile:"", Impersonate:rest.ImpersonationConfig{UserName:"", UID:"", Groups:[]string(nil), Extra:map[string][]string(nil)}, AuthProvider:<nil>, AuthConfigPersister:rest.AuthProviderConfigPersister(nil), ExecProvider:<nil>, TLSClientConfig:rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/client.crt", KeyFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/client.key", CAFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt", CertData:[]uint8(nil), KeyData:[]uint8(nil), CAData:[]uint8(nil), NextProtos:[]string(nil)}
, UserAgent:"", DisableCompression:false, Transport:http.RoundTripper(nil), WrapTransport:(transport.WrapperFunc)(0x673a720), QPS:0, Burst:0, RateLimiter:flowcontrol.RateLimiter(nil), WarningHandler:rest.WarningHandler(nil), Timeout:0, Dial:(func(context.Context, string, string) (net.Conn, error))(nil), Proxy:(func(*http.Request) (*url.URL, error))(nil)}
W0917 02:12:00.944614 4110 kubeadm.go:483] Overriding stale ClientConfig host https://192.169.0.254:8443 with https://192.169.0.5:8443
I0917 02:12:00.944789 4110 node_ready.go:35] waiting up to 6m0s for node "ha-857000-m03" to be "Ready" ...
I0917 02:12:00.944851 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:00.944858 4110 round_trippers.go:469] Request Headers:
I0917 02:12:00.944867 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:00.944872 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:00.946764 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:00.947061 4110 node_ready.go:49] node "ha-857000-m03" has status "Ready":"True"
I0917 02:12:00.947072 4110 node_ready.go:38] duration metric: took 2.273862ms for node "ha-857000-m03" to be "Ready" ...
I0917 02:12:00.947078 4110 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0917 02:12:00.947127 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods
I0917 02:12:00.947133 4110 round_trippers.go:469] Request Headers:
I0917 02:12:00.947139 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:00.947143 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:00.950970 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:00.956449 4110 pod_ready.go:79] waiting up to 6m0s for pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace to be "Ready" ...
I0917 02:12:00.956504 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:00.956511 4110 round_trippers.go:469] Request Headers:
I0917 02:12:00.956518 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:00.956526 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:00.959279 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:00.959653 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:00.959660 4110 round_trippers.go:469] Request Headers:
I0917 02:12:00.959666 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:00.959669 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:00.961657 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:00.962160 4110 pod_ready.go:93] pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:00.962170 4110 pod_ready.go:82] duration metric: took 5.706294ms for pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace to be "Ready" ...
I0917 02:12:00.962176 4110 pod_ready.go:79] waiting up to 6m0s for pod "coredns-7c65d6cfc9-nl5j5" in "kube-system" namespace to be "Ready" ...
I0917 02:12:00.962215 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-nl5j5
I0917 02:12:00.962221 4110 round_trippers.go:469] Request Headers:
I0917 02:12:00.962226 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:00.962230 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:00.966635 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:12:00.967113 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:00.967122 4110 round_trippers.go:469] Request Headers:
I0917 02:12:00.967128 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:00.967131 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:00.969231 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:00.969585 4110 pod_ready.go:93] pod "coredns-7c65d6cfc9-nl5j5" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:00.969594 4110 pod_ready.go:82] duration metric: took 7.413149ms for pod "coredns-7c65d6cfc9-nl5j5" in "kube-system" namespace to be "Ready" ...
I0917 02:12:00.969601 4110 pod_ready.go:79] waiting up to 6m0s for pod "etcd-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:00.969645 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/etcd-ha-857000
I0917 02:12:00.969650 4110 round_trippers.go:469] Request Headers:
I0917 02:12:00.969655 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:00.969659 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:00.971799 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:00.972247 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:00.972254 4110 round_trippers.go:469] Request Headers:
I0917 02:12:00.972264 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:00.972267 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:00.974411 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:00.974879 4110 pod_ready.go:93] pod "etcd-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:00.974888 4110 pod_ready.go:82] duration metric: took 5.282457ms for pod "etcd-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:00.974895 4110 pod_ready.go:79] waiting up to 6m0s for pod "etcd-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:00.974931 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/etcd-ha-857000-m02
I0917 02:12:00.974936 4110 round_trippers.go:469] Request Headers:
I0917 02:12:00.974941 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:00.974945 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:00.977288 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:00.977952 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:00.977959 4110 round_trippers.go:469] Request Headers:
I0917 02:12:00.977964 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:00.977966 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:00.980610 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:00.981051 4110 pod_ready.go:93] pod "etcd-ha-857000-m02" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:00.981061 4110 pod_ready.go:82] duration metric: took 6.161283ms for pod "etcd-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:00.981068 4110 pod_ready.go:79] waiting up to 6m0s for pod "etcd-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:01.146340 4110 request.go:632] Waited for 165.222252ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/etcd-ha-857000-m03
I0917 02:12:01.146408 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/etcd-ha-857000-m03
I0917 02:12:01.146414 4110 round_trippers.go:469] Request Headers:
I0917 02:12:01.146420 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:01.146423 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:01.148663 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:01.345119 4110 request.go:632] Waited for 196.038973ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:01.345177 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:01.345186 4110 round_trippers.go:469] Request Headers:
I0917 02:12:01.345198 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:01.345210 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:01.348611 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:01.349143 4110 pod_ready.go:93] pod "etcd-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:01.349154 4110 pod_ready.go:82] duration metric: took 368.067559ms for pod "etcd-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:01.349166 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:01.545007 4110 request.go:632] Waited for 195.782486ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000
I0917 02:12:01.545050 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000
I0917 02:12:01.545055 4110 round_trippers.go:469] Request Headers:
I0917 02:12:01.545061 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:01.545066 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:01.547602 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:01.745603 4110 request.go:632] Waited for 197.630153ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:01.745661 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:01.745667 4110 round_trippers.go:469] Request Headers:
I0917 02:12:01.745673 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:01.745676 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:01.748299 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:01.748902 4110 pod_ready.go:93] pod "kube-apiserver-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:01.748919 4110 pod_ready.go:82] duration metric: took 399.734114ms for pod "kube-apiserver-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:01.748926 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:01.945883 4110 request.go:632] Waited for 196.866004ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m02
I0917 02:12:01.945945 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m02
I0917 02:12:01.945954 4110 round_trippers.go:469] Request Headers:
I0917 02:12:01.945964 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:01.945969 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:01.951958 4110 round_trippers.go:574] Response Status: 200 OK in 5 milliseconds
I0917 02:12:02.145413 4110 request.go:632] Waited for 192.798684ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:02.145468 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:02.145478 4110 round_trippers.go:469] Request Headers:
I0917 02:12:02.145511 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:02.145520 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:02.148357 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:02.149190 4110 pod_ready.go:93] pod "kube-apiserver-ha-857000-m02" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:02.149203 4110 pod_ready.go:82] duration metric: took 400.265258ms for pod "kube-apiserver-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:02.149211 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:02.345683 4110 request.go:632] Waited for 196.426528ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:02.345728 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:02.345736 4110 round_trippers.go:469] Request Headers:
I0917 02:12:02.345744 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:02.345751 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:02.348508 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:02.544925 4110 request.go:632] Waited for 196.020856ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:02.544994 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:02.545000 4110 round_trippers.go:469] Request Headers:
I0917 02:12:02.545006 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:02.545009 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:02.547483 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:02.744993 4110 request.go:632] Waited for 95.563815ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:02.745048 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:02.745054 4110 round_trippers.go:469] Request Headers:
I0917 02:12:02.745061 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:02.745065 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:02.747122 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:02.945441 4110 request.go:632] Waited for 197.559126ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:02.945475 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:02.945480 4110 round_trippers.go:469] Request Headers:
I0917 02:12:02.945486 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:02.945491 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:02.948036 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:03.150936 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:03.150968 4110 round_trippers.go:469] Request Headers:
I0917 02:12:03.150975 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:03.150980 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:03.153272 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:03.346424 4110 request.go:632] Waited for 192.442992ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:03.346514 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:03.346521 4110 round_trippers.go:469] Request Headers:
I0917 02:12:03.346528 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:03.346533 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:03.350998 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:12:03.649774 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:03.649809 4110 round_trippers.go:469] Request Headers:
I0917 02:12:03.649818 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:03.649823 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:03.652931 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:03.744972 4110 request.go:632] Waited for 90.967061ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:03.745023 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:03.745029 4110 round_trippers.go:469] Request Headers:
I0917 02:12:03.745034 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:03.745039 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:03.747431 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:04.149979 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:04.150024 4110 round_trippers.go:469] Request Headers:
I0917 02:12:04.150033 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:04.150037 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:04.153328 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:04.153812 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:04.153822 4110 round_trippers.go:469] Request Headers:
I0917 02:12:04.153828 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:04.153832 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:04.156074 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:04.156716 4110 pod_ready.go:103] pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:04.650904 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:04.650924 4110 round_trippers.go:469] Request Headers:
I0917 02:12:04.650931 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:04.650946 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:04.653820 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:04.654378 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:04.654386 4110 round_trippers.go:469] Request Headers:
I0917 02:12:04.654393 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:04.654396 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:04.656654 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:05.151431 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:05.151485 4110 round_trippers.go:469] Request Headers:
I0917 02:12:05.151499 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:05.151506 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:05.154809 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:05.155323 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:05.155331 4110 round_trippers.go:469] Request Headers:
I0917 02:12:05.155337 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:05.155340 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:05.156965 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:05.650343 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:05.650367 4110 round_trippers.go:469] Request Headers:
I0917 02:12:05.650413 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:05.650421 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:05.653876 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:05.654508 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:05.654516 4110 round_trippers.go:469] Request Headers:
I0917 02:12:05.654522 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:05.654525 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:05.656260 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:06.149952 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:06.149982 4110 round_trippers.go:469] Request Headers:
I0917 02:12:06.149989 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:06.149994 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:06.152142 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:06.152594 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:06.152602 4110 round_trippers.go:469] Request Headers:
I0917 02:12:06.152608 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:06.152611 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:06.154378 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:06.650007 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:06.650040 4110 round_trippers.go:469] Request Headers:
I0917 02:12:06.650049 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:06.650053 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:06.652517 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:06.653131 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:06.653138 4110 round_trippers.go:469] Request Headers:
I0917 02:12:06.653144 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:06.653148 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:06.655153 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:06.655511 4110 pod_ready.go:103] pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:07.150612 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:07.150642 4110 round_trippers.go:469] Request Headers:
I0917 02:12:07.150678 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:07.150687 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:07.153805 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:07.154498 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:07.154508 4110 round_trippers.go:469] Request Headers:
I0917 02:12:07.154516 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:07.154521 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:07.156264 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:07.650356 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:07.650381 4110 round_trippers.go:469] Request Headers:
I0917 02:12:07.650392 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:07.650401 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:07.653535 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:07.653958 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:07.653966 4110 round_trippers.go:469] Request Headers:
I0917 02:12:07.653972 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:07.653975 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:07.656337 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:08.150386 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:08.150440 4110 round_trippers.go:469] Request Headers:
I0917 02:12:08.150452 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:08.150460 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:08.153584 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:08.155108 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:08.155123 4110 round_trippers.go:469] Request Headers:
I0917 02:12:08.155132 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:08.155143 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:08.157038 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:08.650349 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:08.650377 4110 round_trippers.go:469] Request Headers:
I0917 02:12:08.650389 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:08.650398 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:08.654034 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:08.654828 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:08.654836 4110 round_trippers.go:469] Request Headers:
I0917 02:12:08.654843 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:08.654846 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:08.656625 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:08.656928 4110 pod_ready.go:103] pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:09.151423 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:09.151447 4110 round_trippers.go:469] Request Headers:
I0917 02:12:09.151459 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:09.151464 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:09.154460 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:09.154947 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:09.154956 4110 round_trippers.go:469] Request Headers:
I0917 02:12:09.154961 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:09.154966 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:09.156555 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:09.650477 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:09.650503 4110 round_trippers.go:469] Request Headers:
I0917 02:12:09.650554 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:09.650568 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:09.653583 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:09.653960 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:09.653967 4110 round_trippers.go:469] Request Headers:
I0917 02:12:09.653973 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:09.653983 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:09.655828 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:10.149696 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:10.149720 4110 round_trippers.go:469] Request Headers:
I0917 02:12:10.149732 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:10.149739 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:10.153151 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:10.153716 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:10.153726 4110 round_trippers.go:469] Request Headers:
I0917 02:12:10.153734 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:10.153739 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:10.155758 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:10.649780 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:10.649830 4110 round_trippers.go:469] Request Headers:
I0917 02:12:10.649844 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:10.649854 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:10.653210 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:10.653938 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:10.653945 4110 round_trippers.go:469] Request Headers:
I0917 02:12:10.653951 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:10.653956 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:10.655718 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:11.149497 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:11.149512 4110 round_trippers.go:469] Request Headers:
I0917 02:12:11.149525 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:11.149530 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:11.151647 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:11.152174 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:11.152181 4110 round_trippers.go:469] Request Headers:
I0917 02:12:11.152187 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:11.152189 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:11.154098 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:11.154423 4110 pod_ready.go:103] pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:11.650969 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:11.650998 4110 round_trippers.go:469] Request Headers:
I0917 02:12:11.651032 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:11.651039 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:11.654171 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:11.654962 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:11.654969 4110 round_trippers.go:469] Request Headers:
I0917 02:12:11.654975 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:11.654979 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:11.656692 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:12.150871 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:12.150884 4110 round_trippers.go:469] Request Headers:
I0917 02:12:12.150890 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:12.150893 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:12.153079 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:12.153733 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:12.153741 4110 round_trippers.go:469] Request Headers:
I0917 02:12:12.153747 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:12.153751 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:12.155608 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:12.650611 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:12.650636 4110 round_trippers.go:469] Request Headers:
I0917 02:12:12.650674 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:12.650684 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:12.654409 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:12.654934 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:12.654941 4110 round_trippers.go:469] Request Headers:
I0917 02:12:12.654951 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:12.654954 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:12.656676 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:12.657136 4110 pod_ready.go:93] pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:12.657145 4110 pod_ready.go:82] duration metric: took 10.507747852s for pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:12.657152 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:12.657184 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000
I0917 02:12:12.657189 4110 round_trippers.go:469] Request Headers:
I0917 02:12:12.657194 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:12.657198 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:12.658893 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:12.659304 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:12.659312 4110 round_trippers.go:469] Request Headers:
I0917 02:12:12.659317 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:12.659321 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:12.660920 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:12.661222 4110 pod_ready.go:93] pod "kube-controller-manager-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:12.661230 4110 pod_ready.go:82] duration metric: took 4.073163ms for pod "kube-controller-manager-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:12.661237 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:12.661269 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m02
I0917 02:12:12.661274 4110 round_trippers.go:469] Request Headers:
I0917 02:12:12.661279 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:12.661282 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:12.662821 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:12.663178 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:12.663186 4110 round_trippers.go:469] Request Headers:
I0917 02:12:12.663192 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:12.663195 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:12.664635 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:12.665084 4110 pod_ready.go:93] pod "kube-controller-manager-ha-857000-m02" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:12.665092 4110 pod_ready.go:82] duration metric: took 3.849688ms for pod "kube-controller-manager-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:12.665098 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:12.665131 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:12:12.665136 4110 round_trippers.go:469] Request Headers:
I0917 02:12:12.665142 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:12.665157 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:12.666924 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:12.667551 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:12.667558 4110 round_trippers.go:469] Request Headers:
I0917 02:12:12.667564 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:12.667566 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:12.669116 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:13.165275 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:12:13.165342 4110 round_trippers.go:469] Request Headers:
I0917 02:12:13.165359 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:13.165367 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:13.168538 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:13.169042 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:13.169049 4110 round_trippers.go:469] Request Headers:
I0917 02:12:13.169054 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:13.169059 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:13.170903 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:13.665896 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:12:13.665914 4110 round_trippers.go:469] Request Headers:
I0917 02:12:13.665923 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:13.665930 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:13.668510 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:13.669059 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:13.669066 4110 round_trippers.go:469] Request Headers:
I0917 02:12:13.669071 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:13.669074 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:13.670842 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:14.165888 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:12:14.165910 4110 round_trippers.go:469] Request Headers:
I0917 02:12:14.165935 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:14.165941 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:14.168473 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:14.169111 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:14.169118 4110 round_trippers.go:469] Request Headers:
I0917 02:12:14.169124 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:14.169137 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:14.170994 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:14.667072 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:12:14.667128 4110 round_trippers.go:469] Request Headers:
I0917 02:12:14.667140 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:14.667151 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:14.670650 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:14.671210 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:14.671217 4110 round_trippers.go:469] Request Headers:
I0917 02:12:14.671222 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:14.671226 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:14.672859 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:14.673218 4110 pod_ready.go:103] pod "kube-controller-manager-ha-857000-m03" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:15.165335 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:12:15.165362 4110 round_trippers.go:469] Request Headers:
I0917 02:12:15.165375 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:15.165382 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:15.169212 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:15.169615 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:15.169623 4110 round_trippers.go:469] Request Headers:
I0917 02:12:15.169629 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:15.169633 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:15.171395 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:15.665422 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:12:15.665483 4110 round_trippers.go:469] Request Headers:
I0917 02:12:15.665498 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:15.665505 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:15.667889 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:15.668348 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:15.668356 4110 round_trippers.go:469] Request Headers:
I0917 02:12:15.668364 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:15.668369 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:15.670115 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:16.166085 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:12:16.166134 4110 round_trippers.go:469] Request Headers:
I0917 02:12:16.166147 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:16.166156 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:16.168879 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:16.169423 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:16.169430 4110 round_trippers.go:469] Request Headers:
I0917 02:12:16.169439 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:16.169442 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:16.171016 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:16.666749 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:12:16.666767 4110 round_trippers.go:469] Request Headers:
I0917 02:12:16.666797 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:16.666802 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:16.669480 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:16.669826 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:16.669832 4110 round_trippers.go:469] Request Headers:
I0917 02:12:16.669838 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:16.669842 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:16.671504 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:16.671930 4110 pod_ready.go:93] pod "kube-controller-manager-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:16.671939 4110 pod_ready.go:82] duration metric: took 4.006767511s for pod "kube-controller-manager-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:16.671955 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-528ht" in "kube-system" namespace to be "Ready" ...
I0917 02:12:16.671990 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-528ht
I0917 02:12:16.671995 4110 round_trippers.go:469] Request Headers:
I0917 02:12:16.672000 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:16.672005 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:16.673862 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:16.674451 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m04
I0917 02:12:16.674459 4110 round_trippers.go:469] Request Headers:
I0917 02:12:16.674464 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:16.674468 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:16.676355 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:16.676667 4110 pod_ready.go:93] pod "kube-proxy-528ht" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:16.676675 4110 pod_ready.go:82] duration metric: took 4.715112ms for pod "kube-proxy-528ht" in "kube-system" namespace to be "Ready" ...
I0917 02:12:16.676682 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-g9wxm" in "kube-system" namespace to be "Ready" ...
I0917 02:12:16.676724 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-g9wxm
I0917 02:12:16.676729 4110 round_trippers.go:469] Request Headers:
I0917 02:12:16.676734 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:16.676738 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:16.678611 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:16.678986 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:16.678993 4110 round_trippers.go:469] Request Headers:
I0917 02:12:16.678999 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:16.679003 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:16.680713 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:16.681034 4110 pod_ready.go:93] pod "kube-proxy-g9wxm" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:16.681043 4110 pod_ready.go:82] duration metric: took 4.356651ms for pod "kube-proxy-g9wxm" in "kube-system" namespace to be "Ready" ...
I0917 02:12:16.681050 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-vskbj" in "kube-system" namespace to be "Ready" ...
I0917 02:12:16.681091 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-vskbj
I0917 02:12:16.681097 4110 round_trippers.go:469] Request Headers:
I0917 02:12:16.681102 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:16.681106 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:16.682940 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:16.683445 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:16.683452 4110 round_trippers.go:469] Request Headers:
I0917 02:12:16.683458 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:16.683462 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:16.685017 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:16.685461 4110 pod_ready.go:93] pod "kube-proxy-vskbj" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:16.685470 4110 pod_ready.go:82] duration metric: took 4.414596ms for pod "kube-proxy-vskbj" in "kube-system" namespace to be "Ready" ...
I0917 02:12:16.685478 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-zrqvr" in "kube-system" namespace to be "Ready" ...
I0917 02:12:16.851971 4110 request.go:632] Waited for 166.418009ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-zrqvr
I0917 02:12:16.852035 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-zrqvr
I0917 02:12:16.852064 4110 round_trippers.go:469] Request Headers:
I0917 02:12:16.852076 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:16.852084 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:16.855683 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:17.050985 4110 request.go:632] Waited for 194.718198ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:17.051089 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:17.051098 4110 round_trippers.go:469] Request Headers:
I0917 02:12:17.051110 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:17.051119 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:17.054384 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:17.054876 4110 pod_ready.go:93] pod "kube-proxy-zrqvr" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:17.054889 4110 pod_ready.go:82] duration metric: took 369.398412ms for pod "kube-proxy-zrqvr" in "kube-system" namespace to be "Ready" ...
I0917 02:12:17.054898 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:17.250755 4110 request.go:632] Waited for 195.811261ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000
I0917 02:12:17.250805 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000
I0917 02:12:17.250817 4110 round_trippers.go:469] Request Headers:
I0917 02:12:17.250830 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:17.250841 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:17.291380 4110 round_trippers.go:574] Response Status: 200 OK in 40 milliseconds
I0917 02:12:17.450914 4110 request.go:632] Waited for 157.443488ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:17.450956 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:17.450990 4110 round_trippers.go:469] Request Headers:
I0917 02:12:17.450996 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:17.450999 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:17.455828 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:12:17.456276 4110 pod_ready.go:93] pod "kube-scheduler-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:17.456286 4110 pod_ready.go:82] duration metric: took 401.376038ms for pod "kube-scheduler-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:17.456294 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:17.651418 4110 request.go:632] Waited for 195.082221ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m02
I0917 02:12:17.651455 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m02
I0917 02:12:17.651461 4110 round_trippers.go:469] Request Headers:
I0917 02:12:17.651471 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:17.651495 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:17.668422 4110 round_trippers.go:574] Response Status: 200 OK in 16 milliseconds
I0917 02:12:17.850764 4110 request.go:632] Waited for 181.996065ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:17.850819 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:17.850825 4110 round_trippers.go:469] Request Headers:
I0917 02:12:17.850832 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:17.850836 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:17.857947 4110 round_trippers.go:574] Response Status: 200 OK in 7 milliseconds
I0917 02:12:17.858420 4110 pod_ready.go:93] pod "kube-scheduler-ha-857000-m02" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:17.858431 4110 pod_ready.go:82] duration metric: took 402.124989ms for pod "kube-scheduler-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:17.858439 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:18.051442 4110 request.go:632] Waited for 192.93696ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m03
I0917 02:12:18.051483 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m03
I0917 02:12:18.051491 4110 round_trippers.go:469] Request Headers:
I0917 02:12:18.051499 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:18.051512 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:18.054127 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:18.250926 4110 request.go:632] Waited for 196.199352ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:18.250961 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:18.250966 4110 round_trippers.go:469] Request Headers:
I0917 02:12:18.251003 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:18.251008 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:18.274920 4110 round_trippers.go:574] Response Status: 200 OK in 23 milliseconds
I0917 02:12:18.275585 4110 pod_ready.go:93] pod "kube-scheduler-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:18.275595 4110 pod_ready.go:82] duration metric: took 417.143356ms for pod "kube-scheduler-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:18.275606 4110 pod_ready.go:39] duration metric: took 17.328217726s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0917 02:12:18.275618 4110 api_server.go:52] waiting for apiserver process to appear ...
I0917 02:12:18.275688 4110 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0917 02:12:18.289040 4110 api_server.go:72] duration metric: took 17.521587147s to wait for apiserver process to appear ...
I0917 02:12:18.289060 4110 api_server.go:88] waiting for apiserver healthz status ...
I0917 02:12:18.289072 4110 api_server.go:253] Checking apiserver healthz at https://192.169.0.5:8443/healthz ...
I0917 02:12:18.292824 4110 api_server.go:279] https://192.169.0.5:8443/healthz returned 200:
ok
I0917 02:12:18.292862 4110 round_trippers.go:463] GET https://192.169.0.5:8443/version
I0917 02:12:18.292866 4110 round_trippers.go:469] Request Headers:
I0917 02:12:18.292872 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:18.292879 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:18.294137 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:18.294247 4110 api_server.go:141] control plane version: v1.31.1
I0917 02:12:18.294257 4110 api_server.go:131] duration metric: took 5.192363ms to wait for apiserver health ...
I0917 02:12:18.294263 4110 system_pods.go:43] waiting for kube-system pods to appear ...
I0917 02:12:18.451185 4110 request.go:632] Waited for 156.882548ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods
I0917 02:12:18.451216 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods
I0917 02:12:18.451222 4110 round_trippers.go:469] Request Headers:
I0917 02:12:18.451248 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:18.451254 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:18.490169 4110 round_trippers.go:574] Response Status: 200 OK in 38 milliseconds
I0917 02:12:18.505194 4110 system_pods.go:59] 26 kube-system pods found
I0917 02:12:18.505219 4110 system_pods.go:61] "coredns-7c65d6cfc9-fg65r" [1690cf49-3cd5-45ba-bcff-6c2947fb1bef] Running / Ready:ContainersNotReady (containers with unready status: [coredns]) / ContainersReady:ContainersNotReady (containers with unready status: [coredns])
I0917 02:12:18.505226 4110 system_pods.go:61] "coredns-7c65d6cfc9-nl5j5" [dad0f1c6-0feb-4024-b0ee-95776c68bae8] Running / Ready:ContainersNotReady (containers with unready status: [coredns]) / ContainersReady:ContainersNotReady (containers with unready status: [coredns])
I0917 02:12:18.505231 4110 system_pods.go:61] "etcd-ha-857000" [e9da37c5-ad0f-47e0-b65f-361d2cb9f204] Running
I0917 02:12:18.505234 4110 system_pods.go:61] "etcd-ha-857000-m02" [f540f85a-9556-44c0-a560-188721a58bd5] Running
I0917 02:12:18.505237 4110 system_pods.go:61] "etcd-ha-857000-m03" [46cb6c7e-73e2-49b0-986f-c63a23ffa29d] Running
I0917 02:12:18.505240 4110 system_pods.go:61] "kindnet-4jk9v" [24a018c6-9cbb-4d17-a295-8fef456534a0] Running
I0917 02:12:18.505244 4110 system_pods.go:61] "kindnet-7pf7v" [eecd1421-3a2f-4e48-b2b2-abcbef7869e7] Running / Ready:ContainersNotReady (containers with unready status: [kindnet-cni]) / ContainersReady:ContainersNotReady (containers with unready status: [kindnet-cni])
I0917 02:12:18.505247 4110 system_pods.go:61] "kindnet-vc6z5" [a92e41cb-74d1-4bd4-bffd-b807c859efd3] Running
I0917 02:12:18.505250 4110 system_pods.go:61] "kindnet-vh2h2" [00d83b6a-e87e-4c36-b073-36e67c76d67d] Running
I0917 02:12:18.505273 4110 system_pods.go:61] "kube-apiserver-ha-857000" [b5451c9b-3be2-4454-bed6-fdc48031180e] Running
I0917 02:12:18.505282 4110 system_pods.go:61] "kube-apiserver-ha-857000-m02" [d043a0e6-6ab2-47b6-bb82-ceff496f8336] Running
I0917 02:12:18.505290 4110 system_pods.go:61] "kube-apiserver-ha-857000-m03" [b3d91c89-8830-4dd9-8c20-4d6f821a3d88] Running
I0917 02:12:18.505313 4110 system_pods.go:61] "kube-controller-manager-ha-857000" [f4b0f56c-8d7d-4567-a4eb-088805c36c54] Running
I0917 02:12:18.505323 4110 system_pods.go:61] "kube-controller-manager-ha-857000-m02" [16670a48-0dc2-4665-9b1d-5fc25337ec58] Running
I0917 02:12:18.505338 4110 system_pods.go:61] "kube-controller-manager-ha-857000-m03" [754fdff4-0e27-4f32-ab12-3a6695924396] Running
I0917 02:12:18.505343 4110 system_pods.go:61] "kube-proxy-528ht" [18b29dac-e4bf-4f26-988f-b1ba4019f9bc] Running
I0917 02:12:18.505351 4110 system_pods.go:61] "kube-proxy-g9wxm" [a5b974f1-ed28-4f42-8c86-6fed0bf32317] Running
I0917 02:12:18.505361 4110 system_pods.go:61] "kube-proxy-vskbj" [7a396757-8954-48d2-b708-dcdfbab21dc7] Running / Ready:ContainersNotReady (containers with unready status: [kube-proxy]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-proxy])
I0917 02:12:18.505367 4110 system_pods.go:61] "kube-proxy-zrqvr" [bb98af15-e15a-4904-b68d-2de3c6e5864c] Running
I0917 02:12:18.505373 4110 system_pods.go:61] "kube-scheduler-ha-857000" [fdce329c-0340-4e0a-8bc1-295e4970708b] Running
I0917 02:12:18.505378 4110 system_pods.go:61] "kube-scheduler-ha-857000-m02" [274256f9-cd98-4dd3-befc-35443dcca033] Running
I0917 02:12:18.505384 4110 system_pods.go:61] "kube-scheduler-ha-857000-m03" [74d4633a-1c51-478d-9c68-d05c964089d9] Running
I0917 02:12:18.505388 4110 system_pods.go:61] "kube-vip-ha-857000" [84b805d8-9a8f-4c6f-b18f-76c98ca4776c] Running
I0917 02:12:18.505392 4110 system_pods.go:61] "kube-vip-ha-857000-m02" [b194d3e4-3b4d-4075-a6b1-f05d219393a0] Running
I0917 02:12:18.505396 4110 system_pods.go:61] "kube-vip-ha-857000-m03" [29be8efa-f99f-460a-80bd-ccc25f608e48] Running
I0917 02:12:18.505399 4110 system_pods.go:61] "storage-provisioner" [d81e7b55-a14e-4dc7-9193-ebe6914cdacf] Running
I0917 02:12:18.505406 4110 system_pods.go:74] duration metric: took 211.134036ms to wait for pod list to return data ...
I0917 02:12:18.505413 4110 default_sa.go:34] waiting for default service account to be created ...
I0917 02:12:18.650733 4110 request.go:632] Waited for 145.255733ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/default/serviceaccounts
I0917 02:12:18.650776 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/default/serviceaccounts
I0917 02:12:18.650782 4110 round_trippers.go:469] Request Headers:
I0917 02:12:18.650793 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:18.650798 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:18.659108 4110 round_trippers.go:574] Response Status: 200 OK in 8 milliseconds
I0917 02:12:18.659203 4110 default_sa.go:45] found service account: "default"
I0917 02:12:18.659217 4110 default_sa.go:55] duration metric: took 153.795915ms for default service account to be created ...
I0917 02:12:18.659227 4110 system_pods.go:116] waiting for k8s-apps to be running ...
I0917 02:12:18.851528 4110 request.go:632] Waited for 192.225662ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods
I0917 02:12:18.851585 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods
I0917 02:12:18.851591 4110 round_trippers.go:469] Request Headers:
I0917 02:12:18.851597 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:18.851600 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:18.855716 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:12:18.861599 4110 system_pods.go:86] 26 kube-system pods found
I0917 02:12:18.861618 4110 system_pods.go:89] "coredns-7c65d6cfc9-fg65r" [1690cf49-3cd5-45ba-bcff-6c2947fb1bef] Running / Ready:ContainersNotReady (containers with unready status: [coredns]) / ContainersReady:ContainersNotReady (containers with unready status: [coredns])
I0917 02:12:18.861630 4110 system_pods.go:89] "coredns-7c65d6cfc9-nl5j5" [dad0f1c6-0feb-4024-b0ee-95776c68bae8] Running / Ready:ContainersNotReady (containers with unready status: [coredns]) / ContainersReady:ContainersNotReady (containers with unready status: [coredns])
I0917 02:12:18.861635 4110 system_pods.go:89] "etcd-ha-857000" [e9da37c5-ad0f-47e0-b65f-361d2cb9f204] Running
I0917 02:12:18.861638 4110 system_pods.go:89] "etcd-ha-857000-m02" [f540f85a-9556-44c0-a560-188721a58bd5] Running
I0917 02:12:18.861642 4110 system_pods.go:89] "etcd-ha-857000-m03" [46cb6c7e-73e2-49b0-986f-c63a23ffa29d] Running
I0917 02:12:18.861645 4110 system_pods.go:89] "kindnet-4jk9v" [24a018c6-9cbb-4d17-a295-8fef456534a0] Running
I0917 02:12:18.861649 4110 system_pods.go:89] "kindnet-7pf7v" [eecd1421-3a2f-4e48-b2b2-abcbef7869e7] Running / Ready:ContainersNotReady (containers with unready status: [kindnet-cni]) / ContainersReady:ContainersNotReady (containers with unready status: [kindnet-cni])
I0917 02:12:18.861653 4110 system_pods.go:89] "kindnet-vc6z5" [a92e41cb-74d1-4bd4-bffd-b807c859efd3] Running
I0917 02:12:18.861657 4110 system_pods.go:89] "kindnet-vh2h2" [00d83b6a-e87e-4c36-b073-36e67c76d67d] Running
I0917 02:12:18.861660 4110 system_pods.go:89] "kube-apiserver-ha-857000" [b5451c9b-3be2-4454-bed6-fdc48031180e] Running
I0917 02:12:18.861663 4110 system_pods.go:89] "kube-apiserver-ha-857000-m02" [d043a0e6-6ab2-47b6-bb82-ceff496f8336] Running
I0917 02:12:18.861666 4110 system_pods.go:89] "kube-apiserver-ha-857000-m03" [b3d91c89-8830-4dd9-8c20-4d6f821a3d88] Running
I0917 02:12:18.861670 4110 system_pods.go:89] "kube-controller-manager-ha-857000" [f4b0f56c-8d7d-4567-a4eb-088805c36c54] Running
I0917 02:12:18.861673 4110 system_pods.go:89] "kube-controller-manager-ha-857000-m02" [16670a48-0dc2-4665-9b1d-5fc25337ec58] Running
I0917 02:12:18.861677 4110 system_pods.go:89] "kube-controller-manager-ha-857000-m03" [754fdff4-0e27-4f32-ab12-3a6695924396] Running
I0917 02:12:18.861682 4110 system_pods.go:89] "kube-proxy-528ht" [18b29dac-e4bf-4f26-988f-b1ba4019f9bc] Running
I0917 02:12:18.861685 4110 system_pods.go:89] "kube-proxy-g9wxm" [a5b974f1-ed28-4f42-8c86-6fed0bf32317] Running
I0917 02:12:18.861690 4110 system_pods.go:89] "kube-proxy-vskbj" [7a396757-8954-48d2-b708-dcdfbab21dc7] Running / Ready:ContainersNotReady (containers with unready status: [kube-proxy]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-proxy])
I0917 02:12:18.861694 4110 system_pods.go:89] "kube-proxy-zrqvr" [bb98af15-e15a-4904-b68d-2de3c6e5864c] Running
I0917 02:12:18.861698 4110 system_pods.go:89] "kube-scheduler-ha-857000" [fdce329c-0340-4e0a-8bc1-295e4970708b] Running
I0917 02:12:18.861701 4110 system_pods.go:89] "kube-scheduler-ha-857000-m02" [274256f9-cd98-4dd3-befc-35443dcca033] Running
I0917 02:12:18.861704 4110 system_pods.go:89] "kube-scheduler-ha-857000-m03" [74d4633a-1c51-478d-9c68-d05c964089d9] Running
I0917 02:12:18.861707 4110 system_pods.go:89] "kube-vip-ha-857000" [c577f2f1-ab99-4fbe-acc1-516a135f0377] Pending
I0917 02:12:18.861710 4110 system_pods.go:89] "kube-vip-ha-857000-m02" [b194d3e4-3b4d-4075-a6b1-f05d219393a0] Running
I0917 02:12:18.861713 4110 system_pods.go:89] "kube-vip-ha-857000-m03" [29be8efa-f99f-460a-80bd-ccc25f608e48] Running
I0917 02:12:18.861715 4110 system_pods.go:89] "storage-provisioner" [d81e7b55-a14e-4dc7-9193-ebe6914cdacf] Running
I0917 02:12:18.861720 4110 system_pods.go:126] duration metric: took 202.461636ms to wait for k8s-apps to be running ...
I0917 02:12:18.861726 4110 system_svc.go:44] waiting for kubelet service to be running ....
I0917 02:12:18.861778 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0917 02:12:18.882032 4110 system_svc.go:56] duration metric: took 20.298661ms WaitForService to wait for kubelet
I0917 02:12:18.882059 4110 kubeadm.go:582] duration metric: took 18.114595178s to wait for: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0917 02:12:18.882083 4110 node_conditions.go:102] verifying NodePressure condition ...
I0917 02:12:19.052878 4110 request.go:632] Waited for 170.643294ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes
I0917 02:12:19.052938 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes
I0917 02:12:19.052951 4110 round_trippers.go:469] Request Headers:
I0917 02:12:19.052966 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:19.052976 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:19.057011 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:12:19.057806 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:12:19.057817 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:12:19.057824 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:12:19.057827 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:12:19.057830 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:12:19.057834 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:12:19.057837 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:12:19.057840 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:12:19.057843 4110 node_conditions.go:105] duration metric: took 175.740836ms to run NodePressure ...
I0917 02:12:19.057851 4110 start.go:241] waiting for startup goroutines ...
I0917 02:12:19.057867 4110 start.go:255] writing updated cluster config ...
I0917 02:12:19.079978 4110 out.go:201]
I0917 02:12:19.117280 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:12:19.117377 4110 profile.go:143] Saving config to /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/config.json ...
I0917 02:12:19.138898 4110 out.go:177] * Starting "ha-857000-m04" worker node in "ha-857000" cluster
I0917 02:12:19.180945 4110 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime docker
I0917 02:12:19.180969 4110 cache.go:56] Caching tarball of preloaded images
I0917 02:12:19.181086 4110 preload.go:172] Found /Users/jenkins/minikube-integration/19648-1025/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0917 02:12:19.181097 4110 cache.go:59] Finished verifying existence of preloaded tar for v1.31.1 on docker
I0917 02:12:19.181167 4110 profile.go:143] Saving config to /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/config.json ...
I0917 02:12:19.181757 4110 start.go:360] acquireMachinesLock for ha-857000-m04: {Name:mkcb27accfc361dc9c175eeb8bb69dc9967aa557 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0917 02:12:19.181807 4110 start.go:364] duration metric: took 37.353µs to acquireMachinesLock for "ha-857000-m04"
I0917 02:12:19.181825 4110 start.go:96] Skipping create...Using existing machine configuration
I0917 02:12:19.181830 4110 fix.go:54] fixHost starting: m04
I0917 02:12:19.182086 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:12:19.182106 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:12:19.191065 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52292
I0917 02:12:19.191452 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:12:19.191850 4110 main.go:141] libmachine: Using API Version 1
I0917 02:12:19.191867 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:12:19.192069 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:12:19.192186 4110 main.go:141] libmachine: (ha-857000-m04) Calling .DriverName
I0917 02:12:19.192279 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetState
I0917 02:12:19.192404 4110 main.go:141] libmachine: (ha-857000-m04) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:12:19.192500 4110 main.go:141] libmachine: (ha-857000-m04) DBG | hyperkit pid from json: 3550
I0917 02:12:19.193450 4110 main.go:141] libmachine: (ha-857000-m04) DBG | hyperkit pid 3550 missing from process table
I0917 02:12:19.193488 4110 fix.go:112] recreateIfNeeded on ha-857000-m04: state=Stopped err=<nil>
I0917 02:12:19.193498 4110 main.go:141] libmachine: (ha-857000-m04) Calling .DriverName
W0917 02:12:19.193587 4110 fix.go:138] unexpected machine state, will restart: <nil>
I0917 02:12:19.214824 4110 out.go:177] * Restarting existing hyperkit VM for "ha-857000-m04" ...
I0917 02:12:19.289023 4110 main.go:141] libmachine: (ha-857000-m04) Calling .Start
I0917 02:12:19.289295 4110 main.go:141] libmachine: (ha-857000-m04) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:12:19.289356 4110 main.go:141] libmachine: (ha-857000-m04) DBG | clean start, hyperkit pid file doesn't exist: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/hyperkit.pid
I0917 02:12:19.289453 4110 main.go:141] libmachine: (ha-857000-m04) DBG | Using UUID 32bc812d-06ce-423b-90a4-5417ea5ec912
I0917 02:12:19.319068 4110 main.go:141] libmachine: (ha-857000-m04) DBG | Generated MAC a:b6:8:34:25:a6
I0917 02:12:19.319111 4110 main.go:141] libmachine: (ha-857000-m04) DBG | Starting with cmdline: loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000
I0917 02:12:19.319291 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 DEBUG: hyperkit: Start &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"32bc812d-06ce-423b-90a4-5417ea5ec912", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc000289980)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/bzimage", Initrd:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os.Process)(nil)}
I0917 02:12:19.319339 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 DEBUG: hyperkit: check &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"32bc812d-06ce-423b-90a4-5417ea5ec912", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc000289980)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/bzimage", Initrd:"/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil), CmdLine:"", process:(*os.Process)(nil)}
I0917 02:12:19.319395 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 DEBUG: hyperkit: Arguments: []string{"-A", "-u", "-F", "/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/hyperkit.pid", "-c", "2", "-m", "2200M", "-s", "0:0,hostbridge", "-s", "31,lpc", "-s", "1:0,virtio-net", "-U", "32bc812d-06ce-423b-90a4-5417ea5ec912", "-s", "2:0,virtio-blk,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/ha-857000-m04.rawdisk", "-s", "3,ahci-cd,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/boot2docker.iso", "-s", "4,virtio-rnd", "-l", "com1,autopty=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/tty,log=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/console-ring", "-f", "kexec,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/bzimage,/Users/jenkins/minikube-integration/19648-1025/.minikube/machine
s/ha-857000-m04/initrd,earlyprintk=serial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000"}
I0917 02:12:19.319498 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 DEBUG: hyperkit: CmdLine: "/usr/local/bin/hyperkit -A -u -F /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/hyperkit.pid -c 2 -m 2200M -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-net -U 32bc812d-06ce-423b-90a4-5417ea5ec912 -s 2:0,virtio-blk,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/ha-857000-m04.rawdisk -s 3,ahci-cd,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/boot2docker.iso -s 4,virtio-rnd -l com1,autopty=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/tty,log=/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/console-ring -f kexec,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/bzimage,/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/initrd,earlyprintk=serial loglevel=3 console=t
tyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=ha-857000"
I0917 02:12:19.319538 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 DEBUG: hyperkit: Redirecting stdout/stderr to logger
I0917 02:12:19.321260 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 DEBUG: hyperkit: Pid is 4161
I0917 02:12:19.321886 4110 main.go:141] libmachine: (ha-857000-m04) DBG | Attempt 0
I0917 02:12:19.321908 4110 main.go:141] libmachine: (ha-857000-m04) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:12:19.321989 4110 main.go:141] libmachine: (ha-857000-m04) DBG | hyperkit pid from json: 4161
I0917 02:12:19.324366 4110 main.go:141] libmachine: (ha-857000-m04) DBG | Searching for a:b6:8:34:25:a6 in /var/db/dhcpd_leases ...
I0917 02:12:19.324461 4110 main.go:141] libmachine: (ha-857000-m04) DBG | Found 7 entries in /var/db/dhcpd_leases!
I0917 02:12:19.324494 4110 main.go:141] libmachine: (ha-857000-m04) DBG | dhcp entry: {Name:minikube IPAddress:192.169.0.7 HWAddress:16:4d:1d:5e:91:c8 ID:1,16:4d:1d:5e:91:c8 Lease:0x66ea9957}
I0917 02:12:19.324519 4110 main.go:141] libmachine: (ha-857000-m04) DBG | dhcp entry: {Name:minikube IPAddress:192.169.0.6 HWAddress:9a:95:4e:4b:65:fe ID:1,9a:95:4e:4b:65:fe Lease:0x66ea991f}
I0917 02:12:19.324537 4110 main.go:141] libmachine: (ha-857000-m04) DBG | dhcp entry: {Name:minikube IPAddress:192.169.0.5 HWAddress:c2:63:2b:63:80:76 ID:1,c2:63:2b:63:80:76 Lease:0x66ea990c}
I0917 02:12:19.324552 4110 main.go:141] libmachine: (ha-857000-m04) DBG | dhcp entry: {Name:minikube IPAddress:192.169.0.8 HWAddress:a:b6:8:34:25:a6 ID:1,a:b6:8:34:25:a6 Lease:0x66e94661}
I0917 02:12:19.324565 4110 main.go:141] libmachine: (ha-857000-m04) DBG | Found match: a:b6:8:34:25:a6
I0917 02:12:19.324580 4110 main.go:141] libmachine: (ha-857000-m04) DBG | IP: 192.169.0.8
I0917 02:12:19.324586 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetConfigRaw
I0917 02:12:19.325317 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetIP
I0917 02:12:19.325565 4110 profile.go:143] Saving config to /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/config.json ...
I0917 02:12:19.326089 4110 machine.go:93] provisionDockerMachine start ...
I0917 02:12:19.326109 4110 main.go:141] libmachine: (ha-857000-m04) Calling .DriverName
I0917 02:12:19.326263 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:19.326401 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:19.326560 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:19.326727 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:19.326852 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:19.327048 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:12:19.327215 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.8 22 <nil> <nil>}
I0917 02:12:19.327223 4110 main.go:141] libmachine: About to run SSH command:
hostname
I0917 02:12:19.329900 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: Using fd 5 for I/O notifications
I0917 02:12:19.339917 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/boot2docker.iso: fcntl(F_PUNCHHOLE) Operation not permitted: block device will not support TRIM/DISCARD
I0917 02:12:19.340861 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0
I0917 02:12:19.340880 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0
I0917 02:12:19.340887 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0
I0917 02:12:19.340906 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0
I0917 02:12:19.732737 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 0
I0917 02:12:19.732752 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 0
I0917 02:12:19.847625 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0
I0917 02:12:19.847643 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0
I0917 02:12:19.847688 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0
I0917 02:12:19.847715 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0
I0917 02:12:19.848483 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: rdmsr to register 0x3a on vcpu 1
I0917 02:12:19.848501 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:19 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 1
I0917 02:12:25.591852 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:25 INFO : hyperkit: stderr: rdmsr to register 0x64d on vcpu 1
I0917 02:12:25.591915 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:25 INFO : hyperkit: stderr: rdmsr to register 0x64e on vcpu 1
I0917 02:12:25.591925 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:25 INFO : hyperkit: stderr: rdmsr to register 0x34 on vcpu 1
I0917 02:12:25.615174 4110 main.go:141] libmachine: (ha-857000-m04) DBG | 2024/09/17 02:12:25 INFO : hyperkit: stderr: rdmsr to register 0xc0011029 on vcpu 1
I0917 02:12:29.572071 4110 main.go:141] libmachine: Error dialing TCP: dial tcp 192.169.0.8:22: connect: connection refused
I0917 02:12:32.627647 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: minikube
I0917 02:12:32.627664 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetMachineName
I0917 02:12:32.627799 4110 buildroot.go:166] provisioning hostname "ha-857000-m04"
I0917 02:12:32.627808 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetMachineName
I0917 02:12:32.627920 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:32.628014 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:32.628110 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:32.628210 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:32.628294 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:32.628431 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:12:32.628580 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.8 22 <nil> <nil>}
I0917 02:12:32.628587 4110 main.go:141] libmachine: About to run SSH command:
sudo hostname ha-857000-m04 && echo "ha-857000-m04" | sudo tee /etc/hostname
I0917 02:12:32.692963 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: ha-857000-m04
I0917 02:12:32.692980 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:32.693102 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:32.693193 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:32.693281 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:32.693375 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:32.693517 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:12:32.693670 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.8 22 <nil> <nil>}
I0917 02:12:32.693680 4110 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sha-857000-m04' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 ha-857000-m04/g' /etc/hosts;
else
echo '127.0.1.1 ha-857000-m04' | sudo tee -a /etc/hosts;
fi
fi
I0917 02:12:32.753597 4110 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0917 02:12:32.753613 4110 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/19648-1025/.minikube CaCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/19648-1025/.minikube}
I0917 02:12:32.753629 4110 buildroot.go:174] setting up certificates
I0917 02:12:32.753635 4110 provision.go:84] configureAuth start
I0917 02:12:32.753642 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetMachineName
I0917 02:12:32.753783 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetIP
I0917 02:12:32.753886 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:32.753973 4110 provision.go:143] copyHostCerts
I0917 02:12:32.754002 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem
I0917 02:12:32.754055 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem, removing ...
I0917 02:12:32.754061 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem
I0917 02:12:32.754199 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.pem (1078 bytes)
I0917 02:12:32.754425 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem
I0917 02:12:32.754455 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem, removing ...
I0917 02:12:32.754465 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem
I0917 02:12:32.754535 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/cert.pem (1123 bytes)
I0917 02:12:32.754684 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem -> /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem
I0917 02:12:32.754713 4110 exec_runner.go:144] found /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem, removing ...
I0917 02:12:32.754717 4110 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem
I0917 02:12:32.754781 4110 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/19648-1025/.minikube/key.pem (1675 bytes)
I0917 02:12:32.754925 4110 provision.go:117] generating server cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem org=jenkins.ha-857000-m04 san=[127.0.0.1 192.169.0.8 ha-857000-m04 localhost minikube]
I0917 02:12:32.886815 4110 provision.go:177] copyRemoteCerts
I0917 02:12:32.886883 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0917 02:12:32.886900 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:32.887049 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:32.887156 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:32.887265 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:32.887345 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.8 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/id_rsa Username:docker}
I0917 02:12:32.921412 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem -> /etc/docker/ca.pem
I0917 02:12:32.921483 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0917 02:12:32.942093 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem -> /etc/docker/server.pem
I0917 02:12:32.942165 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes)
I0917 02:12:32.962202 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem -> /etc/docker/server-key.pem
I0917 02:12:32.962278 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0917 02:12:32.982539 4110 provision.go:87] duration metric: took 228.892121ms to configureAuth
I0917 02:12:32.982555 4110 buildroot.go:189] setting minikube options for container-runtime
I0917 02:12:32.982734 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:12:32.982747 4110 main.go:141] libmachine: (ha-857000-m04) Calling .DriverName
I0917 02:12:32.982882 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:32.982965 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:32.983053 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:32.983146 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:32.983222 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:32.983341 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:12:32.983471 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.8 22 <nil> <nil>}
I0917 02:12:32.983479 4110 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0917 02:12:33.039112 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs
I0917 02:12:33.039126 4110 buildroot.go:70] root file system type: tmpfs
I0917 02:12:33.039209 4110 provision.go:314] Updating docker unit: /lib/systemd/system/docker.service ...
I0917 02:12:33.039225 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:33.039356 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:33.039463 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:33.039553 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:33.039642 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:33.039765 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:12:33.039901 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.8 22 <nil> <nil>}
I0917 02:12:33.039948 4110 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
Environment="NO_PROXY=192.169.0.5"
Environment="NO_PROXY=192.169.0.5,192.169.0.6"
Environment="NO_PROXY=192.169.0.5,192.169.0.6,192.169.0.7"
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0917 02:12:33.105290 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
Environment=NO_PROXY=192.169.0.5
Environment=NO_PROXY=192.169.0.5,192.169.0.6
Environment=NO_PROXY=192.169.0.5,192.169.0.6,192.169.0.7
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0917 02:12:33.105311 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:33.105463 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:33.105557 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:33.105679 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:33.105803 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:33.106006 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:12:33.106166 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.8 22 <nil> <nil>}
I0917 02:12:33.106179 4110 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0917 02:12:34.690044 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
I0917 02:12:34.690061 4110 machine.go:96] duration metric: took 15.363692529s to provisionDockerMachine
I0917 02:12:34.690069 4110 start.go:293] postStartSetup for "ha-857000-m04" (driver="hyperkit")
I0917 02:12:34.690105 4110 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0917 02:12:34.690128 4110 main.go:141] libmachine: (ha-857000-m04) Calling .DriverName
I0917 02:12:34.690331 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0917 02:12:34.690344 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:34.690448 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:34.690550 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:34.690643 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:34.690734 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.8 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/id_rsa Username:docker}
I0917 02:12:34.729693 4110 ssh_runner.go:195] Run: cat /etc/os-release
I0917 02:12:34.733386 4110 info.go:137] Remote host: Buildroot 2023.02.9
I0917 02:12:34.733399 4110 filesync.go:126] Scanning /Users/jenkins/minikube-integration/19648-1025/.minikube/addons for local assets ...
I0917 02:12:34.733491 4110 filesync.go:126] Scanning /Users/jenkins/minikube-integration/19648-1025/.minikube/files for local assets ...
I0917 02:12:34.733629 4110 filesync.go:149] local asset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> 15602.pem in /etc/ssl/certs
I0917 02:12:34.733635 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> /etc/ssl/certs/15602.pem
I0917 02:12:34.733801 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0917 02:12:34.743555 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem --> /etc/ssl/certs/15602.pem (1708 bytes)
I0917 02:12:34.777005 4110 start.go:296] duration metric: took 86.908647ms for postStartSetup
I0917 02:12:34.777029 4110 main.go:141] libmachine: (ha-857000-m04) Calling .DriverName
I0917 02:12:34.777213 4110 ssh_runner.go:195] Run: sudo ls --almost-all -1 /var/lib/minikube/backup
I0917 02:12:34.777227 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:34.777324 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:34.777401 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:34.777484 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:34.777560 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.8 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/id_rsa Username:docker}
I0917 02:12:34.811015 4110 machine.go:197] restoring vm config from /var/lib/minikube/backup: [etc]
I0917 02:12:34.811085 4110 ssh_runner.go:195] Run: sudo rsync --archive --update /var/lib/minikube/backup/etc /
I0917 02:12:34.865249 4110 fix.go:56] duration metric: took 15.683145042s for fixHost
I0917 02:12:34.865277 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:34.865435 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:34.865528 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:34.865626 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:34.865720 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:34.865866 4110 main.go:141] libmachine: Using SSH client type: native
I0917 02:12:34.866008 4110 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x5064820] 0x5067500 <nil> [] 0s} 192.169.0.8 22 <nil> <nil>}
I0917 02:12:34.866017 4110 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0917 02:12:34.922683 4110 main.go:141] libmachine: SSH cmd err, output: <nil>: 1726564355.020144093
I0917 02:12:34.922697 4110 fix.go:216] guest clock: 1726564355.020144093
I0917 02:12:34.922703 4110 fix.go:229] Guest: 2024-09-17 02:12:35.020144093 -0700 PDT Remote: 2024-09-17 02:12:34.865267 -0700 PDT m=+127.793621612 (delta=154.877093ms)
I0917 02:12:34.922714 4110 fix.go:200] guest clock delta is within tolerance: 154.877093ms
I0917 02:12:34.922718 4110 start.go:83] releasing machines lock for "ha-857000-m04", held for 15.740632652s
I0917 02:12:34.922744 4110 main.go:141] libmachine: (ha-857000-m04) Calling .DriverName
I0917 02:12:34.922875 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetIP
I0917 02:12:34.945234 4110 out.go:177] * Found network options:
I0917 02:12:34.965134 4110 out.go:177] - NO_PROXY=192.169.0.5,192.169.0.6,192.169.0.7
W0917 02:12:34.986412 4110 proxy.go:119] fail to check proxy env: Error ip not in block
W0917 02:12:34.986446 4110 proxy.go:119] fail to check proxy env: Error ip not in block
W0917 02:12:34.986459 4110 proxy.go:119] fail to check proxy env: Error ip not in block
I0917 02:12:34.986477 4110 main.go:141] libmachine: (ha-857000-m04) Calling .DriverName
I0917 02:12:34.987363 4110 main.go:141] libmachine: (ha-857000-m04) Calling .DriverName
I0917 02:12:34.987619 4110 main.go:141] libmachine: (ha-857000-m04) Calling .DriverName
I0917 02:12:34.987838 4110 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
W0917 02:12:34.987863 4110 proxy.go:119] fail to check proxy env: Error ip not in block
I0917 02:12:34.987882 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
W0917 02:12:34.987901 4110 proxy.go:119] fail to check proxy env: Error ip not in block
W0917 02:12:34.987917 4110 proxy.go:119] fail to check proxy env: Error ip not in block
I0917 02:12:34.988015 4110 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0917 02:12:34.988040 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHHostname
I0917 02:12:34.988144 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:34.988241 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHPort
I0917 02:12:34.988362 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:34.988430 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHKeyPath
I0917 02:12:34.988562 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:34.988636 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetSSHUsername
I0917 02:12:34.988712 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.8 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/id_rsa Username:docker}
I0917 02:12:34.988798 4110 sshutil.go:53] new ssh client: &{IP:192.169.0.8 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/19648-1025/.minikube/machines/ha-857000-m04/id_rsa Username:docker}
W0917 02:12:35.089466 4110 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0917 02:12:35.089538 4110 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0917 02:12:35.103798 4110 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0917 02:12:35.103814 4110 start.go:495] detecting cgroup driver to use...
I0917 02:12:35.103888 4110 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0917 02:12:35.122855 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0917 02:12:35.131456 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0917 02:12:35.140120 4110 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0917 02:12:35.140187 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0917 02:12:35.148614 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0917 02:12:35.156897 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0917 02:12:35.165192 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0917 02:12:35.173754 4110 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0917 02:12:35.182471 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0917 02:12:35.191008 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0917 02:12:35.199448 4110 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0917 02:12:35.207926 4110 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0917 02:12:35.216411 4110 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0917 02:12:35.228568 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:12:35.327014 4110 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0917 02:12:35.346549 4110 start.go:495] detecting cgroup driver to use...
I0917 02:12:35.346628 4110 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0917 02:12:35.370011 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0917 02:12:35.382502 4110 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0917 02:12:35.397499 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0917 02:12:35.408840 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0917 02:12:35.420206 4110 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0917 02:12:35.442422 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0917 02:12:35.453508 4110 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/cri-dockerd.sock
" | sudo tee /etc/crictl.yaml"
I0917 02:12:35.468375 4110 ssh_runner.go:195] Run: which cri-dockerd
I0917 02:12:35.471279 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I0917 02:12:35.479407 4110 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (190 bytes)
I0917 02:12:35.492955 4110 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0917 02:12:35.593589 4110 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0917 02:12:35.695477 4110 docker.go:574] configuring docker to use "cgroupfs" as cgroup driver...
I0917 02:12:35.695504 4110 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)
I0917 02:12:35.710594 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:12:35.826600 4110 ssh_runner.go:195] Run: sudo systemctl restart docker
I0917 02:12:38.101010 4110 ssh_runner.go:235] Completed: sudo systemctl restart docker: (2.274345081s)
I0917 02:12:38.101138 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.socket
I0917 02:12:38.113882 4110 ssh_runner.go:195] Run: sudo systemctl stop cri-docker.socket
I0917 02:12:38.128373 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0917 02:12:38.140107 4110 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket
I0917 02:12:38.249684 4110 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket
I0917 02:12:38.361672 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:12:38.469978 4110 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket
I0917 02:12:38.489760 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0917 02:12:38.502395 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:12:38.604591 4110 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.service
I0917 02:12:38.669590 4110 start.go:542] Will wait 60s for socket path /var/run/cri-dockerd.sock
I0917 02:12:38.669684 4110 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock
I0917 02:12:38.674420 4110 start.go:563] Will wait 60s for crictl version
I0917 02:12:38.674483 4110 ssh_runner.go:195] Run: which crictl
I0917 02:12:38.677707 4110 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0917 02:12:38.702126 4110 start.go:579] Version: 0.1.0
RuntimeName: docker
RuntimeVersion: 27.2.1
RuntimeApiVersion: v1
I0917 02:12:38.702225 4110 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0917 02:12:38.719390 4110 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0917 02:12:38.757457 4110 out.go:235] * Preparing Kubernetes v1.31.1 on Docker 27.2.1 ...
I0917 02:12:38.799117 4110 out.go:177] - env NO_PROXY=192.169.0.5
I0917 02:12:38.819990 4110 out.go:177] - env NO_PROXY=192.169.0.5,192.169.0.6
I0917 02:12:38.841085 4110 out.go:177] - env NO_PROXY=192.169.0.5,192.169.0.6,192.169.0.7
I0917 02:12:38.862007 4110 main.go:141] libmachine: (ha-857000-m04) Calling .GetIP
I0917 02:12:38.862240 4110 ssh_runner.go:195] Run: grep 192.169.0.1 host.minikube.internal$ /etc/hosts
I0917 02:12:38.865326 4110 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.169.0.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0917 02:12:38.874823 4110 mustload.go:65] Loading cluster: ha-857000
I0917 02:12:38.875009 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:12:38.875239 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:12:38.875265 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:12:38.884252 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52315
I0917 02:12:38.884596 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:12:38.885007 4110 main.go:141] libmachine: Using API Version 1
I0917 02:12:38.885024 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:12:38.885217 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:12:38.885327 4110 main.go:141] libmachine: (ha-857000) Calling .GetState
I0917 02:12:38.885411 4110 main.go:141] libmachine: (ha-857000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0917 02:12:38.885502 4110 main.go:141] libmachine: (ha-857000) DBG | hyperkit pid from json: 4124
I0917 02:12:38.886472 4110 host.go:66] Checking if "ha-857000" exists ...
I0917 02:12:38.886740 4110 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0917 02:12:38.886764 4110 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0917 02:12:38.895399 4110 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52317
I0917 02:12:38.895752 4110 main.go:141] libmachine: () Calling .GetVersion
I0917 02:12:38.896084 4110 main.go:141] libmachine: Using API Version 1
I0917 02:12:38.896095 4110 main.go:141] libmachine: () Calling .SetConfigRaw
I0917 02:12:38.896312 4110 main.go:141] libmachine: () Calling .GetMachineName
I0917 02:12:38.896445 4110 main.go:141] libmachine: (ha-857000) Calling .DriverName
I0917 02:12:38.896532 4110 certs.go:68] Setting up /Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000 for IP: 192.169.0.8
I0917 02:12:38.896538 4110 certs.go:194] generating shared ca certs ...
I0917 02:12:38.896550 4110 certs.go:226] acquiring lock for ca certs: {Name:mkbc37b68ace578bc430db43facbca466a5a1602 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0917 02:12:38.896701 4110 certs.go:235] skipping valid "minikubeCA" ca cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key
I0917 02:12:38.896754 4110 certs.go:235] skipping valid "proxyClientCA" ca cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key
I0917 02:12:38.896764 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt -> /var/lib/minikube/certs/ca.crt
I0917 02:12:38.896789 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key -> /var/lib/minikube/certs/ca.key
I0917 02:12:38.896809 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.crt -> /var/lib/minikube/certs/proxy-client-ca.crt
I0917 02:12:38.896826 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key -> /var/lib/minikube/certs/proxy-client-ca.key
I0917 02:12:38.896910 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem (1338 bytes)
W0917 02:12:38.896963 4110 certs.go:480] ignoring /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560_empty.pem, impossibly tiny 0 bytes
I0917 02:12:38.896974 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca-key.pem (1679 bytes)
I0917 02:12:38.897008 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/ca.pem (1078 bytes)
I0917 02:12:38.897042 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/cert.pem (1123 bytes)
I0917 02:12:38.897070 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/key.pem (1675 bytes)
I0917 02:12:38.897139 4110 certs.go:484] found cert: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem (1708 bytes)
I0917 02:12:38.897176 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem -> /usr/share/ca-certificates/1560.pem
I0917 02:12:38.897196 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem -> /usr/share/ca-certificates/15602.pem
I0917 02:12:38.897214 4110 vm_assets.go:164] NewFileAsset: /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt -> /usr/share/ca-certificates/minikubeCA.pem
I0917 02:12:38.897242 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0917 02:12:38.917488 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0917 02:12:38.937120 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0917 02:12:38.956856 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0917 02:12:38.976762 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/certs/1560.pem --> /usr/share/ca-certificates/1560.pem (1338 bytes)
I0917 02:12:38.997198 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/files/etc/ssl/certs/15602.pem --> /usr/share/ca-certificates/15602.pem (1708 bytes)
I0917 02:12:39.018037 4110 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0917 02:12:39.040033 4110 ssh_runner.go:195] Run: openssl version
I0917 02:12:39.044757 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/15602.pem && ln -fs /usr/share/ca-certificates/15602.pem /etc/ssl/certs/15602.pem"
I0917 02:12:39.053844 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/15602.pem
I0917 02:12:39.057290 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 17 08:55 /usr/share/ca-certificates/15602.pem
I0917 02:12:39.057337 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/15602.pem
I0917 02:12:39.061592 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/15602.pem /etc/ssl/certs/3ec20f2e.0"
I0917 02:12:39.070092 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0917 02:12:39.078554 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0917 02:12:39.082016 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 17 08:38 /usr/share/ca-certificates/minikubeCA.pem
I0917 02:12:39.082086 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0917 02:12:39.086282 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0917 02:12:39.094779 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1560.pem && ln -fs /usr/share/ca-certificates/1560.pem /etc/ssl/certs/1560.pem"
I0917 02:12:39.103890 4110 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1560.pem
I0917 02:12:39.107498 4110 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 17 08:55 /usr/share/ca-certificates/1560.pem
I0917 02:12:39.107551 4110 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1560.pem
I0917 02:12:39.111799 4110 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1560.pem /etc/ssl/certs/51391683.0"
I0917 02:12:39.120941 4110 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0917 02:12:39.124549 4110 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0917 02:12:39.124586 4110 kubeadm.go:934] updating node {m04 192.169.0.8 0 v1.31.1 docker false true} ...
I0917 02:12:39.124645 4110 kubeadm.go:946] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=ha-857000-m04 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.169.0.8
[Install]
config:
{KubernetesVersion:v1.31.1 ClusterName:ha-857000 Namespace:default APIServerHAVIP:192.169.0.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0917 02:12:39.124713 4110 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.1
I0917 02:12:39.132685 4110 binaries.go:44] Found k8s binaries, skipping transfer
I0917 02:12:39.132752 4110 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system
I0917 02:12:39.140189 4110 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (311 bytes)
I0917 02:12:39.153737 4110 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0917 02:12:39.167480 4110 ssh_runner.go:195] Run: grep 192.169.0.254 control-plane.minikube.internal$ /etc/hosts
I0917 02:12:39.170335 4110 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.169.0.254 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0917 02:12:39.180131 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:12:39.274978 4110 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0917 02:12:39.290344 4110 start.go:235] Will wait 6m0s for node &{Name:m04 IP:192.169.0.8 Port:0 KubernetesVersion:v1.31.1 ContainerRuntime:docker ControlPlane:false Worker:true}
I0917 02:12:39.290539 4110 config.go:182] Loaded profile config "ha-857000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.31.1
I0917 02:12:39.312606 4110 out.go:177] * Verifying Kubernetes components...
I0917 02:12:39.332523 4110 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0917 02:12:39.447567 4110 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0917 02:12:39.466307 4110 loader.go:395] Config loaded from file: /Users/jenkins/minikube-integration/19648-1025/kubeconfig
I0917 02:12:39.466524 4110 kapi.go:59] client config for ha-857000: &rest.Config{Host:"https://192.169.0.254:8443", APIPath:"", ContentConfig:rest.ContentConfig{AcceptContentTypes:"", ContentType:"", GroupVersion:(*schema.GroupVersion)(nil), NegotiatedSerializer:runtime.NegotiatedSerializer(nil)}, Username:"", Password:"", BearerToken:"", BearerTokenFile:"", Impersonate:rest.ImpersonationConfig{UserName:"", UID:"", Groups:[]string(nil), Extra:map[string][]string(nil)}, AuthProvider:<nil>, AuthConfigPersister:rest.AuthProviderConfigPersister(nil), ExecProvider:<nil>, TLSClientConfig:rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/client.crt", KeyFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/profiles/ha-857000/client.key", CAFile:"/Users/jenkins/minikube-integration/19648-1025/.minikube/ca.crt", CertData:[]uint8(nil), KeyData:[]uint8(nil), CAData:[]uint8(nil), NextProtos:[]string(nil)}
, UserAgent:"", DisableCompression:false, Transport:http.RoundTripper(nil), WrapTransport:(transport.WrapperFunc)(0x673a720), QPS:0, Burst:0, RateLimiter:flowcontrol.RateLimiter(nil), WarningHandler:rest.WarningHandler(nil), Timeout:0, Dial:(func(context.Context, string, string) (net.Conn, error))(nil), Proxy:(func(*http.Request) (*url.URL, error))(nil)}
W0917 02:12:39.466571 4110 kubeadm.go:483] Overriding stale ClientConfig host https://192.169.0.254:8443 with https://192.169.0.5:8443
I0917 02:12:39.467449 4110 node_ready.go:35] waiting up to 6m0s for node "ha-857000-m04" to be "Ready" ...
I0917 02:12:39.467568 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m04
I0917 02:12:39.467575 4110 round_trippers.go:469] Request Headers:
I0917 02:12:39.467585 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:39.467591 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:39.470632 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:39.969561 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m04
I0917 02:12:39.969576 4110 round_trippers.go:469] Request Headers:
I0917 02:12:39.969585 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:39.969590 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:39.972203 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:39.972562 4110 node_ready.go:49] node "ha-857000-m04" has status "Ready":"True"
I0917 02:12:39.972573 4110 node_ready.go:38] duration metric: took 505.091961ms for node "ha-857000-m04" to be "Ready" ...
I0917 02:12:39.972579 4110 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0917 02:12:39.972614 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods
I0917 02:12:39.972619 4110 round_trippers.go:469] Request Headers:
I0917 02:12:39.972625 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:39.972629 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:39.976988 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:12:39.982728 4110 pod_ready.go:79] waiting up to 6m0s for pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace to be "Ready" ...
I0917 02:12:39.982773 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:39.982778 4110 round_trippers.go:469] Request Headers:
I0917 02:12:39.982795 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:39.982801 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:39.985018 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:39.985518 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:39.985526 4110 round_trippers.go:469] Request Headers:
I0917 02:12:39.985532 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:39.985536 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:39.987300 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:40.482877 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:40.482889 4110 round_trippers.go:469] Request Headers:
I0917 02:12:40.482894 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:40.482898 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:40.485392 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:40.485952 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:40.485960 4110 round_trippers.go:469] Request Headers:
I0917 02:12:40.485965 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:40.485972 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:40.487726 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:40.984290 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:40.984330 4110 round_trippers.go:469] Request Headers:
I0917 02:12:40.984337 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:40.984340 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:40.986636 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:40.987126 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:40.987134 4110 round_trippers.go:469] Request Headers:
I0917 02:12:40.987140 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:40.987144 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:40.989077 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:41.483798 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:41.483813 4110 round_trippers.go:469] Request Headers:
I0917 02:12:41.483838 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:41.483842 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:41.485913 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:41.486349 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:41.486357 4110 round_trippers.go:469] Request Headers:
I0917 02:12:41.486363 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:41.486366 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:41.487997 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:41.984399 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:41.984423 4110 round_trippers.go:469] Request Headers:
I0917 02:12:41.984436 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:41.984441 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:41.987692 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:41.988563 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:41.988571 4110 round_trippers.go:469] Request Headers:
I0917 02:12:41.988576 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:41.988580 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:41.990387 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:41.990837 4110 pod_ready.go:103] pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:42.483597 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:42.483651 4110 round_trippers.go:469] Request Headers:
I0917 02:12:42.483720 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:42.483731 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:42.486451 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:42.487002 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:42.487009 4110 round_trippers.go:469] Request Headers:
I0917 02:12:42.487015 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:42.487019 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:42.488735 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:42.984178 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:42.984202 4110 round_trippers.go:469] Request Headers:
I0917 02:12:42.984244 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:42.984250 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:42.987573 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:42.988040 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:42.988049 4110 round_trippers.go:469] Request Headers:
I0917 02:12:42.988056 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:42.988060 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:42.989664 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:43.484870 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:43.484884 4110 round_trippers.go:469] Request Headers:
I0917 02:12:43.484891 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:43.484894 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:43.487141 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:43.487687 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:43.487695 4110 round_trippers.go:469] Request Headers:
I0917 02:12:43.487701 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:43.487705 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:43.489384 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:43.985004 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:43.985028 4110 round_trippers.go:469] Request Headers:
I0917 02:12:43.985040 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:43.985047 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:43.988376 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:43.989251 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:43.989258 4110 round_trippers.go:469] Request Headers:
I0917 02:12:43.989264 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:43.989274 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:43.991010 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:43.991366 4110 pod_ready.go:103] pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:44.483323 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:44.483341 4110 round_trippers.go:469] Request Headers:
I0917 02:12:44.483350 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:44.483355 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:44.486151 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:44.486714 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:44.486722 4110 round_trippers.go:469] Request Headers:
I0917 02:12:44.486727 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:44.486732 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:44.488452 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:44.984530 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:44.984557 4110 round_trippers.go:469] Request Headers:
I0917 02:12:44.984569 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:44.984574 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:44.987518 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:44.988156 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:44.988163 4110 round_trippers.go:469] Request Headers:
I0917 02:12:44.988169 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:44.988173 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:44.989906 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:45.484413 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:45.484429 4110 round_trippers.go:469] Request Headers:
I0917 02:12:45.484436 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:45.484438 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:45.486664 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:45.487158 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:45.487166 4110 round_trippers.go:469] Request Headers:
I0917 02:12:45.487172 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:45.487180 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:45.488811 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:45.983568 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:45.983588 4110 round_trippers.go:469] Request Headers:
I0917 02:12:45.983597 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:45.983601 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:45.986094 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:45.986663 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:45.986670 4110 round_trippers.go:469] Request Headers:
I0917 02:12:45.986676 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:45.986681 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:45.988390 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:46.484237 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:46.484252 4110 round_trippers.go:469] Request Headers:
I0917 02:12:46.484258 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:46.484262 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:46.486548 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:46.487112 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:46.487120 4110 round_trippers.go:469] Request Headers:
I0917 02:12:46.487126 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:46.487130 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:46.488764 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:46.489074 4110 pod_ready.go:103] pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:46.984666 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:46.984685 4110 round_trippers.go:469] Request Headers:
I0917 02:12:46.984693 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:46.984699 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:46.987277 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:46.987747 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:46.987754 4110 round_trippers.go:469] Request Headers:
I0917 02:12:46.987760 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:46.987764 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:46.989871 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:47.483189 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:47.483204 4110 round_trippers.go:469] Request Headers:
I0917 02:12:47.483220 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:47.483225 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:47.485536 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:47.486040 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:47.486048 4110 round_trippers.go:469] Request Headers:
I0917 02:12:47.486053 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:47.486077 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:47.487968 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:47.983218 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:47.983261 4110 round_trippers.go:469] Request Headers:
I0917 02:12:47.983271 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:47.983276 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:47.985959 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:47.986467 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:47.986476 4110 round_trippers.go:469] Request Headers:
I0917 02:12:47.986480 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:47.986483 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:47.988256 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:48.483839 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:48.483855 4110 round_trippers.go:469] Request Headers:
I0917 02:12:48.483877 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:48.483881 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:48.486127 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:48.486742 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:48.486750 4110 round_trippers.go:469] Request Headers:
I0917 02:12:48.486756 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:48.486763 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:48.488482 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:48.983104 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:48.983116 4110 round_trippers.go:469] Request Headers:
I0917 02:12:48.983123 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:48.983126 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:48.986541 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:48.986974 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:48.986982 4110 round_trippers.go:469] Request Headers:
I0917 02:12:48.986988 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:48.987000 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:48.989572 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:48.989840 4110 pod_ready.go:103] pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:49.483113 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:49.483127 4110 round_trippers.go:469] Request Headers:
I0917 02:12:49.483135 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:49.483138 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:49.485418 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:49.485944 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:49.485952 4110 round_trippers.go:469] Request Headers:
I0917 02:12:49.485958 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:49.485965 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:49.488051 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:49.983392 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:49.983418 4110 round_trippers.go:469] Request Headers:
I0917 02:12:49.983430 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:49.983435 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:49.990100 4110 round_trippers.go:574] Response Status: 200 OK in 6 milliseconds
I0917 02:12:49.990521 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:49.990528 4110 round_trippers.go:469] Request Headers:
I0917 02:12:49.990534 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:49.990551 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:49.995841 4110 round_trippers.go:574] Response Status: 200 OK in 5 milliseconds
I0917 02:12:50.484489 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:50.484507 4110 round_trippers.go:469] Request Headers:
I0917 02:12:50.484516 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:50.484519 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:50.487282 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:50.487803 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:50.487815 4110 round_trippers.go:469] Request Headers:
I0917 02:12:50.487821 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:50.487826 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:50.489538 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:50.984752 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:50.984776 4110 round_trippers.go:469] Request Headers:
I0917 02:12:50.984788 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:50.984796 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:50.988059 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:50.988580 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:50.988587 4110 round_trippers.go:469] Request Headers:
I0917 02:12:50.988593 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:50.988597 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:50.990162 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:50.990537 4110 pod_ready.go:103] pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:51.483827 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:51.483847 4110 round_trippers.go:469] Request Headers:
I0917 02:12:51.483864 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:51.483902 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:51.487323 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:51.487924 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:51.487932 4110 round_trippers.go:469] Request Headers:
I0917 02:12:51.487937 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:51.487942 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:51.489844 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:51.983451 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:51.983470 4110 round_trippers.go:469] Request Headers:
I0917 02:12:51.983482 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:51.983488 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:51.986994 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:51.987525 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:51.987535 4110 round_trippers.go:469] Request Headers:
I0917 02:12:51.987543 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:51.987548 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:51.989115 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:52.483263 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:52.483288 4110 round_trippers.go:469] Request Headers:
I0917 02:12:52.483325 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:52.483332 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:52.486347 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:52.486988 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:52.486995 4110 round_trippers.go:469] Request Headers:
I0917 02:12:52.487001 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:52.487005 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:52.488688 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:52.983765 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:52.983790 4110 round_trippers.go:469] Request Headers:
I0917 02:12:52.983801 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:52.983810 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:52.986675 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:52.987089 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:52.987119 4110 round_trippers.go:469] Request Headers:
I0917 02:12:52.987125 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:52.987129 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:52.988627 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:53.484927 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:53.484941 4110 round_trippers.go:469] Request Headers:
I0917 02:12:53.484948 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:53.484951 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:53.487216 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:53.487660 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:53.487667 4110 round_trippers.go:469] Request Headers:
I0917 02:12:53.487673 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:53.487676 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:53.489219 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:53.489560 4110 pod_ready.go:103] pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:53.984242 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:53.984261 4110 round_trippers.go:469] Request Headers:
I0917 02:12:53.984274 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:53.984280 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:53.986802 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:53.987318 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:53.987326 4110 round_trippers.go:469] Request Headers:
I0917 02:12:53.987333 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:53.987336 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:53.989152 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:54.483277 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:54.483309 4110 round_trippers.go:469] Request Headers:
I0917 02:12:54.483353 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:54.483368 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:54.486304 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:54.486703 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:54.486709 4110 round_trippers.go:469] Request Headers:
I0917 02:12:54.486715 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:54.486718 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:54.488409 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:54.984401 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:54.984421 4110 round_trippers.go:469] Request Headers:
I0917 02:12:54.984432 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:54.984436 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:54.987150 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:54.987731 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:54.987739 4110 round_trippers.go:469] Request Headers:
I0917 02:12:54.987745 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:54.987762 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:54.990093 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:55.484219 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:55.484245 4110 round_trippers.go:469] Request Headers:
I0917 02:12:55.484263 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:55.484270 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:55.487478 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:55.488038 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:55.488046 4110 round_trippers.go:469] Request Headers:
I0917 02:12:55.488052 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:55.488055 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:55.489736 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:55.490063 4110 pod_ready.go:103] pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace has status "Ready":"False"
I0917 02:12:55.983721 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:55.983738 4110 round_trippers.go:469] Request Headers:
I0917 02:12:55.983747 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:55.983751 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:55.986467 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:55.986910 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:55.986918 4110 round_trippers.go:469] Request Headers:
I0917 02:12:55.986924 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:55.986927 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:55.988668 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:56.483680 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:56.483698 4110 round_trippers.go:469] Request Headers:
I0917 02:12:56.483705 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:56.483708 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:56.486006 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:56.486509 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:56.486517 4110 round_trippers.go:469] Request Headers:
I0917 02:12:56.486523 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:56.486526 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:56.488267 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:56.984953 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:56.984979 4110 round_trippers.go:469] Request Headers:
I0917 02:12:56.984991 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:56.984998 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:56.988958 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:56.989556 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:56.989567 4110 round_trippers.go:469] Request Headers:
I0917 02:12:56.989575 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:56.989580 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:56.991555 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:57.483204 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-fg65r
I0917 02:12:57.483220 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.483244 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.483257 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.489651 4110 round_trippers.go:574] Response Status: 200 OK in 6 milliseconds
I0917 02:12:57.491669 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:57.491685 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.491693 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.491697 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.500745 4110 round_trippers.go:574] Response Status: 200 OK in 9 milliseconds
I0917 02:12:57.502366 4110 pod_ready.go:93] pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:57.502386 4110 pod_ready.go:82] duration metric: took 17.519343583s for pod "coredns-7c65d6cfc9-fg65r" in "kube-system" namespace to be "Ready" ...
I0917 02:12:57.502398 4110 pod_ready.go:79] waiting up to 6m0s for pod "coredns-7c65d6cfc9-nl5j5" in "kube-system" namespace to be "Ready" ...
I0917 02:12:57.502483 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/coredns-7c65d6cfc9-nl5j5
I0917 02:12:57.502497 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.502507 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.502512 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.512509 4110 round_trippers.go:574] Response Status: 200 OK in 9 milliseconds
I0917 02:12:57.513793 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:57.513807 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.513817 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.513823 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.522244 4110 round_trippers.go:574] Response Status: 200 OK in 8 milliseconds
I0917 02:12:57.522585 4110 pod_ready.go:93] pod "coredns-7c65d6cfc9-nl5j5" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:57.522595 4110 pod_ready.go:82] duration metric: took 20.190892ms for pod "coredns-7c65d6cfc9-nl5j5" in "kube-system" namespace to be "Ready" ...
I0917 02:12:57.522609 4110 pod_ready.go:79] waiting up to 6m0s for pod "etcd-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:57.522650 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/etcd-ha-857000
I0917 02:12:57.522656 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.522662 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.522666 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.527526 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:12:57.528075 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:57.528084 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.528089 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.528100 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.530647 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:57.531009 4110 pod_ready.go:93] pod "etcd-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:57.531019 4110 pod_ready.go:82] duration metric: took 8.403704ms for pod "etcd-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:57.531025 4110 pod_ready.go:79] waiting up to 6m0s for pod "etcd-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:57.531068 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/etcd-ha-857000-m02
I0917 02:12:57.531073 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.531082 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.531087 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.533324 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:57.533687 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:57.533694 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.533700 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.533704 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.535601 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:57.535875 4110 pod_ready.go:93] pod "etcd-ha-857000-m02" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:57.535883 4110 pod_ready.go:82] duration metric: took 4.853562ms for pod "etcd-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:57.535902 4110 pod_ready.go:79] waiting up to 6m0s for pod "etcd-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:57.535944 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/etcd-ha-857000-m03
I0917 02:12:57.535950 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.535956 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.535960 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.537587 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:57.537964 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:57.537972 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.537978 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.537982 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.539462 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:12:57.539797 4110 pod_ready.go:93] pod "etcd-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:57.539805 4110 pod_ready.go:82] duration metric: took 3.894392ms for pod "etcd-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:57.539816 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:57.684040 4110 request.go:632] Waited for 144.185674ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000
I0917 02:12:57.684081 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000
I0917 02:12:57.684104 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.684125 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.684132 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.686547 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:57.883303 4110 request.go:632] Waited for 196.17665ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:57.883378 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:57.883388 4110 round_trippers.go:469] Request Headers:
I0917 02:12:57.883398 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:57.883406 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:57.886942 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:57.887555 4110 pod_ready.go:93] pod "kube-apiserver-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:57.887569 4110 pod_ready.go:82] duration metric: took 347.737487ms for pod "kube-apiserver-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:57.887576 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:58.083903 4110 request.go:632] Waited for 196.258589ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m02
I0917 02:12:58.084076 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m02
I0917 02:12:58.084095 4110 round_trippers.go:469] Request Headers:
I0917 02:12:58.084104 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:58.084111 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:58.087323 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:58.284752 4110 request.go:632] Waited for 196.829301ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:58.284841 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:58.284851 4110 round_trippers.go:469] Request Headers:
I0917 02:12:58.284863 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:58.284871 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:58.287836 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:58.288234 4110 pod_ready.go:93] pod "kube-apiserver-ha-857000-m02" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:58.288243 4110 pod_ready.go:82] duration metric: took 400.655079ms for pod "kube-apiserver-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:58.288251 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:58.484581 4110 request.go:632] Waited for 196.285151ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:58.484627 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-ha-857000-m03
I0917 02:12:58.484634 4110 round_trippers.go:469] Request Headers:
I0917 02:12:58.484670 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:58.484676 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:58.487401 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:58.683590 4110 request.go:632] Waited for 195.669934ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:58.683635 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:58.683643 4110 round_trippers.go:469] Request Headers:
I0917 02:12:58.683695 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:58.683709 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:58.687024 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:58.687397 4110 pod_ready.go:93] pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:58.687407 4110 pod_ready.go:82] duration metric: took 399.144074ms for pod "kube-apiserver-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:58.687414 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:58.884795 4110 request.go:632] Waited for 197.34012ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000
I0917 02:12:58.884845 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000
I0917 02:12:58.884854 4110 round_trippers.go:469] Request Headers:
I0917 02:12:58.884862 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:58.884886 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:58.887327 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:59.083807 4110 request.go:632] Waited for 195.949253ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:59.083945 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:12:59.083961 4110 round_trippers.go:469] Request Headers:
I0917 02:12:59.083973 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:59.083980 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:59.087431 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:59.087851 4110 pod_ready.go:93] pod "kube-controller-manager-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:59.087864 4110 pod_ready.go:82] duration metric: took 400.438219ms for pod "kube-controller-manager-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:12:59.087874 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:59.283487 4110 request.go:632] Waited for 195.551174ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m02
I0917 02:12:59.283570 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m02
I0917 02:12:59.283587 4110 round_trippers.go:469] Request Headers:
I0917 02:12:59.283598 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:59.283604 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:59.286668 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:12:59.483240 4110 request.go:632] Waited for 196.050684ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:59.483272 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:12:59.483277 4110 round_trippers.go:469] Request Headers:
I0917 02:12:59.483284 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:59.483287 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:59.485481 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:59.485790 4110 pod_ready.go:93] pod "kube-controller-manager-ha-857000-m02" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:59.485799 4110 pod_ready.go:82] duration metric: took 397.912163ms for pod "kube-controller-manager-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:12:59.485808 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:59.684196 4110 request.go:632] Waited for 198.346846ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:12:59.684283 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-ha-857000-m03
I0917 02:12:59.684289 4110 round_trippers.go:469] Request Headers:
I0917 02:12:59.684295 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:59.684299 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:59.686349 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:12:59.883921 4110 request.go:632] Waited for 197.130794ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:59.883972 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:12:59.883980 4110 round_trippers.go:469] Request Headers:
I0917 02:12:59.884030 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:12:59.884039 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:12:59.888316 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:12:59.888770 4110 pod_ready.go:93] pod "kube-controller-manager-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:12:59.888788 4110 pod_ready.go:82] duration metric: took 402.964156ms for pod "kube-controller-manager-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:12:59.888815 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-528ht" in "kube-system" namespace to be "Ready" ...
I0917 02:13:00.083631 4110 request.go:632] Waited for 194.730555ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-528ht
I0917 02:13:00.083713 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-528ht
I0917 02:13:00.083720 4110 round_trippers.go:469] Request Headers:
I0917 02:13:00.083728 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:00.083732 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:00.086353 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:13:00.285261 4110 request.go:632] Waited for 198.400376ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m04
I0917 02:13:00.285348 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m04
I0917 02:13:00.285356 4110 round_trippers.go:469] Request Headers:
I0917 02:13:00.285364 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:00.285370 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:00.287853 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:13:00.288149 4110 pod_ready.go:93] pod "kube-proxy-528ht" in "kube-system" namespace has status "Ready":"True"
I0917 02:13:00.288159 4110 pod_ready.go:82] duration metric: took 399.322905ms for pod "kube-proxy-528ht" in "kube-system" namespace to be "Ready" ...
I0917 02:13:00.288167 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-g9wxm" in "kube-system" namespace to be "Ready" ...
I0917 02:13:00.484621 4110 request.go:632] Waited for 196.39101ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-g9wxm
I0917 02:13:00.484716 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-g9wxm
I0917 02:13:00.484727 4110 round_trippers.go:469] Request Headers:
I0917 02:13:00.484737 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:00.484744 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:00.488045 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:13:00.685321 4110 request.go:632] Waited for 196.686181ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:13:00.685381 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:13:00.685438 4110 round_trippers.go:469] Request Headers:
I0917 02:13:00.685455 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:00.685464 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:00.688919 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:13:00.689362 4110 pod_ready.go:93] pod "kube-proxy-g9wxm" in "kube-system" namespace has status "Ready":"True"
I0917 02:13:00.689374 4110 pod_ready.go:82] duration metric: took 401.194339ms for pod "kube-proxy-g9wxm" in "kube-system" namespace to be "Ready" ...
I0917 02:13:00.689383 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-vskbj" in "kube-system" namespace to be "Ready" ...
I0917 02:13:00.884950 4110 request.go:632] Waited for 195.521785ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-vskbj
I0917 02:13:00.884994 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-vskbj
I0917 02:13:00.885018 4110 round_trippers.go:469] Request Headers:
I0917 02:13:00.885025 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:00.885034 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:00.887231 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:13:01.084761 4110 request.go:632] Waited for 197.012037ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:13:01.084795 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:13:01.084800 4110 round_trippers.go:469] Request Headers:
I0917 02:13:01.084806 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:01.084810 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:01.088892 4110 round_trippers.go:574] Response Status: 200 OK in 4 milliseconds
I0917 02:13:01.089243 4110 pod_ready.go:93] pod "kube-proxy-vskbj" in "kube-system" namespace has status "Ready":"True"
I0917 02:13:01.089253 4110 pod_ready.go:82] duration metric: took 399.857039ms for pod "kube-proxy-vskbj" in "kube-system" namespace to be "Ready" ...
I0917 02:13:01.089261 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-zrqvr" in "kube-system" namespace to be "Ready" ...
I0917 02:13:01.284602 4110 request.go:632] Waited for 195.290385ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-zrqvr
I0917 02:13:01.284640 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-proxy-zrqvr
I0917 02:13:01.284645 4110 round_trippers.go:469] Request Headers:
I0917 02:13:01.284672 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:01.284680 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:01.286636 4110 round_trippers.go:574] Response Status: 200 OK in 1 milliseconds
I0917 02:13:01.483312 4110 request.go:632] Waited for 196.269648ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:13:01.483391 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:13:01.483403 4110 round_trippers.go:469] Request Headers:
I0917 02:13:01.483413 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:01.483434 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:01.486551 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:13:01.486934 4110 pod_ready.go:93] pod "kube-proxy-zrqvr" in "kube-system" namespace has status "Ready":"True"
I0917 02:13:01.486943 4110 pod_ready.go:82] duration metric: took 397.670619ms for pod "kube-proxy-zrqvr" in "kube-system" namespace to be "Ready" ...
I0917 02:13:01.486950 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:13:01.683659 4110 request.go:632] Waited for 196.646108ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000
I0917 02:13:01.683796 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000
I0917 02:13:01.683807 4110 round_trippers.go:469] Request Headers:
I0917 02:13:01.683819 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:01.683825 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:01.686996 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:13:01.884224 4110 request.go:632] Waited for 196.55945ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:13:01.884363 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000
I0917 02:13:01.884374 4110 round_trippers.go:469] Request Headers:
I0917 02:13:01.884385 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:01.884393 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:01.888135 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:13:01.888538 4110 pod_ready.go:93] pod "kube-scheduler-ha-857000" in "kube-system" namespace has status "Ready":"True"
I0917 02:13:01.888551 4110 pod_ready.go:82] duration metric: took 401.588084ms for pod "kube-scheduler-ha-857000" in "kube-system" namespace to be "Ready" ...
I0917 02:13:01.888559 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:13:02.083387 4110 request.go:632] Waited for 194.732026ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m02
I0917 02:13:02.083482 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m02
I0917 02:13:02.083493 4110 round_trippers.go:469] Request Headers:
I0917 02:13:02.083503 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:02.083512 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:02.087127 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:13:02.284704 4110 request.go:632] Waited for 197.205174ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:13:02.284756 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m02
I0917 02:13:02.284761 4110 round_trippers.go:469] Request Headers:
I0917 02:13:02.284768 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:02.284773 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:02.287752 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:13:02.288038 4110 pod_ready.go:93] pod "kube-scheduler-ha-857000-m02" in "kube-system" namespace has status "Ready":"True"
I0917 02:13:02.288049 4110 pod_ready.go:82] duration metric: took 399.476957ms for pod "kube-scheduler-ha-857000-m02" in "kube-system" namespace to be "Ready" ...
I0917 02:13:02.288056 4110 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:13:02.485154 4110 request.go:632] Waited for 197.02881ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m03
I0917 02:13:02.485191 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-ha-857000-m03
I0917 02:13:02.485198 4110 round_trippers.go:469] Request Headers:
I0917 02:13:02.485206 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:02.485211 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:02.487672 4110 round_trippers.go:574] Response Status: 200 OK in 2 milliseconds
I0917 02:13:02.685336 4110 request.go:632] Waited for 197.331043ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:13:02.685388 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes/ha-857000-m03
I0917 02:13:02.685397 4110 round_trippers.go:469] Request Headers:
I0917 02:13:02.685411 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:02.685417 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:02.688565 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:13:02.688910 4110 pod_ready.go:93] pod "kube-scheduler-ha-857000-m03" in "kube-system" namespace has status "Ready":"True"
I0917 02:13:02.688918 4110 pod_ready.go:82] duration metric: took 400.85077ms for pod "kube-scheduler-ha-857000-m03" in "kube-system" namespace to be "Ready" ...
I0917 02:13:02.688929 4110 pod_ready.go:39] duration metric: took 22.715951136s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0917 02:13:02.688942 4110 system_svc.go:44] waiting for kubelet service to be running ....
I0917 02:13:02.689000 4110 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0917 02:13:02.699631 4110 system_svc.go:56] duration metric: took 10.684367ms WaitForService to wait for kubelet
I0917 02:13:02.699646 4110 kubeadm.go:582] duration metric: took 23.408872965s to wait for: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0917 02:13:02.699663 4110 node_conditions.go:102] verifying NodePressure condition ...
I0917 02:13:02.884773 4110 request.go:632] Waited for 185.024169ms due to client-side throttling, not priority and fairness, request: GET:https://192.169.0.5:8443/api/v1/nodes
I0917 02:13:02.884858 4110 round_trippers.go:463] GET https://192.169.0.5:8443/api/v1/nodes
I0917 02:13:02.884867 4110 round_trippers.go:469] Request Headers:
I0917 02:13:02.884878 4110 round_trippers.go:473] User-Agent: minikube-darwin-amd64/v0.0.0 (darwin/amd64) kubernetes/$Format
I0917 02:13:02.884887 4110 round_trippers.go:473] Accept: application/json, */*
I0917 02:13:02.888704 4110 round_trippers.go:574] Response Status: 200 OK in 3 milliseconds
I0917 02:13:02.889505 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:13:02.889516 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:13:02.889528 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:13:02.889534 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:13:02.889537 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:13:02.889540 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:13:02.889543 4110 node_conditions.go:122] node storage ephemeral capacity is 17734596Ki
I0917 02:13:02.889545 4110 node_conditions.go:123] node cpu capacity is 2
I0917 02:13:02.889549 4110 node_conditions.go:105] duration metric: took 189.878189ms to run NodePressure ...
I0917 02:13:02.889557 4110 start.go:241] waiting for startup goroutines ...
I0917 02:13:02.889572 4110 start.go:255] writing updated cluster config ...
I0917 02:13:02.889954 4110 ssh_runner.go:195] Run: rm -f paused
I0917 02:13:02.930446 4110 start.go:600] kubectl: 1.29.2, cluster: 1.31.1 (minor skew: 2)
I0917 02:13:02.983109 4110 out.go:201]
W0917 02:13:03.020673 4110 out.go:270] ! /usr/local/bin/kubectl is version 1.29.2, which may have incompatibilities with Kubernetes 1.31.1.
I0917 02:13:03.057789 4110 out.go:177] - Want kubectl v1.31.1? Try 'minikube kubectl -- get pods -A'
I0917 02:13:03.135680 4110 out.go:177] * Done! kubectl is now configured to use "ha-857000" cluster and "default" namespace by default
==> Docker <==
Sep 17 09:12:18 ha-857000 cri-dockerd[1413]: time="2024-09-17T09:12:18Z" level=info msg="Will attempt to re-write config file /var/lib/docker/containers/fc1d198ffe0b277108e5042e64604ceb887f7f17cd5814893fbf789f1ce180f0/resolv.conf as [nameserver 10.96.0.10 search default.svc.cluster.local svc.cluster.local cluster.local options ndots:5]"
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.316039322Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.316201907Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.316216597Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.316284213Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.356401685Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.356591613Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.356646706Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.356901392Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.358210462Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.358271414Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.358284287Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.358347315Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.361819988Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.361879924Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.361892293Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Sep 17 09:12:18 ha-857000 dockerd[1166]: time="2024-09-17T09:12:18.361954784Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Sep 17 09:12:48 ha-857000 dockerd[1160]: time="2024-09-17T09:12:48.289404793Z" level=info msg="ignoring event" container=67814a4514b10a43fcb805320685eed14ae6c928c03c950cc35f4abd031401e7 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Sep 17 09:12:48 ha-857000 dockerd[1166]: time="2024-09-17T09:12:48.290629069Z" level=info msg="shim disconnected" id=67814a4514b10a43fcb805320685eed14ae6c928c03c950cc35f4abd031401e7 namespace=moby
Sep 17 09:12:48 ha-857000 dockerd[1166]: time="2024-09-17T09:12:48.290966877Z" level=warning msg="cleaning up after shim disconnected" id=67814a4514b10a43fcb805320685eed14ae6c928c03c950cc35f4abd031401e7 namespace=moby
Sep 17 09:12:48 ha-857000 dockerd[1166]: time="2024-09-17T09:12:48.291008241Z" level=info msg="cleaning up dead shim" namespace=moby
Sep 17 09:13:00 ha-857000 dockerd[1166]: time="2024-09-17T09:13:00.269678049Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Sep 17 09:13:00 ha-857000 dockerd[1166]: time="2024-09-17T09:13:00.269745426Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Sep 17 09:13:00 ha-857000 dockerd[1166]: time="2024-09-17T09:13:00.269758363Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Sep 17 09:13:00 ha-857000 dockerd[1166]: time="2024-09-17T09:13:00.269841312Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
d940d576a500a 6e38f40d628db 9 seconds ago Running storage-provisioner 2 6fb8068a5c29f storage-provisioner
119f2deb32f13 8c811b4aec35f 51 seconds ago Running busybox 1 fc1d198ffe0b2 busybox-7dff88458-4jzg8
b7aa83ae3a822 c69fa2e9cbf5f 51 seconds ago Running coredns 1 f4e7a7b3c65e5 coredns-7c65d6cfc9-nl5j5
c37a677e31180 60c005f310ff3 51 seconds ago Running kube-proxy 1 5294422217d99 kube-proxy-vskbj
3d889c7c8da7e 12968670680f4 51 seconds ago Running kindnet-cni 1 80326e6e99372 kindnet-7pf7v
7b8b62bf7340c c69fa2e9cbf5f 51 seconds ago Running coredns 1 f4cf87ea66207 coredns-7c65d6cfc9-fg65r
67814a4514b10 6e38f40d628db 52 seconds ago Exited storage-provisioner 1 6fb8068a5c29f storage-provisioner
ca7fe8ccd4c53 175ffd71cce3d About a minute ago Running kube-controller-manager 6 77f536a07a3a6 kube-controller-manager-ha-857000
475dedee37228 6bab7719df100 About a minute ago Running kube-apiserver 6 0968090389d54 kube-apiserver-ha-857000
37d6d6479e30b 38af8ddebf499 2 minutes ago Running kube-vip 1 2842ed202c474 kube-vip-ha-857000
00ff29c213716 9aa1fad941575 2 minutes ago Running kube-scheduler 2 309841a63d772 kube-scheduler-ha-857000
13b7f8a93ad49 175ffd71cce3d 2 minutes ago Exited kube-controller-manager 5 77f536a07a3a6 kube-controller-manager-ha-857000
8c0804e78de8f 2e96e5913fc06 2 minutes ago Running etcd 2 6cfb11ed1d6ba etcd-ha-857000
a18a6b023cd60 6bab7719df100 2 minutes ago Exited kube-apiserver 5 0968090389d54 kube-apiserver-ha-857000
034279696db8f 38af8ddebf499 6 minutes ago Exited kube-vip 0 4205e70bfa1bb kube-vip-ha-857000
d9fae1497b048 9aa1fad941575 6 minutes ago Exited kube-scheduler 1 37d9fe68f2e59 kube-scheduler-ha-857000
f4f59b8c76404 2e96e5913fc06 6 minutes ago Exited etcd 1 a23094a650513 etcd-ha-857000
fe908ac73b00f gcr.io/k8s-minikube/busybox@sha256:9afb80db71730dbb303fe00765cbf34bddbdc6b66e49897fc2e1861967584b12 9 minutes ago Exited busybox 0 80864159ef38e busybox-7dff88458-4jzg8
521527f17691c c69fa2e9cbf5f 11 minutes ago Exited coredns 0 aa21641a5b16e coredns-7c65d6cfc9-nl5j5
f991c8e956d90 c69fa2e9cbf5f 11 minutes ago Exited coredns 0 da08087b51cd9 coredns-7c65d6cfc9-fg65r
5d84a01abd3e7 kindest/kindnetd@sha256:e59a687ca28ae274a2fc92f1e2f5f1c739f353178a43a23aafc71adb802ed166 12 minutes ago Exited kindnet-cni 0 38db6fab73655 kindnet-7pf7v
0b03e5e488939 60c005f310ff3 12 minutes ago Exited kube-proxy 0 067bc1b2ad7fa kube-proxy-vskbj
==> coredns [521527f17691] <==
[INFO] 10.244.2.2:33230 - 5 "AAAA IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 147 0.000100028s
[INFO] 10.244.2.2:37727 - 7 "A IN kubernetes.default.default.svc.cluster.local. udp 62 false 512" NXDOMAIN qr,aa,rd 155 0.000077614s
[INFO] 10.244.2.2:51233 - 8 "A IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 106 0.000090375s
[INFO] 10.244.1.2:43082 - 2 "PTR IN 10.0.96.10.in-addr.arpa. udp 41 false 512" NOERROR qr,aa,rd 116 0.000115984s
[INFO] 10.244.1.2:45048 - 3 "AAAA IN kubernetes.default. udp 36 false 512" NXDOMAIN qr,aa,rd,ra 111 0.000071244s
[INFO] 10.244.1.2:48877 - 5 "AAAA IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 147 0.000106601s
[INFO] 10.244.1.2:59235 - 6 "A IN kubernetes.default. udp 36 false 512" NXDOMAIN qr,aa,rd,ra 111 0.000068348s
[INFO] 10.244.1.2:53808 - 7 "A IN kubernetes.default.default.svc.cluster.local. udp 62 false 512" NXDOMAIN qr,aa,rd 155 0.000064222s
[INFO] 10.244.1.2:54982 - 8 "A IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 106 0.000064992s
[INFO] 10.244.0.4:59177 - 2 "PTR IN 10.0.96.10.in-addr.arpa. udp 41 false 512" NOERROR qr,aa,rd 116 0.00012236s
[INFO] 10.244.0.4:59468 - 3 "AAAA IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 147 0.000096608s
[INFO] 10.244.0.4:49953 - 4 "A IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 106 0.000108018s
[INFO] 10.244.2.2:36658 - 3 "AAAA IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 147 0.000081427s
[INFO] 10.244.1.2:53166 - 2 "PTR IN 10.0.96.10.in-addr.arpa. udp 41 false 512" NOERROR qr,aa,rd 116 0.000140458s
[INFO] 10.244.1.2:60442 - 3 "AAAA IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 147 0.000069729s
[INFO] 10.244.0.4:60564 - 4 "A IN host.minikube.internal. udp 40 false 512" NOERROR qr,aa,rd 78 0.00007076s
[INFO] 10.244.0.4:57696 - 5 "PTR IN 1.0.169.192.in-addr.arpa. udp 42 false 512" NOERROR qr,aa,rd 102 0.000125726s
[INFO] 10.244.2.2:33447 - 2 "PTR IN 10.0.96.10.in-addr.arpa. udp 41 false 512" NOERROR qr,aa,rd 116 0.000114855s
[INFO] 10.244.2.2:49647 - 4 "A IN host.minikube.internal. udp 40 false 512" NOERROR qr,aa,rd 78 0.000058138s
[INFO] 10.244.2.2:55869 - 5 "PTR IN 1.0.169.192.in-addr.arpa. udp 42 false 512" NOERROR qr,aa,rd 102 0.00009725s
[INFO] 10.244.1.2:49826 - 2 "PTR IN 10.0.96.10.in-addr.arpa. udp 41 false 512" NOERROR qr,aa,rd 116 0.000096631s
[INFO] 10.244.1.2:33376 - 4 "A IN host.minikube.internal. udp 40 false 512" NOERROR qr,aa,rd 78 0.000046366s
[INFO] 10.244.1.2:47184 - 5 "PTR IN 1.0.169.192.in-addr.arpa. udp 42 false 512" NOERROR qr,aa,rd 102 0.000084276s
[INFO] SIGTERM: Shutting down servers then terminating
[INFO] plugin/health: Going into lameduck mode for 5s
==> coredns [7b8b62bf7340] <==
[INFO] plugin/kubernetes: waiting for Kubernetes API before starting server
[WARNING] plugin/kubernetes: starting server with unsynced Kubernetes API
.:53
[INFO] plugin/reload: Running configuration SHA512 = 257e111468ef6f1e36f10df061303186c353cd0e51aed8f50f4e4fd21cec02687aef97084fe1f82262f5cee88179d311670a6ae21ae185759728216fc264125f
CoreDNS-1.11.3
linux/amd64, go1.21.11, a6338e9
[INFO] 127.0.0.1:40424 - 46793 "HINFO IN 2652948645074262826.4033840954787183129. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.019948501s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[INFO] plugin/kubernetes: Trace[345670875]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229 (17-Sep-2024 09:12:18.718) (total time: 30000ms):
Trace[345670875]: ---"Objects listed" error:Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout 30000ms (09:12:48.718)
Trace[345670875]: [30.000647992s] [30.000647992s] END
[INFO] plugin/kubernetes: Trace[990255223]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229 (17-Sep-2024 09:12:18.716) (total time: 30002ms):
Trace[990255223]: ---"Objects listed" error:Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout 30001ms (09:12:48.718)
Trace[990255223]: [30.002122547s] [30.002122547s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[INFO] plugin/kubernetes: Trace[1561533284]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229 (17-Sep-2024 09:12:18.716) (total time: 30004ms):
Trace[1561533284]: ---"Objects listed" error:Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout 30004ms (09:12:48.720)
Trace[1561533284]: [30.004471134s] [30.004471134s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
==> coredns [b7aa83ae3a82] <==
[INFO] plugin/kubernetes: waiting for Kubernetes API before starting server
[WARNING] plugin/kubernetes: starting server with unsynced Kubernetes API
.:53
[INFO] plugin/reload: Running configuration SHA512 = 257e111468ef6f1e36f10df061303186c353cd0e51aed8f50f4e4fd21cec02687aef97084fe1f82262f5cee88179d311670a6ae21ae185759728216fc264125f
CoreDNS-1.11.3
linux/amd64, go1.21.11, a6338e9
[INFO] 127.0.0.1:48468 - 41934 "HINFO IN 5248560894606224369.8303849678443807322. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.019682687s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[INFO] plugin/kubernetes: Trace[134011415]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229 (17-Sep-2024 09:12:18.720) (total time: 30000ms):
Trace[134011415]: ---"Objects listed" error:Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout 30000ms (09:12:48.721)
Trace[134011415]: [30.000772699s] [30.000772699s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[INFO] plugin/kubernetes: Trace[1931337556]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229 (17-Sep-2024 09:12:18.720) (total time: 30001ms):
Trace[1931337556]: ---"Objects listed" error:Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout 30001ms (09:12:48.721)
Trace[1931337556]: [30.001621273s] [30.001621273s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[INFO] plugin/kubernetes: Trace[2093896532]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229 (17-Sep-2024 09:12:18.720) (total time: 30001ms):
Trace[2093896532]: ---"Objects listed" error:Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout 30001ms (09:12:48.721)
Trace[2093896532]: [30.001436763s] [30.001436763s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.29.3/tools/cache/reflector.go:229: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
==> coredns [f991c8e956d9] <==
[INFO] 10.244.1.2:36169 - 2 "PTR IN 10.0.96.10.in-addr.arpa. udp 41 false 512" NOERROR qr,aa,rd 116 0.000206963s
[INFO] 10.244.1.2:33814 - 3 "AAAA IN kubernetes.io. udp 31 false 512" NOERROR qr,aa,rd,ra 140 0.000088589s
[INFO] 10.244.1.2:57385 - 5 "PTR IN 148.40.75.147.in-addr.arpa. udp 44 false 512" NXDOMAIN qr,rd,ra 140 0.000535008s
[INFO] 10.244.0.4:54856 - 2 "PTR IN 10.0.96.10.in-addr.arpa. udp 41 false 512" NOERROR qr,aa,rd 116 0.000135529s
[INFO] 10.244.0.4:47831 - 3 "AAAA IN kubernetes.default. udp 36 false 512" NXDOMAIN qr,rd,ra 111 0.019088159s
[INFO] 10.244.0.4:46325 - 4 "AAAA IN kubernetes.default.default.svc.cluster.local. udp 62 false 512" NXDOMAIN qr,aa,rd 155 0.000201714s
[INFO] 10.244.0.4:45239 - 5 "AAAA IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 147 0.000255383s
[INFO] 10.244.0.4:55042 - 8 "A IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 106 0.000141827s
[INFO] 10.244.2.2:47888 - 2 "PTR IN 10.0.96.10.in-addr.arpa. udp 41 false 512" NOERROR qr,aa,rd 116 0.000108401s
[INFO] 10.244.2.2:41486 - 6 "A IN kubernetes.default. udp 36 false 512" NXDOMAIN qr,rd,ra 111 0.00044994s
[INFO] 10.244.2.2:50623 - 9 "PTR IN 1.0.96.10.in-addr.arpa. udp 40 false 512" NOERROR qr,aa,rd 112 0.000082841s
[INFO] 10.244.1.2:54143 - 4 "AAAA IN kubernetes.default.default.svc.cluster.local. udp 62 false 512" NXDOMAIN qr,aa,rd 155 0.000098038s
[INFO] 10.244.1.2:38802 - 9 "PTR IN 1.0.96.10.in-addr.arpa. udp 40 false 512" NOERROR qr,aa,rd 112 0.000046632s
[INFO] 10.244.0.4:39532 - 5 "PTR IN 1.0.96.10.in-addr.arpa. udp 40 false 512" NOERROR qr,aa,rd 112 0.002579505s
[INFO] 10.244.2.2:53978 - 2 "PTR IN 10.0.96.10.in-addr.arpa. udp 41 false 512" NOERROR qr,aa,rd 116 0.000077749s
[INFO] 10.244.2.2:60710 - 4 "A IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 106 0.000092889s
[INFO] 10.244.2.2:51255 - 5 "PTR IN 1.0.96.10.in-addr.arpa. udp 40 false 512" NOERROR qr,aa,rd 112 0.000044117s
[INFO] 10.244.1.2:36996 - 4 "A IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd 106 0.000056219s
[INFO] 10.244.1.2:39487 - 5 "PTR IN 1.0.96.10.in-addr.arpa. udp 40 false 512" NOERROR qr,aa,rd 112 0.000090704s
[INFO] 10.244.0.4:39831 - 2 "PTR IN 10.0.96.10.in-addr.arpa. udp 41 false 512" NOERROR qr,aa,rd 116 0.000131192s
[INFO] 10.244.0.4:35770 - 3 "AAAA IN host.minikube.internal. udp 40 false 512" NOERROR qr,aa,rd 40 0.000154922s
[INFO] 10.244.2.2:45820 - 3 "AAAA IN host.minikube.internal. udp 40 false 512" NOERROR qr,aa,rd 40 0.000113973s
[INFO] 10.244.1.2:44519 - 3 "AAAA IN host.minikube.internal. udp 40 false 512" NOERROR qr,aa,rd 40 0.000120184s
[INFO] SIGTERM: Shutting down servers then terminating
[INFO] plugin/health: Going into lameduck mode for 5s
==> describe nodes <==
Name: ha-857000
Roles: control-plane
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=ha-857000
kubernetes.io/os=linux
minikube.k8s.io/commit=9256ba43b41ea130fa48757ddb8d93db00574f61
minikube.k8s.io/name=ha-857000
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_09_17T02_00_55_0700
minikube.k8s.io/version=v1.34.0
node-role.kubernetes.io/control-plane=
node.kubernetes.io/exclude-from-external-load-balancers=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Tue, 17 Sep 2024 09:00:51 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: ha-857000
AcquireTime: <unset>
RenewTime: Tue, 17 Sep 2024 09:13:02 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Tue, 17 Sep 2024 09:11:41 +0000 Tue, 17 Sep 2024 09:00:50 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Tue, 17 Sep 2024 09:11:41 +0000 Tue, 17 Sep 2024 09:00:50 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Tue, 17 Sep 2024 09:11:41 +0000 Tue, 17 Sep 2024 09:00:50 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Tue, 17 Sep 2024 09:11:41 +0000 Tue, 17 Sep 2024 09:01:16 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.169.0.5
Hostname: ha-857000
Capacity:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164336Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164336Ki
pods: 110
System Info:
Machine ID: 54854ca4cf93431694d9ad27a68ef89d
System UUID: f6fb40b6-0000-0000-91c0-dbf4ea1b682c
Boot ID: a1af0517-f4c2-4eae-96db-f7479d049a6a
Kernel Version: 5.10.207
OS Image: Buildroot 2023.02.9
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://27.2.1
Kubelet Version: v1.31.1
Kube-Proxy Version: v1.31.1
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (11 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox-7dff88458-4jzg8 0 (0%) 0 (0%) 0 (0%) 0 (0%) 9m36s
kube-system coredns-7c65d6cfc9-fg65r 100m (5%) 0 (0%) 70Mi (3%) 170Mi (8%) 12m
kube-system coredns-7c65d6cfc9-nl5j5 100m (5%) 0 (0%) 70Mi (3%) 170Mi (8%) 12m
kube-system etcd-ha-857000 100m (5%) 0 (0%) 100Mi (4%) 0 (0%) 12m
kube-system kindnet-7pf7v 100m (5%) 100m (5%) 50Mi (2%) 50Mi (2%) 12m
kube-system kube-apiserver-ha-857000 250m (12%) 0 (0%) 0 (0%) 0 (0%) 12m
kube-system kube-controller-manager-ha-857000 200m (10%) 0 (0%) 0 (0%) 0 (0%) 12m
kube-system kube-proxy-vskbj 0 (0%) 0 (0%) 0 (0%) 0 (0%) 12m
kube-system kube-scheduler-ha-857000 100m (5%) 0 (0%) 0 (0%) 0 (0%) 12m
kube-system kube-vip-ha-857000 0 (0%) 0 (0%) 0 (0%) 0 (0%) 51s
kube-system storage-provisioner 0 (0%) 0 (0%) 0 (0%) 0 (0%) 12m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%) 100m (5%)
memory 290Mi (13%) 390Mi (18%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 12m kube-proxy
Normal Starting 50s kube-proxy
Normal Starting 12m kubelet Starting kubelet.
Normal NodeAllocatableEnforced 12m kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 12m kubelet Node ha-857000 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 12m kubelet Node ha-857000 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 12m kubelet Node ha-857000 status is now: NodeHasSufficientPID
Normal RegisteredNode 12m node-controller Node ha-857000 event: Registered Node ha-857000 in Controller
Normal NodeReady 11m kubelet Node ha-857000 status is now: NodeReady
Normal RegisteredNode 11m node-controller Node ha-857000 event: Registered Node ha-857000 in Controller
Normal RegisteredNode 9m57s node-controller Node ha-857000 event: Registered Node ha-857000 in Controller
Normal RegisteredNode 7m48s node-controller Node ha-857000 event: Registered Node ha-857000 in Controller
Normal NodeHasSufficientMemory 2m24s (x8 over 2m24s) kubelet Node ha-857000 status is now: NodeHasSufficientMemory
Normal Starting 2m24s kubelet Starting kubelet.
Normal NodeHasNoDiskPressure 2m24s (x8 over 2m24s) kubelet Node ha-857000 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 2m24s (x7 over 2m24s) kubelet Node ha-857000 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 2m24s kubelet Updated Node Allocatable limit across pods
Normal RegisteredNode 90s node-controller Node ha-857000 event: Registered Node ha-857000 in Controller
Normal RegisteredNode 89s node-controller Node ha-857000 event: Registered Node ha-857000 in Controller
Normal RegisteredNode 61s node-controller Node ha-857000 event: Registered Node ha-857000 in Controller
Name: ha-857000-m02
Roles: control-plane
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=ha-857000-m02
kubernetes.io/os=linux
minikube.k8s.io/commit=9256ba43b41ea130fa48757ddb8d93db00574f61
minikube.k8s.io/name=ha-857000
minikube.k8s.io/primary=false
minikube.k8s.io/updated_at=2024_09_17T02_01_55_0700
minikube.k8s.io/version=v1.34.0
node-role.kubernetes.io/control-plane=
node.kubernetes.io/exclude-from-external-load-balancers=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Tue, 17 Sep 2024 09:01:53 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: ha-857000-m02
AcquireTime: <unset>
RenewTime: Tue, 17 Sep 2024 09:13:00 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Tue, 17 Sep 2024 09:11:41 +0000 Tue, 17 Sep 2024 09:01:53 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Tue, 17 Sep 2024 09:11:41 +0000 Tue, 17 Sep 2024 09:01:53 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Tue, 17 Sep 2024 09:11:41 +0000 Tue, 17 Sep 2024 09:01:53 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Tue, 17 Sep 2024 09:11:41 +0000 Tue, 17 Sep 2024 09:11:41 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.169.0.6
Hostname: ha-857000-m02
Capacity:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164336Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164336Ki
pods: 110
System Info:
Machine ID: 39fe1ffb0a9e4afb9fa3c09c6b13fed7
System UUID: 19404b28-0000-0000-842d-d4858a62cbd3
Boot ID: 625329b0-bed9-4da5-90fd-2859c5b852dc
Kernel Version: 5.10.207
OS Image: Buildroot 2023.02.9
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://27.2.1
Kubelet Version: v1.31.1
Kube-Proxy Version: v1.31.1
PodCIDR: 10.244.1.0/24
PodCIDRs: 10.244.1.0/24
Non-terminated Pods: (8 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox-7dff88458-mhjf6 0 (0%) 0 (0%) 0 (0%) 0 (0%) 9m37s
kube-system etcd-ha-857000-m02 100m (5%) 0 (0%) 100Mi (4%) 0 (0%) 11m
kube-system kindnet-vh2h2 100m (5%) 100m (5%) 50Mi (2%) 50Mi (2%) 11m
kube-system kube-apiserver-ha-857000-m02 250m (12%) 0 (0%) 0 (0%) 0 (0%) 11m
kube-system kube-controller-manager-ha-857000-m02 200m (10%) 0 (0%) 0 (0%) 0 (0%) 11m
kube-system kube-proxy-zrqvr 0 (0%) 0 (0%) 0 (0%) 0 (0%) 11m
kube-system kube-scheduler-ha-857000-m02 100m (5%) 0 (0%) 0 (0%) 0 (0%) 11m
kube-system kube-vip-ha-857000-m02 0 (0%) 0 (0%) 0 (0%) 0 (0%) 11m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 750m (37%) 100m (5%)
memory 150Mi (7%) 50Mi (2%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 87s kube-proxy
Normal Starting 7m52s kube-proxy
Normal Starting 11m kube-proxy
Normal NodeAllocatableEnforced 11m kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 11m (x8 over 11m) kubelet Node ha-857000-m02 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 11m (x8 over 11m) kubelet Node ha-857000-m02 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 11m (x7 over 11m) kubelet Node ha-857000-m02 status is now: NodeHasSufficientPID
Normal RegisteredNode 11m node-controller Node ha-857000-m02 event: Registered Node ha-857000-m02 in Controller
Normal RegisteredNode 11m node-controller Node ha-857000-m02 event: Registered Node ha-857000-m02 in Controller
Normal RegisteredNode 9m58s node-controller Node ha-857000-m02 event: Registered Node ha-857000-m02 in Controller
Normal Starting 7m57s kubelet Starting kubelet.
Warning Rebooted 7m57s kubelet Node ha-857000-m02 has been rebooted, boot id: b4c87c19-d878-45a1-b0c5-442ae4d2861b
Normal NodeHasSufficientPID 7m57s kubelet Node ha-857000-m02 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 7m57s kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 7m57s kubelet Node ha-857000-m02 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 7m57s kubelet Node ha-857000-m02 status is now: NodeHasNoDiskPressure
Normal RegisteredNode 7m49s node-controller Node ha-857000-m02 event: Registered Node ha-857000-m02 in Controller
Normal Starting 103s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 103s (x8 over 103s) kubelet Node ha-857000-m02 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 103s (x8 over 103s) kubelet Node ha-857000-m02 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 103s (x7 over 103s) kubelet Node ha-857000-m02 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 103s kubelet Updated Node Allocatable limit across pods
Normal RegisteredNode 91s node-controller Node ha-857000-m02 event: Registered Node ha-857000-m02 in Controller
Normal RegisteredNode 90s node-controller Node ha-857000-m02 event: Registered Node ha-857000-m02 in Controller
Normal RegisteredNode 62s node-controller Node ha-857000-m02 event: Registered Node ha-857000-m02 in Controller
Name: ha-857000-m03
Roles: control-plane
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=ha-857000-m03
kubernetes.io/os=linux
minikube.k8s.io/commit=9256ba43b41ea130fa48757ddb8d93db00574f61
minikube.k8s.io/name=ha-857000
minikube.k8s.io/primary=false
minikube.k8s.io/updated_at=2024_09_17T02_03_07_0700
minikube.k8s.io/version=v1.34.0
node-role.kubernetes.io/control-plane=
node.kubernetes.io/exclude-from-external-load-balancers=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Tue, 17 Sep 2024 09:03:04 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: ha-857000-m03
AcquireTime: <unset>
RenewTime: Tue, 17 Sep 2024 09:13:02 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Tue, 17 Sep 2024 09:12:01 +0000 Tue, 17 Sep 2024 09:03:04 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Tue, 17 Sep 2024 09:12:01 +0000 Tue, 17 Sep 2024 09:03:04 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Tue, 17 Sep 2024 09:12:01 +0000 Tue, 17 Sep 2024 09:03:04 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Tue, 17 Sep 2024 09:12:01 +0000 Tue, 17 Sep 2024 09:03:26 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.169.0.7
Hostname: ha-857000-m03
Capacity:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164336Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164336Ki
pods: 110
System Info:
Machine ID: 69dae176c7914316a8660d135e30666c
System UUID: 3d8f47ea-0000-0000-a80b-a24a99cad96e
Boot ID: e1620cd8-3a62-4426-a27e-eeaf7b39756d
Kernel Version: 5.10.207
OS Image: Buildroot 2023.02.9
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://27.2.1
Kubelet Version: v1.31.1
Kube-Proxy Version: v1.31.1
PodCIDR: 10.244.2.0/24
PodCIDRs: 10.244.2.0/24
Non-terminated Pods: (8 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox-7dff88458-5x9l8 0 (0%) 0 (0%) 0 (0%) 0 (0%) 9m37s
kube-system etcd-ha-857000-m03 100m (5%) 0 (0%) 100Mi (4%) 0 (0%) 10m
kube-system kindnet-vc6z5 100m (5%) 100m (5%) 50Mi (2%) 50Mi (2%) 10m
kube-system kube-apiserver-ha-857000-m03 250m (12%) 0 (0%) 0 (0%) 0 (0%) 10m
kube-system kube-controller-manager-ha-857000-m03 200m (10%) 0 (0%) 0 (0%) 0 (0%) 10m
kube-system kube-proxy-g9wxm 0 (0%) 0 (0%) 0 (0%) 0 (0%) 10m
kube-system kube-scheduler-ha-857000-m03 100m (5%) 0 (0%) 0 (0%) 0 (0%) 10m
kube-system kube-vip-ha-857000-m03 0 (0%) 0 (0%) 0 (0%) 0 (0%) 10m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 750m (37%) 100m (5%)
memory 150Mi (7%) 50Mi (2%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 65s kube-proxy
Normal Starting 10m kube-proxy
Normal NodeAllocatableEnforced 10m kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 10m (x8 over 10m) kubelet Node ha-857000-m03 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 10m (x8 over 10m) kubelet Node ha-857000-m03 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 10m (x7 over 10m) kubelet Node ha-857000-m03 status is now: NodeHasSufficientPID
Normal RegisteredNode 10m node-controller Node ha-857000-m03 event: Registered Node ha-857000-m03 in Controller
Normal RegisteredNode 10m node-controller Node ha-857000-m03 event: Registered Node ha-857000-m03 in Controller
Normal RegisteredNode 9m58s node-controller Node ha-857000-m03 event: Registered Node ha-857000-m03 in Controller
Normal RegisteredNode 7m49s node-controller Node ha-857000-m03 event: Registered Node ha-857000-m03 in Controller
Normal RegisteredNode 91s node-controller Node ha-857000-m03 event: Registered Node ha-857000-m03 in Controller
Normal RegisteredNode 90s node-controller Node ha-857000-m03 event: Registered Node ha-857000-m03 in Controller
Normal Starting 70s kubelet Starting kubelet.
Normal NodeAllocatableEnforced 69s kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 69s kubelet Node ha-857000-m03 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 69s kubelet Node ha-857000-m03 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 69s kubelet Node ha-857000-m03 status is now: NodeHasSufficientPID
Warning Rebooted 69s kubelet Node ha-857000-m03 has been rebooted, boot id: e1620cd8-3a62-4426-a27e-eeaf7b39756d
Normal RegisteredNode 62s node-controller Node ha-857000-m03 event: Registered Node ha-857000-m03 in Controller
Name: ha-857000-m04
Roles: <none>
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=ha-857000-m04
kubernetes.io/os=linux
minikube.k8s.io/commit=9256ba43b41ea130fa48757ddb8d93db00574f61
minikube.k8s.io/name=ha-857000
minikube.k8s.io/primary=false
minikube.k8s.io/updated_at=2024_09_17T02_04_06_0700
minikube.k8s.io/version=v1.34.0
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Tue, 17 Sep 2024 09:04:05 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: ha-857000-m04
AcquireTime: <unset>
RenewTime: Tue, 17 Sep 2024 09:12:59 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Tue, 17 Sep 2024 09:12:39 +0000 Tue, 17 Sep 2024 09:12:39 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Tue, 17 Sep 2024 09:12:39 +0000 Tue, 17 Sep 2024 09:12:39 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Tue, 17 Sep 2024 09:12:39 +0000 Tue, 17 Sep 2024 09:12:39 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Tue, 17 Sep 2024 09:12:39 +0000 Tue, 17 Sep 2024 09:12:39 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.169.0.8
Hostname: ha-857000-m04
Capacity:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164336Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164336Ki
pods: 110
System Info:
Machine ID: 15c3f15f82fe4af0a76f2083dcf53a13
System UUID: 32bc423b-0000-0000-90a4-5417ea5ec912
Boot ID: cd10fc3d-989b-457a-8925-881b38fed37e
Kernel Version: 5.10.207
OS Image: Buildroot 2023.02.9
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://27.2.1
Kubelet Version: v1.31.1
Kube-Proxy Version: v1.31.1
PodCIDR: 10.244.3.0/24
PodCIDRs: 10.244.3.0/24
Non-terminated Pods: (2 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
kube-system kindnet-4jk9v 100m (5%) 100m (5%) 50Mi (2%) 50Mi (2%) 9m5s
kube-system kube-proxy-528ht 0 (0%) 0 (0%) 0 (0%) 0 (0%) 9m5s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 100m (5%) 100m (5%)
memory 50Mi (2%) 50Mi (2%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 8m58s kube-proxy
Normal Starting 28s kube-proxy
Normal NodeHasSufficientMemory 9m5s (x2 over 9m5s) kubelet Node ha-857000-m04 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 9m5s (x2 over 9m5s) kubelet Node ha-857000-m04 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 9m5s (x2 over 9m5s) kubelet Node ha-857000-m04 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 9m5s kubelet Updated Node Allocatable limit across pods
Normal RegisteredNode 9m4s node-controller Node ha-857000-m04 event: Registered Node ha-857000-m04 in Controller
Normal RegisteredNode 9m3s node-controller Node ha-857000-m04 event: Registered Node ha-857000-m04 in Controller
Normal RegisteredNode 9m3s node-controller Node ha-857000-m04 event: Registered Node ha-857000-m04 in Controller
Normal NodeReady 8m42s kubelet Node ha-857000-m04 status is now: NodeReady
Normal RegisteredNode 7m49s node-controller Node ha-857000-m04 event: Registered Node ha-857000-m04 in Controller
Normal RegisteredNode 91s node-controller Node ha-857000-m04 event: Registered Node ha-857000-m04 in Controller
Normal RegisteredNode 90s node-controller Node ha-857000-m04 event: Registered Node ha-857000-m04 in Controller
Normal RegisteredNode 62s node-controller Node ha-857000-m04 event: Registered Node ha-857000-m04 in Controller
Normal NodeNotReady 51s node-controller Node ha-857000-m04 status is now: NodeNotReady
Normal Starting 31s kubelet Starting kubelet.
Normal NodeAllocatableEnforced 31s kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 31s (x3 over 31s) kubelet Node ha-857000-m04 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 31s (x3 over 31s) kubelet Node ha-857000-m04 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 31s (x3 over 31s) kubelet Node ha-857000-m04 status is now: NodeHasSufficientPID
Warning Rebooted 31s (x2 over 31s) kubelet Node ha-857000-m04 has been rebooted, boot id: cd10fc3d-989b-457a-8925-881b38fed37e
Normal NodeReady 31s (x2 over 31s) kubelet Node ha-857000-m04 status is now: NodeReady
==> dmesg <==
[ +0.000000] Unless you actually understand what nomodeset does, you should reboot without enabling it
[ +0.035828] ACPI BIOS Warning (bug): Incorrect checksum in table [DSDT] - 0xBE, should be 0x1B (20200925/tbprint-173)
[ +0.007970] RETBleed: WARNING: Spectre v2 mitigation leaves CPU vulnerable to RETBleed attacks, data leaks possible!
[ +5.690889] ACPI Error: Could not enable RealTimeClock event (20200925/evxfevnt-182)
[ +0.000002] ACPI Warning: Could not enable fixed event - RealTimeClock (4) (20200925/evxface-618)
[ +0.006763] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[ +2.660573] systemd-fstab-generator[127]: Ignoring "noauto" option for root device
[ +2.226234] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[ +0.000004] NFSD: unable to find recovery directory /var/lib/nfs/v4recovery
[ +0.000001] NFSD: Unable to initialize client recovery tracking! (-2)
[ +0.530337] systemd-fstab-generator[461]: Ignoring "noauto" option for root device
[ +0.102427] systemd-fstab-generator[473]: Ignoring "noauto" option for root device
[ +1.905407] systemd-fstab-generator[1088]: Ignoring "noauto" option for root device
[ +0.264183] systemd-fstab-generator[1126]: Ignoring "noauto" option for root device
[ +0.055811] kauditd_printk_skb: 101 callbacks suppressed
[ +0.051134] systemd-fstab-generator[1138]: Ignoring "noauto" option for root device
[ +0.114709] systemd-fstab-generator[1152]: Ignoring "noauto" option for root device
[ +2.420834] systemd-fstab-generator[1366]: Ignoring "noauto" option for root device
[ +0.093862] systemd-fstab-generator[1378]: Ignoring "noauto" option for root device
[ +0.101457] systemd-fstab-generator[1390]: Ignoring "noauto" option for root device
[ +0.112591] systemd-fstab-generator[1405]: Ignoring "noauto" option for root device
[ +0.460313] systemd-fstab-generator[1565]: Ignoring "noauto" option for root device
[ +6.769000] kauditd_printk_skb: 212 callbacks suppressed
[Sep17 09:11] kauditd_printk_skb: 40 callbacks suppressed
[Sep17 09:12] kauditd_printk_skb: 78 callbacks suppressed
==> etcd [8c0804e78de8] <==
{"level":"warn","ts":"2024-09-17T09:11:52.642983Z","caller":"rafthttp/peer.go:267","msg":"dropped internal Raft message since sending buffer is full (overloaded network)","message-type":"MsgHeartbeat","local-member-id":"b8c6c7563d17d844","from":"b8c6c7563d17d844","remote-peer-id":"4843c5334ac100b7","remote-peer-name":"pipeline","remote-peer-active":false}
{"level":"warn","ts":"2024-09-17T09:11:52.663148Z","caller":"rafthttp/peer.go:267","msg":"dropped internal Raft message since sending buffer is full (overloaded network)","message-type":"MsgHeartbeat","local-member-id":"b8c6c7563d17d844","from":"b8c6c7563d17d844","remote-peer-id":"4843c5334ac100b7","remote-peer-name":"pipeline","remote-peer-active":false}
{"level":"warn","ts":"2024-09-17T09:11:52.743398Z","caller":"rafthttp/peer.go:267","msg":"dropped internal Raft message since sending buffer is full (overloaded network)","message-type":"MsgHeartbeat","local-member-id":"b8c6c7563d17d844","from":"b8c6c7563d17d844","remote-peer-id":"4843c5334ac100b7","remote-peer-name":"pipeline","remote-peer-active":false}
{"level":"warn","ts":"2024-09-17T09:11:52.834019Z","caller":"etcdserver/cluster_util.go:294","msg":"failed to reach the peer URL","address":"https://192.169.0.7:2380/version","remote-member-id":"4843c5334ac100b7","error":"Get \"https://192.169.0.7:2380/version\": dial tcp 192.169.0.7:2380: connect: connection refused"}
{"level":"warn","ts":"2024-09-17T09:11:52.834231Z","caller":"etcdserver/cluster_util.go:158","msg":"failed to get version","remote-member-id":"4843c5334ac100b7","error":"Get \"https://192.169.0.7:2380/version\": dial tcp 192.169.0.7:2380: connect: connection refused"}
{"level":"warn","ts":"2024-09-17T09:11:56.836371Z","caller":"etcdserver/cluster_util.go:294","msg":"failed to reach the peer URL","address":"https://192.169.0.7:2380/version","remote-member-id":"4843c5334ac100b7","error":"Get \"https://192.169.0.7:2380/version\": dial tcp 192.169.0.7:2380: connect: connection refused"}
{"level":"warn","ts":"2024-09-17T09:11:56.836465Z","caller":"etcdserver/cluster_util.go:158","msg":"failed to get version","remote-member-id":"4843c5334ac100b7","error":"Get \"https://192.169.0.7:2380/version\": dial tcp 192.169.0.7:2380: connect: connection refused"}
{"level":"warn","ts":"2024-09-17T09:11:57.474154Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"4843c5334ac100b7","rtt":"0s","error":"dial tcp 192.169.0.7:2380: connect: connection refused"}
{"level":"warn","ts":"2024-09-17T09:11:57.474326Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"4843c5334ac100b7","rtt":"0s","error":"dial tcp 192.169.0.7:2380: connect: connection refused"}
{"level":"warn","ts":"2024-09-17T09:12:00.837987Z","caller":"etcdserver/cluster_util.go:294","msg":"failed to reach the peer URL","address":"https://192.169.0.7:2380/version","remote-member-id":"4843c5334ac100b7","error":"Get \"https://192.169.0.7:2380/version\": dial tcp 192.169.0.7:2380: connect: connection refused"}
{"level":"warn","ts":"2024-09-17T09:12:00.838171Z","caller":"etcdserver/cluster_util.go:158","msg":"failed to get version","remote-member-id":"4843c5334ac100b7","error":"Get \"https://192.169.0.7:2380/version\": dial tcp 192.169.0.7:2380: connect: connection refused"}
{"level":"warn","ts":"2024-09-17T09:12:02.474909Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"4843c5334ac100b7","rtt":"0s","error":"dial tcp 192.169.0.7:2380: connect: connection refused"}
{"level":"warn","ts":"2024-09-17T09:12:02.474924Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"4843c5334ac100b7","rtt":"0s","error":"dial tcp 192.169.0.7:2380: connect: connection refused"}
{"level":"info","ts":"2024-09-17T09:12:02.527934Z","caller":"rafthttp/peer_status.go:53","msg":"peer became active","peer-id":"4843c5334ac100b7"}
{"level":"info","ts":"2024-09-17T09:12:02.528179Z","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream Message","local-member-id":"b8c6c7563d17d844","remote-peer-id":"4843c5334ac100b7"}
{"level":"info","ts":"2024-09-17T09:12:02.553614Z","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"b8c6c7563d17d844","remote-peer-id":"4843c5334ac100b7"}
{"level":"info","ts":"2024-09-17T09:12:02.656074Z","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"b8c6c7563d17d844","to":"4843c5334ac100b7","stream-type":"stream MsgApp v2"}
{"level":"info","ts":"2024-09-17T09:12:02.656117Z","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"b8c6c7563d17d844","remote-peer-id":"4843c5334ac100b7"}
{"level":"info","ts":"2024-09-17T09:12:02.671567Z","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"b8c6c7563d17d844","to":"4843c5334ac100b7","stream-type":"stream Message"}
{"level":"info","ts":"2024-09-17T09:12:02.671803Z","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"b8c6c7563d17d844","remote-peer-id":"4843c5334ac100b7"}
{"level":"info","ts":"2024-09-17T09:12:03.645158Z","caller":"traceutil/trace.go:171","msg":"trace[1621339428] linearizableReadLoop","detail":"{readStateIndex:2219; appliedIndex:2219; }","duration":"123.347982ms","start":"2024-09-17T09:12:03.521794Z","end":"2024-09-17T09:12:03.645142Z","steps":["trace[1621339428] 'read index received' (duration: 123.341929ms)","trace[1621339428] 'applied index is now lower than readState.Index' (duration: 4.903µs)"],"step_count":2}
{"level":"warn","ts":"2024-09-17T09:12:03.645527Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"123.681467ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/kube-proxy-g9wxm\" ","response":"range_response_count:1 size:5191"}
{"level":"info","ts":"2024-09-17T09:12:03.645594Z","caller":"traceutil/trace.go:171","msg":"trace[2012729741] range","detail":"{range_begin:/registry/pods/kube-system/kube-proxy-g9wxm; range_end:; response_count:1; response_revision:1897; }","duration":"123.79703ms","start":"2024-09-17T09:12:03.521791Z","end":"2024-09-17T09:12:03.645588Z","steps":["trace[2012729741] 'agreement among raft nodes before linearized reading' (duration: 123.482937ms)"],"step_count":1}
{"level":"info","ts":"2024-09-17T09:12:03.647767Z","caller":"traceutil/trace.go:171","msg":"trace[1450641964] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1898; }","duration":"121.988853ms","start":"2024-09-17T09:12:03.525765Z","end":"2024-09-17T09:12:03.647754Z","steps":["trace[1450641964] 'process raft request' (duration: 121.923204ms)"],"step_count":1}
{"level":"info","ts":"2024-09-17T09:13:08.579639Z","caller":"traceutil/trace.go:171","msg":"trace[2135392401] transaction","detail":"{read_only:false; response_revision:2205; number_of_response:1; }","duration":"108.477653ms","start":"2024-09-17T09:13:08.471150Z","end":"2024-09-17T09:13:08.579628Z","steps":["trace[2135392401] 'process raft request' (duration: 108.403212ms)"],"step_count":1}
==> etcd [f4f59b8c7640] <==
{"level":"info","ts":"2024-09-17T09:10:21.875702Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 is starting a new election at term 2"}
{"level":"info","ts":"2024-09-17T09:10:21.875849Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 became pre-candidate at term 2"}
{"level":"info","ts":"2024-09-17T09:10:21.875892Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 received MsgPreVoteResp from b8c6c7563d17d844 at term 2"}
{"level":"info","ts":"2024-09-17T09:10:21.875927Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 [logterm: 2, index: 1893] sent MsgPreVote request to 4843c5334ac100b7 at term 2"}
{"level":"info","ts":"2024-09-17T09:10:21.875956Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 [logterm: 2, index: 1893] sent MsgPreVote request to ff70cdb626651bff at term 2"}
{"level":"info","ts":"2024-09-17T09:10:23.177744Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 is starting a new election at term 2"}
{"level":"info","ts":"2024-09-17T09:10:23.177829Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 became pre-candidate at term 2"}
{"level":"info","ts":"2024-09-17T09:10:23.177849Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 received MsgPreVoteResp from b8c6c7563d17d844 at term 2"}
{"level":"info","ts":"2024-09-17T09:10:23.177868Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 [logterm: 2, index: 1893] sent MsgPreVote request to 4843c5334ac100b7 at term 2"}
{"level":"info","ts":"2024-09-17T09:10:23.177879Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 [logterm: 2, index: 1893] sent MsgPreVote request to ff70cdb626651bff at term 2"}
{"level":"warn","ts":"2024-09-17T09:10:23.692511Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":15583741136013275704,"retry-timeout":"500ms"}
{"level":"warn","ts":"2024-09-17T09:10:24.194017Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":15583741136013275704,"retry-timeout":"500ms"}
{"level":"warn","ts":"2024-09-17T09:10:24.278276Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"ff70cdb626651bff","rtt":"0s","error":"dial tcp 192.169.0.6:2380: connect: connection refused"}
{"level":"warn","ts":"2024-09-17T09:10:24.278324Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"ff70cdb626651bff","rtt":"0s","error":"dial tcp 192.169.0.6:2380: connect: connection refused"}
{"level":"warn","ts":"2024-09-17T09:10:24.301258Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"4843c5334ac100b7","rtt":"0s","error":"dial tcp 192.169.0.7:2380: connect: no route to host"}
{"level":"warn","ts":"2024-09-17T09:10:24.301488Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"4843c5334ac100b7","rtt":"0s","error":"dial tcp 192.169.0.7:2380: connect: no route to host"}
{"level":"info","ts":"2024-09-17T09:10:24.470887Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 is starting a new election at term 2"}
{"level":"info","ts":"2024-09-17T09:10:24.470976Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 became pre-candidate at term 2"}
{"level":"info","ts":"2024-09-17T09:10:24.470995Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 received MsgPreVoteResp from b8c6c7563d17d844 at term 2"}
{"level":"info","ts":"2024-09-17T09:10:24.471013Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 [logterm: 2, index: 1893] sent MsgPreVote request to 4843c5334ac100b7 at term 2"}
{"level":"info","ts":"2024-09-17T09:10:24.471022Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8c6c7563d17d844 [logterm: 2, index: 1893] sent MsgPreVote request to ff70cdb626651bff at term 2"}
{"level":"warn","ts":"2024-09-17T09:10:24.694867Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":15583741136013275704,"retry-timeout":"500ms"}
{"level":"warn","ts":"2024-09-17T09:10:24.938557Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.746471868s","expected-duration":"100ms","prefix":"read-only range ","request":"limit:1 keys_only:true ","response":"","error":"context canceled"}
{"level":"info","ts":"2024-09-17T09:10:24.938607Z","caller":"traceutil/trace.go:171","msg":"trace[802347161] range","detail":"{range_begin:; range_end:; }","duration":"1.746534049s","start":"2024-09-17T09:10:23.192066Z","end":"2024-09-17T09:10:24.938600Z","steps":["trace[802347161] 'agreement among raft nodes before linearized reading' (duration: 1.746469617s)"],"step_count":1}
{"level":"error","ts":"2024-09-17T09:10:24.938646Z","caller":"etcdhttp/health.go:367","msg":"Health check error","path":"/readyz","reason":"[-]linearizable_read failed: context canceled\n[+]data_corruption ok\n[+]serializable_read ok\n","status-code":503,"stacktrace":"go.etcd.io/etcd/server/v3/etcdserver/api/etcdhttp.(*CheckRegistry).installRootHttpEndpoint.newHealthHandler.func2\n\tgo.etcd.io/etcd/server/v3/etcdserver/api/etcdhttp/health.go:367\nnet/http.HandlerFunc.ServeHTTP\n\tnet/http/server.go:2141\nnet/http.(*ServeMux).ServeHTTP\n\tnet/http/server.go:2519\nnet/http.serverHandler.ServeHTTP\n\tnet/http/server.go:2943\nnet/http.(*conn).serve\n\tnet/http/server.go:2014"}
==> kernel <==
09:13:10 up 2 min, 0 users, load average: 1.00, 0.42, 0.15
Linux ha-857000 5.10.207 #1 SMP Sun Sep 15 20:39:46 UTC 2024 x86_64 GNU/Linux
PRETTY_NAME="Buildroot 2023.02.9"
==> kindnet [3d889c7c8da7] <==
I0917 09:12:39.612978 1 main.go:322] Node ha-857000-m04 has CIDR [10.244.3.0/24]
I0917 09:12:49.606629 1 main.go:295] Handling node with IPs: map[192.169.0.5:{}]
I0917 09:12:49.606712 1 main.go:299] handling current node
I0917 09:12:49.606742 1 main.go:295] Handling node with IPs: map[192.169.0.6:{}]
I0917 09:12:49.606793 1 main.go:322] Node ha-857000-m02 has CIDR [10.244.1.0/24]
I0917 09:12:49.606920 1 main.go:295] Handling node with IPs: map[192.169.0.7:{}]
I0917 09:12:49.606967 1 main.go:322] Node ha-857000-m03 has CIDR [10.244.2.0/24]
I0917 09:12:49.607060 1 main.go:295] Handling node with IPs: map[192.169.0.8:{}]
I0917 09:12:49.607108 1 main.go:322] Node ha-857000-m04 has CIDR [10.244.3.0/24]
I0917 09:12:59.612269 1 main.go:295] Handling node with IPs: map[192.169.0.5:{}]
I0917 09:12:59.612291 1 main.go:299] handling current node
I0917 09:12:59.612301 1 main.go:295] Handling node with IPs: map[192.169.0.6:{}]
I0917 09:12:59.612305 1 main.go:322] Node ha-857000-m02 has CIDR [10.244.1.0/24]
I0917 09:12:59.612392 1 main.go:295] Handling node with IPs: map[192.169.0.7:{}]
I0917 09:12:59.612417 1 main.go:322] Node ha-857000-m03 has CIDR [10.244.2.0/24]
I0917 09:12:59.612453 1 main.go:295] Handling node with IPs: map[192.169.0.8:{}]
I0917 09:12:59.612507 1 main.go:322] Node ha-857000-m04 has CIDR [10.244.3.0/24]
I0917 09:13:09.611849 1 main.go:295] Handling node with IPs: map[192.169.0.5:{}]
I0917 09:13:09.611906 1 main.go:299] handling current node
I0917 09:13:09.611932 1 main.go:295] Handling node with IPs: map[192.169.0.6:{}]
I0917 09:13:09.611940 1 main.go:322] Node ha-857000-m02 has CIDR [10.244.1.0/24]
I0917 09:13:09.612064 1 main.go:295] Handling node with IPs: map[192.169.0.7:{}]
I0917 09:13:09.612072 1 main.go:322] Node ha-857000-m03 has CIDR [10.244.2.0/24]
I0917 09:13:09.612157 1 main.go:295] Handling node with IPs: map[192.169.0.8:{}]
I0917 09:13:09.612166 1 main.go:322] Node ha-857000-m04 has CIDR [10.244.3.0/24]
==> kindnet [5d84a01abd3e] <==
I0917 09:05:22.964948 1 main.go:322] Node ha-857000-m04 has CIDR [10.244.3.0/24]
I0917 09:05:32.966280 1 main.go:295] Handling node with IPs: map[192.169.0.5:{}]
I0917 09:05:32.966503 1 main.go:299] handling current node
I0917 09:05:32.966605 1 main.go:295] Handling node with IPs: map[192.169.0.6:{}]
I0917 09:05:32.966739 1 main.go:322] Node ha-857000-m02 has CIDR [10.244.1.0/24]
I0917 09:05:32.966951 1 main.go:295] Handling node with IPs: map[192.169.0.7:{}]
I0917 09:05:32.967059 1 main.go:322] Node ha-857000-m03 has CIDR [10.244.2.0/24]
I0917 09:05:32.967333 1 main.go:295] Handling node with IPs: map[192.169.0.8:{}]
I0917 09:05:32.967449 1 main.go:322] Node ha-857000-m04 has CIDR [10.244.3.0/24]
I0917 09:05:42.964585 1 main.go:295] Handling node with IPs: map[192.169.0.5:{}]
I0917 09:05:42.964999 1 main.go:299] handling current node
I0917 09:05:42.965252 1 main.go:295] Handling node with IPs: map[192.169.0.6:{}]
I0917 09:05:42.965422 1 main.go:322] Node ha-857000-m02 has CIDR [10.244.1.0/24]
I0917 09:05:42.965746 1 main.go:295] Handling node with IPs: map[192.169.0.7:{}]
I0917 09:05:42.965829 1 main.go:322] Node ha-857000-m03 has CIDR [10.244.2.0/24]
I0917 09:05:42.966204 1 main.go:295] Handling node with IPs: map[192.169.0.8:{}]
I0917 09:05:42.966357 1 main.go:322] Node ha-857000-m04 has CIDR [10.244.3.0/24]
I0917 09:05:52.965279 1 main.go:295] Handling node with IPs: map[192.169.0.6:{}]
I0917 09:05:52.965376 1 main.go:322] Node ha-857000-m02 has CIDR [10.244.1.0/24]
I0917 09:05:52.965533 1 main.go:295] Handling node with IPs: map[192.169.0.7:{}]
I0917 09:05:52.965592 1 main.go:322] Node ha-857000-m03 has CIDR [10.244.2.0/24]
I0917 09:05:52.965673 1 main.go:295] Handling node with IPs: map[192.169.0.8:{}]
I0917 09:05:52.965753 1 main.go:322] Node ha-857000-m04 has CIDR [10.244.3.0/24]
I0917 09:05:52.965812 1 main.go:295] Handling node with IPs: map[192.169.0.5:{}]
I0917 09:05:52.965902 1 main.go:299] handling current node
==> kube-apiserver [475dedee3722] <==
I0917 09:11:36.333360 1 dynamic_cafile_content.go:160] "Starting controller" name="client-ca-bundle::/var/lib/minikube/certs/ca.crt"
I0917 09:11:36.335609 1 dynamic_cafile_content.go:160] "Starting controller" name="request-header::/var/lib/minikube/certs/front-proxy-ca.crt"
I0917 09:11:36.383731 1 apf_controller.go:382] Running API Priority and Fairness config worker
I0917 09:11:36.383763 1 apf_controller.go:385] Running API Priority and Fairness periodic rebalancing process
I0917 09:11:36.384428 1 cache.go:39] Caches are synced for LocalAvailability controller
I0917 09:11:36.385090 1 shared_informer.go:320] Caches are synced for configmaps
I0917 09:11:36.385168 1 shared_informer.go:320] Caches are synced for crd-autoregister
I0917 09:11:36.385606 1 aggregator.go:171] initial CRD sync complete...
I0917 09:11:36.385745 1 autoregister_controller.go:144] Starting autoregister controller
I0917 09:11:36.386077 1 cache.go:32] Waiting for caches to sync for autoregister controller
I0917 09:11:36.386187 1 cache.go:39] Caches are synced for autoregister controller
I0917 09:11:36.388938 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller
I0917 09:11:36.396198 1 cache.go:39] Caches are synced for RemoteAvailability controller
I0917 09:11:36.396611 1 handler_discovery.go:450] Starting ResourceDiscoveryManager
I0917 09:11:36.396812 1 shared_informer.go:320] Caches are synced for cluster_authentication_trust_controller
W0917 09:11:36.438133 1 lease.go:265] Resetting endpoints for master service "kubernetes" to [192.169.0.6]
I0917 09:11:36.461867 1 shared_informer.go:320] Caches are synced for node_authorizer
I0917 09:11:36.465355 1 shared_informer.go:320] Caches are synced for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
I0917 09:11:36.465387 1 policy_source.go:224] refreshing policies
I0917 09:11:36.484251 1 controller.go:615] quota admission added evaluator for: leases.coordination.k8s.io
I0917 09:11:36.540432 1 controller.go:615] quota admission added evaluator for: endpoints
I0917 09:11:36.548136 1 controller.go:615] quota admission added evaluator for: endpointslices.discovery.k8s.io
E0917 09:11:36.554355 1 controller.go:95] Found stale data, removed previous endpoints on kubernetes service, apiserver didn't exit successfully previously
I0917 09:11:37.296848 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist.
W0917 09:11:37.666999 1 lease.go:265] Resetting endpoints for master service "kubernetes" to [192.169.0.5]
==> kube-apiserver [a18a6b023cd6] <==
I0917 09:10:52.375949 1 options.go:228] external host was not specified, using 192.169.0.5
I0917 09:10:52.377617 1 server.go:142] Version: v1.31.1
I0917 09:10:52.377684 1 server.go:144] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0917 09:10:52.824178 1 shared_informer.go:313] Waiting for caches to sync for node_authorizer
I0917 09:10:52.824356 1 shared_informer.go:313] Waiting for caches to sync for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
I0917 09:10:52.826684 1 plugins.go:157] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0917 09:10:52.828510 1 plugins.go:160] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota.
I0917 09:10:52.829505 1 instance.go:232] Using reconciler: lease
W0917 09:11:12.810788 1 logging.go:55] [core] [Channel #1 SubChannel #3]grpc: addrConn.createTransport failed to connect to {Addr: "127.0.0.1:2379", ServerName: "127.0.0.1:2379", }. Err: connection error: desc = "transport: authentication handshake failed: context canceled"
W0917 09:11:12.813364 1 logging.go:55] [core] [Channel #2 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: "127.0.0.1:2379", ServerName: "127.0.0.1:2379", }. Err: connection error: desc = "transport: authentication handshake failed: context canceled"
W0917 09:11:12.831731 1 logging.go:55] [core] [Channel #5 SubChannel #6]grpc: addrConn.createTransport failed to connect to {Addr: "127.0.0.1:2379", ServerName: "127.0.0.1:2379", }. Err: connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
F0917 09:11:12.831919 1 instance.go:225] Error creating leases: error creating storage factory: context deadline exceeded
==> kube-controller-manager [13b7f8a93ad4] <==
I0917 09:10:53.058887 1 serving.go:386] Generated self-signed cert in-memory
I0917 09:10:53.469010 1 controllermanager.go:197] "Starting" version="v1.31.1"
I0917 09:10:53.469133 1 controllermanager.go:199] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0917 09:10:53.478660 1 tlsconfig.go:243] "Starting DynamicServingCertificateController"
I0917 09:10:53.478827 1 dynamic_cafile_content.go:160] "Starting controller" name="request-header::/var/lib/minikube/certs/front-proxy-ca.crt"
I0917 09:10:53.478677 1 secure_serving.go:213] Serving securely on 127.0.0.1:10257
I0917 09:10:53.479256 1 dynamic_cafile_content.go:160] "Starting controller" name="client-ca-bundle::/var/lib/minikube/certs/ca.crt"
E0917 09:11:13.838538 1 controllermanager.go:242] "Error building controller context" err="failed to wait for apiserver being healthy: timed out waiting for the condition: failed to get apiserver /healthz status: Get \"https://192.169.0.5:8443/healthz\": dial tcp 192.169.0.5:8443: connect: connection refused"
==> kube-controller-manager [ca7fe8ccd4c5] <==
I0917 09:12:17.473758 1 replica_set.go:679] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="kube-system/coredns-7c65d6cfc9" duration="14.760263ms"
I0917 09:12:17.473945 1 replica_set.go:679] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="kube-system/coredns-7c65d6cfc9" duration="34.651µs"
I0917 09:12:18.632033 1 replica_set.go:679] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="kube-system/coredns-7c65d6cfc9" duration="35.896µs"
I0917 09:12:18.776005 1 replica_set.go:679] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="default/busybox-7dff88458" duration="29.253969ms"
I0917 09:12:18.776119 1 replica_set.go:679] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="default/busybox-7dff88458" duration="71.789µs"
I0917 09:12:18.785648 1 replica_set.go:679] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="kube-system/coredns-7c65d6cfc9" duration="39.503µs"
I0917 09:12:18.798097 1 endpointslice_controller.go:344] "Error syncing endpoint slices for service, retrying" logger="endpointslice-controller" key="kube-system/kube-dns" err="failed to update kube-dns-f4rqd EndpointSlice for Service kube-system/kube-dns: Operation cannot be fulfilled on endpointslices.discovery.k8s.io \"kube-dns-f4rqd\": the object has been modified; please apply your changes to the latest version and try again"
I0917 09:12:18.798477 1 event.go:377] Event(v1.ObjectReference{Kind:"Service", Namespace:"kube-system", Name:"kube-dns", UID:"fbe3dede-bdc6-453b-baec-6a20140ca1b1", APIVersion:"v1", ResourceVersion:"261", FieldPath:""}): type: 'Warning' reason: 'FailedToUpdateEndpointSlices' Error updating Endpoint Slices for Service kube-system/kube-dns: failed to update kube-dns-f4rqd EndpointSlice for Service kube-system/kube-dns: Operation cannot be fulfilled on endpointslices.discovery.k8s.io "kube-dns-f4rqd": the object has been modified; please apply your changes to the latest version and try again
I0917 09:12:19.953163 1 range_allocator.go:241] "Successfully synced" logger="node-ipam-controller" key="ha-857000-m04"
I0917 09:12:19.967434 1 range_allocator.go:241] "Successfully synced" logger="node-ipam-controller" key="ha-857000-m04"
I0917 09:12:20.682851 1 range_allocator.go:241] "Successfully synced" logger="node-ipam-controller" key="ha-857000-m04"
I0917 09:12:23.128380 1 range_allocator.go:241] "Successfully synced" logger="node-ipam-controller" key="ha-857000-m04"
I0917 09:12:25.083746 1 range_allocator.go:241] "Successfully synced" logger="node-ipam-controller" key="ha-857000-m04"
I0917 09:12:39.721967 1 topologycache.go:237] "Can't get CPU or zone information for node" logger="endpointslice-controller" node="ha-857000-m04"
I0917 09:12:39.722197 1 range_allocator.go:241] "Successfully synced" logger="node-ipam-controller" key="ha-857000-m04"
I0917 09:12:39.733466 1 range_allocator.go:241] "Successfully synced" logger="node-ipam-controller" key="ha-857000-m04"
I0917 09:12:40.010916 1 range_allocator.go:241] "Successfully synced" logger="node-ipam-controller" key="ha-857000-m04"
I0917 09:12:57.587381 1 endpointslice_controller.go:344] "Error syncing endpoint slices for service, retrying" logger="endpointslice-controller" key="kube-system/kube-dns" err="failed to update kube-dns-f4rqd EndpointSlice for Service kube-system/kube-dns: Operation cannot be fulfilled on endpointslices.discovery.k8s.io \"kube-dns-f4rqd\": the object has been modified; please apply your changes to the latest version and try again"
I0917 09:12:57.588538 1 event.go:377] Event(v1.ObjectReference{Kind:"Service", Namespace:"kube-system", Name:"kube-dns", UID:"fbe3dede-bdc6-453b-baec-6a20140ca1b1", APIVersion:"v1", ResourceVersion:"261", FieldPath:""}): type: 'Warning' reason: 'FailedToUpdateEndpointSlices' Error updating Endpoint Slices for Service kube-system/kube-dns: failed to update kube-dns-f4rqd EndpointSlice for Service kube-system/kube-dns: Operation cannot be fulfilled on endpointslices.discovery.k8s.io "kube-dns-f4rqd": the object has been modified; please apply your changes to the latest version and try again
I0917 09:12:57.619018 1 replica_set.go:679] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="kube-system/coredns-7c65d6cfc9" duration="58.719907ms"
E0917 09:12:57.619070 1 replica_set.go:560] "Unhandled Error" err="sync \"kube-system/coredns-7c65d6cfc9\" failed with Operation cannot be fulfilled on replicasets.apps \"coredns-7c65d6cfc9\": the object has been modified; please apply your changes to the latest version and try again" logger="UnhandledError"
I0917 09:12:57.620470 1 replica_set.go:679] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="kube-system/coredns-7c65d6cfc9" duration="97.988µs"
I0917 09:12:57.624100 1 endpointslice_controller.go:344] "Error syncing endpoint slices for service, retrying" logger="endpointslice-controller" key="kube-system/kube-dns" err="failed to update kube-dns-f4rqd EndpointSlice for Service kube-system/kube-dns: Operation cannot be fulfilled on endpointslices.discovery.k8s.io \"kube-dns-f4rqd\": the object has been modified; please apply your changes to the latest version and try again"
I0917 09:12:57.624538 1 event.go:377] Event(v1.ObjectReference{Kind:"Service", Namespace:"kube-system", Name:"kube-dns", UID:"fbe3dede-bdc6-453b-baec-6a20140ca1b1", APIVersion:"v1", ResourceVersion:"261", FieldPath:""}): type: 'Warning' reason: 'FailedToUpdateEndpointSlices' Error updating Endpoint Slices for Service kube-system/kube-dns: failed to update kube-dns-f4rqd EndpointSlice for Service kube-system/kube-dns: Operation cannot be fulfilled on endpointslices.discovery.k8s.io "kube-dns-f4rqd": the object has been modified; please apply your changes to the latest version and try again
I0917 09:12:57.625793 1 replica_set.go:679] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="kube-system/coredns-7c65d6cfc9" duration="62.927µs"
==> kube-proxy [0b03e5e48893] <==
add table ip kube-proxy
^^^^^^^^^^^^^^^^^^^^^^^^
>
E0917 09:00:59.069869 1 proxier.go:734] "Error cleaning up nftables rules" err=<
could not run nftables command: /dev/stdin:1:1-25: Error: Could not process rule: Operation not supported
add table ip6 kube-proxy
^^^^^^^^^^^^^^^^^^^^^^^^^
>
I0917 09:00:59.079118 1 server.go:677] "Successfully retrieved node IP(s)" IPs=["192.169.0.5"]
E0917 09:00:59.079199 1 server.go:234] "Kube-proxy configuration may be incomplete or incorrect" err="nodePortAddresses is unset; NodePort connections will be accepted on all local IPs. Consider using `--nodeport-addresses primary`"
I0917 09:00:59.109184 1 server_linux.go:146] "No iptables support for family" ipFamily="IPv6"
I0917 09:00:59.109227 1 server.go:245] "kube-proxy running in single-stack mode" ipFamily="IPv4"
I0917 09:00:59.109245 1 server_linux.go:169] "Using iptables Proxier"
I0917 09:00:59.111661 1 proxier.go:255] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses" ipFamily="IPv4"
I0917 09:00:59.111847 1 server.go:483] "Version info" version="v1.31.1"
I0917 09:00:59.111876 1 server.go:485] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0917 09:00:59.112952 1 config.go:199] "Starting service config controller"
I0917 09:00:59.112979 1 shared_informer.go:313] Waiting for caches to sync for service config
I0917 09:00:59.112995 1 config.go:105] "Starting endpoint slice config controller"
I0917 09:00:59.112998 1 shared_informer.go:313] Waiting for caches to sync for endpoint slice config
I0917 09:00:59.113603 1 config.go:328] "Starting node config controller"
I0917 09:00:59.113673 1 shared_informer.go:313] Waiting for caches to sync for node config
I0917 09:00:59.213587 1 shared_informer.go:320] Caches are synced for endpoint slice config
I0917 09:00:59.213649 1 shared_informer.go:320] Caches are synced for service config
I0917 09:00:59.213808 1 shared_informer.go:320] Caches are synced for node config
==> kube-proxy [c37a677e3118] <==
add table ip kube-proxy
^^^^^^^^^^^^^^^^^^^^^^^^
>
E0917 09:12:19.054558 1 proxier.go:734] "Error cleaning up nftables rules" err=<
could not run nftables command: /dev/stdin:1:1-25: Error: Could not process rule: Operation not supported
add table ip6 kube-proxy
^^^^^^^^^^^^^^^^^^^^^^^^^
>
I0917 09:12:19.080090 1 server.go:677] "Successfully retrieved node IP(s)" IPs=["192.169.0.5"]
E0917 09:12:19.080297 1 server.go:234] "Kube-proxy configuration may be incomplete or incorrect" err="nodePortAddresses is unset; NodePort connections will be accepted on all local IPs. Consider using `--nodeport-addresses primary`"
I0917 09:12:19.208559 1 server_linux.go:146] "No iptables support for family" ipFamily="IPv6"
I0917 09:12:19.208589 1 server.go:245] "kube-proxy running in single-stack mode" ipFamily="IPv4"
I0917 09:12:19.208607 1 server_linux.go:169] "Using iptables Proxier"
I0917 09:12:19.212603 1 proxier.go:255] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses" ipFamily="IPv4"
I0917 09:12:19.213076 1 server.go:483] "Version info" version="v1.31.1"
I0917 09:12:19.213105 1 server.go:485] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0917 09:12:19.216919 1 config.go:199] "Starting service config controller"
I0917 09:12:19.217067 1 config.go:105] "Starting endpoint slice config controller"
I0917 09:12:19.217988 1 shared_informer.go:313] Waiting for caches to sync for endpoint slice config
I0917 09:12:19.218116 1 shared_informer.go:313] Waiting for caches to sync for service config
I0917 09:12:19.228165 1 config.go:328] "Starting node config controller"
I0917 09:12:19.228196 1 shared_informer.go:313] Waiting for caches to sync for node config
I0917 09:12:19.319175 1 shared_informer.go:320] Caches are synced for endpoint slice config
I0917 09:12:19.319361 1 shared_informer.go:320] Caches are synced for service config
I0917 09:12:19.328396 1 shared_informer.go:320] Caches are synced for node config
==> kube-scheduler [00ff29c21371] <==
W0917 09:11:36.373943 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0917 09:11:36.373983 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User \"system:kube-scheduler\" cannot list resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError"
W0917 09:11:36.374259 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0917 09:11:36.374300 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User \"system:kube-scheduler\" cannot list resource \"persistentvolumeclaims\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0917 09:11:36.376668 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0917 09:11:36.376725 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User \"system:kube-scheduler\" cannot list resource \"persistentvolumes\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0917 09:11:36.376996 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0917 09:11:36.377204 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User \"system:kube-scheduler\" cannot list resource \"poddisruptionbudgets\" in API group \"policy\" at the cluster scope" logger="UnhandledError"
W0917 09:11:36.377457 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
E0917 09:11:36.377528 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:kube-scheduler\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError"
W0917 09:11:36.378762 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0917 09:11:36.378803 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User \"system:kube-scheduler\" cannot list resource \"pods\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0917 09:11:36.381567 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0917 09:11:36.381612 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User \"system:kube-scheduler\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0917 09:11:36.381875 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope
E0917 09:11:36.382055 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User \"system:kube-scheduler\" cannot list resource \"csistoragecapacities\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError"
W0917 09:11:36.382318 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0917 09:11:36.382353 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User \"system:kube-scheduler\" cannot list resource \"csinodes\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError"
W0917 09:11:36.382449 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0917 09:11:36.382484 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User \"system:kube-scheduler\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0917 09:11:36.382718 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope
E0917 09:11:36.382767 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User \"system:kube-scheduler\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0917 09:11:36.382890 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0917 09:11:36.383104 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User \"system:kube-scheduler\" cannot list resource \"statefulsets\" in API group \"apps\" at the cluster scope" logger="UnhandledError"
I0917 09:11:36.446439 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [d9fae1497b04] <==
E0917 09:09:54.047035 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: Get \"https://192.169.0.5:8443/api/v1/persistentvolumes?limit=500&resourceVersion=0\": dial tcp 192.169.0.5:8443: connect: connection refused" logger="UnhandledError"
W0917 09:10:01.417081 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSIStorageCapacity: Get "https://192.169.0.5:8443/apis/storage.k8s.io/v1/csistoragecapacities?limit=500&resourceVersion=0": dial tcp 192.169.0.5:8443: connect: connection refused
E0917 09:10:01.417178 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: Get \"https://192.169.0.5:8443/apis/storage.k8s.io/v1/csistoragecapacities?limit=500&resourceVersion=0\": dial tcp 192.169.0.5:8443: connect: connection refused" logger="UnhandledError"
W0917 09:10:02.586956 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Namespace: Get "https://192.169.0.5:8443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 192.169.0.5:8443: connect: connection refused
E0917 09:10:02.587049 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get \"https://192.169.0.5:8443/api/v1/namespaces?limit=500&resourceVersion=0\": dial tcp 192.169.0.5:8443: connect: connection refused" logger="UnhandledError"
W0917 09:10:09.339944 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.ReplicationController: Get "https://192.169.0.5:8443/api/v1/replicationcontrollers?limit=500&resourceVersion=0": dial tcp 192.169.0.5:8443: connect: connection refused
E0917 09:10:09.340160 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: Get \"https://192.169.0.5:8443/api/v1/replicationcontrollers?limit=500&resourceVersion=0\": dial tcp 192.169.0.5:8443: connect: connection refused" logger="UnhandledError"
W0917 09:10:12.375946 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSINode: Get "https://192.169.0.5:8443/apis/storage.k8s.io/v1/csinodes?limit=500&resourceVersion=0": dial tcp 192.169.0.5:8443: connect: connection refused
E0917 09:10:12.375997 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSINode: failed to list *v1.CSINode: Get \"https://192.169.0.5:8443/apis/storage.k8s.io/v1/csinodes?limit=500&resourceVersion=0\": dial tcp 192.169.0.5:8443: connect: connection refused" logger="UnhandledError"
W0917 09:10:14.579545 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: Get "https://192.169.0.5:8443/api/v1/nodes?limit=500&resourceVersion=0": dial tcp 192.169.0.5:8443: connect: connection refused
E0917 09:10:14.579979 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get \"https://192.169.0.5:8443/api/v1/nodes?limit=500&resourceVersion=0\": dial tcp 192.169.0.5:8443: connect: connection refused" logger="UnhandledError"
W0917 09:10:18.357149 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: Get "https://192.169.0.5:8443/api/v1/services?limit=500&resourceVersion=0": dial tcp 192.169.0.5:8443: connect: connection refused
E0917 09:10:18.357192 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: Get \"https://192.169.0.5:8443/api/v1/services?limit=500&resourceVersion=0\": dial tcp 192.169.0.5:8443: connect: connection refused" logger="UnhandledError"
W0917 09:10:19.971293 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.PodDisruptionBudget: Get "https://192.169.0.5:8443/apis/policy/v1/poddisruptionbudgets?limit=500&resourceVersion=0": dial tcp 192.169.0.5:8443: connect: connection refused
E0917 09:10:19.971663 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: Get \"https://192.169.0.5:8443/apis/policy/v1/poddisruptionbudgets?limit=500&resourceVersion=0\": dial tcp 192.169.0.5:8443: connect: connection refused" logger="UnhandledError"
W0917 09:10:22.259174 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Pod: Get "https://192.169.0.5:8443/api/v1/pods?fieldSelector=status.phase%21%3DSucceeded%2Cstatus.phase%21%3DFailed&limit=500&resourceVersion=0": dial tcp 192.169.0.5:8443: connect: connection refused
E0917 09:10:22.259229 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Pod: failed to list *v1.Pod: Get \"https://192.169.0.5:8443/api/v1/pods?fieldSelector=status.phase%21%3DSucceeded%2Cstatus.phase%21%3DFailed&limit=500&resourceVersion=0\": dial tcp 192.169.0.5:8443: connect: connection refused" logger="UnhandledError"
W0917 09:10:24.413900 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSIDriver: Get "https://192.169.0.5:8443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 192.169.0.5:8443: connect: connection refused
E0917 09:10:24.413975 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get \"https://192.169.0.5:8443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0\": dial tcp 192.169.0.5:8443: connect: connection refused" logger="UnhandledError"
I0917 09:10:24.953479 1 tlsconfig.go:258] "Shutting down DynamicServingCertificateController"
I0917 09:10:24.953762 1 secure_serving.go:258] Stopped listening on 127.0.0.1:10259
E0917 09:10:24.953909 1 shared_informer.go:316] "Unhandled Error" err="unable to sync caches for client-ca::kube-system::extension-apiserver-authentication::client-ca-file" logger="UnhandledError"
E0917 09:10:24.953957 1 server.go:267] "waiting for handlers to sync" err="context canceled"
I0917 09:10:24.955052 1 configmap_cafile_content.go:213] "Shutting down controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
E0917 09:10:24.955061 1 run.go:72] "command failed" err="finished without leader elect"
==> kubelet <==
Sep 17 09:12:17 ha-857000 kubelet[1572]: E0917 09:12:17.230909 1572 kubelet.go:1915] "Failed creating a mirror pod for" err="pods \"kube-vip-ha-857000\" already exists" pod="kube-system/kube-vip-ha-857000"
Sep 17 09:12:17 ha-857000 kubelet[1572]: I0917 09:12:17.239224 1572 apiserver.go:52] "Watching apiserver"
Sep 17 09:12:17 ha-857000 kubelet[1572]: I0917 09:12:17.296247 1572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 09:12:17 ha-857000 kubelet[1572]: I0917 09:12:17.363699 1572 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eecd1421-3a2f-4e48-b2b2-abcbef7869e7-lib-modules\") pod \"kindnet-7pf7v\" (UID: \"eecd1421-3a2f-4e48-b2b2-abcbef7869e7\") " pod="kube-system/kindnet-7pf7v"
Sep 17 09:12:17 ha-857000 kubelet[1572]: I0917 09:12:17.363849 1572 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"xtables-lock\" (UniqueName: \"kubernetes.io/host-path/eecd1421-3a2f-4e48-b2b2-abcbef7869e7-xtables-lock\") pod \"kindnet-7pf7v\" (UID: \"eecd1421-3a2f-4e48-b2b2-abcbef7869e7\") " pod="kube-system/kindnet-7pf7v"
Sep 17 09:12:17 ha-857000 kubelet[1572]: I0917 09:12:17.363896 1572 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-cfg\" (UniqueName: \"kubernetes.io/host-path/eecd1421-3a2f-4e48-b2b2-abcbef7869e7-cni-cfg\") pod \"kindnet-7pf7v\" (UID: \"eecd1421-3a2f-4e48-b2b2-abcbef7869e7\") " pod="kube-system/kindnet-7pf7v"
Sep 17 09:12:17 ha-857000 kubelet[1572]: I0917 09:12:17.363942 1572 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"xtables-lock\" (UniqueName: \"kubernetes.io/host-path/7a396757-8954-48d2-b708-dcdfbab21dc7-xtables-lock\") pod \"kube-proxy-vskbj\" (UID: \"7a396757-8954-48d2-b708-dcdfbab21dc7\") " pod="kube-system/kube-proxy-vskbj"
Sep 17 09:12:17 ha-857000 kubelet[1572]: I0917 09:12:17.363979 1572 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7a396757-8954-48d2-b708-dcdfbab21dc7-lib-modules\") pod \"kube-proxy-vskbj\" (UID: \"7a396757-8954-48d2-b708-dcdfbab21dc7\") " pod="kube-system/kube-proxy-vskbj"
Sep 17 09:12:17 ha-857000 kubelet[1572]: I0917 09:12:17.364021 1572 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/host-path/d81e7b55-a14e-4dc7-9193-ebe6914cdacf-tmp\") pod \"storage-provisioner\" (UID: \"d81e7b55-a14e-4dc7-9193-ebe6914cdacf\") " pod="kube-system/storage-provisioner"
Sep 17 09:12:17 ha-857000 kubelet[1572]: I0917 09:12:17.381710 1572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 09:12:18 ha-857000 kubelet[1572]: I0917 09:12:18.732394 1572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc1d198ffe0b277108e5042e64604ceb887f7f17cd5814893fbf789f1ce180f0"
Sep 17 09:12:18 ha-857000 kubelet[1572]: I0917 09:12:18.754870 1572 kubelet.go:1895] "Trying to delete pod" pod="kube-system/kube-vip-ha-857000" podUID="84b805d8-9a8f-4c6f-b18f-76c98ca4776c"
Sep 17 09:12:18 ha-857000 kubelet[1572]: I0917 09:12:18.779039 1572 kubelet.go:1900] "Deleted mirror pod because it is outdated" pod="kube-system/kube-vip-ha-857000"
Sep 17 09:12:19 ha-857000 kubelet[1572]: I0917 09:12:19.228668 1572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ca8e5543181b6f9996b6d7e435c3947" path="/var/lib/kubelet/pods/3ca8e5543181b6f9996b6d7e435c3947/volumes"
Sep 17 09:12:19 ha-857000 kubelet[1572]: I0917 09:12:19.846405 1572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-vip-ha-857000" podStartSLOduration=1.846388448 podStartE2EDuration="1.846388448s" podCreationTimestamp="2024-09-17 09:12:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2024-09-17 09:12:19.829429782 +0000 UTC m=+94.772487592" watchObservedRunningTime="2024-09-17 09:12:19.846388448 +0000 UTC m=+94.789446258"
Sep 17 09:12:45 ha-857000 kubelet[1572]: E0917 09:12:45.245854 1572 iptables.go:577] "Could not set up iptables canary" err=<
Sep 17 09:12:45 ha-857000 kubelet[1572]: error creating chain "KUBE-KUBELET-CANARY": exit status 3: Ignoring deprecated --wait-interval option.
Sep 17 09:12:45 ha-857000 kubelet[1572]: ip6tables v1.8.9 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)
Sep 17 09:12:45 ha-857000 kubelet[1572]: Perhaps ip6tables or your kernel needs to be upgraded.
Sep 17 09:12:45 ha-857000 kubelet[1572]: > table="nat" chain="KUBE-KUBELET-CANARY"
Sep 17 09:12:45 ha-857000 kubelet[1572]: I0917 09:12:45.363926 1572 scope.go:117] "RemoveContainer" containerID="fcb7038a6ac9ef515ab38df1dab73586ab93030767bab4f0d4d141f34bac886f"
Sep 17 09:12:49 ha-857000 kubelet[1572]: I0917 09:12:49.092301 1572 scope.go:117] "RemoveContainer" containerID="611759af4bf7a8b48c2739f53afaeba3cb10af70a80bf85bfb78eebe6230c491"
Sep 17 09:12:49 ha-857000 kubelet[1572]: I0917 09:12:49.092548 1572 scope.go:117] "RemoveContainer" containerID="67814a4514b10a43fcb805320685eed14ae6c928c03c950cc35f4abd031401e7"
Sep 17 09:12:49 ha-857000 kubelet[1572]: E0917 09:12:49.092633 1572 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-provisioner\" with CrashLoopBackOff: \"back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(d81e7b55-a14e-4dc7-9193-ebe6914cdacf)\"" pod="kube-system/storage-provisioner" podUID="d81e7b55-a14e-4dc7-9193-ebe6914cdacf"
Sep 17 09:13:00 ha-857000 kubelet[1572]: I0917 09:13:00.226410 1572 scope.go:117] "RemoveContainer" containerID="67814a4514b10a43fcb805320685eed14ae6c928c03c950cc35f4abd031401e7"
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-darwin-amd64 status --format={{.APIServer}} -p ha-857000 -n ha-857000
helpers_test.go:261: (dbg) Run: kubectl --context ha-857000 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:285: <<< TestMultiControlPlane/serial/DegradedAfterClusterRestart FAILED: end of post-mortem logs <<<
helpers_test.go:286: ---------------------/post-mortem---------------------------------
--- FAIL: TestMultiControlPlane/serial/DegradedAfterClusterRestart (4.60s)