=== RUN TestStoppedBinaryUpgrade/Upgrade
version_upgrade_test.go:195: (dbg) Run: /var/folders/vq/yhv778t970xgml0dzm5fdwlr0000gp/T/minikube-v1.6.2.2860910170.exe start -p stopped-upgrade-494000 --memory=2200 --vm-driver=hyperkit
E0520 08:40:37.177477 1473 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/16543-1012/.minikube/profiles/skaffold-170000/client.crt: no such file or directory
version_upgrade_test.go:195: (dbg) Done: /var/folders/vq/yhv778t970xgml0dzm5fdwlr0000gp/T/minikube-v1.6.2.2860910170.exe start -p stopped-upgrade-494000 --memory=2200 --vm-driver=hyperkit : (1m30.587428735s)
version_upgrade_test.go:204: (dbg) Run: /var/folders/vq/yhv778t970xgml0dzm5fdwlr0000gp/T/minikube-v1.6.2.2860910170.exe -p stopped-upgrade-494000 stop
E0520 08:41:57.273691 1473 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/16543-1012/.minikube/profiles/ingress-addon-legacy-507000/client.crt: no such file or directory
E0520 08:41:59.097891 1473 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/16543-1012/.minikube/profiles/skaffold-170000/client.crt: no such file or directory
version_upgrade_test.go:204: (dbg) Done: /var/folders/vq/yhv778t970xgml0dzm5fdwlr0000gp/T/minikube-v1.6.2.2860910170.exe -p stopped-upgrade-494000 stop: (8.096533925s)
version_upgrade_test.go:210: (dbg) Run: out/minikube-darwin-amd64 start -p stopped-upgrade-494000 --memory=2200 --alsologtostderr -v=1 --driver=hyperkit
E0520 08:42:10.183088 1473 cert_rotation.go:168] key failed with : open /Users/jenkins/minikube-integration/16543-1012/.minikube/profiles/addons-146000/client.crt: no such file or directory
version_upgrade_test.go:210: (dbg) Non-zero exit: out/minikube-darwin-amd64 start -p stopped-upgrade-494000 --memory=2200 --alsologtostderr -v=1 --driver=hyperkit : exit status 90 (21.914447704s)
-- stdout --
* [stopped-upgrade-494000] minikube v1.30.1 on Darwin 13.3.1
- MINIKUBE_LOCATION=16543
- KUBECONFIG=/Users/jenkins/minikube-integration/16543-1012/kubeconfig
- MINIKUBE_BIN=out/minikube-darwin-amd64
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- MINIKUBE_HOME=/Users/jenkins/minikube-integration/16543-1012/.minikube
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.27.2 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.27.2
* Using the hyperkit driver based on existing profile
* Starting control plane node stopped-upgrade-494000 in cluster stopped-upgrade-494000
* Restarting existing hyperkit VM for "stopped-upgrade-494000" ...
-- /stdout --
** stderr **
I0520 08:42:02.318905 5910 out.go:296] Setting OutFile to fd 1 ...
I0520 08:42:02.319091 5910 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0520 08:42:02.319096 5910 out.go:309] Setting ErrFile to fd 2...
I0520 08:42:02.319100 5910 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0520 08:42:02.319207 5910 root.go:336] Updating PATH: /Users/jenkins/minikube-integration/16543-1012/.minikube/bin
I0520 08:42:02.320706 5910 out.go:303] Setting JSON to false
I0520 08:42:02.341427 5910 start.go:125] hostinfo: {"hostname":"MacOS-Agent-2.local","uptime":2493,"bootTime":1684594829,"procs":406,"os":"darwin","platform":"darwin","platformFamily":"Standalone Workstation","platformVersion":"13.3.1","kernelVersion":"22.4.0","kernelArch":"x86_64","virtualizationSystem":"","virtualizationRole":"","hostId":"2965c349-98a5-5970-aaa9-9eedd3ae5959"}
W0520 08:42:02.341518 5910 start.go:133] gopshost.Virtualization returned error: not implemented yet
I0520 08:42:02.361933 5910 out.go:177] * [stopped-upgrade-494000] minikube v1.30.1 on Darwin 13.3.1
I0520 08:42:02.404222 5910 out.go:177] - MINIKUBE_LOCATION=16543
I0520 08:42:02.404228 5910 notify.go:220] Checking for updates...
I0520 08:42:02.425034 5910 out.go:177] - KUBECONFIG=/Users/jenkins/minikube-integration/16543-1012/kubeconfig
I0520 08:42:02.467103 5910 out.go:177] - MINIKUBE_BIN=out/minikube-darwin-amd64
I0520 08:42:02.509129 5910 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0520 08:42:02.551186 5910 out.go:177] - MINIKUBE_HOME=/Users/jenkins/minikube-integration/16543-1012/.minikube
I0520 08:42:02.593074 5910 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0520 08:42:02.619375 5910 config.go:182] Loaded profile config "stopped-upgrade-494000": Driver=, ContainerRuntime=docker, KubernetesVersion=v1.17.0
I0520 08:42:02.619396 5910 start_flags.go:683] config upgrade: Driver=hyperkit
I0520 08:42:02.619402 5910 start_flags.go:695] config upgrade: KicBaseImage=gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1684523789-16533@sha256:ed200ff6d686f303885e8aaf964442d08018856d63a8e23f7acdc068766ea82b
I0520 08:42:02.619464 5910 profile.go:148] Saving config to /Users/jenkins/minikube-integration/16543-1012/.minikube/profiles/stopped-upgrade-494000/config.json ...
I0520 08:42:02.620152 5910 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0520 08:42:02.620201 5910 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0520 08:42:02.628652 5910 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52770
I0520 08:42:02.629055 5910 main.go:141] libmachine: () Calling .GetVersion
I0520 08:42:02.629509 5910 main.go:141] libmachine: Using API Version 1
I0520 08:42:02.629521 5910 main.go:141] libmachine: () Calling .SetConfigRaw
I0520 08:42:02.629737 5910 main.go:141] libmachine: () Calling .GetMachineName
I0520 08:42:02.629855 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .DriverName
I0520 08:42:02.651132 5910 out.go:177] * Kubernetes 1.27.2 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.27.2
I0520 08:42:02.671948 5910 driver.go:375] Setting default libvirt URI to qemu:///system
I0520 08:42:02.672255 5910 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0520 08:42:02.672281 5910 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0520 08:42:02.680757 5910 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52772
I0520 08:42:02.681170 5910 main.go:141] libmachine: () Calling .GetVersion
I0520 08:42:02.681543 5910 main.go:141] libmachine: Using API Version 1
I0520 08:42:02.681563 5910 main.go:141] libmachine: () Calling .SetConfigRaw
I0520 08:42:02.681789 5910 main.go:141] libmachine: () Calling .GetMachineName
I0520 08:42:02.681905 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .DriverName
I0520 08:42:02.730806 5910 out.go:177] * Using the hyperkit driver based on existing profile
I0520 08:42:02.751890 5910 start.go:295] selected driver: hyperkit
I0520 08:42:02.751905 5910 start.go:870] validating driver "hyperkit" against &{Name:stopped-upgrade-494000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.6.0.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1684523789-16533@sha256:ed200ff6d686f303885e8aaf964442d08018856d63a8e23f7acdc068766ea82b Memory:2200 CPUs:2 DiskSize:20000 VMDriver:hyperkit Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:0 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser: SSHKey: SSHPort:0 KubernetesConfig:{KubernetesVersion:v
1.17.0 ClusterName: Namespace: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name:minikube IP:192.168.64.27 Port:8443 KubernetesVersion:v1.17.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[] StartHostTimeout:0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString: Mount9PVersion: MountGID: MountIP: MountMSize:0 MountOptions:[] MountPort:0 MountType: MountUID: BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: Stat
icIP:}
I0520 08:42:02.752007 5910 start.go:881] status for hyperkit: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0520 08:42:02.754745 5910 install.go:52] acquiring lock: {Name:mk4023283b30b374c3f04c8805d539e68824c0b8 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 08:42:02.754875 5910 install.go:117] Validating docker-machine-driver-hyperkit, PATH=/Users/jenkins/minikube-integration/16543-1012/.minikube/bin:/Users/jenkins/workspace/out/:/usr/bin:/bin:/usr/sbin:/sbin:/Users/jenkins/google-cloud-sdk/bin:/usr/local/bin/:/usr/local/go/bin/:/Users/jenkins/go/bin
I0520 08:42:02.762364 5910 install.go:137] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit version is 1.30.1
I0520 08:42:02.765902 5910 install.go:79] stdout: /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0520 08:42:02.765927 5910 install.go:81] /Users/jenkins/workspace/out/docker-machine-driver-hyperkit looks good
I0520 08:42:02.766010 5910 cni.go:84] Creating CNI manager for ""
I0520 08:42:02.766026 5910 cni.go:161] CNI unnecessary in this configuration, recommending no CNI
I0520 08:42:02.766035 5910 start_flags.go:319] config:
{Name:stopped-upgrade-494000 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.6.0.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1684523789-16533@sha256:ed200ff6d686f303885e8aaf964442d08018856d63a8e23f7acdc068766ea82b Memory:2200 CPUs:2 DiskSize:20000 VMDriver:hyperkit Driver:hyperkit HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:0 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser: SSHKey: SSHPort:0 KubernetesConfig:{KubernetesVersion:v1.17.0 ClusterName: Namespace: APIServerName:minikubeCA APIServerNames:[] APIServe
rIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name:minikube IP:192.168.64.27 Port:8443 KubernetesVersion:v1.17.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[] StartHostTimeout:0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString: Mount9PVersion: MountGID: MountIP: MountMSize:0 MountOptions:[] MountPort:0 MountType: MountUID: BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:}
I0520 08:42:02.766188 5910 iso.go:125] acquiring lock: {Name:mk812398b71fc7dea8b0811ed01428528fb404a8 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 08:42:02.807786 5910 out.go:177] * Starting control plane node stopped-upgrade-494000 in cluster stopped-upgrade-494000
I0520 08:42:02.828721 5910 preload.go:132] Checking if preload exists for k8s version v1.17.0 and runtime docker
W0520 08:42:02.938701 5910 preload.go:115] https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.17.0/preloaded-images-k8s-v18-v1.17.0-docker-overlay2-amd64.tar.lz4 status code: 404
I0520 08:42:02.938828 5910 profile.go:148] Saving config to /Users/jenkins/minikube-integration/16543-1012/.minikube/profiles/stopped-upgrade-494000/config.json ...
I0520 08:42:02.938895 5910 cache.go:107] acquiring lock: {Name:mk9bcd09850a89db07b3375c71f3bc0a8dac4aad Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 08:42:02.938917 5910 cache.go:107] acquiring lock: {Name:mk2f17e30a8857e26779a79d72654773d414fdfd Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 08:42:02.938926 5910 cache.go:107] acquiring lock: {Name:mke78730a6ea54d6bc6eaaac3b116fd676c4e772 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 08:42:02.938898 5910 cache.go:107] acquiring lock: {Name:mk8c3b70f2187f253b6d5413d26139c6742d4787 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 08:42:02.938978 5910 cache.go:107] acquiring lock: {Name:mk6af16deb85580c427a803d1f14b22fc4ebc0a0 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 08:42:02.939027 5910 cache.go:115] /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/gcr.io/k8s-minikube/storage-provisioner_v5 exists
I0520 08:42:02.939041 5910 cache.go:115] /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.17.0 exists
I0520 08:42:02.939039 5910 cache.go:96] cache image "gcr.io/k8s-minikube/storage-provisioner:v5" -> "/Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/gcr.io/k8s-minikube/storage-provisioner_v5" took 163.129µs
I0520 08:42:02.939006 5910 cache.go:107] acquiring lock: {Name:mk7e0d0036b5a6bd76c8c5e1657254e75041eb2f Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 08:42:02.939054 5910 cache.go:80] save to tar file gcr.io/k8s-minikube/storage-provisioner:v5 -> /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/gcr.io/k8s-minikube/storage-provisioner_v5 succeeded
I0520 08:42:02.939053 5910 cache.go:115] /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.17.0 exists
I0520 08:42:02.939052 5910 cache.go:96] cache image "registry.k8s.io/kube-controller-manager:v1.17.0" -> "/Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.17.0" took 153.028µs
I0520 08:42:02.939039 5910 cache.go:107] acquiring lock: {Name:mk12f762c0c52f54e2cd38f12ec39aecf4546319 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 08:42:02.939071 5910 cache.go:80] save to tar file registry.k8s.io/kube-controller-manager:v1.17.0 -> /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.17.0 succeeded
I0520 08:42:02.939071 5910 cache.go:96] cache image "registry.k8s.io/kube-scheduler:v1.17.0" -> "/Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.17.0" took 148.09µs
I0520 08:42:02.939082 5910 cache.go:80] save to tar file registry.k8s.io/kube-scheduler:v1.17.0 -> /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.17.0 succeeded
I0520 08:42:02.939056 5910 cache.go:107] acquiring lock: {Name:mkf82eb3a97573a01ecb3d776fa86c3c155339e4 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 08:42:02.939144 5910 cache.go:115] /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.17.0 exists
I0520 08:42:02.939160 5910 cache.go:96] cache image "registry.k8s.io/kube-proxy:v1.17.0" -> "/Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.17.0" took 266.811µs
I0520 08:42:02.939168 5910 cache.go:80] save to tar file registry.k8s.io/kube-proxy:v1.17.0 -> /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.17.0 succeeded
I0520 08:42:02.939168 5910 cache.go:115] /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.3-0 exists
I0520 08:42:02.939175 5910 cache.go:115] /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.6.5 exists
I0520 08:42:02.939179 5910 cache.go:115] /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/pause_3.1 exists
I0520 08:42:02.939182 5910 cache.go:115] /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.17.0 exists
I0520 08:42:02.939180 5910 cache.go:96] cache image "registry.k8s.io/etcd:3.4.3-0" -> "/Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.3-0" took 210.487µs
I0520 08:42:02.939182 5910 cache.go:96] cache image "registry.k8s.io/coredns:1.6.5" -> "/Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.6.5" took 198.847µs
I0520 08:42:02.939198 5910 cache.go:80] save to tar file registry.k8s.io/etcd:3.4.3-0 -> /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.4.3-0 succeeded
I0520 08:42:02.939206 5910 cache.go:80] save to tar file registry.k8s.io/coredns:1.6.5 -> /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/coredns_1.6.5 succeeded
I0520 08:42:02.939188 5910 cache.go:96] cache image "registry.k8s.io/pause:3.1" -> "/Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/pause_3.1" took 163.376µs
I0520 08:42:02.939223 5910 cache.go:80] save to tar file registry.k8s.io/pause:3.1 -> /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/pause_3.1 succeeded
I0520 08:42:02.939199 5910 cache.go:96] cache image "registry.k8s.io/kube-apiserver:v1.17.0" -> "/Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.17.0" took 264.835µs
I0520 08:42:02.939229 5910 cache.go:80] save to tar file registry.k8s.io/kube-apiserver:v1.17.0 -> /Users/jenkins/minikube-integration/16543-1012/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.17.0 succeeded
I0520 08:42:02.939233 5910 cache.go:87] Successfully saved all images to host disk.
I0520 08:42:02.939315 5910 cache.go:195] Successfully downloaded all kic artifacts
I0520 08:42:02.939345 5910 start.go:364] acquiring machines lock for stopped-upgrade-494000: {Name:mka6d506048574344e8a31e8d1b1dae0a9a9a412 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0520 08:42:02.939390 5910 start.go:368] acquired machines lock for "stopped-upgrade-494000" in 36.535µs
I0520 08:42:02.939407 5910 start.go:96] Skipping create...Using existing machine configuration
I0520 08:42:02.939416 5910 fix.go:55] fixHost starting: minikube
I0520 08:42:02.939654 5910 main.go:141] libmachine: Found binary path at /Users/jenkins/workspace/out/docker-machine-driver-hyperkit
I0520 08:42:02.939674 5910 main.go:141] libmachine: Launching plugin server for driver hyperkit
I0520 08:42:02.947394 5910 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:52774
I0520 08:42:02.947741 5910 main.go:141] libmachine: () Calling .GetVersion
I0520 08:42:02.948125 5910 main.go:141] libmachine: Using API Version 1
I0520 08:42:02.948143 5910 main.go:141] libmachine: () Calling .SetConfigRaw
I0520 08:42:02.948337 5910 main.go:141] libmachine: () Calling .GetMachineName
I0520 08:42:02.948436 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .DriverName
I0520 08:42:02.948527 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetState
I0520 08:42:02.948610 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0520 08:42:02.948675 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | hyperkit pid from json: 5668
I0520 08:42:02.949572 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | hyperkit pid 5668 missing from process table
I0520 08:42:02.949615 5910 fix.go:103] recreateIfNeeded on stopped-upgrade-494000: state=Stopped err=<nil>
I0520 08:42:02.949637 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .DriverName
W0520 08:42:02.949711 5910 fix.go:129] unexpected machine state, will restart: <nil>
I0520 08:42:02.989866 5910 out.go:177] * Restarting existing hyperkit VM for "stopped-upgrade-494000" ...
I0520 08:42:03.010888 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .Start
I0520 08:42:03.011070 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0520 08:42:03.011128 5910 main.go:141] libmachine: (stopped-upgrade-494000) minikube might have been shutdown in an unclean way, the hyperkit pid file still exists: /Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/hyperkit.pid
I0520 08:42:03.012296 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | hyperkit pid 5668 missing from process table
I0520 08:42:03.012310 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | pid 5668 is in state "Stopped"
I0520 08:42:03.012337 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | Removing stale pid file /Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/hyperkit.pid...
I0520 08:42:03.012512 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | Using UUID a2cb0202-f724-11ed-9597-f01898ef957c
I0520 08:42:03.035135 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | Generated MAC 5e:62:70:29:a2:39
I0520 08:42:03.035161 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | Starting with cmdline: loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=stopped-upgrade-494000
I0520 08:42:03.035338 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 DEBUG: hyperkit: Start &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"a2cb0202-f724-11ed-9597-f01898ef957c", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc0003d82d0)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/bzimage", Initrd:"/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil),
CmdLine:"", process:(*os.Process)(nil)}
I0520 08:42:03.035373 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 DEBUG: hyperkit: check &hyperkit.HyperKit{HyperKit:"/usr/local/bin/hyperkit", Argv0:"", StateDir:"/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000", VPNKitSock:"", VPNKitUUID:"", VPNKitPreferredIPv4:"", UUID:"a2cb0202-f724-11ed-9597-f01898ef957c", Disks:[]hyperkit.Disk{(*hyperkit.RawDisk)(0xc0003d82d0)}, ISOImages:[]string{"/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/boot2docker.iso"}, VSock:false, VSockDir:"", VSockPorts:[]int(nil), VSockGuestCID:3, VMNet:true, Sockets9P:[]hyperkit.Socket9P(nil), Kernel:"/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/bzimage", Initrd:"/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/initrd", Bootrom:"", CPUs:2, Memory:2200, Console:1, Serials:[]hyperkit.Serial(nil), Pid:0, Arguments:[]string(nil),
CmdLine:"", process:(*os.Process)(nil)}
I0520 08:42:03.035466 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 DEBUG: hyperkit: Arguments: []string{"-A", "-u", "-F", "/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/hyperkit.pid", "-c", "2", "-m", "2200M", "-s", "0:0,hostbridge", "-s", "31,lpc", "-s", "1:0,virtio-net", "-U", "a2cb0202-f724-11ed-9597-f01898ef957c", "-s", "2:0,virtio-blk,/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/stopped-upgrade-494000.rawdisk", "-s", "3,ahci-cd,/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/boot2docker.iso", "-s", "4,virtio-rnd", "-l", "com1,autopty=/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/tty,log=/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/console-ring", "-f", "kexec,/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/
bzimage,/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/initrd,earlyprintk=serial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=stopped-upgrade-494000"}
I0520 08:42:03.035522 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 DEBUG: hyperkit: CmdLine: "/usr/local/bin/hyperkit -A -u -F /Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/hyperkit.pid -c 2 -m 2200M -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-net -U a2cb0202-f724-11ed-9597-f01898ef957c -s 2:0,virtio-blk,/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/stopped-upgrade-494000.rawdisk -s 3,ahci-cd,/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/boot2docker.iso -s 4,virtio-rnd -l com1,autopty=/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/tty,log=/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/console-ring -f kexec,/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/bzimage,/Users/jenkins/minikube-integration/16543-1012/.miniku
be/machines/stopped-upgrade-494000/initrd,earlyprintk=serial loglevel=3 console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes random.trust_cpu=on hw_rng_model=virtio base host=stopped-upgrade-494000"
I0520 08:42:03.035544 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 DEBUG: hyperkit: Redirecting stdout/stderr to logger
I0520 08:42:03.036904 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 DEBUG: hyperkit: Pid is 5921
I0520 08:42:03.037303 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | Attempt 0
I0520 08:42:03.037321 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | exe=/Users/jenkins/workspace/out/docker-machine-driver-hyperkit uid=0
I0520 08:42:03.037439 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | hyperkit pid from json: 5921
I0520 08:42:03.039071 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | Searching for 5e:62:70:29:a2:39 in /var/db/dhcpd_leases ...
I0520 08:42:03.039135 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | Found 26 entries in /var/db/dhcpd_leases!
I0520 08:42:03.039154 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | dhcp entry: {Name:minikube IPAddress:192.168.64.26 HWAddress:d2:47:62:6:aa:cb ID:1,d2:47:62:6:aa:cb Lease:0x646a3bc0}
I0520 08:42:03.039175 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | dhcp entry: {Name:minikube IPAddress:192.168.64.27 HWAddress:5e:62:70:29:a2:39 ID:1,5e:62:70:29:a2:39 Lease:0x646a3b79}
I0520 08:42:03.039195 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | Found match: 5e:62:70:29:a2:39
I0520 08:42:03.039214 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | IP: 192.168.64.27
I0520 08:42:03.039267 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetConfigRaw
I0520 08:42:03.039914 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetIP
I0520 08:42:03.040061 5910 profile.go:148] Saving config to /Users/jenkins/minikube-integration/16543-1012/.minikube/profiles/stopped-upgrade-494000/config.json ...
I0520 08:42:03.040408 5910 machine.go:88] provisioning docker machine ...
I0520 08:42:03.040417 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .DriverName
I0520 08:42:03.040538 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetMachineName
I0520 08:42:03.040637 5910 buildroot.go:166] provisioning hostname "stopped-upgrade-494000"
I0520 08:42:03.040649 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetMachineName
I0520 08:42:03.040742 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHHostname
I0520 08:42:03.040827 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHPort
I0520 08:42:03.040929 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:03.041024 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:03.041104 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHUsername
I0520 08:42:03.041237 5910 main.go:141] libmachine: Using SSH client type: native
I0520 08:42:03.041615 5910 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140c4a0] 0x140f540 <nil> [] 0s} 192.168.64.27 22 <nil> <nil>}
I0520 08:42:03.041625 5910 main.go:141] libmachine: About to run SSH command:
sudo hostname stopped-upgrade-494000 && echo "stopped-upgrade-494000" | sudo tee /etc/hostname
I0520 08:42:03.044527 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: Using fd 5 for I/O notifications
I0520 08:42:03.053601 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: /Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/boot2docker.iso: fcntl(F_PUNCHHOLE) Operation not permitted: block device will not support TRIM/DISCARD
I0520 08:42:03.054460 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0
I0520 08:42:03.054501 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0
I0520 08:42:03.054523 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0
I0520 08:42:03.054563 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0
I0520 08:42:03.444286 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 0
I0520 08:42:03.549696 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 2 bit: 22 unspecified don't care: bit is 0
I0520 08:42:03.549742 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 12 unspecified don't care: bit is 0
I0520 08:42:03.549769 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 4 bit: 20 unspecified don't care: bit is 0
I0520 08:42:03.549786 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: vmx_set_ctlreg: cap_field: 3 bit: 13 unspecified don't care: bit is 0
I0520 08:42:03.550506 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:03 INFO : hyperkit: stderr: rdmsr to register 0x140 on vcpu 1
I0520 08:42:17.798470 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:17 INFO : hyperkit: stderr: rdmsr to register 0x64e on vcpu 1
I0520 08:42:17.798484 5910 main.go:141] libmachine: (stopped-upgrade-494000) DBG | 2023/05/20 08:42:17 INFO : hyperkit: stderr: rdmsr to register 0x34 on vcpu 1
I0520 08:42:22.622251 5910 main.go:141] libmachine: SSH cmd err, output: <nil>: stopped-upgrade-494000
I0520 08:42:22.622271 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHHostname
I0520 08:42:22.622417 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHPort
I0520 08:42:22.622552 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:22.622646 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:22.622757 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHUsername
I0520 08:42:22.622891 5910 main.go:141] libmachine: Using SSH client type: native
I0520 08:42:22.623199 5910 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140c4a0] 0x140f540 <nil> [] 0s} 192.168.64.27 22 <nil> <nil>}
I0520 08:42:22.623212 5910 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sstopped-upgrade-494000' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 stopped-upgrade-494000/g' /etc/hosts;
else
echo '127.0.1.1 stopped-upgrade-494000' | sudo tee -a /etc/hosts;
fi
fi
I0520 08:42:22.684124 5910 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0520 08:42:22.684140 5910 buildroot.go:172] set auth options {CertDir:/Users/jenkins/minikube-integration/16543-1012/.minikube CaCertPath:/Users/jenkins/minikube-integration/16543-1012/.minikube/certs/ca.pem CaPrivateKeyPath:/Users/jenkins/minikube-integration/16543-1012/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/server.pem ServerKeyPath:/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/server-key.pem ClientKeyPath:/Users/jenkins/minikube-integration/16543-1012/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/Users/jenkins/minikube-integration/16543-1012/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/Users/jenkins/minikube-integration/16543-1012/.minikube}
I0520 08:42:22.684161 5910 buildroot.go:174] setting up certificates
I0520 08:42:22.684173 5910 provision.go:83] configureAuth start
I0520 08:42:22.684180 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetMachineName
I0520 08:42:22.684302 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetIP
I0520 08:42:22.684401 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHHostname
I0520 08:42:22.684498 5910 provision.go:138] copyHostCerts
I0520 08:42:22.684575 5910 exec_runner.go:144] found /Users/jenkins/minikube-integration/16543-1012/.minikube/ca.pem, removing ...
I0520 08:42:22.684584 5910 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/16543-1012/.minikube/ca.pem
I0520 08:42:22.684695 5910 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/16543-1012/.minikube/certs/ca.pem --> /Users/jenkins/minikube-integration/16543-1012/.minikube/ca.pem (1082 bytes)
I0520 08:42:22.684915 5910 exec_runner.go:144] found /Users/jenkins/minikube-integration/16543-1012/.minikube/cert.pem, removing ...
I0520 08:42:22.684921 5910 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/16543-1012/.minikube/cert.pem
I0520 08:42:22.684985 5910 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/16543-1012/.minikube/certs/cert.pem --> /Users/jenkins/minikube-integration/16543-1012/.minikube/cert.pem (1123 bytes)
I0520 08:42:22.685138 5910 exec_runner.go:144] found /Users/jenkins/minikube-integration/16543-1012/.minikube/key.pem, removing ...
I0520 08:42:22.685144 5910 exec_runner.go:203] rm: /Users/jenkins/minikube-integration/16543-1012/.minikube/key.pem
I0520 08:42:22.685202 5910 exec_runner.go:151] cp: /Users/jenkins/minikube-integration/16543-1012/.minikube/certs/key.pem --> /Users/jenkins/minikube-integration/16543-1012/.minikube/key.pem (1679 bytes)
I0520 08:42:22.685333 5910 provision.go:112] generating server cert: /Users/jenkins/minikube-integration/16543-1012/.minikube/machines/server.pem ca-key=/Users/jenkins/minikube-integration/16543-1012/.minikube/certs/ca.pem private-key=/Users/jenkins/minikube-integration/16543-1012/.minikube/certs/ca-key.pem org=jenkins.stopped-upgrade-494000 san=[192.168.64.27 192.168.64.27 localhost 127.0.0.1 minikube stopped-upgrade-494000]
I0520 08:42:22.753584 5910 provision.go:172] copyRemoteCerts
I0520 08:42:22.753651 5910 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0520 08:42:22.753668 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHHostname
I0520 08:42:22.753817 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHPort
I0520 08:42:22.753912 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:22.754000 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHUsername
I0520 08:42:22.754095 5910 sshutil.go:53] new ssh client: &{IP:192.168.64.27 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/id_rsa Username:docker}
I0520 08:42:22.785112 5910 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16543-1012/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0520 08:42:22.793978 5910 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16543-1012/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I0520 08:42:22.802825 5910 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16543-1012/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0520 08:42:22.811980 5910 provision.go:86] duration metric: configureAuth took 127.795155ms
I0520 08:42:22.811994 5910 buildroot.go:189] setting minikube options for container-runtime
I0520 08:42:22.812129 5910 config.go:182] Loaded profile config "stopped-upgrade-494000": Driver=hyperkit, ContainerRuntime=docker, KubernetesVersion=v1.17.0
I0520 08:42:22.812142 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .DriverName
I0520 08:42:22.812291 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHHostname
I0520 08:42:22.812389 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHPort
I0520 08:42:22.812480 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:22.812574 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:22.812673 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHUsername
I0520 08:42:22.812793 5910 main.go:141] libmachine: Using SSH client type: native
I0520 08:42:22.813094 5910 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140c4a0] 0x140f540 <nil> [] 0s} 192.168.64.27 22 <nil> <nil>}
I0520 08:42:22.813103 5910 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0520 08:42:22.870032 5910 main.go:141] libmachine: SSH cmd err, output: <nil>: tmpfs
I0520 08:42:22.870044 5910 buildroot.go:70] root file system type: tmpfs
I0520 08:42:22.870129 5910 provision.go:309] Updating docker unit: /lib/systemd/system/docker.service ...
I0520 08:42:22.870142 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHHostname
I0520 08:42:22.870272 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHPort
I0520 08:42:22.870365 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:22.870445 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:22.870547 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHUsername
I0520 08:42:22.870672 5910 main.go:141] libmachine: Using SSH client type: native
I0520 08:42:22.870974 5910 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140c4a0] 0x140f540 <nil> [] 0s} 192.168.64.27 22 <nil> <nil>}
I0520 08:42:22.871018 5910 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0520 08:42:22.933983 5910 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target minikube-automount.service docker.socket
Requires= minikube-automount.service docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0520 08:42:22.934007 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHHostname
I0520 08:42:22.934150 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHPort
I0520 08:42:22.934278 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:22.934367 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:22.934477 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHUsername
I0520 08:42:22.934624 5910 main.go:141] libmachine: Using SSH client type: native
I0520 08:42:22.934940 5910 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140c4a0] 0x140f540 <nil> [] 0s} 192.168.64.27 22 <nil> <nil>}
I0520 08:42:22.934953 5910 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0520 08:42:23.400884 5910 main.go:141] libmachine: SSH cmd err, output: <nil>: diff: can't stat '/lib/systemd/system/docker.service': No such file or directory
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
I0520 08:42:23.400898 5910 machine.go:91] provisioned docker machine in 20.360883212s
I0520 08:42:23.400911 5910 start.go:300] post-start starting for "stopped-upgrade-494000" (driver="hyperkit")
I0520 08:42:23.400915 5910 start.go:328] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0520 08:42:23.400925 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .DriverName
I0520 08:42:23.401107 5910 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0520 08:42:23.401126 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHHostname
I0520 08:42:23.401210 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHPort
I0520 08:42:23.401281 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:23.401382 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHUsername
I0520 08:42:23.401459 5910 sshutil.go:53] new ssh client: &{IP:192.168.64.27 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/id_rsa Username:docker}
I0520 08:42:23.433983 5910 ssh_runner.go:195] Run: cat /etc/os-release
I0520 08:42:23.436444 5910 info.go:137] Remote host: Buildroot 2019.02.7
I0520 08:42:23.436458 5910 filesync.go:126] Scanning /Users/jenkins/minikube-integration/16543-1012/.minikube/addons for local assets ...
I0520 08:42:23.436535 5910 filesync.go:126] Scanning /Users/jenkins/minikube-integration/16543-1012/.minikube/files for local assets ...
I0520 08:42:23.436684 5910 filesync.go:149] local asset: /Users/jenkins/minikube-integration/16543-1012/.minikube/files/etc/ssl/certs/14732.pem -> 14732.pem in /etc/ssl/certs
I0520 08:42:23.436842 5910 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0520 08:42:23.440497 5910 ssh_runner.go:362] scp /Users/jenkins/minikube-integration/16543-1012/.minikube/files/etc/ssl/certs/14732.pem --> /etc/ssl/certs/14732.pem (1708 bytes)
I0520 08:42:23.449308 5910 start.go:303] post-start completed in 48.391392ms
I0520 08:42:23.449321 5910 fix.go:57] fixHost completed within 20.51031222s
I0520 08:42:23.449336 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHHostname
I0520 08:42:23.449462 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHPort
I0520 08:42:23.449558 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:23.449636 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:23.449739 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHUsername
I0520 08:42:23.449864 5910 main.go:141] libmachine: Using SSH client type: native
I0520 08:42:23.450174 5910 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x140c4a0] 0x140f540 <nil> [] 0s} 192.168.64.27 22 <nil> <nil>}
I0520 08:42:23.450182 5910 main.go:141] libmachine: About to run SSH command:
date +%s.%N
I0520 08:42:23.508560 5910 main.go:141] libmachine: SSH cmd err, output: <nil>: 1684597343.025649940
I0520 08:42:23.508574 5910 fix.go:207] guest clock: 1684597343.025649940
I0520 08:42:23.508580 5910 fix.go:220] Guest: 2023-05-20 08:42:23.02564994 -0700 PDT Remote: 2023-05-20 08:42:23.449326 -0700 PDT m=+21.163000470 (delta=-423.67606ms)
I0520 08:42:23.508593 5910 fix.go:191] guest clock delta is within tolerance: -423.67606ms
I0520 08:42:23.508598 5910 start.go:83] releasing machines lock for "stopped-upgrade-494000", held for 20.569607681s
I0520 08:42:23.508621 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .DriverName
I0520 08:42:23.508791 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetIP
I0520 08:42:23.508894 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .DriverName
I0520 08:42:23.509243 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .DriverName
I0520 08:42:23.509369 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .DriverName
I0520 08:42:23.509478 5910 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0520 08:42:23.509509 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHHostname
I0520 08:42:23.509529 5910 ssh_runner.go:195] Run: cat /version.json
I0520 08:42:23.509543 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHHostname
I0520 08:42:23.509663 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHPort
I0520 08:42:23.509683 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHPort
I0520 08:42:23.509787 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:23.509804 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHKeyPath
I0520 08:42:23.509900 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHUsername
I0520 08:42:23.509921 5910 main.go:141] libmachine: (stopped-upgrade-494000) Calling .GetSSHUsername
I0520 08:42:23.510010 5910 sshutil.go:53] new ssh client: &{IP:192.168.64.27 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/id_rsa Username:docker}
I0520 08:42:23.510018 5910 sshutil.go:53] new ssh client: &{IP:192.168.64.27 Port:22 SSHKeyPath:/Users/jenkins/minikube-integration/16543-1012/.minikube/machines/stopped-upgrade-494000/id_rsa Username:docker}
W0520 08:42:23.540785 5910 start.go:409] Unable to open version.json: cat /version.json: Process exited with status 1
stdout:
stderr:
cat: /version.json: No such file or directory
I0520 08:42:23.540853 5910 ssh_runner.go:195] Run: systemctl --version
I0520 08:42:23.590636 5910 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0520 08:42:23.594075 5910 cni.go:208] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0520 08:42:23.594144 5910 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I0520 08:42:23.597680 5910 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I0520 08:42:23.601100 5910 cni.go:304] no active bridge cni configs found in "/etc/cni/net.d" - nothing to configure
I0520 08:42:23.601113 5910 preload.go:132] Checking if preload exists for k8s version v1.17.0 and runtime docker
I0520 08:42:23.601126 5910 start.go:481] detecting cgroup driver to use...
I0520 08:42:23.601185 5910 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0520 08:42:23.608711 5910 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.1"|' /etc/containerd/config.toml"
I0520 08:42:23.612988 5910 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0520 08:42:23.617046 5910 containerd.go:145] configuring containerd to use "cgroupfs" as cgroup driver...
I0520 08:42:23.617092 5910 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0520 08:42:23.621201 5910 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0520 08:42:23.625480 5910 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0520 08:42:23.629677 5910 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0520 08:42:23.633893 5910 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0520 08:42:23.638491 5910 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0520 08:42:23.642547 5910 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0520 08:42:23.645947 5910 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0520 08:42:23.649499 5910 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0520 08:42:23.711941 5910 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0520 08:42:23.723731 5910 start.go:481] detecting cgroup driver to use...
I0520 08:42:23.723806 5910 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0520 08:42:23.730170 5910 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0520 08:42:23.736143 5910 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0520 08:42:23.755393 5910 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0520 08:42:23.763508 5910 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0520 08:42:23.771922 5910 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0520 08:42:23.779884 5910 ssh_runner.go:195] Run: which cri-dockerd
I0520 08:42:23.782094 5910 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I0520 08:42:23.786112 5910 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)
I0520 08:42:23.792658 5910 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0520 08:42:23.853438 5910 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0520 08:42:23.912669 5910 docker.go:532] configuring docker to use "cgroupfs" as cgroup driver...
I0520 08:42:23.912682 5910 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (144 bytes)
I0520 08:42:23.919626 5910 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0520 08:42:23.983103 5910 ssh_runner.go:195] Run: sudo systemctl restart docker
I0520 08:42:24.050166 5910 out.go:177]
W0520 08:42:24.070950 5910 out.go:239] X Exiting due to RUNTIME_ENABLE: Failed to enable container runtime: sudo systemctl restart docker: Process exited with status 1
stdout:
stderr:
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
X Exiting due to RUNTIME_ENABLE: Failed to enable container runtime: sudo systemctl restart docker: Process exited with status 1
stdout:
stderr:
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
W0520 08:42:24.070961 5910 out.go:239] *
*
W0520 08:42:24.071645 5910 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0520 08:42:24.170881 5910 out.go:177]
** /stderr **
version_upgrade_test.go:212: upgrade from v1.6.2 to HEAD failed: out/minikube-darwin-amd64 start -p stopped-upgrade-494000 --memory=2200 --alsologtostderr -v=1 --driver=hyperkit : exit status 90
--- FAIL: TestStoppedBinaryUpgrade/Upgrade (120.61s)