=== RUN TestMissingContainerUpgrade
=== PAUSE TestMissingContainerUpgrade
=== CONT TestMissingContainerUpgrade
version_upgrade_test.go:309: (dbg) Run: /tmp/minikube-v1.32.0.4070784972 start -p missing-upgrade-381064 --memory=3072 --driver=docker --container-runtime=docker
E1006 22:23:30.684729 376634 cert_rotation.go:172] "Loading client cert failed" err="open /home/jenkins/minikube-integration/21681-373053/.minikube/profiles/addons-329840/client.crt: no such file or directory" logger="tls-transport-cache.UnhandledError" key="key"
version_upgrade_test.go:309: (dbg) Done: /tmp/minikube-v1.32.0.4070784972 start -p missing-upgrade-381064 --memory=3072 --driver=docker --container-runtime=docker: (39.590274617s)
version_upgrade_test.go:318: (dbg) Run: docker stop missing-upgrade-381064
version_upgrade_test.go:318: (dbg) Done: docker stop missing-upgrade-381064: (10.406385839s)
version_upgrade_test.go:323: (dbg) Run: docker rm missing-upgrade-381064
version_upgrade_test.go:329: (dbg) Run: out/minikube-linux-amd64 start -p missing-upgrade-381064 --memory=3072 --alsologtostderr -v=1 --driver=docker --container-runtime=docker
version_upgrade_test.go:329: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p missing-upgrade-381064 --memory=3072 --alsologtostderr -v=1 --driver=docker --container-runtime=docker: exit status 90 (48.441769649s)
-- stdout --
* [missing-upgrade-381064] minikube v1.37.0 on Ubuntu 22.04 (kvm/amd64)
- MINIKUBE_LOCATION=21681
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/21681-373053/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/21681-373053/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.34.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.34.1
* Using the docker driver based on existing profile
* Starting "missing-upgrade-381064" primary control-plane node in "missing-upgrade-381064" cluster
* Pulling base image v0.0.48-1759745255-21703 ...
* docker "missing-upgrade-381064" container is missing, will recreate.
-- /stdout --
** stderr **
I1006 22:24:18.307804 672451 out.go:360] Setting OutFile to fd 1 ...
I1006 22:24:18.308150 672451 out.go:408] TERM=,COLORTERM=, which probably does not support color
I1006 22:24:18.308166 672451 out.go:374] Setting ErrFile to fd 2...
I1006 22:24:18.308172 672451 out.go:408] TERM=,COLORTERM=, which probably does not support color
I1006 22:24:18.308533 672451 root.go:338] Updating PATH: /home/jenkins/minikube-integration/21681-373053/.minikube/bin
I1006 22:24:18.309249 672451 out.go:368] Setting JSON to false
I1006 22:24:18.310799 672451 start.go:131] hostinfo: {"hostname":"ubuntu-20-agent-6","uptime":7601,"bootTime":1759781857,"procs":287,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"22.04","kernelVersion":"6.8.0-1041-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I1006 22:24:18.310887 672451 start.go:141] virtualization: kvm guest
I1006 22:24:18.312906 672451 out.go:179] * [missing-upgrade-381064] minikube v1.37.0 on Ubuntu 22.04 (kvm/amd64)
I1006 22:24:18.314132 672451 out.go:179] - MINIKUBE_LOCATION=21681
I1006 22:24:18.314131 672451 notify.go:220] Checking for updates...
I1006 22:24:18.316678 672451 out.go:179] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I1006 22:24:18.317887 672451 out.go:179] - KUBECONFIG=/home/jenkins/minikube-integration/21681-373053/kubeconfig
I1006 22:24:18.319118 672451 out.go:179] - MINIKUBE_HOME=/home/jenkins/minikube-integration/21681-373053/.minikube
I1006 22:24:18.320494 672451 out.go:179] - MINIKUBE_BIN=out/minikube-linux-amd64
I1006 22:24:18.322113 672451 out.go:179] - MINIKUBE_FORCE_SYSTEMD=
I1006 22:24:18.325437 672451 config.go:182] Loaded profile config "missing-upgrade-381064": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.3
I1006 22:24:18.328043 672451 out.go:179] * Kubernetes 1.34.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.34.1
I1006 22:24:18.329890 672451 driver.go:421] Setting default libvirt URI to qemu:///system
I1006 22:24:18.361203 672451 docker.go:123] docker version: linux-28.5.0:Docker Engine - Community
I1006 22:24:18.361323 672451 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1006 22:24:18.430441 672451 info.go:266] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:3 ContainersRunning:2 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:69 OomKillDisable:false NGoroutines:76 SystemTime:2025-10-06 22:24:18.419493826 +0000 UTC LoggingDriver:json-file CgroupDriver:systemd NEventsListener:0 KernelVersion:6.8.0-1041-gcp OperatingSystem:Ubuntu 22.04.5 LTS OSType:linux Architecture:x
86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[::1/128 127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33652174848 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-6 Labels:[] ExperimentalBuild:false ServerVersion:28.5.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:b98a3aace656320842a23f4a392a33f46af97866 Expected:} RuncCommit:{ID:v1.3.0-0-g4ca628d1 Expected:} InitCommit:{ID:de40ad0 Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=builtin name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[m
ap[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.29.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.40.0] map[Name:model Path:/usr/libexec/docker/cli-plugins/docker-model SchemaVersion:0.1.0 ShortDescription:Docker Model Runner Vendor:Docker Inc. Version:v0.1.42] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I1006 22:24:18.430580 672451 docker.go:318] overlay module found
I1006 22:24:18.434178 672451 out.go:179] * Using the docker driver based on existing profile
I1006 22:24:18.435511 672451 start.go:305] selected driver: docker
I1006 22:24:18.435532 672451 start.go:925] validating driver "docker" against &{Name:missing-upgrade-381064 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.3 ClusterName:missing-upgrade-381064 Namespace:default APIServerHAVIP: APIServerName:minikubeCA API
ServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.103.2 Port:8443 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirm
warePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I1006 22:24:18.435628 672451 start.go:936] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I1006 22:24:18.436157 672451 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1006 22:24:18.505172 672451 info.go:266] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:3 ContainersRunning:3 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:66 OomKillDisable:false NGoroutines:78 SystemTime:2025-10-06 22:24:18.49317381 +0000 UTC LoggingDriver:json-file CgroupDriver:systemd NEventsListener:0 KernelVersion:6.8.0-1041-gcp OperatingSystem:Ubuntu 22.04.5 LTS OSType:linux Architecture:x8
6_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[::1/128 127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33652174848 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-6 Labels:[] ExperimentalBuild:false ServerVersion:28.5.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:b98a3aace656320842a23f4a392a33f46af97866 Expected:} RuncCommit:{ID:v1.3.0-0-g4ca628d1 Expected:} InitCommit:{ID:de40ad0 Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=builtin name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[ma
p[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.29.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.40.0] map[Name:model Path:/usr/libexec/docker/cli-plugins/docker-model SchemaVersion:0.1.0 ShortDescription:Docker Model Runner Vendor:Docker Inc. Version:v0.1.42] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I1006 22:24:18.505587 672451 cni.go:84] Creating CNI manager for ""
I1006 22:24:18.505671 672451 cni.go:158] "docker" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge
I1006 22:24:18.505740 672451 start.go:349] cluster config:
{Name:missing-upgrade-381064 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.3 ClusterName:missing-upgrade-381064 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker C
RISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.103.2 Port:8443 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgen
tPID:0 GPUs: AutoPauseInterval:1m0s}
I1006 22:24:18.507593 672451 out.go:179] * Starting "missing-upgrade-381064" primary control-plane node in "missing-upgrade-381064" cluster
I1006 22:24:18.509323 672451 cache.go:123] Beginning downloading kic base image for docker with docker
I1006 22:24:18.510438 672451 out.go:179] * Pulling base image v0.0.48-1759745255-21703 ...
I1006 22:24:18.512127 672451 preload.go:183] Checking if preload exists for k8s version v1.28.3 and runtime docker
I1006 22:24:18.512189 672451 preload.go:198] Found local preload: /home/jenkins/minikube-integration/21681-373053/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4
I1006 22:24:18.512213 672451 cache.go:58] Caching tarball of preloaded images
I1006 22:24:18.512282 672451 image.go:81] Checking for gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 in local docker daemon
I1006 22:24:18.512342 672451 preload.go:233] Found /home/jenkins/minikube-integration/21681-373053/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I1006 22:24:18.512353 672451 cache.go:61] Finished verifying existence of preloaded tar for v1.28.3 on docker
I1006 22:24:18.512510 672451 profile.go:143] Saving config to /home/jenkins/minikube-integration/21681-373053/.minikube/profiles/missing-upgrade-381064/config.json ...
I1006 22:24:18.538731 672451 image.go:100] Found gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 in local docker daemon, skipping pull
I1006 22:24:18.538758 672451 cache.go:147] gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 exists in daemon, skipping load
I1006 22:24:18.538778 672451 cache.go:232] Successfully downloaded all kic artifacts
I1006 22:24:18.538814 672451 start.go:360] acquireMachinesLock for missing-upgrade-381064: {Name:mk7dca380665ba5f6523e311e0914c39b2dbfa39 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I1006 22:24:18.538917 672451 start.go:364] duration metric: took 55.615µs to acquireMachinesLock for "missing-upgrade-381064"
I1006 22:24:18.538940 672451 start.go:96] Skipping create...Using existing machine configuration
I1006 22:24:18.538947 672451 fix.go:54] fixHost starting:
I1006 22:24:18.539253 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:18.561266 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
I1006 22:24:18.561342 672451 fix.go:112] recreateIfNeeded on missing-upgrade-381064: state= err=unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:18.561383 672451 fix.go:117] machineExists: false. err=machine does not exist
I1006 22:24:18.566877 672451 out.go:179] * docker "missing-upgrade-381064" container is missing, will recreate.
I1006 22:24:18.568554 672451 delete.go:124] DEMOLISHING missing-upgrade-381064 ...
I1006 22:24:18.568729 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:18.591507 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
W1006 22:24:18.591573 672451 stop.go:83] unable to get state: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:18.591603 672451 delete.go:128] stophost failed (probably ok): ssh power off: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:18.591978 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:18.613019 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
I1006 22:24:18.613102 672451 delete.go:82] Unable to get host status for missing-upgrade-381064, assuming it has already been deleted: state: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:18.613175 672451 cli_runner.go:164] Run: docker container inspect -f {{.Id}} missing-upgrade-381064
W1006 22:24:18.637562 672451 cli_runner.go:211] docker container inspect -f {{.Id}} missing-upgrade-381064 returned with exit code 1
I1006 22:24:18.637618 672451 kic.go:371] could not find the container missing-upgrade-381064 to remove it. will try anyways
I1006 22:24:18.637677 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:18.657025 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
W1006 22:24:18.657105 672451 oci.go:84] error getting container status, will try to delete anyways: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:18.657175 672451 cli_runner.go:164] Run: docker exec --privileged -t missing-upgrade-381064 /bin/bash -c "sudo init 0"
W1006 22:24:18.685960 672451 cli_runner.go:211] docker exec --privileged -t missing-upgrade-381064 /bin/bash -c "sudo init 0" returned with exit code 1
I1006 22:24:18.686011 672451 oci.go:659] error shutdown missing-upgrade-381064: docker exec --privileged -t missing-upgrade-381064 /bin/bash -c "sudo init 0": exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:19.686252 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:19.704634 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
I1006 22:24:19.704706 672451 oci.go:671] temporary error verifying shutdown: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:19.704724 672451 oci.go:673] temporary error: container missing-upgrade-381064 status is but expect it to be exited
I1006 22:24:19.704760 672451 retry.go:31] will retry after 725.267462ms: couldn't verify container is exited. %v: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:20.430623 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:20.450016 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
I1006 22:24:20.450078 672451 oci.go:671] temporary error verifying shutdown: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:20.450090 672451 oci.go:673] temporary error: container missing-upgrade-381064 status is but expect it to be exited
I1006 22:24:20.450126 672451 retry.go:31] will retry after 844.540513ms: couldn't verify container is exited. %v: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:21.295701 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:21.313144 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
I1006 22:24:21.313206 672451 oci.go:671] temporary error verifying shutdown: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:21.313227 672451 oci.go:673] temporary error: container missing-upgrade-381064 status is but expect it to be exited
I1006 22:24:21.313256 672451 retry.go:31] will retry after 753.137335ms: couldn't verify container is exited. %v: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:22.066611 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:22.084978 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
I1006 22:24:22.085055 672451 oci.go:671] temporary error verifying shutdown: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:22.085071 672451 oci.go:673] temporary error: container missing-upgrade-381064 status is but expect it to be exited
I1006 22:24:22.085113 672451 retry.go:31] will retry after 1.601062319s: couldn't verify container is exited. %v: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:23.687211 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:23.708679 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
I1006 22:24:23.708750 672451 oci.go:671] temporary error verifying shutdown: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:23.708765 672451 oci.go:673] temporary error: container missing-upgrade-381064 status is but expect it to be exited
I1006 22:24:23.708803 672451 retry.go:31] will retry after 2.727006471s: couldn't verify container is exited. %v: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:26.437614 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:26.456139 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
I1006 22:24:26.456205 672451 oci.go:671] temporary error verifying shutdown: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:26.456235 672451 oci.go:673] temporary error: container missing-upgrade-381064 status is but expect it to be exited
I1006 22:24:26.456265 672451 retry.go:31] will retry after 4.297235192s: couldn't verify container is exited. %v: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:30.754641 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:30.775622 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
I1006 22:24:30.775731 672451 oci.go:671] temporary error verifying shutdown: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:30.775745 672451 oci.go:673] temporary error: container missing-upgrade-381064 status is but expect it to be exited
I1006 22:24:30.775798 672451 retry.go:31] will retry after 5.691681854s: couldn't verify container is exited. %v: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:36.470284 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:36.487978 672451 cli_runner.go:211] docker container inspect missing-upgrade-381064 --format={{.State.Status}} returned with exit code 1
I1006 22:24:36.488046 672451 oci.go:671] temporary error verifying shutdown: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:36.488056 672451 oci.go:673] temporary error: container missing-upgrade-381064 status is but expect it to be exited
I1006 22:24:36.488089 672451 oci.go:88] couldn't shut down missing-upgrade-381064 (might be okay): verify shutdown: couldn't verify container is exited. %v: unknown state "missing-upgrade-381064": docker container inspect missing-upgrade-381064 --format={{.State.Status}}: exit status 1
stdout:
stderr:
Error response from daemon: No such container: missing-upgrade-381064
I1006 22:24:36.488134 672451 cli_runner.go:164] Run: docker rm -f -v missing-upgrade-381064
I1006 22:24:36.505918 672451 cli_runner.go:164] Run: docker container inspect -f {{.Id}} missing-upgrade-381064
W1006 22:24:36.524229 672451 cli_runner.go:211] docker container inspect -f {{.Id}} missing-upgrade-381064 returned with exit code 1
I1006 22:24:36.524314 672451 cli_runner.go:164] Run: docker network inspect missing-upgrade-381064 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I1006 22:24:36.542183 672451 cli_runner.go:164] Run: docker network rm missing-upgrade-381064
I1006 22:24:36.686667 672451 fix.go:124] Sleeping 1 second for extra luck!
I1006 22:24:37.686807 672451 start.go:125] createHost starting for "" (driver="docker")
I1006 22:24:37.688891 672451 out.go:252] * Creating docker container (CPUs=2, Memory=3072MB) ...
I1006 22:24:37.689059 672451 start.go:159] libmachine.API.Create for "missing-upgrade-381064" (driver="docker")
I1006 22:24:37.689100 672451 client.go:168] LocalClient.Create starting
I1006 22:24:37.689226 672451 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca.pem
I1006 22:24:37.689281 672451 main.go:141] libmachine: Decoding PEM data...
I1006 22:24:37.689309 672451 main.go:141] libmachine: Parsing certificate...
I1006 22:24:37.689396 672451 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/21681-373053/.minikube/certs/cert.pem
I1006 22:24:37.689424 672451 main.go:141] libmachine: Decoding PEM data...
I1006 22:24:37.689441 672451 main.go:141] libmachine: Parsing certificate...
I1006 22:24:37.689757 672451 cli_runner.go:164] Run: docker network inspect missing-upgrade-381064 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W1006 22:24:37.708171 672451 cli_runner.go:211] docker network inspect missing-upgrade-381064 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I1006 22:24:37.708265 672451 network_create.go:284] running [docker network inspect missing-upgrade-381064] to gather additional debugging logs...
I1006 22:24:37.708289 672451 cli_runner.go:164] Run: docker network inspect missing-upgrade-381064
W1006 22:24:37.725874 672451 cli_runner.go:211] docker network inspect missing-upgrade-381064 returned with exit code 1
I1006 22:24:37.725911 672451 network_create.go:287] error running [docker network inspect missing-upgrade-381064]: docker network inspect missing-upgrade-381064: exit status 1
stdout:
[]
stderr:
Error response from daemon: network missing-upgrade-381064 not found
I1006 22:24:37.725928 672451 network_create.go:289] output of [docker network inspect missing-upgrade-381064]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network missing-upgrade-381064 not found
** /stderr **
I1006 22:24:37.726084 672451 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I1006 22:24:37.744035 672451 network.go:211] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName:br-3692094b8a9f IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:76:50:41:ed:fe:41} reservation:<nil>}
I1006 22:24:37.744758 672451 network.go:211] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName:br-f8b0bebc7358 IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:d6:8b:b4:39:f6:d2} reservation:<nil>}
I1006 22:24:37.745205 672451 network.go:211] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName:br-98a13c1ce7bc IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:0e:6d:07:61:3d:49} reservation:<nil>}
I1006 22:24:37.745747 672451 network.go:211] skipping subnet 192.168.76.0/24 that is taken: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName:br-b93d95213c7c IfaceIPv4:192.168.76.1 IfaceMTU:1500 IfaceMAC:b6:58:72:1e:6a:8d} reservation:<nil>}
I1006 22:24:37.746427 672451 network.go:211] skipping subnet 192.168.85.0/24 that is taken: &{IP:192.168.85.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.85.0/24 Gateway:192.168.85.1 ClientMin:192.168.85.2 ClientMax:192.168.85.254 Broadcast:192.168.85.255 IsPrivate:true Interface:{IfaceName:br-8e3eac84e698 IfaceIPv4:192.168.85.1 IfaceMTU:1500 IfaceMAC:be:4a:1b:7a:35:d3} reservation:<nil>}
I1006 22:24:37.747130 672451 network.go:206] using free private subnet 192.168.94.0/24: &{IP:192.168.94.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.94.0/24 Gateway:192.168.94.1 ClientMin:192.168.94.2 ClientMax:192.168.94.254 Broadcast:192.168.94.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0xc001fb0ad0}
I1006 22:24:37.747152 672451 network_create.go:124] attempt to create docker network missing-upgrade-381064 192.168.94.0/24 with gateway 192.168.94.1 and MTU of 1500 ...
I1006 22:24:37.747202 672451 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.94.0/24 --gateway=192.168.94.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=missing-upgrade-381064 missing-upgrade-381064
I1006 22:24:37.807038 672451 network_create.go:108] docker network missing-upgrade-381064 192.168.94.0/24 created
I1006 22:24:37.807073 672451 kic.go:121] calculated static IP "192.168.94.2" for the "missing-upgrade-381064" container
I1006 22:24:37.807133 672451 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I1006 22:24:37.825512 672451 cli_runner.go:164] Run: docker volume create missing-upgrade-381064 --label name.minikube.sigs.k8s.io=missing-upgrade-381064 --label created_by.minikube.sigs.k8s.io=true
I1006 22:24:37.843362 672451 oci.go:103] Successfully created a docker volume missing-upgrade-381064
I1006 22:24:37.843451 672451 cli_runner.go:164] Run: docker run --rm --name missing-upgrade-381064-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=missing-upgrade-381064 --entrypoint /usr/bin/test -v missing-upgrade-381064:/var gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 -d /var/lib
I1006 22:24:38.119267 672451 oci.go:107] Successfully prepared a docker volume missing-upgrade-381064
I1006 22:24:38.119326 672451 preload.go:183] Checking if preload exists for k8s version v1.28.3 and runtime docker
I1006 22:24:38.119359 672451 kic.go:194] Starting extracting preloaded images to volume ...
I1006 22:24:38.119510 672451 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/21681-373053/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v missing-upgrade-381064:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 -I lz4 -xf /preloaded.tar -C /extractDir
I1006 22:24:43.308253 672451 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/21681-373053/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v missing-upgrade-381064:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 -I lz4 -xf /preloaded.tar -C /extractDir: (5.188658973s)
I1006 22:24:43.308289 672451 kic.go:203] duration metric: took 5.188928228s to extract preloaded images to volume ...
W1006 22:24:43.308387 672451 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
W1006 22:24:43.308426 672451 oci.go:252] Your kernel does not support CPU cfs period/quota or the cgroup is not mounted.
I1006 22:24:43.308485 672451 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I1006 22:24:43.380429 672451 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname missing-upgrade-381064 --name missing-upgrade-381064 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=missing-upgrade-381064 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=missing-upgrade-381064 --network missing-upgrade-381064 --ip 192.168.94.2 --volume missing-upgrade-381064:/var --security-opt apparmor=unconfined --memory=3072mb -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0
I1006 22:24:43.850491 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Running}}
I1006 22:24:43.874549 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
I1006 22:24:43.897314 672451 cli_runner.go:164] Run: docker exec missing-upgrade-381064 stat /var/lib/dpkg/alternatives/iptables
I1006 22:24:43.948511 672451 oci.go:144] the created container "missing-upgrade-381064" has a running status.
I1006 22:24:43.948542 672451 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/21681-373053/.minikube/machines/missing-upgrade-381064/id_rsa...
I1006 22:24:44.236652 672451 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/21681-373053/.minikube/machines/missing-upgrade-381064/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I1006 22:24:44.275690 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
I1006 22:24:44.299201 672451 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I1006 22:24:44.299227 672451 kic_runner.go:114] Args: [docker exec --privileged missing-upgrade-381064 chown docker:docker /home/docker/.ssh/authorized_keys]
I1006 22:24:44.355521 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
I1006 22:24:44.378635 672451 machine.go:93] provisionDockerMachine start ...
I1006 22:24:44.378756 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:44.406487 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:44.407017 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:44.407052 672451 main.go:141] libmachine: About to run SSH command:
hostname
I1006 22:24:44.543960 672451 main.go:141] libmachine: SSH cmd err, output: <nil>: missing-upgrade-381064
I1006 22:24:44.543991 672451 ubuntu.go:182] provisioning hostname "missing-upgrade-381064"
I1006 22:24:44.544065 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:44.567727 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:44.567956 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:44.567972 672451 main.go:141] libmachine: About to run SSH command:
sudo hostname missing-upgrade-381064 && echo "missing-upgrade-381064" | sudo tee /etc/hostname
I1006 22:24:44.715481 672451 main.go:141] libmachine: SSH cmd err, output: <nil>: missing-upgrade-381064
I1006 22:24:44.715592 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:44.738328 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:44.738652 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:44.738690 672451 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\smissing-upgrade-381064' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 missing-upgrade-381064/g' /etc/hosts;
else
echo '127.0.1.1 missing-upgrade-381064' | sudo tee -a /etc/hosts;
fi
fi
I1006 22:24:44.861287 672451 main.go:141] libmachine: SSH cmd err, output: <nil>:
I1006 22:24:44.861322 672451 ubuntu.go:188] set auth options {CertDir:/home/jenkins/minikube-integration/21681-373053/.minikube CaCertPath:/home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/21681-373053/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/21681-373053/.minikube}
I1006 22:24:44.861394 672451 ubuntu.go:190] setting up certificates
I1006 22:24:44.861406 672451 provision.go:84] configureAuth start
I1006 22:24:44.861507 672451 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" missing-upgrade-381064
I1006 22:24:44.886668 672451 provision.go:143] copyHostCerts
I1006 22:24:44.886730 672451 exec_runner.go:144] found /home/jenkins/minikube-integration/21681-373053/.minikube/ca.pem, removing ...
I1006 22:24:44.886739 672451 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21681-373053/.minikube/ca.pem
I1006 22:24:44.886814 672451 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/21681-373053/.minikube/ca.pem (1078 bytes)
I1006 22:24:44.887000 672451 exec_runner.go:144] found /home/jenkins/minikube-integration/21681-373053/.minikube/cert.pem, removing ...
I1006 22:24:44.887015 672451 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21681-373053/.minikube/cert.pem
I1006 22:24:44.887061 672451 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21681-373053/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/21681-373053/.minikube/cert.pem (1123 bytes)
I1006 22:24:44.887177 672451 exec_runner.go:144] found /home/jenkins/minikube-integration/21681-373053/.minikube/key.pem, removing ...
I1006 22:24:44.887185 672451 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21681-373053/.minikube/key.pem
I1006 22:24:44.887228 672451 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21681-373053/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/21681-373053/.minikube/key.pem (1679 bytes)
I1006 22:24:44.887311 672451 provision.go:117] generating server cert: /home/jenkins/minikube-integration/21681-373053/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca-key.pem org=jenkins.missing-upgrade-381064 san=[127.0.0.1 192.168.94.2 localhost minikube missing-upgrade-381064]
I1006 22:24:44.981432 672451 provision.go:177] copyRemoteCerts
I1006 22:24:44.981541 672451 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I1006 22:24:44.981596 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:45.003134 672451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33427 SSHKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/missing-upgrade-381064/id_rsa Username:docker}
I1006 22:24:45.095758 672451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I1006 22:24:45.215837 672451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21681-373053/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I1006 22:24:45.243475 672451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21681-373053/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I1006 22:24:45.270233 672451 provision.go:87] duration metric: took 408.80616ms to configureAuth
I1006 22:24:45.270272 672451 ubuntu.go:206] setting minikube options for container-runtime
I1006 22:24:45.270495 672451 config.go:182] Loaded profile config "missing-upgrade-381064": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.3
I1006 22:24:45.270577 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:45.290251 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:45.290611 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:45.290633 672451 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I1006 22:24:45.408351 672451 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I1006 22:24:45.408377 672451 ubuntu.go:71] root file system type: overlay
I1006 22:24:45.408553 672451 provision.go:314] Updating docker unit: /lib/systemd/system/docker.service ...
I1006 22:24:45.408627 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:45.428607 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:45.428888 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:45.428997 672451 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target nss-lookup.target docker.socket firewalld.service containerd.service time-set.target
Wants=network-online.target containerd.service
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=always
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 \
-H fd:// --containerd=/run/containerd/containerd.sock \
-H unix:///var/run/docker.sock \
--default-ulimit=nofile=1048576:1048576 \
--tlsverify \
--tlscacert /etc/docker/ca.pem \
--tlscert /etc/docker/server.pem \
--tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I1006 22:24:45.573593 672451 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target nss-lookup.target docker.socket firewalld.service containerd.service time-set.target
Wants=network-online.target containerd.service
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=always
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd=/run/containerd/containerd.sock -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
I1006 22:24:45.573701 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:45.598655 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:45.599013 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:45.599046 672451 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I1006 22:24:47.631837 672451 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2023-10-26 09:06:22.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2025-10-06 22:24:45.570697750 +0000
@@ -1,30 +1,31 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service time-set.target
+After=network-online.target nss-lookup.target docker.socket firewalld.service containerd.service time-set.target
Wants=network-online.target containerd.service
Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutStartSec=0
-RestartSec=2
Restart=always
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd=/run/containerd/containerd.sock -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,9 +33,10 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I1006 22:24:47.631912 672451 machine.go:96] duration metric: took 3.253249874s to provisionDockerMachine
I1006 22:24:47.631927 672451 client.go:171] duration metric: took 9.942811415s to LocalClient.Create
I1006 22:24:47.631948 672451 start.go:167] duration metric: took 9.942891868s to libmachine.API.Create "missing-upgrade-381064"
I1006 22:24:47.631961 672451 start.go:293] postStartSetup for "missing-upgrade-381064" (driver="docker")
I1006 22:24:47.631977 672451 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I1006 22:24:47.632049 672451 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I1006 22:24:47.632101 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:47.652077 672451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33427 SSHKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/missing-upgrade-381064/id_rsa Username:docker}
I1006 22:24:47.746902 672451 ssh_runner.go:195] Run: cat /etc/os-release
I1006 22:24:47.750626 672451 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I1006 22:24:47.750667 672451 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I1006 22:24:47.750678 672451 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I1006 22:24:47.750687 672451 info.go:137] Remote host: Ubuntu 22.04.3 LTS
I1006 22:24:47.750701 672451 filesync.go:126] Scanning /home/jenkins/minikube-integration/21681-373053/.minikube/addons for local assets ...
I1006 22:24:47.750757 672451 filesync.go:126] Scanning /home/jenkins/minikube-integration/21681-373053/.minikube/files for local assets ...
I1006 22:24:47.750850 672451 filesync.go:149] local asset: /home/jenkins/minikube-integration/21681-373053/.minikube/files/etc/ssl/certs/3766342.pem -> 3766342.pem in /etc/ssl/certs
I1006 22:24:47.750975 672451 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I1006 22:24:47.760492 672451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21681-373053/.minikube/files/etc/ssl/certs/3766342.pem --> /etc/ssl/certs/3766342.pem (1708 bytes)
I1006 22:24:47.790018 672451 start.go:296] duration metric: took 158.026069ms for postStartSetup
I1006 22:24:47.790419 672451 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" missing-upgrade-381064
I1006 22:24:47.815874 672451 profile.go:143] Saving config to /home/jenkins/minikube-integration/21681-373053/.minikube/profiles/missing-upgrade-381064/config.json ...
I1006 22:24:47.816192 672451 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I1006 22:24:47.816254 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:47.835787 672451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33427 SSHKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/missing-upgrade-381064/id_rsa Username:docker}
I1006 22:24:47.922848 672451 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I1006 22:24:47.929031 672451 start.go:128] duration metric: took 10.242176518s to createHost
I1006 22:24:47.929125 672451 cli_runner.go:164] Run: docker container inspect missing-upgrade-381064 --format={{.State.Status}}
W1006 22:24:47.956976 672451 fix.go:138] unexpected machine state, will restart: <nil>
I1006 22:24:47.957011 672451 machine.go:93] provisionDockerMachine start ...
I1006 22:24:47.957097 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:47.986496 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:47.986802 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:47.986821 672451 main.go:141] libmachine: About to run SSH command:
hostname
I1006 22:24:48.119635 672451 main.go:141] libmachine: SSH cmd err, output: <nil>: missing-upgrade-381064
I1006 22:24:48.119668 672451 ubuntu.go:182] provisioning hostname "missing-upgrade-381064"
I1006 22:24:48.119742 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:48.141738 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:48.142033 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:48.142057 672451 main.go:141] libmachine: About to run SSH command:
sudo hostname missing-upgrade-381064 && echo "missing-upgrade-381064" | sudo tee /etc/hostname
I1006 22:24:48.279986 672451 main.go:141] libmachine: SSH cmd err, output: <nil>: missing-upgrade-381064
I1006 22:24:48.280072 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:48.301281 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:48.301510 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:48.301528 672451 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\smissing-upgrade-381064' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 missing-upgrade-381064/g' /etc/hosts;
else
echo '127.0.1.1 missing-upgrade-381064' | sudo tee -a /etc/hosts;
fi
fi
I1006 22:24:48.420141 672451 main.go:141] libmachine: SSH cmd err, output: <nil>:
I1006 22:24:48.420171 672451 ubuntu.go:188] set auth options {CertDir:/home/jenkins/minikube-integration/21681-373053/.minikube CaCertPath:/home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/21681-373053/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/21681-373053/.minikube}
I1006 22:24:48.420193 672451 ubuntu.go:190] setting up certificates
I1006 22:24:48.420204 672451 provision.go:84] configureAuth start
I1006 22:24:48.420257 672451 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" missing-upgrade-381064
I1006 22:24:48.440095 672451 provision.go:143] copyHostCerts
I1006 22:24:48.440157 672451 exec_runner.go:144] found /home/jenkins/minikube-integration/21681-373053/.minikube/key.pem, removing ...
I1006 22:24:48.440166 672451 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21681-373053/.minikube/key.pem
I1006 22:24:48.440222 672451 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21681-373053/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/21681-373053/.minikube/key.pem (1679 bytes)
I1006 22:24:48.440311 672451 exec_runner.go:144] found /home/jenkins/minikube-integration/21681-373053/.minikube/ca.pem, removing ...
I1006 22:24:48.440322 672451 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21681-373053/.minikube/ca.pem
I1006 22:24:48.440345 672451 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/21681-373053/.minikube/ca.pem (1078 bytes)
I1006 22:24:48.440411 672451 exec_runner.go:144] found /home/jenkins/minikube-integration/21681-373053/.minikube/cert.pem, removing ...
I1006 22:24:48.440418 672451 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21681-373053/.minikube/cert.pem
I1006 22:24:48.440436 672451 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21681-373053/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/21681-373053/.minikube/cert.pem (1123 bytes)
I1006 22:24:48.440521 672451 provision.go:117] generating server cert: /home/jenkins/minikube-integration/21681-373053/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca-key.pem org=jenkins.missing-upgrade-381064 san=[127.0.0.1 192.168.94.2 localhost minikube missing-upgrade-381064]
I1006 22:24:48.511940 672451 provision.go:177] copyRemoteCerts
I1006 22:24:48.511994 672451 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I1006 22:24:48.512029 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:48.530712 672451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33427 SSHKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/missing-upgrade-381064/id_rsa Username:docker}
I1006 22:24:48.620580 672451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21681-373053/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I1006 22:24:48.649107 672451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21681-373053/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I1006 22:24:48.674854 672451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21681-373053/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I1006 22:24:48.702329 672451 provision.go:87] duration metric: took 282.108762ms to configureAuth
I1006 22:24:48.702360 672451 ubuntu.go:206] setting minikube options for container-runtime
I1006 22:24:48.702550 672451 config.go:182] Loaded profile config "missing-upgrade-381064": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.3
I1006 22:24:48.702612 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:48.721401 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:48.721651 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:48.721668 672451 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I1006 22:24:48.840856 672451 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I1006 22:24:48.840883 672451 ubuntu.go:71] root file system type: overlay
I1006 22:24:48.841060 672451 provision.go:314] Updating docker unit: /lib/systemd/system/docker.service ...
I1006 22:24:48.841138 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:48.859356 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:48.859677 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:48.859791 672451 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target nss-lookup.target docker.socket firewalld.service containerd.service time-set.target
Wants=network-online.target containerd.service
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=always
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 \
-H fd:// --containerd=/run/containerd/containerd.sock \
-H unix:///var/run/docker.sock \
--default-ulimit=nofile=1048576:1048576 \
--tlsverify \
--tlscacert /etc/docker/ca.pem \
--tlscert /etc/docker/server.pem \
--tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I1006 22:24:48.992789 672451 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target nss-lookup.target docker.socket firewalld.service containerd.service time-set.target
Wants=network-online.target containerd.service
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=always
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd=/run/containerd/containerd.sock -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
I1006 22:24:48.992892 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:49.013144 672451 main.go:141] libmachine: Using SSH client type: native
I1006 22:24:49.013415 672451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x840040] 0x842d40 <nil> [] 0s} 127.0.0.1 33427 <nil> <nil>}
I1006 22:24:49.013436 672451 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I1006 22:24:49.138432 672451 main.go:141] libmachine: SSH cmd err, output: <nil>:
I1006 22:24:49.138492 672451 machine.go:96] duration metric: took 1.181471465s to provisionDockerMachine
I1006 22:24:49.138515 672451 start.go:293] postStartSetup for "missing-upgrade-381064" (driver="docker")
I1006 22:24:49.138531 672451 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I1006 22:24:49.138597 672451 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I1006 22:24:49.138645 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:49.158135 672451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33427 SSHKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/missing-upgrade-381064/id_rsa Username:docker}
I1006 22:24:49.246915 672451 ssh_runner.go:195] Run: cat /etc/os-release
I1006 22:24:49.250633 672451 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I1006 22:24:49.250659 672451 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I1006 22:24:49.250667 672451 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I1006 22:24:49.250674 672451 info.go:137] Remote host: Ubuntu 22.04.3 LTS
I1006 22:24:49.250684 672451 filesync.go:126] Scanning /home/jenkins/minikube-integration/21681-373053/.minikube/addons for local assets ...
I1006 22:24:49.250728 672451 filesync.go:126] Scanning /home/jenkins/minikube-integration/21681-373053/.minikube/files for local assets ...
I1006 22:24:49.250813 672451 filesync.go:149] local asset: /home/jenkins/minikube-integration/21681-373053/.minikube/files/etc/ssl/certs/3766342.pem -> 3766342.pem in /etc/ssl/certs
I1006 22:24:49.250954 672451 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I1006 22:24:49.259778 672451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21681-373053/.minikube/files/etc/ssl/certs/3766342.pem --> /etc/ssl/certs/3766342.pem (1708 bytes)
I1006 22:24:49.287961 672451 start.go:296] duration metric: took 149.426671ms for postStartSetup
I1006 22:24:49.288066 672451 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I1006 22:24:49.288115 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:49.309623 672451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33427 SSHKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/missing-upgrade-381064/id_rsa Username:docker}
I1006 22:24:49.392732 672451 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I1006 22:24:49.397209 672451 fix.go:56] duration metric: took 30.858256057s for fixHost
I1006 22:24:49.397236 672451 start.go:83] releasing machines lock for "missing-upgrade-381064", held for 30.858304766s
I1006 22:24:49.397319 672451 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" missing-upgrade-381064
I1006 22:24:49.416630 672451 ssh_runner.go:195] Run: cat /version.json
I1006 22:24:49.416692 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:49.416711 672451 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I1006 22:24:49.416785 672451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" missing-upgrade-381064
I1006 22:24:49.436980 672451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33427 SSHKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/missing-upgrade-381064/id_rsa Username:docker}
I1006 22:24:49.437346 672451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33427 SSHKeyPath:/home/jenkins/minikube-integration/21681-373053/.minikube/machines/missing-upgrade-381064/id_rsa Username:docker}
W1006 22:24:49.629087 672451 out.go:285] ! Image was not built for the current minikube version. To resolve this you can delete and recreate your minikube cluster using the latest images. Expected minikube version: v1.32.0 -> Actual minikube version: v1.37.0
! Image was not built for the current minikube version. To resolve this you can delete and recreate your minikube cluster using the latest images. Expected minikube version: v1.32.0 -> Actual minikube version: v1.37.0
I1006 22:24:49.629181 672451 ssh_runner.go:195] Run: systemctl --version
I1006 22:24:49.634155 672451 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I1006 22:24:49.639055 672451 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I1006 22:24:49.670847 672451 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I1006 22:24:49.670929 672451 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I1006 22:24:49.701089 672451 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I1006 22:24:49.701114 672451 start.go:495] detecting cgroup driver to use...
I1006 22:24:49.701150 672451 detect.go:190] detected "systemd" cgroup driver on host os
I1006 22:24:49.701253 672451 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I1006 22:24:49.722702 672451 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"
I1006 22:24:49.736661 672451 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I1006 22:24:49.747871 672451 containerd.go:146] configuring containerd to use "systemd" as cgroup driver...
I1006 22:24:49.747933 672451 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = true|g' /etc/containerd/config.toml"
I1006 22:24:49.758793 672451 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I1006 22:24:49.769321 672451 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I1006 22:24:49.780294 672451 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I1006 22:24:49.790957 672451 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I1006 22:24:49.802143 672451 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I1006 22:24:49.814079 672451 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I1006 22:24:49.825497 672451 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I1006 22:24:49.839074 672451 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I1006 22:24:49.848453 672451 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I1006 22:24:49.858182 672451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1006 22:24:49.933957 672451 ssh_runner.go:195] Run: sudo systemctl restart containerd
I1006 22:24:50.014207 672451 start.go:495] detecting cgroup driver to use...
I1006 22:24:50.014261 672451 detect.go:190] detected "systemd" cgroup driver on host os
I1006 22:24:50.014311 672451 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I1006 22:24:50.031690 672451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I1006 22:24:50.048439 672451 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I1006 22:24:50.065311 672451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I1006 22:24:50.078074 672451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I1006 22:24:50.091048 672451 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/cri-dockerd.sock
" | sudo tee /etc/crictl.yaml"
I1006 22:24:50.110030 672451 ssh_runner.go:195] Run: which cri-dockerd
I1006 22:24:50.114086 672451 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I1006 22:24:50.130897 672451 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)
I1006 22:24:50.151054 672451 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I1006 22:24:50.226731 672451 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I1006 22:24:50.309012 672451 docker.go:575] configuring docker to use "systemd" as cgroup driver...
I1006 22:24:50.309133 672451 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (129 bytes)
I1006 22:24:50.330151 672451 ssh_runner.go:195] Run: sudo systemctl reset-failed docker
I1006 22:24:50.344101 672451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1006 22:24:50.428367 672451 ssh_runner.go:195] Run: sudo systemctl restart docker
I1006 22:24:50.560105 672451 ssh_runner.go:195] Run: sudo journalctl --no-pager -u docker
I1006 22:24:50.572788 672451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I1006 22:24:50.584770 672451 ssh_runner.go:195] Run: sudo systemctl reset-failed docker
I1006 22:24:50.596690 672451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1006 22:24:50.669798 672451 ssh_runner.go:195] Run: sudo systemctl restart docker
I1006 22:24:50.735656 672451 ssh_runner.go:195] Run: sudo journalctl --no-pager -u docker
I1006 22:24:50.750233 672451 retry.go:31] will retry after 929.950443ms: docker not running
I1006 22:24:51.680730 672451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I1006 22:24:51.695413 672451 ssh_runner.go:195] Run: sudo systemctl reset-failed docker
I1006 22:24:51.708875 672451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1006 22:24:51.788039 672451 ssh_runner.go:195] Run: sudo systemctl restart docker
I1006 22:24:51.868025 672451 ssh_runner.go:195] Run: sudo journalctl --no-pager -u docker
I1006 22:24:51.885222 672451 retry.go:31] will retry after 1.847341517s: docker not running
I1006 22:24:53.733736 672451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I1006 22:24:53.747901 672451 ssh_runner.go:195] Run: sudo systemctl reset-failed docker
I1006 22:24:53.761219 672451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1006 22:24:53.844819 672451 ssh_runner.go:195] Run: sudo systemctl restart docker
I1006 22:24:53.936858 672451 ssh_runner.go:195] Run: sudo journalctl --no-pager -u docker
I1006 22:24:53.952274 672451 retry.go:31] will retry after 3.199365102s: docker not running
I1006 22:24:57.154609 672451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I1006 22:24:57.169617 672451 ssh_runner.go:195] Run: sudo systemctl reset-failed docker
I1006 22:24:57.185133 672451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1006 22:24:57.256818 672451 ssh_runner.go:195] Run: sudo systemctl restart docker
I1006 22:24:57.347512 672451 ssh_runner.go:195] Run: sudo journalctl --no-pager -u docker
I1006 22:24:57.367533 672451 retry.go:31] will retry after 2.384871454s: docker not running
I1006 22:24:59.752644 672451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I1006 22:24:59.770069 672451 ssh_runner.go:195] Run: sudo systemctl reset-failed docker
I1006 22:24:59.786988 672451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1006 22:24:59.885261 672451 ssh_runner.go:195] Run: sudo systemctl restart docker
I1006 22:24:59.974737 672451 ssh_runner.go:195] Run: sudo journalctl --no-pager -u docker
I1006 22:24:59.995409 672451 retry.go:31] will retry after 6.488796229s: docker not running
I1006 22:25:06.487294 672451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I1006 22:25:06.500188 672451 ssh_runner.go:195] Run: sudo systemctl reset-failed docker
I1006 22:25:06.512586 672451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1006 22:25:06.584926 672451 ssh_runner.go:195] Run: sudo systemctl restart docker
I1006 22:25:06.658515 672451 ssh_runner.go:195] Run: sudo journalctl --no-pager -u docker
I1006 22:25:06.676545 672451 out.go:203]
W1006 22:25:06.678074 672451 out.go:285] X Exiting due to RUNTIME_ENABLE: Failed to enable container runtime: docker not running
X Exiting due to RUNTIME_ENABLE: Failed to enable container runtime: docker not running
W1006 22:25:06.678093 672451 out.go:285] *
*
W1006 22:25:06.679839 672451 out.go:308] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I1006 22:25:06.682074 672451 out.go:203]
** /stderr **
version_upgrade_test.go:331: failed missing container upgrade from v1.32.0. args: out/minikube-linux-amd64 start -p missing-upgrade-381064 --memory=3072 --alsologtostderr -v=1 --driver=docker --container-runtime=docker : exit status 90
version_upgrade_test.go:333: *** TestMissingContainerUpgrade FAILED at 2025-10-06 22:25:06.727394379 +0000 UTC m=+2350.809046190
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:223: ======> post-mortem[TestMissingContainerUpgrade]: network settings <======
helpers_test.go:230: HOST ENV snapshots: PROXY env: HTTP_PROXY="<empty>" HTTPS_PROXY="<empty>" NO_PROXY="<empty>"
helpers_test.go:238: ======> post-mortem[TestMissingContainerUpgrade]: docker inspect <======
helpers_test.go:239: (dbg) Run: docker inspect missing-upgrade-381064
helpers_test.go:243: (dbg) docker inspect missing-upgrade-381064:
-- stdout --
[
{
"Id": "79a391223c93c316ee3347b56aff48772e4af1d4f4d3456e5dce4ebb8ddd887f",
"Created": "2025-10-06T22:24:43.39912246Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 679225,
"ExitCode": 0,
"Error": "",
"StartedAt": "2025-10-06T22:24:43.480661151Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:dbc648475405a75e8c472743ce721cb0b74db98d9501831a17a27a54e2bd3e47",
"ResolvConfPath": "/var/lib/docker/containers/79a391223c93c316ee3347b56aff48772e4af1d4f4d3456e5dce4ebb8ddd887f/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/79a391223c93c316ee3347b56aff48772e4af1d4f4d3456e5dce4ebb8ddd887f/hostname",
"HostsPath": "/var/lib/docker/containers/79a391223c93c316ee3347b56aff48772e4af1d4f4d3456e5dce4ebb8ddd887f/hosts",
"LogPath": "/var/lib/docker/containers/79a391223c93c316ee3347b56aff48772e4af1d4f4d3456e5dce4ebb8ddd887f/79a391223c93c316ee3347b56aff48772e4af1d4f4d3456e5dce4ebb8ddd887f-json.log",
"Name": "/missing-upgrade-381064",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"missing-upgrade-381064:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {
"max-size": "100m"
}
},
"NetworkMode": "missing-upgrade-381064",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "private",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 3221225472,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 6442450944,
"MemorySwappiness": null,
"OomKillDisable": null,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"ID": "79a391223c93c316ee3347b56aff48772e4af1d4f4d3456e5dce4ebb8ddd887f",
"LowerDir": "/var/lib/docker/overlay2/edaa5fb93402e47b7dd7165b0cee792a8cdd228629c3f84c645c47f7c76e6ca9-init/diff:/var/lib/docker/overlay2/265b4299b008e14db84bafe04f82f71091e672cc5b7cce5179299681e16664b5/diff",
"MergedDir": "/var/lib/docker/overlay2/edaa5fb93402e47b7dd7165b0cee792a8cdd228629c3f84c645c47f7c76e6ca9/merged",
"UpperDir": "/var/lib/docker/overlay2/edaa5fb93402e47b7dd7165b0cee792a8cdd228629c3f84c645c47f7c76e6ca9/diff",
"WorkDir": "/var/lib/docker/overlay2/edaa5fb93402e47b7dd7165b0cee792a8cdd228629c3f84c645c47f7c76e6ca9/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "missing-upgrade-381064",
"Source": "/var/lib/docker/volumes/missing-upgrade-381064/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "missing-upgrade-381064",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "missing-upgrade-381064",
"name.minikube.sigs.k8s.io": "missing-upgrade-381064",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "d7d697c305ebd157790bc2a7de26d79faf458f59d5a0192814b045d12cb1ded6",
"SandboxKey": "/var/run/docker/netns/d7d697c305eb",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33427"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33428"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33431"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33429"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33430"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"missing-upgrade-381064": {
"IPAMConfig": {
"IPv4Address": "192.168.94.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "9e:8b:87:00:27:87",
"DriverOpts": null,
"GwPriority": 0,
"NetworkID": "63d9ab314a2b057b8e0fc02cf1a4cd3d2b72a9fa4e4777204d323b3b5b34f724",
"EndpointID": "d026e447a811f32e9ae4a5435ffa08ff0c051979056ba43e6a59121b15c566e1",
"Gateway": "192.168.94.1",
"IPAddress": "192.168.94.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"missing-upgrade-381064",
"79a391223c93"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:247: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p missing-upgrade-381064 -n missing-upgrade-381064
helpers_test.go:247: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p missing-upgrade-381064 -n missing-upgrade-381064: exit status 6 (291.378204ms)
-- stdout --
Running
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E1006 22:25:07.022836 688307 status.go:458] kubeconfig endpoint: got: 192.168.103.2:8443, want: 192.168.94.2:8443
** /stderr **
helpers_test.go:247: status error: exit status 6 (may be ok)
helpers_test.go:252: <<< TestMissingContainerUpgrade FAILED: start of post-mortem logs <<<
helpers_test.go:253: ======> post-mortem[TestMissingContainerUpgrade]: minikube logs <======
helpers_test.go:255: (dbg) Run: out/minikube-linux-amd64 -p missing-upgrade-381064 logs -n 25
helpers_test.go:260: TestMissingContainerUpgrade logs:
-- stdout --
==> Audit <==
┌─────────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬───────────────────────────┬─────────┬─────────┬─────────────────────┬─────────────────────┐
│ COMMAND │ ARGS │ PROFILE │ USER │ VERSION │ START TIME │ END TIME │
├─────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────┼─────────┼─────────┼─────────────────────┼─────────────────────┤
│ delete │ -p NoKubernetes-947649 │ NoKubernetes-947649 │ jenkins │ v1.37.0 │ 06 Oct 25 22:22 UTC │ 06 Oct 25 22:22 UTC │
│ start │ -p NoKubernetes-947649 --no-kubernetes --memory=3072 --alsologtostderr -v=5 --driver=docker --container-runtime=docker │ NoKubernetes-947649 │ jenkins │ v1.37.0 │ 06 Oct 25 22:22 UTC │ 06 Oct 25 22:23 UTC │
│ ssh │ -p NoKubernetes-947649 sudo systemctl is-active --quiet service kubelet │ NoKubernetes-947649 │ jenkins │ v1.37.0 │ 06 Oct 25 22:23 UTC │ │
│ stop │ -p NoKubernetes-947649 │ NoKubernetes-947649 │ jenkins │ v1.37.0 │ 06 Oct 25 22:23 UTC │ 06 Oct 25 22:23 UTC │
│ start │ -p NoKubernetes-947649 --driver=docker --container-runtime=docker │ NoKubernetes-947649 │ jenkins │ v1.37.0 │ 06 Oct 25 22:23 UTC │ 06 Oct 25 22:23 UTC │
│ ssh │ cert-options-924273 ssh openssl x509 -text -noout -in /var/lib/minikube/certs/apiserver.crt │ cert-options-924273 │ jenkins │ v1.37.0 │ 06 Oct 25 22:23 UTC │ 06 Oct 25 22:23 UTC │
│ ssh │ -p cert-options-924273 -- sudo cat /etc/kubernetes/admin.conf │ cert-options-924273 │ jenkins │ v1.37.0 │ 06 Oct 25 22:23 UTC │ 06 Oct 25 22:23 UTC │
│ delete │ -p cert-options-924273 │ cert-options-924273 │ jenkins │ v1.37.0 │ 06 Oct 25 22:23 UTC │ 06 Oct 25 22:23 UTC │
│ start │ -p stopped-upgrade-852858 --memory=3072 --vm-driver=docker --container-runtime=docker │ stopped-upgrade-852858 │ jenkins │ v1.32.0 │ 06 Oct 25 22:23 UTC │ 06 Oct 25 22:24 UTC │
│ ssh │ -p NoKubernetes-947649 sudo systemctl is-active --quiet service kubelet │ NoKubernetes-947649 │ jenkins │ v1.37.0 │ 06 Oct 25 22:23 UTC │ │
│ delete │ -p NoKubernetes-947649 │ NoKubernetes-947649 │ jenkins │ v1.37.0 │ 06 Oct 25 22:23 UTC │ 06 Oct 25 22:23 UTC │
│ delete │ -p offline-docker-905409 │ offline-docker-905409 │ jenkins │ v1.37.0 │ 06 Oct 25 22:23 UTC │ 06 Oct 25 22:23 UTC │
│ start │ -p force-systemd-env-611588 --memory=3072 --alsologtostderr -v=5 --driver=docker --container-runtime=docker │ force-systemd-env-611588 │ jenkins │ v1.37.0 │ 06 Oct 25 22:23 UTC │ 06 Oct 25 22:24 UTC │
│ start │ -p missing-upgrade-381064 --memory=3072 --driver=docker --container-runtime=docker │ missing-upgrade-381064 │ jenkins │ v1.32.0 │ 06 Oct 25 22:23 UTC │ 06 Oct 25 22:24 UTC │
│ ssh │ force-systemd-env-611588 ssh docker info --format {{.CgroupDriver}} │ force-systemd-env-611588 │ jenkins │ v1.37.0 │ 06 Oct 25 22:24 UTC │ 06 Oct 25 22:24 UTC │
│ stop │ stopped-upgrade-852858 stop │ stopped-upgrade-852858 │ jenkins │ v1.32.0 │ 06 Oct 25 22:24 UTC │ 06 Oct 25 22:24 UTC │
│ delete │ -p force-systemd-env-611588 │ force-systemd-env-611588 │ jenkins │ v1.37.0 │ 06 Oct 25 22:24 UTC │ 06 Oct 25 22:24 UTC │
│ start │ -p kubernetes-upgrade-446559 --memory=3072 --kubernetes-version=v1.28.0 --alsologtostderr -v=1 --driver=docker --container-runtime=docker │ kubernetes-upgrade-446559 │ jenkins │ v1.37.0 │ 06 Oct 25 22:24 UTC │ 06 Oct 25 22:24 UTC │
│ start │ -p stopped-upgrade-852858 --memory=3072 --alsologtostderr -v=1 --driver=docker --container-runtime=docker │ stopped-upgrade-852858 │ jenkins │ v1.37.0 │ 06 Oct 25 22:24 UTC │ 06 Oct 25 22:24 UTC │
│ start │ -p missing-upgrade-381064 --memory=3072 --alsologtostderr -v=1 --driver=docker --container-runtime=docker │ missing-upgrade-381064 │ jenkins │ v1.37.0 │ 06 Oct 25 22:24 UTC │ │
│ stop │ -p kubernetes-upgrade-446559 │ kubernetes-upgrade-446559 │ jenkins │ v1.37.0 │ 06 Oct 25 22:24 UTC │ 06 Oct 25 22:24 UTC │
│ delete │ -p stopped-upgrade-852858 │ stopped-upgrade-852858 │ jenkins │ v1.37.0 │ 06 Oct 25 22:24 UTC │ 06 Oct 25 22:24 UTC │
│ start │ -p running-upgrade-975038 --memory=3072 --vm-driver=docker --container-runtime=docker │ running-upgrade-975038 │ jenkins │ v1.32.0 │ 06 Oct 25 22:24 UTC │ 06 Oct 25 22:25 UTC │
│ start │ -p kubernetes-upgrade-446559 --memory=3072 --kubernetes-version=v1.34.1 --alsologtostderr -v=1 --driver=docker --container-runtime=docker │ kubernetes-upgrade-446559 │ jenkins │ v1.37.0 │ 06 Oct 25 22:24 UTC │ │
│ start │ -p running-upgrade-975038 --memory=3072 --alsologtostderr -v=1 --driver=docker --container-runtime=docker │ running-upgrade-975038 │ jenkins │ v1.37.0 │ 06 Oct 25 22:25 UTC │ │
└─────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴───────────────────────────┴─────────┴─────────┴─────────────────────┴─────────────────────┘
==> Last Start <==
Log file created at: 2025/10/06 22:25:04
Running on machine: ubuntu-20-agent-6
Binary: Built with gc go1.24.6 for linux/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I1006 22:25:04.322609 687634 out.go:360] Setting OutFile to fd 1 ...
I1006 22:25:04.322898 687634 out.go:408] TERM=,COLORTERM=, which probably does not support color
I1006 22:25:04.322909 687634 out.go:374] Setting ErrFile to fd 2...
I1006 22:25:04.322913 687634 out.go:408] TERM=,COLORTERM=, which probably does not support color
I1006 22:25:04.323164 687634 root.go:338] Updating PATH: /home/jenkins/minikube-integration/21681-373053/.minikube/bin
I1006 22:25:04.323672 687634 out.go:368] Setting JSON to false
I1006 22:25:04.325076 687634 start.go:131] hostinfo: {"hostname":"ubuntu-20-agent-6","uptime":7647,"bootTime":1759781857,"procs":333,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"22.04","kernelVersion":"6.8.0-1041-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I1006 22:25:04.325197 687634 start.go:141] virtualization: kvm guest
I1006 22:25:04.327562 687634 out.go:179] * [running-upgrade-975038] minikube v1.37.0 on Ubuntu 22.04 (kvm/amd64)
I1006 22:25:04.329137 687634 notify.go:220] Checking for updates...
I1006 22:25:04.329142 687634 out.go:179] - MINIKUBE_LOCATION=21681
I1006 22:25:04.330664 687634 out.go:179] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I1006 22:25:04.332190 687634 out.go:179] - KUBECONFIG=/home/jenkins/minikube-integration/21681-373053/kubeconfig
I1006 22:25:04.333601 687634 out.go:179] - MINIKUBE_HOME=/home/jenkins/minikube-integration/21681-373053/.minikube
I1006 22:25:04.334853 687634 out.go:179] - MINIKUBE_BIN=out/minikube-linux-amd64
I1006 22:25:04.336154 687634 out.go:179] - MINIKUBE_FORCE_SYSTEMD=
I1006 22:25:04.338109 687634 config.go:182] Loaded profile config "running-upgrade-975038": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.28.3
I1006 22:25:04.339900 687634 out.go:179] * Kubernetes 1.34.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.34.1
I1006 22:25:04.341362 687634 driver.go:421] Setting default libvirt URI to qemu:///system
I1006 22:25:04.367318 687634 docker.go:123] docker version: linux-28.5.0:Docker Engine - Community
I1006 22:25:04.367440 687634 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1006 22:25:04.429143 687634 info.go:266] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:4 ContainersRunning:4 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:79 OomKillDisable:false NGoroutines:86 SystemTime:2025-10-06 22:25:04.417641495 +0000 UTC LoggingDriver:json-file CgroupDriver:systemd NEventsListener:0 KernelVersion:6.8.0-1041-gcp OperatingSystem:Ubuntu 22.04.5 LTS OSType:linux Architecture:x
86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[::1/128 127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33652174848 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-6 Labels:[] ExperimentalBuild:false ServerVersion:28.5.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:b98a3aace656320842a23f4a392a33f46af97866 Expected:} RuncCommit:{ID:v1.3.0-0-g4ca628d1 Expected:} InitCommit:{ID:de40ad0 Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=builtin name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[m
ap[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.29.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.40.0] map[Name:model Path:/usr/libexec/docker/cli-plugins/docker-model SchemaVersion:0.1.0 ShortDescription:Docker Model Runner Vendor:Docker Inc. Version:v0.1.42] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I1006 22:25:04.429281 687634 docker.go:318] overlay module found
I1006 22:25:04.432170 687634 out.go:179] * Using the docker driver based on existing profile
I1006 22:25:04.433602 687634 start.go:305] selected driver: docker
I1006 22:25:04.433623 687634 start.go:925] validating driver "docker" against &{Name:running-upgrade-975038 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.3 ClusterName:running-upgrade-975038 Namespace:default APIServerHAVIP: APIServerName:minikubeCA API
ServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.103.2 Port:8443 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirm
warePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I1006 22:25:04.433711 687634 start.go:936] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I1006 22:25:04.434411 687634 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1006 22:25:04.496585 687634 info.go:266] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:4 ContainersRunning:4 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:false CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:79 OomKillDisable:false NGoroutines:86 SystemTime:2025-10-06 22:25:04.485449801 +0000 UTC LoggingDriver:json-file CgroupDriver:systemd NEventsListener:0 KernelVersion:6.8.0-1041-gcp OperatingSystem:Ubuntu 22.04.5 LTS OSType:linux Architecture:x
86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[::1/128 127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33652174848 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-6 Labels:[] ExperimentalBuild:false ServerVersion:28.5.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:b98a3aace656320842a23f4a392a33f46af97866 Expected:} RuncCommit:{ID:v1.3.0-0-g4ca628d1 Expected:} InitCommit:{ID:de40ad0 Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=builtin name=cgroupns] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[m
ap[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.29.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.40.0] map[Name:model Path:/usr/libexec/docker/cli-plugins/docker-model SchemaVersion:0.1.0 ShortDescription:Docker Model Runner Vendor:Docker Inc. Version:v0.1.42] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.23.0]] Warnings:<nil>}}
I1006 22:25:04.496850 687634 cni.go:84] Creating CNI manager for ""
I1006 22:25:04.496914 687634 cni.go:158] "docker" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge
I1006 22:25:04.496948 687634 start.go:349] cluster config:
{Name:running-upgrade-975038 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.3 ClusterName:running-upgrade-975038 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker C
RISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.103.2 Port:8443 KubernetesVersion:v1.28.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgen
tPID:0 GPUs: AutoPauseInterval:1m0s}
I1006 22:25:04.498880 687634 out.go:179] * Starting "running-upgrade-975038" primary control-plane node in "running-upgrade-975038" cluster
I1006 22:25:04.500098 687634 cache.go:123] Beginning downloading kic base image for docker with docker
I1006 22:25:04.501590 687634 out.go:179] * Pulling base image v0.0.48-1759745255-21703 ...
I1006 22:25:04.502724 687634 preload.go:183] Checking if preload exists for k8s version v1.28.3 and runtime docker
I1006 22:25:04.502781 687634 preload.go:198] Found local preload: /home/jenkins/minikube-integration/21681-373053/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4
I1006 22:25:04.502797 687634 cache.go:58] Caching tarball of preloaded images
I1006 22:25:04.502824 687634 image.go:81] Checking for gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 in local docker daemon
I1006 22:25:04.502935 687634 preload.go:233] Found /home/jenkins/minikube-integration/21681-373053/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.3-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I1006 22:25:04.502953 687634 cache.go:61] Finished verifying existence of preloaded tar for v1.28.3 on docker
I1006 22:25:04.503067 687634 profile.go:143] Saving config to /home/jenkins/minikube-integration/21681-373053/.minikube/profiles/running-upgrade-975038/config.json ...
I1006 22:25:04.525153 687634 image.go:100] Found gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 in local docker daemon, skipping pull
I1006 22:25:04.525178 687634 cache.go:147] gcr.io/k8s-minikube/kicbase:v0.0.42@sha256:d35ac07dfda971cabee05e0deca8aeac772f885a5348e1a0c0b0a36db20fcfc0 exists in daemon, skipping load
I1006 22:25:04.525200 687634 cache.go:232] Successfully downloaded all kic artifacts
I1006 22:25:04.525233 687634 start.go:360] acquireMachinesLock for running-upgrade-975038: {Name:mkde543495db72f193f0cbbec80956d374040eea Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I1006 22:25:04.525356 687634 start.go:364] duration metric: took 96.593µs to acquireMachinesLock for "running-upgrade-975038"
I1006 22:25:04.525397 687634 start.go:96] Skipping create...Using existing machine configuration
I1006 22:25:04.525405 687634 fix.go:54] fixHost starting:
I1006 22:25:04.525690 687634 cli_runner.go:164] Run: docker container inspect running-upgrade-975038 --format={{.State.Status}}
I1006 22:25:04.545554 687634 fix.go:112] recreateIfNeeded on running-upgrade-975038: state=Running err=<nil>
W1006 22:25:04.545599 687634 fix.go:138] unexpected machine state, will restart: <nil>
I1006 22:25:06.487294 672451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I1006 22:25:06.500188 672451 ssh_runner.go:195] Run: sudo systemctl reset-failed docker
I1006 22:25:06.512586 672451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1006 22:25:06.584926 672451 ssh_runner.go:195] Run: sudo systemctl restart docker
I1006 22:25:06.658515 672451 ssh_runner.go:195] Run: sudo journalctl --no-pager -u docker
I1006 22:25:06.676545 672451 out.go:203]
W1006 22:25:06.678074 672451 out.go:285] X Exiting due to RUNTIME_ENABLE: Failed to enable container runtime: docker not running
W1006 22:25:06.678093 672451 out.go:285] *
W1006 22:25:06.679839 672451 out.go:308] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I1006 22:25:06.682074 672451 out.go:203]
==> Docker <==
Oct 06 22:25:00 missing-upgrade-381064 systemd[1]: Failed to start Docker Application Container Engine.
Oct 06 22:25:06 missing-upgrade-381064 systemd[1]: Starting Docker Application Container Engine...
Oct 06 22:25:06 missing-upgrade-381064 dockerd[1394]: invalid TLS configuration: Error reading X509 key pair (cert: "/etc/docker/server.pem", key: "/etc/docker/server-key.pem"): tls: private key does not match public key. Make sure the key is not encrypted.
Oct 06 22:25:06 missing-upgrade-381064 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Oct 06 22:25:06 missing-upgrade-381064 systemd[1]: docker.service: Failed with result 'exit-code'.
Oct 06 22:25:06 missing-upgrade-381064 systemd[1]: Failed to start Docker Application Container Engine.
Oct 06 22:25:06 missing-upgrade-381064 systemd[1]: docker.service: Scheduled restart job, restart counter is at 1.
Oct 06 22:25:06 missing-upgrade-381064 systemd[1]: Stopped Docker Application Container Engine.
Oct 06 22:25:06 missing-upgrade-381064 systemd[1]: Starting Docker Application Container Engine...
Oct 06 22:25:06 missing-upgrade-381064 dockerd[1411]: invalid TLS configuration: Error reading X509 key pair (cert: "/etc/docker/server.pem", key: "/etc/docker/server-key.pem"): tls: private key does not match public key. Make sure the key is not encrypted.
Oct 06 22:25:06 missing-upgrade-381064 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Oct 06 22:25:06 missing-upgrade-381064 systemd[1]: docker.service: Failed with result 'exit-code'.
Oct 06 22:25:06 missing-upgrade-381064 systemd[1]: Failed to start Docker Application Container Engine.
Oct 06 22:25:07 missing-upgrade-381064 systemd[1]: docker.service: Scheduled restart job, restart counter is at 2.
Oct 06 22:25:07 missing-upgrade-381064 systemd[1]: Stopped Docker Application Container Engine.
Oct 06 22:25:07 missing-upgrade-381064 systemd[1]: Starting Docker Application Container Engine...
Oct 06 22:25:07 missing-upgrade-381064 dockerd[1431]: invalid TLS configuration: Error reading X509 key pair (cert: "/etc/docker/server.pem", key: "/etc/docker/server-key.pem"): tls: private key does not match public key. Make sure the key is not encrypted.
Oct 06 22:25:07 missing-upgrade-381064 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Oct 06 22:25:07 missing-upgrade-381064 systemd[1]: docker.service: Failed with result 'exit-code'.
Oct 06 22:25:07 missing-upgrade-381064 systemd[1]: Failed to start Docker Application Container Engine.
Oct 06 22:25:07 missing-upgrade-381064 systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
Oct 06 22:25:07 missing-upgrade-381064 systemd[1]: Stopped Docker Application Container Engine.
Oct 06 22:25:07 missing-upgrade-381064 systemd[1]: docker.service: Start request repeated too quickly.
Oct 06 22:25:07 missing-upgrade-381064 systemd[1]: docker.service: Failed with result 'exit-code'.
Oct 06 22:25:07 missing-upgrade-381064 systemd[1]: Failed to start Docker Application Container Engine.
==> container status <==
command /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a" failed with error: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a": Process exited with status 1
stdout:
stderr:
time="2025-10-06T22:25:07Z" level=fatal msg="unable to determine runtime API version: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial unix /var/run/cri-dockerd.sock: connect: no such file or directory\""
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
==> describe nodes <==
command /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.3/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" failed with error: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.3/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
E1006 22:25:07.602959 1536 memcache.go:265] couldn't get current server API group list: Get "https://localhost:8443/api?timeout=32s": dial tcp [::1]:8443: connect: connection refused
E1006 22:25:07.603478 1536 memcache.go:265] couldn't get current server API group list: Get "https://localhost:8443/api?timeout=32s": dial tcp [::1]:8443: connect: connection refused
E1006 22:25:07.605114 1536 memcache.go:265] couldn't get current server API group list: Get "https://localhost:8443/api?timeout=32s": dial tcp [::1]:8443: connect: connection refused
E1006 22:25:07.605551 1536 memcache.go:265] couldn't get current server API group list: Get "https://localhost:8443/api?timeout=32s": dial tcp [::1]:8443: connect: connection refused
E1006 22:25:07.607127 1536 memcache.go:265] couldn't get current server API group list: Get "https://localhost:8443/api?timeout=32s": dial tcp [::1]:8443: connect: connection refused
The connection to the server localhost:8443 was refused - did you specify the right host or port?
==> dmesg <==
[ +42.025773] IPv4: martian source 10.244.0.1 from 10.244.0.3, on dev eth0
[ +0.000007] ll header: 00000000: ff ff ff ff ff ff 92 e0 4f 10 bf 25 08 06
[Oct 6 21:53] IPv4: martian source 10.244.0.1 from 10.244.0.4, on dev eth0
[ +0.000009] ll header: 00000000: ff ff ff ff ff ff 86 cc 46 20 11 91 08 06
[Oct 6 21:54] IPv4: martian source 10.244.0.1 from 10.244.0.12, on dev eth0
[ +0.000010] ll header: 00000000: ff ff ff ff ff ff b6 78 6a 31 44 af 08 06
[Oct 6 21:58] overlayfs: lowerdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
[Oct 6 22:07] IPv4: martian source 10.244.0.1 from 10.244.0.2, on dev eth0
[ +0.000009] ll header: 00000000: ff ff ff ff ff ff 52 6e 7e ae 28 e3 08 06
[Oct 6 22:10] IPv4: martian source 10.244.0.1 from 10.244.0.2, on dev eth0
[ +0.000009] ll header: 00000000: ff ff ff ff ff ff 7e dc 09 a9 12 57 08 06
[Oct 6 22:16] IPv4: martian source 10.244.0.1 from 10.244.0.2, on dev eth0
[ +0.000007] ll header: 00000000: ff ff ff ff ff ff 2a b6 b1 46 e4 3e 08 06
[Oct 6 22:18] IPv4: martian source 10.244.0.1 from 10.244.0.3, on dev eth0
[ +0.000007] ll header: 00000000: ff ff ff ff ff ff f2 b3 16 95 47 81 08 06
[Oct 6 22:19] IPv4: martian source 10.244.0.1 from 10.244.0.2, on dev eth0
[ +0.000008] ll header: 00000000: ff ff ff ff ff ff 6a c4 de ea fa 0b 08 06
[Oct 6 22:21] IPv4: martian source 10.244.0.1 from 10.244.0.2, on dev eth0
[ +0.000009] ll header: 00000000: ff ff ff ff ff ff 66 e9 41 58 8f c5 08 06
[Oct 6 22:22] IPv4: martian source 10.244.0.1 from 10.244.0.2, on dev eth0
[ +0.000008] ll header: 00000000: ff ff ff ff ff ff 66 28 60 26 f8 74 08 06
[ +0.080502] IPv4: martian source 10.244.0.1 from 10.244.0.3, on dev eth0
[ +0.000008] ll header: 00000000: ff ff ff ff ff ff 0a f3 a4 2c 50 22 08 06
[Oct 6 22:23] IPv4: martian source 10.244.0.1 from 10.244.0.2, on dev eth0
[ +0.000008] ll header: 00000000: ff ff ff ff ff ff c6 6d 9e b0 f5 73 08 06
==> kernel <==
22:25:07 up 2:07, 0 users, load average: 4.68, 3.58, 2.55
Linux missing-upgrade-381064 6.8.0-1041-gcp #43~22.04.1-Ubuntu SMP Wed Sep 24 23:11:19 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.3 LTS"
==> kubelet <==
-- No entries --
-- /stdout --
** stderr **
E1006 22:25:07.319194 688427 logs.go:279] Failed to list containers for "kube-apiserver": docker: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}: Process exited with status 1
stdout:
stderr:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
E1006 22:25:07.338975 688427 logs.go:279] Failed to list containers for "etcd": docker: docker ps -a --filter=name=k8s_etcd --format={{.ID}}: Process exited with status 1
stdout:
stderr:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
E1006 22:25:07.359979 688427 logs.go:279] Failed to list containers for "coredns": docker: docker ps -a --filter=name=k8s_coredns --format={{.ID}}: Process exited with status 1
stdout:
stderr:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
E1006 22:25:07.379444 688427 logs.go:279] Failed to list containers for "kube-scheduler": docker: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}: Process exited with status 1
stdout:
stderr:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
E1006 22:25:07.400380 688427 logs.go:279] Failed to list containers for "kube-proxy": docker: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}: Process exited with status 1
stdout:
stderr:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
E1006 22:25:07.421026 688427 logs.go:279] Failed to list containers for "kube-controller-manager": docker: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}: Process exited with status 1
stdout:
stderr:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
E1006 22:25:07.439864 688427 logs.go:279] Failed to list containers for "kindnet": docker: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}: Process exited with status 1
stdout:
stderr:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
E1006 22:25:07.459486 688427 logs.go:279] Failed to list containers for "storage-provisioner": docker: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}: Process exited with status 1
stdout:
stderr:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
** /stderr **
helpers_test.go:262: (dbg) Run: out/minikube-linux-amd64 status --format={{.APIServer}} -p missing-upgrade-381064 -n missing-upgrade-381064
helpers_test.go:262: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.APIServer}} -p missing-upgrade-381064 -n missing-upgrade-381064: exit status 6 (284.005169ms)
-- stdout --
Stopped
WARNING: Your kubectl is pointing to stale minikube-vm.
To fix the kubectl context, run `minikube update-context`
-- /stdout --
** stderr **
E1006 22:25:07.943295 688648 status.go:458] kubeconfig endpoint: got: 192.168.103.2:8443, want: 192.168.94.2:8443
** /stderr **
helpers_test.go:262: status error: exit status 6 (may be ok)
helpers_test.go:264: "missing-upgrade-381064" apiserver is not running, skipping kubectl commands (state="Stopped")
helpers_test.go:175: Cleaning up "missing-upgrade-381064" profile ...
helpers_test.go:178: (dbg) Run: out/minikube-linux-amd64 delete -p missing-upgrade-381064
helpers_test.go:178: (dbg) Done: out/minikube-linux-amd64 delete -p missing-upgrade-381064: (1.801926895s)
--- FAIL: TestMissingContainerUpgrade (104.10s)