=== RUN TestKubernetesUpgrade
=== PAUSE TestKubernetesUpgrade
=== CONT TestKubernetesUpgrade
version_upgrade_test.go:222: (dbg) Run: out/minikube-linux-arm64 start -p kubernetes-upgrade-068009 --memory=3072 --kubernetes-version=v1.28.0 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd
version_upgrade_test.go:222: (dbg) Done: out/minikube-linux-arm64 start -p kubernetes-upgrade-068009 --memory=3072 --kubernetes-version=v1.28.0 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd: (35.075346178s)
version_upgrade_test.go:227: (dbg) Run: out/minikube-linux-arm64 stop -p kubernetes-upgrade-068009
version_upgrade_test.go:227: (dbg) Done: out/minikube-linux-arm64 stop -p kubernetes-upgrade-068009: (1.260689755s)
version_upgrade_test.go:232: (dbg) Run: out/minikube-linux-arm64 -p kubernetes-upgrade-068009 status --format={{.Host}}
version_upgrade_test.go:232: (dbg) Non-zero exit: out/minikube-linux-arm64 -p kubernetes-upgrade-068009 status --format={{.Host}}: exit status 7 (145.945196ms)
-- stdout --
Stopped
-- /stdout --
version_upgrade_test.go:234: status error: exit status 7 (may be ok)
version_upgrade_test.go:243: (dbg) Run: out/minikube-linux-arm64 start -p kubernetes-upgrade-068009 --memory=3072 --kubernetes-version=v1.34.0 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd
E0920 04:58:53.562809 295509 cert_rotation.go:172] "Loading client cert failed" err="open /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/addons-115966/client.crt: no such file or directory" logger="tls-transport-cache.UnhandledError" key="key"
version_upgrade_test.go:243: (dbg) Done: out/minikube-linux-arm64 start -p kubernetes-upgrade-068009 --memory=3072 --kubernetes-version=v1.34.0 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd: (35.952324258s)
version_upgrade_test.go:248: (dbg) Run: kubectl --context kubernetes-upgrade-068009 version --output=json
version_upgrade_test.go:267: Attempting to downgrade Kubernetes (should fail)
version_upgrade_test.go:269: (dbg) Run: out/minikube-linux-arm64 start -p kubernetes-upgrade-068009 --memory=3072 --kubernetes-version=v1.28.0 --driver=docker --container-runtime=containerd
version_upgrade_test.go:269: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p kubernetes-upgrade-068009 --memory=3072 --kubernetes-version=v1.28.0 --driver=docker --container-runtime=containerd: exit status 106 (146.72513ms)
-- stdout --
* [kubernetes-upgrade-068009] minikube v1.37.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=21599
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/21599-293644/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/21599-293644/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
-- /stdout --
** stderr **
X Exiting due to K8S_DOWNGRADE_UNSUPPORTED: Unable to safely downgrade existing Kubernetes v1.34.0 cluster to v1.28.0
* Suggestion:
1) Recreate the cluster with Kubernetes 1.28.0, by running:
minikube delete -p kubernetes-upgrade-068009
minikube start -p kubernetes-upgrade-068009 --kubernetes-version=v1.28.0
2) Create a second cluster with Kubernetes 1.28.0, by running:
minikube start -p kubernetes-upgrade-0680092 --kubernetes-version=v1.28.0
3) Use the existing cluster at version Kubernetes 1.34.0, by running:
minikube start -p kubernetes-upgrade-068009 --kubernetes-version=v1.34.0
** /stderr **
version_upgrade_test.go:273: Attempting restart after unsuccessful downgrade
version_upgrade_test.go:275: (dbg) Run: out/minikube-linux-arm64 start -p kubernetes-upgrade-068009 --memory=3072 --kubernetes-version=v1.34.0 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd
version_upgrade_test.go:275: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p kubernetes-upgrade-068009 --memory=3072 --kubernetes-version=v1.34.0 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd: exit status 80 (7m30.099318339s)
-- stdout --
* [kubernetes-upgrade-068009] minikube v1.37.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=21599
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/21599-293644/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/21599-293644/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Using the docker driver based on existing profile
* Starting "kubernetes-upgrade-068009" primary control-plane node in "kubernetes-upgrade-068009" cluster
* Pulling base image v0.0.48 ...
* Preparing Kubernetes v1.34.0 on containerd 1.7.27 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
* Enabled addons:
-- /stdout --
** stderr **
I0920 04:59:21.616392 467239 out.go:360] Setting OutFile to fd 1 ...
I0920 04:59:21.616643 467239 out.go:408] TERM=,COLORTERM=, which probably does not support color
I0920 04:59:21.616672 467239 out.go:374] Setting ErrFile to fd 2...
I0920 04:59:21.616691 467239 out.go:408] TERM=,COLORTERM=, which probably does not support color
I0920 04:59:21.616991 467239 root.go:338] Updating PATH: /home/jenkins/minikube-integration/21599-293644/.minikube/bin
I0920 04:59:21.617496 467239 out.go:368] Setting JSON to false
I0920 04:59:21.622157 467239 start.go:130] hostinfo: {"hostname":"ip-172-31-24-2","uptime":6114,"bootTime":1758338248,"procs":195,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1084-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"6d436adf-771e-4269-b9a3-c25fd4fca4f5"}
I0920 04:59:21.622288 467239 start.go:140] virtualization:
I0920 04:59:21.627464 467239 out.go:179] * [kubernetes-upgrade-068009] minikube v1.37.0 on Ubuntu 20.04 (arm64)
I0920 04:59:21.630557 467239 out.go:179] - MINIKUBE_LOCATION=21599
I0920 04:59:21.630645 467239 notify.go:220] Checking for updates...
I0920 04:59:21.636831 467239 out.go:179] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0920 04:59:21.639976 467239 out.go:179] - KUBECONFIG=/home/jenkins/minikube-integration/21599-293644/kubeconfig
I0920 04:59:21.642865 467239 out.go:179] - MINIKUBE_HOME=/home/jenkins/minikube-integration/21599-293644/.minikube
I0920 04:59:21.645965 467239 out.go:179] - MINIKUBE_BIN=out/minikube-linux-arm64
I0920 04:59:21.648884 467239 out.go:179] - MINIKUBE_FORCE_SYSTEMD=
I0920 04:59:21.652310 467239 config.go:182] Loaded profile config "kubernetes-upgrade-068009": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.34.0
I0920 04:59:21.652894 467239 driver.go:421] Setting default libvirt URI to qemu:///system
I0920 04:59:21.719099 467239 docker.go:123] docker version: linux-28.1.1:Docker Engine - Community
I0920 04:59:21.719325 467239 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0920 04:59:21.850125 467239 info.go:266] docker info: {ID:J4M5:W6MX:GOX4:4LAQ:VI7E:VJNF:J3OP:OPBH:GF7G:PPY4:WQWD:7N4L Containers:2 ContainersRunning:2 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:52 OomKillDisable:true NGoroutines:63 SystemTime:2025-09-20 04:59:21.834668119 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1084-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[::1/128 127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-24-2 Labels:[] ExperimentalBuild:false ServerVersion:28.1.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:05044ec0a9a75232cad458027ca83437aae3f4da Expected:} RuncCommit:{ID:v1.2.5-0-g59923ef Expected:} InitCommit:{ID:de40ad0 Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Pat
h:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.23.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.35.1]] Warnings:<nil>}}
I0920 04:59:21.850239 467239 docker.go:318] overlay module found
I0920 04:59:21.853627 467239 out.go:179] * Using the docker driver based on existing profile
I0920 04:59:21.856542 467239 start.go:304] selected driver: docker
I0920 04:59:21.856564 467239 start.go:924] validating driver "docker" against &{Name:kubernetes-upgrade-068009 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.0 ClusterName:kubernetes-upgrade-068009 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.34.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: So
cketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0920 04:59:21.856669 467239 start.go:935] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0920 04:59:21.857422 467239 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0920 04:59:21.944024 467239 info.go:266] docker info: {ID:J4M5:W6MX:GOX4:4LAQ:VI7E:VJNF:J3OP:OPBH:GF7G:PPY4:WQWD:7N4L Containers:2 ContainersRunning:2 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:52 OomKillDisable:true NGoroutines:63 SystemTime:2025-09-20 04:59:21.934212028 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1084-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[::1/128 127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-24-2 Labels:[] ExperimentalBuild:false ServerVersion:28.1.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:05044ec0a9a75232cad458027ca83437aae3f4da Expected:} RuncCommit:{ID:v1.2.5-0-g59923ef Expected:} InitCommit:{ID:de40ad0 Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Pat
h:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.23.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.35.1]] Warnings:<nil>}}
I0920 04:59:21.944361 467239 cni.go:84] Creating CNI manager for ""
I0920 04:59:21.944424 467239 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0920 04:59:21.944461 467239 start.go:348] cluster config:
{Name:kubernetes-upgrade-068009 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.0 ClusterName:kubernetes-upgrade-068009 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime
:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.34.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs:
AutoPauseInterval:1m0s}
I0920 04:59:21.948134 467239 out.go:179] * Starting "kubernetes-upgrade-068009" primary control-plane node in "kubernetes-upgrade-068009" cluster
I0920 04:59:21.950863 467239 cache.go:123] Beginning downloading kic base image for docker with containerd
I0920 04:59:21.953937 467239 out.go:179] * Pulling base image v0.0.48 ...
I0920 04:59:21.957472 467239 preload.go:131] Checking if preload exists for k8s version v1.34.0 and runtime containerd
I0920 04:59:21.957542 467239 preload.go:146] Found local preload: /home/jenkins/minikube-integration/21599-293644/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.34.0-containerd-overlay2-arm64.tar.lz4
I0920 04:59:21.957551 467239 cache.go:58] Caching tarball of preloaded images
I0920 04:59:21.957637 467239 preload.go:172] Found /home/jenkins/minikube-integration/21599-293644/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.34.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0920 04:59:21.957645 467239 cache.go:61] Finished verifying existence of preloaded tar for v1.34.0 on containerd
I0920 04:59:21.957714 467239 image.go:81] Checking for gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 in local docker daemon
I0920 04:59:21.957744 467239 profile.go:143] Saving config to /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/config.json ...
I0920 04:59:21.986007 467239 image.go:100] Found gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 in local docker daemon, skipping pull
I0920 04:59:21.986123 467239 cache.go:147] gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 exists in daemon, skipping load
I0920 04:59:21.986150 467239 cache.go:232] Successfully downloaded all kic artifacts
I0920 04:59:21.986176 467239 start.go:360] acquireMachinesLock for kubernetes-upgrade-068009: {Name:mk3194005108dd00b33f651d322315abd349024a Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0920 04:59:21.986249 467239 start.go:364] duration metric: took 54.245µs to acquireMachinesLock for "kubernetes-upgrade-068009"
I0920 04:59:21.986271 467239 start.go:96] Skipping create...Using existing machine configuration
I0920 04:59:21.986276 467239 fix.go:54] fixHost starting:
I0920 04:59:21.986542 467239 cli_runner.go:164] Run: docker container inspect kubernetes-upgrade-068009 --format={{.State.Status}}
I0920 04:59:22.006434 467239 fix.go:112] recreateIfNeeded on kubernetes-upgrade-068009: state=Running err=<nil>
W0920 04:59:22.006467 467239 fix.go:138] unexpected machine state, will restart: <nil>
I0920 04:59:22.009086 467239 out.go:252] * Updating the running docker "kubernetes-upgrade-068009" container ...
I0920 04:59:22.009154 467239 machine.go:93] provisionDockerMachine start ...
I0920 04:59:22.009275 467239 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-068009
I0920 04:59:22.032290 467239 main.go:141] libmachine: Using SSH client type: native
I0920 04:59:22.032637 467239 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ef1e0] 0x3f19a0 <nil> [] 0s} 127.0.0.1 33378 <nil> <nil>}
I0920 04:59:22.032652 467239 main.go:141] libmachine: About to run SSH command:
hostname
I0920 04:59:22.189224 467239 main.go:141] libmachine: SSH cmd err, output: <nil>: kubernetes-upgrade-068009
I0920 04:59:22.189259 467239 ubuntu.go:182] provisioning hostname "kubernetes-upgrade-068009"
I0920 04:59:22.189323 467239 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-068009
I0920 04:59:22.212912 467239 main.go:141] libmachine: Using SSH client type: native
I0920 04:59:22.213295 467239 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ef1e0] 0x3f19a0 <nil> [] 0s} 127.0.0.1 33378 <nil> <nil>}
I0920 04:59:22.213319 467239 main.go:141] libmachine: About to run SSH command:
sudo hostname kubernetes-upgrade-068009 && echo "kubernetes-upgrade-068009" | sudo tee /etc/hostname
I0920 04:59:22.368501 467239 main.go:141] libmachine: SSH cmd err, output: <nil>: kubernetes-upgrade-068009
I0920 04:59:22.368593 467239 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-068009
I0920 04:59:22.386369 467239 main.go:141] libmachine: Using SSH client type: native
I0920 04:59:22.386729 467239 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ef1e0] 0x3f19a0 <nil> [] 0s} 127.0.0.1 33378 <nil> <nil>}
I0920 04:59:22.386756 467239 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\skubernetes-upgrade-068009' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 kubernetes-upgrade-068009/g' /etc/hosts;
else
echo '127.0.1.1 kubernetes-upgrade-068009' | sudo tee -a /etc/hosts;
fi
fi
I0920 04:59:22.538379 467239 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0920 04:59:22.538448 467239 ubuntu.go:188] set auth options {CertDir:/home/jenkins/minikube-integration/21599-293644/.minikube CaCertPath:/home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/21599-293644/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/21599-293644/.minikube}
I0920 04:59:22.538496 467239 ubuntu.go:190] setting up certificates
I0920 04:59:22.538531 467239 provision.go:84] configureAuth start
I0920 04:59:22.538614 467239 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" kubernetes-upgrade-068009
I0920 04:59:22.566591 467239 provision.go:143] copyHostCerts
I0920 04:59:22.566665 467239 exec_runner.go:144] found /home/jenkins/minikube-integration/21599-293644/.minikube/ca.pem, removing ...
I0920 04:59:22.566680 467239 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21599-293644/.minikube/ca.pem
I0920 04:59:22.566758 467239 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/21599-293644/.minikube/ca.pem (1078 bytes)
I0920 04:59:22.566854 467239 exec_runner.go:144] found /home/jenkins/minikube-integration/21599-293644/.minikube/cert.pem, removing ...
I0920 04:59:22.566859 467239 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21599-293644/.minikube/cert.pem
I0920 04:59:22.566890 467239 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/21599-293644/.minikube/cert.pem (1123 bytes)
I0920 04:59:22.566939 467239 exec_runner.go:144] found /home/jenkins/minikube-integration/21599-293644/.minikube/key.pem, removing ...
I0920 04:59:22.566943 467239 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21599-293644/.minikube/key.pem
I0920 04:59:22.566966 467239 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/21599-293644/.minikube/key.pem (1675 bytes)
I0920 04:59:22.567010 467239 provision.go:117] generating server cert: /home/jenkins/minikube-integration/21599-293644/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca-key.pem org=jenkins.kubernetes-upgrade-068009 san=[127.0.0.1 192.168.76.2 kubernetes-upgrade-068009 localhost minikube]
I0920 04:59:22.991943 467239 provision.go:177] copyRemoteCerts
I0920 04:59:22.992012 467239 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0920 04:59:22.992064 467239 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-068009
I0920 04:59:23.018064 467239 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33378 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/kubernetes-upgrade-068009/id_rsa Username:docker}
I0920 04:59:23.130868 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0920 04:59:23.172042 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/machines/server.pem --> /etc/docker/server.pem (1241 bytes)
I0920 04:59:23.202393 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0920 04:59:23.235362 467239 provision.go:87] duration metric: took 696.797399ms to configureAuth
I0920 04:59:23.235386 467239 ubuntu.go:206] setting minikube options for container-runtime
I0920 04:59:23.235559 467239 config.go:182] Loaded profile config "kubernetes-upgrade-068009": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.34.0
I0920 04:59:23.235567 467239 machine.go:96] duration metric: took 1.226405349s to provisionDockerMachine
I0920 04:59:23.235574 467239 start.go:293] postStartSetup for "kubernetes-upgrade-068009" (driver="docker")
I0920 04:59:23.235584 467239 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0920 04:59:23.235645 467239 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0920 04:59:23.235684 467239 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-068009
I0920 04:59:23.253694 467239 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33378 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/kubernetes-upgrade-068009/id_rsa Username:docker}
I0920 04:59:23.359755 467239 ssh_runner.go:195] Run: cat /etc/os-release
I0920 04:59:23.365912 467239 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0920 04:59:23.365950 467239 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0920 04:59:23.365964 467239 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0920 04:59:23.365971 467239 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I0920 04:59:23.365981 467239 filesync.go:126] Scanning /home/jenkins/minikube-integration/21599-293644/.minikube/addons for local assets ...
I0920 04:59:23.366045 467239 filesync.go:126] Scanning /home/jenkins/minikube-integration/21599-293644/.minikube/files for local assets ...
I0920 04:59:23.366158 467239 filesync.go:149] local asset: /home/jenkins/minikube-integration/21599-293644/.minikube/files/etc/ssl/certs/2955092.pem -> 2955092.pem in /etc/ssl/certs
I0920 04:59:23.366292 467239 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0920 04:59:23.382530 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/files/etc/ssl/certs/2955092.pem --> /etc/ssl/certs/2955092.pem (1708 bytes)
I0920 04:59:23.427584 467239 start.go:296] duration metric: took 191.994556ms for postStartSetup
I0920 04:59:23.427726 467239 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0920 04:59:23.427792 467239 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-068009
I0920 04:59:23.446289 467239 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33378 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/kubernetes-upgrade-068009/id_rsa Username:docker}
I0920 04:59:23.546617 467239 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0920 04:59:23.551568 467239 fix.go:56] duration metric: took 1.565283913s for fixHost
I0920 04:59:23.551595 467239 start.go:83] releasing machines lock for "kubernetes-upgrade-068009", held for 1.565336187s
I0920 04:59:23.551665 467239 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" kubernetes-upgrade-068009
I0920 04:59:23.575304 467239 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0920 04:59:23.575387 467239 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-068009
I0920 04:59:23.575502 467239 ssh_runner.go:195] Run: cat /version.json
I0920 04:59:23.575549 467239 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-068009
I0920 04:59:23.619944 467239 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33378 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/kubernetes-upgrade-068009/id_rsa Username:docker}
I0920 04:59:23.621929 467239 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33378 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/kubernetes-upgrade-068009/id_rsa Username:docker}
I0920 04:59:23.944826 467239 ssh_runner.go:195] Run: systemctl --version
I0920 04:59:23.950640 467239 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0920 04:59:23.955923 467239 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0920 04:59:24.015985 467239 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0920 04:59:24.016070 467239 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0920 04:59:24.028436 467239 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0920 04:59:24.028518 467239 start.go:495] detecting cgroup driver to use...
I0920 04:59:24.028569 467239 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0920 04:59:24.028650 467239 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0920 04:59:24.043380 467239 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0920 04:59:24.056590 467239 docker.go:218] disabling cri-docker service (if available) ...
I0920 04:59:24.056651 467239 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0920 04:59:24.078076 467239 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0920 04:59:24.097458 467239 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0920 04:59:24.272888 467239 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0920 04:59:24.438162 467239 docker.go:234] disabling docker service ...
I0920 04:59:24.438225 467239 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0920 04:59:24.461277 467239 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0920 04:59:24.478721 467239 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0920 04:59:24.676497 467239 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0920 04:59:24.951409 467239 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0920 04:59:24.974117 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0920 04:59:25.107045 467239 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10.1"|' /etc/containerd/config.toml"
I0920 04:59:25.178130 467239 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0920 04:59:25.220598 467239 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0920 04:59:25.220661 467239 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0920 04:59:25.265970 467239 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0920 04:59:25.324232 467239 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0920 04:59:25.350729 467239 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0920 04:59:25.361174 467239 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0920 04:59:25.391160 467239 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0920 04:59:25.421269 467239 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0920 04:59:25.443674 467239 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0920 04:59:25.458526 467239 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0920 04:59:25.477611 467239 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0920 04:59:25.491014 467239 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0920 04:59:25.679504 467239 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0920 04:59:26.010328 467239 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0920 04:59:26.010404 467239 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0920 04:59:26.016961 467239 start.go:563] Will wait 60s for crictl version
I0920 04:59:26.017023 467239 ssh_runner.go:195] Run: which crictl
I0920 04:59:26.021810 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0920 04:59:26.072566 467239 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.27
RuntimeApiVersion: v1
I0920 04:59:26.072652 467239 ssh_runner.go:195] Run: containerd --version
I0920 04:59:26.103916 467239 ssh_runner.go:195] Run: containerd --version
I0920 04:59:26.154616 467239 out.go:179] * Preparing Kubernetes v1.34.0 on containerd 1.7.27 ...
I0920 04:59:26.157815 467239 cli_runner.go:164] Run: docker network inspect kubernetes-upgrade-068009 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0920 04:59:26.208408 467239 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0920 04:59:26.213334 467239 kubeadm.go:875] updating cluster {Name:kubernetes-upgrade-068009 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.0 ClusterName:kubernetes-upgrade-068009 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNa
mes:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.34.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketVMnetClientPa
th: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0920 04:59:26.213478 467239 preload.go:131] Checking if preload exists for k8s version v1.34.0 and runtime containerd
I0920 04:59:26.213533 467239 ssh_runner.go:195] Run: sudo crictl images --output json
I0920 04:59:26.261326 467239 containerd.go:623] couldn't find preloaded image for "registry.k8s.io/etcd:3.6.4-0". assuming images are not preloaded.
I0920 04:59:26.261418 467239 ssh_runner.go:195] Run: which lz4
I0920 04:59:26.265012 467239 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0920 04:59:26.269078 467239 ssh_runner.go:356] copy: skipping /preloaded.tar.lz4 (exists)
I0920 04:59:26.269103 467239 containerd.go:563] duration metric: took 4.13826ms to copy over tarball
I0920 04:59:26.269161 467239 ssh_runner.go:195] Run: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4
I0920 04:59:31.523098 467239 ssh_runner.go:235] Completed: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: (5.253902289s)
I0920 04:59:31.523172 467239 kubeadm.go:901] preload failed, will try to load cached images: extracting tarball:
** stderr **
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Europe: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Brazil: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Canada: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Antarctica: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Chile: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Etc: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Pacific: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Mexico: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Australia: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/US: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Asia: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Atlantic: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/America: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Arctic: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Africa: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Indian: Cannot open: File exists
tar: Exiting with failure status due to previous errors
** /stderr **: sudo tar --xattrs --xattrs-include security.capability -I lz4 -C /var -xf /preloaded.tar.lz4: Process exited with status 2
stdout:
stderr:
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Europe: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Brazil: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Canada: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Antarctica: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Chile: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Etc: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Pacific: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Mexico: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Australia: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/US: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Asia: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Atlantic: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/America: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Arctic: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Africa: Cannot open: File exists
tar: ./lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/3/fs/usr/share/zoneinfo/posix/Indian: Cannot open: File exists
tar: Exiting with failure status due to previous errors
I0920 04:59:31.523255 467239 ssh_runner.go:195] Run: sudo crictl images --output json
I0920 04:59:31.579397 467239 containerd.go:623] couldn't find preloaded image for "registry.k8s.io/etcd:3.6.4-0". assuming images are not preloaded.
I0920 04:59:31.579424 467239 cache_images.go:89] LoadCachedImages start: [registry.k8s.io/kube-apiserver:v1.34.0 registry.k8s.io/kube-controller-manager:v1.34.0 registry.k8s.io/kube-scheduler:v1.34.0 registry.k8s.io/kube-proxy:v1.34.0 registry.k8s.io/pause:3.10.1 registry.k8s.io/etcd:3.6.4-0 registry.k8s.io/coredns/coredns:v1.12.1 gcr.io/k8s-minikube/storage-provisioner:v5]
I0920 04:59:31.579492 467239 image.go:138] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0920 04:59:31.579795 467239 image.go:138] retrieving image: registry.k8s.io/kube-apiserver:v1.34.0
I0920 04:59:31.579925 467239 image.go:138] retrieving image: registry.k8s.io/kube-controller-manager:v1.34.0
I0920 04:59:31.580028 467239 image.go:138] retrieving image: registry.k8s.io/kube-scheduler:v1.34.0
I0920 04:59:31.580125 467239 image.go:138] retrieving image: registry.k8s.io/kube-proxy:v1.34.0
I0920 04:59:31.580241 467239 image.go:138] retrieving image: registry.k8s.io/pause:3.10.1
I0920 04:59:31.580336 467239 image.go:138] retrieving image: registry.k8s.io/etcd:3.6.4-0
I0920 04:59:31.580423 467239 image.go:138] retrieving image: registry.k8s.io/coredns/coredns:v1.12.1
I0920 04:59:31.581604 467239 image.go:181] daemon lookup for registry.k8s.io/pause:3.10.1: Error response from daemon: No such image: registry.k8s.io/pause:3.10.1
I0920 04:59:31.582193 467239 image.go:181] daemon lookup for registry.k8s.io/kube-apiserver:v1.34.0: Error response from daemon: No such image: registry.k8s.io/kube-apiserver:v1.34.0
I0920 04:59:31.582404 467239 image.go:181] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error response from daemon: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0920 04:59:31.582722 467239 image.go:181] daemon lookup for registry.k8s.io/etcd:3.6.4-0: Error response from daemon: No such image: registry.k8s.io/etcd:3.6.4-0
I0920 04:59:31.582942 467239 image.go:181] daemon lookup for registry.k8s.io/coredns/coredns:v1.12.1: Error response from daemon: No such image: registry.k8s.io/coredns/coredns:v1.12.1
I0920 04:59:31.583110 467239 image.go:181] daemon lookup for registry.k8s.io/kube-proxy:v1.34.0: Error response from daemon: No such image: registry.k8s.io/kube-proxy:v1.34.0
I0920 04:59:31.586207 467239 image.go:181] daemon lookup for registry.k8s.io/kube-scheduler:v1.34.0: Error response from daemon: No such image: registry.k8s.io/kube-scheduler:v1.34.0
I0920 04:59:31.586572 467239 image.go:181] daemon lookup for registry.k8s.io/kube-controller-manager:v1.34.0: Error response from daemon: No such image: registry.k8s.io/kube-controller-manager:v1.34.0
I0920 04:59:31.899451 467239 containerd.go:267] Checking existence of image with name "registry.k8s.io/etcd:3.6.4-0" and sha "a1894772a478e07c67a56e8bf32335fdbe1dd4ec96976a5987083164bd00bc0e"
I0920 04:59:31.899553 467239 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images ls name==registry.k8s.io/etcd:3.6.4-0
I0920 04:59:31.915278 467239 containerd.go:267] Checking existence of image with name "registry.k8s.io/kube-scheduler:v1.34.0" and sha "a25f5ef9c34c37c649f3b4f9631a169221ac2d6f41d9767c7588cd355f76f9ee"
I0920 04:59:31.915344 467239 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images ls name==registry.k8s.io/kube-scheduler:v1.34.0
I0920 04:59:31.927549 467239 containerd.go:267] Checking existence of image with name "registry.k8s.io/kube-controller-manager:v1.34.0" and sha "996be7e86d9b3a549d718de63713d9fea9db1f45ac44863a6770292d7b463570"
I0920 04:59:31.927622 467239 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images ls name==registry.k8s.io/kube-controller-manager:v1.34.0
I0920 04:59:31.935622 467239 containerd.go:267] Checking existence of image with name "registry.k8s.io/kube-proxy:v1.34.0" and sha "6fc32d66c141152245438e6512df788cb52d64a1617e33561950b0e7a4675abf"
I0920 04:59:31.935695 467239 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images ls name==registry.k8s.io/kube-proxy:v1.34.0
I0920 04:59:31.938886 467239 containerd.go:267] Checking existence of image with name "registry.k8s.io/coredns/coredns:v1.12.1" and sha "138784d87c9c50f8e59412544da4cf4928d61ccbaf93b9f5898a3ba406871bfc"
I0920 04:59:31.939005 467239 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images ls name==registry.k8s.io/coredns/coredns:v1.12.1
I0920 04:59:31.944105 467239 containerd.go:267] Checking existence of image with name "registry.k8s.io/pause:3.10.1" and sha "d7b100cd9a77ba782c5e428c8dd5a1df4a1e79d4cb6294acd7d01290ab3babbd"
I0920 04:59:31.944225 467239 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images ls name==registry.k8s.io/pause:3.10.1
I0920 04:59:31.995561 467239 containerd.go:267] Checking existence of image with name "registry.k8s.io/kube-apiserver:v1.34.0" and sha "d291939e994064911484215449d0ab96c535b02adc4fc5d0ad4e438cf71465be"
I0920 04:59:31.995713 467239 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images ls name==registry.k8s.io/kube-apiserver:v1.34.0
I0920 04:59:32.123137 467239 cache_images.go:117] "registry.k8s.io/etcd:3.6.4-0" needs transfer: "registry.k8s.io/etcd:3.6.4-0" does not exist at hash "a1894772a478e07c67a56e8bf32335fdbe1dd4ec96976a5987083164bd00bc0e" in container runtime
I0920 04:59:32.123227 467239 cri.go:218] Removing image: registry.k8s.io/etcd:3.6.4-0
I0920 04:59:32.123307 467239 ssh_runner.go:195] Run: which crictl
I0920 04:59:32.135146 467239 cache_images.go:117] "registry.k8s.io/kube-controller-manager:v1.34.0" needs transfer: "registry.k8s.io/kube-controller-manager:v1.34.0" does not exist at hash "996be7e86d9b3a549d718de63713d9fea9db1f45ac44863a6770292d7b463570" in container runtime
I0920 04:59:32.135237 467239 cri.go:218] Removing image: registry.k8s.io/kube-controller-manager:v1.34.0
I0920 04:59:32.135348 467239 ssh_runner.go:195] Run: which crictl
I0920 04:59:32.135501 467239 cache_images.go:117] "registry.k8s.io/kube-scheduler:v1.34.0" needs transfer: "registry.k8s.io/kube-scheduler:v1.34.0" does not exist at hash "a25f5ef9c34c37c649f3b4f9631a169221ac2d6f41d9767c7588cd355f76f9ee" in container runtime
I0920 04:59:32.135552 467239 cri.go:218] Removing image: registry.k8s.io/kube-scheduler:v1.34.0
I0920 04:59:32.135594 467239 ssh_runner.go:195] Run: which crictl
I0920 04:59:32.135850 467239 cache_images.go:117] "registry.k8s.io/kube-proxy:v1.34.0" needs transfer: "registry.k8s.io/kube-proxy:v1.34.0" does not exist at hash "6fc32d66c141152245438e6512df788cb52d64a1617e33561950b0e7a4675abf" in container runtime
I0920 04:59:32.135906 467239 cri.go:218] Removing image: registry.k8s.io/kube-proxy:v1.34.0
I0920 04:59:32.135966 467239 ssh_runner.go:195] Run: which crictl
I0920 04:59:32.141311 467239 cache_images.go:117] "registry.k8s.io/coredns/coredns:v1.12.1" needs transfer: "registry.k8s.io/coredns/coredns:v1.12.1" does not exist at hash "138784d87c9c50f8e59412544da4cf4928d61ccbaf93b9f5898a3ba406871bfc" in container runtime
I0920 04:59:32.141500 467239 cri.go:218] Removing image: registry.k8s.io/coredns/coredns:v1.12.1
I0920 04:59:32.141283 467239 cache_images.go:117] "registry.k8s.io/pause:3.10.1" needs transfer: "registry.k8s.io/pause:3.10.1" does not exist at hash "d7b100cd9a77ba782c5e428c8dd5a1df4a1e79d4cb6294acd7d01290ab3babbd" in container runtime
I0920 04:59:32.141578 467239 cri.go:218] Removing image: registry.k8s.io/pause:3.10.1
I0920 04:59:32.141651 467239 ssh_runner.go:195] Run: which crictl
I0920 04:59:32.141792 467239 ssh_runner.go:195] Run: which crictl
I0920 04:59:32.156183 467239 cache_images.go:117] "registry.k8s.io/kube-apiserver:v1.34.0" needs transfer: "registry.k8s.io/kube-apiserver:v1.34.0" does not exist at hash "d291939e994064911484215449d0ab96c535b02adc4fc5d0ad4e438cf71465be" in container runtime
I0920 04:59:32.156229 467239 cri.go:218] Removing image: registry.k8s.io/kube-apiserver:v1.34.0
I0920 04:59:32.156278 467239 ssh_runner.go:195] Run: which crictl
I0920 04:59:32.156365 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.6.4-0
I0920 04:59:32.156473 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.34.0
I0920 04:59:32.156619 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.34.0
I0920 04:59:32.156763 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.34.0
I0920 04:59:32.156907 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns/coredns:v1.12.1
I0920 04:59:32.157029 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.10.1
I0920 04:59:32.488938 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns/coredns:v1.12.1
I0920 04:59:32.489038 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.34.0
I0920 04:59:32.489103 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.6.4-0
I0920 04:59:32.489139 467239 cache_images.go:290] Loading image from: /home/jenkins/minikube-integration/21599-293644/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.34.0
I0920 04:59:32.489162 467239 cache_images.go:290] Loading image from: /home/jenkins/minikube-integration/21599-293644/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.34.0
I0920 04:59:32.489180 467239 cache_images.go:290] Loading image from: /home/jenkins/minikube-integration/21599-293644/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.34.0
I0920 04:59:32.489202 467239 cache_images.go:290] Loading image from: /home/jenkins/minikube-integration/21599-293644/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10.1
I0920 04:59:32.638436 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.6.4-0
I0920 04:59:32.638531 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns/coredns:v1.12.1
I0920 04:59:32.638575 467239 cache_images.go:290] Loading image from: /home/jenkins/minikube-integration/21599-293644/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.34.0
I0920 04:59:32.727734 467239 cache_images.go:290] Loading image from: /home/jenkins/minikube-integration/21599-293644/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.12.1
I0920 04:59:32.727800 467239 cache_images.go:290] Loading image from: /home/jenkins/minikube-integration/21599-293644/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.6.4-0
W0920 04:59:33.046490 467239 image.go:286] image gcr.io/k8s-minikube/storage-provisioner:v5 arch mismatch: want arm64 got amd64. fixing
I0920 04:59:33.046659 467239 containerd.go:267] Checking existence of image with name "gcr.io/k8s-minikube/storage-provisioner:v5" and sha "66749159455b3f08c8318fe0233122f54d0f5889f9c5fdfb73c3fd9d99895b51"
I0920 04:59:33.046734 467239 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images ls name==gcr.io/k8s-minikube/storage-provisioner:v5
I0920 04:59:33.081588 467239 cache_images.go:117] "gcr.io/k8s-minikube/storage-provisioner:v5" needs transfer: "gcr.io/k8s-minikube/storage-provisioner:v5" does not exist at hash "66749159455b3f08c8318fe0233122f54d0f5889f9c5fdfb73c3fd9d99895b51" in container runtime
I0920 04:59:33.081651 467239 cri.go:218] Removing image: gcr.io/k8s-minikube/storage-provisioner:v5
I0920 04:59:33.081711 467239 ssh_runner.go:195] Run: which crictl
I0920 04:59:33.085907 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi gcr.io/k8s-minikube/storage-provisioner:v5
I0920 04:59:33.132822 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi gcr.io/k8s-minikube/storage-provisioner:v5
I0920 04:59:33.194293 467239 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi gcr.io/k8s-minikube/storage-provisioner:v5
I0920 04:59:33.252964 467239 cache_images.go:290] Loading image from: /home/jenkins/minikube-integration/21599-293644/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5
I0920 04:59:33.253107 467239 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/storage-provisioner_v5
I0920 04:59:33.261055 467239 ssh_runner.go:356] copy: skipping /var/lib/minikube/images/storage-provisioner_v5 (exists)
I0920 04:59:33.261089 467239 containerd.go:285] Loading image: /var/lib/minikube/images/storage-provisioner_v5
I0920 04:59:33.261142 467239 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/storage-provisioner_v5
I0920 04:59:33.526162 467239 cache_images.go:322] Transferred and loaded /home/jenkins/minikube-integration/21599-293644/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 from cache
I0920 04:59:33.526240 467239 cache_images.go:93] duration metric: took 1.946799789s to LoadCachedImages
W0920 04:59:33.526330 467239 out.go:285] X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/21599-293644/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.34.0: no such file or directory
X Unable to load cached images: LoadCachedImages: stat /home/jenkins/minikube-integration/21599-293644/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.34.0: no such file or directory
I0920 04:59:33.526347 467239 kubeadm.go:926] updating node { 192.168.76.2 8443 v1.34.0 containerd true true} ...
I0920 04:59:33.526572 467239 kubeadm.go:938] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.34.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=kubernetes-upgrade-068009 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.34.0 ClusterName:kubernetes-upgrade-068009 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0920 04:59:33.526662 467239 ssh_runner.go:195] Run: sudo crictl info
I0920 04:59:33.587435 467239 cni.go:84] Creating CNI manager for ""
I0920 04:59:33.587472 467239 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0920 04:59:33.587483 467239 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0920 04:59:33.587505 467239 kubeadm.go:189] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.34.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:kubernetes-upgrade-068009 NodeName:kubernetes-upgrade-068009 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/c
a.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0920 04:59:33.587650 467239 kubeadm.go:195] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "kubernetes-upgrade-068009"
kubeletExtraArgs:
- name: "node-ip"
value: "192.168.76.2"
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
- name: "enable-admission-plugins"
value: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
- name: "allocate-node-cidrs"
value: "true"
- name: "leader-elect"
value: "false"
scheduler:
extraArgs:
- name: "leader-elect"
value: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
kubernetesVersion: v1.34.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0920 04:59:33.587737 467239 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.34.0
I0920 04:59:33.601680 467239 binaries.go:44] Found k8s binaries, skipping transfer
I0920 04:59:33.601785 467239 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0920 04:59:33.611781 467239 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (329 bytes)
I0920 04:59:33.639060 467239 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0920 04:59:33.672531 467239 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2238 bytes)
I0920 04:59:33.697208 467239 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0920 04:59:33.702212 467239 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0920 04:59:33.839168 467239 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0920 04:59:33.854307 467239 certs.go:68] Setting up /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009 for IP: 192.168.76.2
I0920 04:59:33.854380 467239 certs.go:194] generating shared ca certs ...
I0920 04:59:33.854412 467239 certs.go:226] acquiring lock for ca certs: {Name:mk5aa50fa7f0dce0a90669167c944409dfea47c0 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 04:59:33.854608 467239 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/21599-293644/.minikube/ca.key
I0920 04:59:33.854701 467239 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/21599-293644/.minikube/proxy-client-ca.key
I0920 04:59:33.854749 467239 certs.go:256] generating profile certs ...
I0920 04:59:33.854875 467239 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/client.key
I0920 04:59:33.854962 467239 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/apiserver.key.8c17df0e
I0920 04:59:33.855037 467239 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/proxy-client.key
I0920 04:59:33.855207 467239 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/295509.pem (1338 bytes)
W0920 04:59:33.855273 467239 certs.go:480] ignoring /home/jenkins/minikube-integration/21599-293644/.minikube/certs/295509_empty.pem, impossibly tiny 0 bytes
I0920 04:59:33.855298 467239 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca-key.pem (1679 bytes)
I0920 04:59:33.855354 467239 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca.pem (1078 bytes)
I0920 04:59:33.855409 467239 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/cert.pem (1123 bytes)
I0920 04:59:33.855462 467239 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/key.pem (1675 bytes)
I0920 04:59:33.855544 467239 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/files/etc/ssl/certs/2955092.pem (1708 bytes)
I0920 04:59:33.856395 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0920 04:59:33.883132 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0920 04:59:33.907927 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0920 04:59:33.933238 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0920 04:59:33.958649 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1436 bytes)
I0920 04:59:33.983275 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0920 04:59:34.010058 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0920 04:59:34.038101 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0920 04:59:34.064301 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0920 04:59:34.092403 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/certs/295509.pem --> /usr/share/ca-certificates/295509.pem (1338 bytes)
I0920 04:59:34.120387 467239 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/files/etc/ssl/certs/2955092.pem --> /usr/share/ca-certificates/2955092.pem (1708 bytes)
I0920 04:59:34.148403 467239 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0920 04:59:34.168656 467239 ssh_runner.go:195] Run: openssl version
I0920 04:59:34.175479 467239 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0920 04:59:34.186332 467239 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0920 04:59:34.190363 467239 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 20 04:21 /usr/share/ca-certificates/minikubeCA.pem
I0920 04:59:34.190474 467239 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0920 04:59:34.198333 467239 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0920 04:59:34.208682 467239 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/295509.pem && ln -fs /usr/share/ca-certificates/295509.pem /etc/ssl/certs/295509.pem"
I0920 04:59:34.219501 467239 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/295509.pem
I0920 04:59:34.223878 467239 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 20 04:28 /usr/share/ca-certificates/295509.pem
I0920 04:59:34.223949 467239 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/295509.pem
I0920 04:59:34.231847 467239 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/295509.pem /etc/ssl/certs/51391683.0"
I0920 04:59:34.241965 467239 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/2955092.pem && ln -fs /usr/share/ca-certificates/2955092.pem /etc/ssl/certs/2955092.pem"
I0920 04:59:34.252479 467239 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/2955092.pem
I0920 04:59:34.256641 467239 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 20 04:28 /usr/share/ca-certificates/2955092.pem
I0920 04:59:34.256707 467239 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/2955092.pem
I0920 04:59:34.264349 467239 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/2955092.pem /etc/ssl/certs/3ec20f2e.0"
I0920 04:59:34.275379 467239 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0920 04:59:34.279600 467239 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0920 04:59:34.287083 467239 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0920 04:59:34.294737 467239 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0920 04:59:34.302419 467239 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0920 04:59:34.310204 467239 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0920 04:59:34.317867 467239 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0920 04:59:34.325614 467239 kubeadm.go:392] StartCluster: {Name:kubernetes-upgrade-068009 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.0 ClusterName:kubernetes-upgrade-068009 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames
:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.34.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketVMnetClientPath:
SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0920 04:59:34.325732 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0920 04:59:34.325839 467239 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0920 04:59:34.392237 467239 cri.go:89] found id: "e18ec467a23e7b723ec44836c3d5e20f04bd1dd8389b806c59c4cbc7002d7e60"
I0920 04:59:34.392275 467239 cri.go:89] found id: ""
I0920 04:59:34.392360 467239 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0920 04:59:34.420631 467239 cri.go:116] JSON = [{"ociVersion":"1.2.0","id":"0bf671a5b6a597e43eaef89daef5f2901f2f5980df95e920e9f628fb78840b66","pid":2320,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/0bf671a5b6a597e43eaef89daef5f2901f2f5980df95e920e9f628fb78840b66","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/0bf671a5b6a597e43eaef89daef5f2901f2f5980df95e920e9f628fb78840b66/rootfs","created":"2025-09-20T04:59:26.930557207Z","annotations":{"io.kubernetes.cri.container-type":"sandbox","io.kubernetes.cri.sandbox-cpu-period":"100000","io.kubernetes.cri.sandbox-cpu-quota":"0","io.kubernetes.cri.sandbox-cpu-shares":"256","io.kubernetes.cri.sandbox-id":"0bf671a5b6a597e43eaef89daef5f2901f2f5980df95e920e9f628fb78840b66","io.kubernetes.cri.sandbox-log-directory":"/var/log/pods/kube-system_kube-apiserver-kubernetes-upgrade-068009_687bffc11a702480173094555b606021","io.kubernetes.cri.sandbox-memory":"0","io.kubernetes.cri.sandbox-name":"kube-apiserver-kubernet
es-upgrade-068009","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"687bffc11a702480173094555b606021"},"owner":"root"},{"ociVersion":"1.2.0","id":"1557ff9f393189a14fd852a0ec90946d91fad7899f2f1c939dd99bc4dc1063c7","pid":1256,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/1557ff9f393189a14fd852a0ec90946d91fad7899f2f1c939dd99bc4dc1063c7","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/1557ff9f393189a14fd852a0ec90946d91fad7899f2f1c939dd99bc4dc1063c7/rootfs","created":"2025-09-20T04:59:04.989515845Z","annotations":{"io.kubernetes.cri.container-type":"sandbox","io.kubernetes.cri.sandbox-cpu-period":"100000","io.kubernetes.cri.sandbox-cpu-quota":"0","io.kubernetes.cri.sandbox-cpu-shares":"102","io.kubernetes.cri.sandbox-id":"1557ff9f393189a14fd852a0ec90946d91fad7899f2f1c939dd99bc4dc1063c7","io.kubernetes.cri.sandbox-log-directory":"/var/log/pods/kube-system_kube-scheduler-kubernetes-upgrade-068009_d2deb90f3760525a93376c0733b4e440",
"io.kubernetes.cri.sandbox-memory":"0","io.kubernetes.cri.sandbox-name":"kube-scheduler-kubernetes-upgrade-068009","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"d2deb90f3760525a93376c0733b4e440"},"owner":"root"},{"ociVersion":"1.2.0","id":"1d50bcf26608004c373426a62f75f79cd7834e12fa5e3eab1bc8eb1983899192","pid":1804,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/1d50bcf26608004c373426a62f75f79cd7834e12fa5e3eab1bc8eb1983899192","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/1d50bcf26608004c373426a62f75f79cd7834e12fa5e3eab1bc8eb1983899192/rootfs","created":"2025-09-20T04:59:24.368630333Z","annotations":{"io.kubernetes.cri.container-type":"sandbox","io.kubernetes.cri.sandbox-cpu-period":"100000","io.kubernetes.cri.sandbox-cpu-quota":"0","io.kubernetes.cri.sandbox-cpu-shares":"2","io.kubernetes.cri.sandbox-id":"1d50bcf26608004c373426a62f75f79cd7834e12fa5e3eab1bc8eb1983899192","io.kubernetes.cri.sandbox-log-directory":"/var/lo
g/pods/kube-system_kube-proxy-d9z5s_9c869634-da45-4dc6-b850-c5dd2aa596a9","io.kubernetes.cri.sandbox-memory":"0","io.kubernetes.cri.sandbox-name":"kube-proxy-d9z5s","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"9c869634-da45-4dc6-b850-c5dd2aa596a9"},"owner":"root"},{"ociVersion":"1.2.0","id":"2fde8768449fdf70f3393ad7e3dffb896ed6684d5695ae59992766196aa7cae0","pid":2307,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/2fde8768449fdf70f3393ad7e3dffb896ed6684d5695ae59992766196aa7cae0","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/2fde8768449fdf70f3393ad7e3dffb896ed6684d5695ae59992766196aa7cae0/rootfs","created":"2025-09-20T04:59:26.907718246Z","annotations":{"io.kubernetes.cri.container-type":"sandbox","io.kubernetes.cri.sandbox-cpu-period":"100000","io.kubernetes.cri.sandbox-cpu-quota":"0","io.kubernetes.cri.sandbox-cpu-shares":"204","io.kubernetes.cri.sandbox-id":"2fde8768449fdf70f3393ad7e3dffb896ed6684d5695ae59992766196aa7
cae0","io.kubernetes.cri.sandbox-log-directory":"/var/log/pods/kube-system_kube-controller-manager-kubernetes-upgrade-068009_68753ed2ff04b7e77be2a6bf67dea66e","io.kubernetes.cri.sandbox-memory":"0","io.kubernetes.cri.sandbox-name":"kube-controller-manager-kubernetes-upgrade-068009","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"68753ed2ff04b7e77be2a6bf67dea66e"},"owner":"root"},{"ociVersion":"1.2.0","id":"32a69748880b508fbc6dbf9d534f83a93857dfaed0d16ce92105661b888bf3ab","pid":1296,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/32a69748880b508fbc6dbf9d534f83a93857dfaed0d16ce92105661b888bf3ab","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/32a69748880b508fbc6dbf9d534f83a93857dfaed0d16ce92105661b888bf3ab/rootfs","created":"2025-09-20T04:59:05.135950489Z","annotations":{"io.kubernetes.cri.container-type":"sandbox","io.kubernetes.cri.sandbox-cpu-period":"100000","io.kubernetes.cri.sandbox-cpu-quota":"0","io.kubernetes.cri.sand
box-cpu-shares":"256","io.kubernetes.cri.sandbox-id":"32a69748880b508fbc6dbf9d534f83a93857dfaed0d16ce92105661b888bf3ab","io.kubernetes.cri.sandbox-log-directory":"/var/log/pods/kube-system_kube-apiserver-kubernetes-upgrade-068009_687bffc11a702480173094555b606021","io.kubernetes.cri.sandbox-memory":"0","io.kubernetes.cri.sandbox-name":"kube-apiserver-kubernetes-upgrade-068009","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"687bffc11a702480173094555b606021"},"owner":"root"},{"ociVersion":"1.2.0","id":"4aa21cf21ac92fc854f4e36c78e7d8984960f8e77b5b0a1ecf15131c103d3eab","pid":1438,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/4aa21cf21ac92fc854f4e36c78e7d8984960f8e77b5b0a1ecf15131c103d3eab","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/4aa21cf21ac92fc854f4e36c78e7d8984960f8e77b5b0a1ecf15131c103d3eab/rootfs","created":"2025-09-20T04:59:07.526065723Z","annotations":{"io.kubernetes.cri.container-name":"kube-controller-manager","
io.kubernetes.cri.container-type":"container","io.kubernetes.cri.image-name":"registry.k8s.io/kube-controller-manager:v1.34.0","io.kubernetes.cri.sandbox-id":"f42462b96763eb110fc7908ee5556d8472cb6052c9f76ec745ed0d185f286113","io.kubernetes.cri.sandbox-name":"kube-controller-manager-kubernetes-upgrade-068009","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"68753ed2ff04b7e77be2a6bf67dea66e"},"owner":"root"},{"ociVersion":"1.2.0","id":"61f36518c995233264ce66c232db3a0ffb85a1751d3ade5bf9dea17481d1c1cd","pid":2060,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/61f36518c995233264ce66c232db3a0ffb85a1751d3ade5bf9dea17481d1c1cd","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/61f36518c995233264ce66c232db3a0ffb85a1751d3ade5bf9dea17481d1c1cd/rootfs","created":"2025-09-20T04:59:25.298339512Z","annotations":{"io.kubernetes.cri.container-name":"storage-provisioner","io.kubernetes.cri.container-type":"container","io.kubernetes.cri.image-na
me":"gcr.io/k8s-minikube/storage-provisioner:v5","io.kubernetes.cri.sandbox-id":"74b8580d6ac90233b68a0658e0b54235299a97f73971e9ad9bdbbc523dcc1e1d","io.kubernetes.cri.sandbox-name":"storage-provisioner","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"3a3b848b-742f-4b34-b59a-6311025f0e7e"},"owner":"root"},{"ociVersion":"1.2.0","id":"74b8580d6ac90233b68a0658e0b54235299a97f73971e9ad9bdbbc523dcc1e1d","pid":1962,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/74b8580d6ac90233b68a0658e0b54235299a97f73971e9ad9bdbbc523dcc1e1d","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/74b8580d6ac90233b68a0658e0b54235299a97f73971e9ad9bdbbc523dcc1e1d/rootfs","created":"2025-09-20T04:59:25.007966854Z","annotations":{"io.kubernetes.cri.container-type":"sandbox","io.kubernetes.cri.sandbox-cpu-period":"100000","io.kubernetes.cri.sandbox-cpu-quota":"0","io.kubernetes.cri.sandbox-cpu-shares":"2","io.kubernetes.cri.sandbox-id":"74b8580d6ac90233b68a0658e
0b54235299a97f73971e9ad9bdbbc523dcc1e1d","io.kubernetes.cri.sandbox-log-directory":"/var/log/pods/kube-system_storage-provisioner_3a3b848b-742f-4b34-b59a-6311025f0e7e","io.kubernetes.cri.sandbox-memory":"0","io.kubernetes.cri.sandbox-name":"storage-provisioner","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"3a3b848b-742f-4b34-b59a-6311025f0e7e"},"owner":"root"},{"ociVersion":"1.2.0","id":"9557866e341938ba6b6198b0d27fce3d0ca1acb1a35828d5593a0620f295d60d","pid":1855,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/9557866e341938ba6b6198b0d27fce3d0ca1acb1a35828d5593a0620f295d60d","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/9557866e341938ba6b6198b0d27fce3d0ca1acb1a35828d5593a0620f295d60d/rootfs","created":"2025-09-20T04:59:24.605493361Z","annotations":{"io.kubernetes.cri.container-type":"sandbox","io.kubernetes.cri.sandbox-cpu-period":"100000","io.kubernetes.cri.sandbox-cpu-quota":"10000","io.kubernetes.cri.sandbox-cpu-share
s":"102","io.kubernetes.cri.sandbox-id":"9557866e341938ba6b6198b0d27fce3d0ca1acb1a35828d5593a0620f295d60d","io.kubernetes.cri.sandbox-log-directory":"/var/log/pods/kube-system_kindnet-q85xj_a09046bb-167b-4ccb-8e31-5cd2809c4ac0","io.kubernetes.cri.sandbox-memory":"52428800","io.kubernetes.cri.sandbox-name":"kindnet-q85xj","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"a09046bb-167b-4ccb-8e31-5cd2809c4ac0"},"owner":"root"},{"ociVersion":"1.2.0","id":"db1604f5029a43668878e0b89fde4b0cd56963c67445f30f5d5564ef34521ff2","pid":1487,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/db1604f5029a43668878e0b89fde4b0cd56963c67445f30f5d5564ef34521ff2","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/db1604f5029a43668878e0b89fde4b0cd56963c67445f30f5d5564ef34521ff2/rootfs","created":"2025-09-20T04:59:09.546161067Z","annotations":{"io.kubernetes.cri.container-name":"kube-scheduler","io.kubernetes.cri.container-type":"container","io.kubernetes.
cri.image-name":"registry.k8s.io/kube-scheduler:v1.34.0","io.kubernetes.cri.sandbox-id":"1557ff9f393189a14fd852a0ec90946d91fad7899f2f1c939dd99bc4dc1063c7","io.kubernetes.cri.sandbox-name":"kube-scheduler-kubernetes-upgrade-068009","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"d2deb90f3760525a93376c0733b4e440"},"owner":"root"},{"ociVersion":"1.2.0","id":"f42462b96763eb110fc7908ee5556d8472cb6052c9f76ec745ed0d185f286113","pid":1280,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/f42462b96763eb110fc7908ee5556d8472cb6052c9f76ec745ed0d185f286113","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/f42462b96763eb110fc7908ee5556d8472cb6052c9f76ec745ed0d185f286113/rootfs","created":"2025-09-20T04:59:05.088081013Z","annotations":{"io.kubernetes.cri.container-type":"sandbox","io.kubernetes.cri.sandbox-cpu-period":"100000","io.kubernetes.cri.sandbox-cpu-quota":"0","io.kubernetes.cri.sandbox-cpu-shares":"204","io.kubernetes.cri.sandbox-id"
:"f42462b96763eb110fc7908ee5556d8472cb6052c9f76ec745ed0d185f286113","io.kubernetes.cri.sandbox-log-directory":"/var/log/pods/kube-system_kube-controller-manager-kubernetes-upgrade-068009_68753ed2ff04b7e77be2a6bf67dea66e","io.kubernetes.cri.sandbox-memory":"0","io.kubernetes.cri.sandbox-name":"kube-controller-manager-kubernetes-upgrade-068009","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"68753ed2ff04b7e77be2a6bf67dea66e"},"owner":"root"},{"ociVersion":"1.2.0","id":"f5684e8f8ad0a074cfe424ef18be9f990a8623756d05e947e13c520bf1560644","pid":1538,"status":"running","bundle":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/f5684e8f8ad0a074cfe424ef18be9f990a8623756d05e947e13c520bf1560644","rootfs":"/run/containerd/io.containerd.runtime.v2.task/k8s.io/f5684e8f8ad0a074cfe424ef18be9f990a8623756d05e947e13c520bf1560644/rootfs","created":"2025-09-20T04:59:12.051136193Z","annotations":{"io.kubernetes.cri.container-name":"kube-apiserver","io.kubernetes.cri.container-type":"containe
r","io.kubernetes.cri.image-name":"registry.k8s.io/kube-apiserver:v1.34.0","io.kubernetes.cri.sandbox-id":"32a69748880b508fbc6dbf9d534f83a93857dfaed0d16ce92105661b888bf3ab","io.kubernetes.cri.sandbox-name":"kube-apiserver-kubernetes-upgrade-068009","io.kubernetes.cri.sandbox-namespace":"kube-system","io.kubernetes.cri.sandbox-uid":"687bffc11a702480173094555b606021"},"owner":"root"}]
I0920 04:59:34.420970 467239 cri.go:126] list returned 12 containers
I0920 04:59:34.420987 467239 cri.go:129] container: {ID:0bf671a5b6a597e43eaef89daef5f2901f2f5980df95e920e9f628fb78840b66 Status:running}
I0920 04:59:34.421021 467239 cri.go:131] skipping 0bf671a5b6a597e43eaef89daef5f2901f2f5980df95e920e9f628fb78840b66 - not in ps
I0920 04:59:34.421027 467239 cri.go:129] container: {ID:1557ff9f393189a14fd852a0ec90946d91fad7899f2f1c939dd99bc4dc1063c7 Status:running}
I0920 04:59:34.421050 467239 cri.go:131] skipping 1557ff9f393189a14fd852a0ec90946d91fad7899f2f1c939dd99bc4dc1063c7 - not in ps
I0920 04:59:34.421061 467239 cri.go:129] container: {ID:1d50bcf26608004c373426a62f75f79cd7834e12fa5e3eab1bc8eb1983899192 Status:running}
I0920 04:59:34.421067 467239 cri.go:131] skipping 1d50bcf26608004c373426a62f75f79cd7834e12fa5e3eab1bc8eb1983899192 - not in ps
I0920 04:59:34.421073 467239 cri.go:129] container: {ID:2fde8768449fdf70f3393ad7e3dffb896ed6684d5695ae59992766196aa7cae0 Status:running}
I0920 04:59:34.421082 467239 cri.go:131] skipping 2fde8768449fdf70f3393ad7e3dffb896ed6684d5695ae59992766196aa7cae0 - not in ps
I0920 04:59:34.421087 467239 cri.go:129] container: {ID:32a69748880b508fbc6dbf9d534f83a93857dfaed0d16ce92105661b888bf3ab Status:running}
I0920 04:59:34.421094 467239 cri.go:131] skipping 32a69748880b508fbc6dbf9d534f83a93857dfaed0d16ce92105661b888bf3ab - not in ps
I0920 04:59:34.421104 467239 cri.go:129] container: {ID:4aa21cf21ac92fc854f4e36c78e7d8984960f8e77b5b0a1ecf15131c103d3eab Status:running}
I0920 04:59:34.421125 467239 cri.go:131] skipping 4aa21cf21ac92fc854f4e36c78e7d8984960f8e77b5b0a1ecf15131c103d3eab - not in ps
I0920 04:59:34.421136 467239 cri.go:129] container: {ID:61f36518c995233264ce66c232db3a0ffb85a1751d3ade5bf9dea17481d1c1cd Status:running}
I0920 04:59:34.421143 467239 cri.go:131] skipping 61f36518c995233264ce66c232db3a0ffb85a1751d3ade5bf9dea17481d1c1cd - not in ps
I0920 04:59:34.421162 467239 cri.go:129] container: {ID:74b8580d6ac90233b68a0658e0b54235299a97f73971e9ad9bdbbc523dcc1e1d Status:running}
I0920 04:59:34.421169 467239 cri.go:131] skipping 74b8580d6ac90233b68a0658e0b54235299a97f73971e9ad9bdbbc523dcc1e1d - not in ps
I0920 04:59:34.421180 467239 cri.go:129] container: {ID:9557866e341938ba6b6198b0d27fce3d0ca1acb1a35828d5593a0620f295d60d Status:running}
I0920 04:59:34.421186 467239 cri.go:131] skipping 9557866e341938ba6b6198b0d27fce3d0ca1acb1a35828d5593a0620f295d60d - not in ps
I0920 04:59:34.421203 467239 cri.go:129] container: {ID:db1604f5029a43668878e0b89fde4b0cd56963c67445f30f5d5564ef34521ff2 Status:running}
I0920 04:59:34.421216 467239 cri.go:131] skipping db1604f5029a43668878e0b89fde4b0cd56963c67445f30f5d5564ef34521ff2 - not in ps
I0920 04:59:34.421220 467239 cri.go:129] container: {ID:f42462b96763eb110fc7908ee5556d8472cb6052c9f76ec745ed0d185f286113 Status:running}
I0920 04:59:34.421234 467239 cri.go:131] skipping f42462b96763eb110fc7908ee5556d8472cb6052c9f76ec745ed0d185f286113 - not in ps
I0920 04:59:34.421244 467239 cri.go:129] container: {ID:f5684e8f8ad0a074cfe424ef18be9f990a8623756d05e947e13c520bf1560644 Status:running}
I0920 04:59:34.421251 467239 cri.go:131] skipping f5684e8f8ad0a074cfe424ef18be9f990a8623756d05e947e13c520bf1560644 - not in ps
I0920 04:59:34.421318 467239 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0920 04:59:34.431999 467239 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0920 04:59:34.432058 467239 kubeadm.go:589] restartPrimaryControlPlane start ...
I0920 04:59:34.432135 467239 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0920 04:59:34.442210 467239 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0920 04:59:34.442832 467239 kubeconfig.go:125] found "kubernetes-upgrade-068009" server: "https://192.168.76.2:8443"
I0920 04:59:34.443524 467239 kapi.go:59] client config for kubernetes-upgrade-068009: &rest.Config{Host:"https://192.168.76.2:8443", APIPath:"", ContentConfig:rest.ContentConfig{AcceptContentTypes:"", ContentType:"", GroupVersion:(*schema.GroupVersion)(nil), NegotiatedSerializer:runtime.NegotiatedSerializer(nil)}, Username:"", Password:"", BearerToken:"", BearerTokenFile:"", Impersonate:rest.ImpersonationConfig{UserName:"", UID:"", Groups:[]string(nil), Extra:map[string][]string(nil)}, AuthProvider:<nil>, AuthConfigPersister:rest.AuthProviderConfigPersister(nil), ExecProvider:<nil>, TLSClientConfig:rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"/home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/client.crt", KeyFile:"/home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/client.key", CAFile:"/home/jenkins/minikube-integration/21599-293644/.minikube/ca.crt", CertData:[]uint8(nil), KeyData:[]uint8(ni
l), CAData:[]uint8(nil), NextProtos:[]string(nil)}, UserAgent:"", DisableCompression:false, Transport:http.RoundTripper(nil), WrapTransport:(transport.WrapperFunc)(0x20f8410), QPS:0, Burst:0, RateLimiter:flowcontrol.RateLimiter(nil), WarningHandler:rest.WarningHandler(nil), WarningHandlerWithContext:rest.WarningHandlerWithContext(nil), Timeout:0, Dial:(func(context.Context, string, string) (net.Conn, error))(nil), Proxy:(func(*http.Request) (*url.URL, error))(nil)}
I0920 04:59:34.444165 467239 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false
I0920 04:59:34.444189 467239 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true
I0920 04:59:34.444391 467239 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false
I0920 04:59:34.444406 467239 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false
I0920 04:59:34.444413 467239 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false
I0920 04:59:34.444785 467239 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0920 04:59:34.456252 467239 kubeadm.go:626] The running cluster does not require reconfiguration: 192.168.76.2
I0920 04:59:34.456316 467239 kubeadm.go:593] duration metric: took 24.228857ms to restartPrimaryControlPlane
I0920 04:59:34.456332 467239 kubeadm.go:394] duration metric: took 130.73025ms to StartCluster
I0920 04:59:34.456350 467239 settings.go:142] acquiring lock: {Name:mk0431f8e6fad2f77306cccf2f2d56ababc5b1b1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 04:59:34.456457 467239 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/21599-293644/kubeconfig
I0920 04:59:34.457178 467239 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21599-293644/kubeconfig: {Name:mka409452b1806eb89337f306620696a06e9e870 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 04:59:34.457705 467239 config.go:182] Loaded profile config "kubernetes-upgrade-068009": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.34.0
I0920 04:59:34.457518 467239 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.34.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0920 04:59:34.457850 467239 addons.go:511] enable addons start: toEnable=map[ambassador:false amd-gpu-device-plugin:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubetail:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0920 04:59:34.458122 467239 addons.go:69] Setting storage-provisioner=true in profile "kubernetes-upgrade-068009"
I0920 04:59:34.458390 467239 addons.go:238] Setting addon storage-provisioner=true in "kubernetes-upgrade-068009"
W0920 04:59:34.458510 467239 addons.go:247] addon storage-provisioner should already be in state true
I0920 04:59:34.458554 467239 host.go:66] Checking if "kubernetes-upgrade-068009" exists ...
I0920 04:59:34.459137 467239 cli_runner.go:164] Run: docker container inspect kubernetes-upgrade-068009 --format={{.State.Status}}
I0920 04:59:34.459336 467239 addons.go:69] Setting default-storageclass=true in profile "kubernetes-upgrade-068009"
I0920 04:59:34.459375 467239 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "kubernetes-upgrade-068009"
I0920 04:59:34.459676 467239 cli_runner.go:164] Run: docker container inspect kubernetes-upgrade-068009 --format={{.State.Status}}
I0920 04:59:34.466392 467239 out.go:179] * Verifying Kubernetes components...
I0920 04:59:34.477998 467239 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0920 04:59:34.494010 467239 kapi.go:59] client config for kubernetes-upgrade-068009: &rest.Config{Host:"https://192.168.76.2:8443", APIPath:"", ContentConfig:rest.ContentConfig{AcceptContentTypes:"", ContentType:"", GroupVersion:(*schema.GroupVersion)(nil), NegotiatedSerializer:runtime.NegotiatedSerializer(nil)}, Username:"", Password:"", BearerToken:"", BearerTokenFile:"", Impersonate:rest.ImpersonationConfig{UserName:"", UID:"", Groups:[]string(nil), Extra:map[string][]string(nil)}, AuthProvider:<nil>, AuthConfigPersister:rest.AuthProviderConfigPersister(nil), ExecProvider:<nil>, TLSClientConfig:rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"/home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/client.crt", KeyFile:"/home/jenkins/minikube-integration/21599-293644/.minikube/profiles/kubernetes-upgrade-068009/client.key", CAFile:"/home/jenkins/minikube-integration/21599-293644/.minikube/ca.crt", CertData:[]uint8(nil), KeyData:[]uint8(ni
l), CAData:[]uint8(nil), NextProtos:[]string(nil)}, UserAgent:"", DisableCompression:false, Transport:http.RoundTripper(nil), WrapTransport:(transport.WrapperFunc)(0x20f8410), QPS:0, Burst:0, RateLimiter:flowcontrol.RateLimiter(nil), WarningHandler:rest.WarningHandler(nil), WarningHandlerWithContext:rest.WarningHandlerWithContext(nil), Timeout:0, Dial:(func(context.Context, string, string) (net.Conn, error))(nil), Proxy:(func(*http.Request) (*url.URL, error))(nil)}
I0920 04:59:34.494312 467239 addons.go:238] Setting addon default-storageclass=true in "kubernetes-upgrade-068009"
W0920 04:59:34.494324 467239 addons.go:247] addon default-storageclass should already be in state true
I0920 04:59:34.494349 467239 host.go:66] Checking if "kubernetes-upgrade-068009" exists ...
I0920 04:59:34.494761 467239 cli_runner.go:164] Run: docker container inspect kubernetes-upgrade-068009 --format={{.State.Status}}
I0920 04:59:34.500333 467239 out.go:179] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0920 04:59:34.503104 467239 addons.go:435] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0920 04:59:34.503130 467239 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0920 04:59:34.503190 467239 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-068009
I0920 04:59:34.518489 467239 addons.go:435] installing /etc/kubernetes/addons/storageclass.yaml
I0920 04:59:34.518517 467239 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0920 04:59:34.518588 467239 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-068009
I0920 04:59:34.565403 467239 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33378 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/kubernetes-upgrade-068009/id_rsa Username:docker}
I0920 04:59:34.566980 467239 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33378 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/kubernetes-upgrade-068009/id_rsa Username:docker}
I0920 04:59:34.698948 467239 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0920 04:59:34.714277 467239 api_server.go:52] waiting for apiserver process to appear ...
I0920 04:59:34.714403 467239 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0920 04:59:34.728174 467239 api_server.go:72] duration metric: took 270.347749ms to wait for apiserver process to appear ...
I0920 04:59:34.728209 467239 api_server.go:88] waiting for apiserver healthz status ...
I0920 04:59:34.728271 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 04:59:34.758377 467239 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0920 04:59:34.763647 467239 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0920 04:59:36.750681 467239 api_server.go:279] https://192.168.76.2:8443/healthz returned 500:
[+]ping ok
[+]log ok
[-]etcd failed: reason withheld
[+]poststarthook/start-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/storage-object-count-tracker-hook ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/start-system-namespaces-controller ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok
[+]poststarthook/start-legacy-token-tracking-controller ok
[+]poststarthook/start-service-ip-repair-controllers ok
[+]poststarthook/rbac/bootstrap-roles ok
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/bootstrap-controller ok
[+]poststarthook/start-kubernetes-service-cidr-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-status-local-available-controller ok
[+]poststarthook/apiservice-status-remote-available-controller ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-discovery-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
W0920 04:59:36.750721 467239 api_server.go:103] status: https://192.168.76.2:8443/healthz returned error 500:
[+]ping ok
[+]log ok
[-]etcd failed: reason withheld
[+]poststarthook/start-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/storage-object-count-tracker-hook ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/start-system-namespaces-controller ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok
[+]poststarthook/start-legacy-token-tracking-controller ok
[+]poststarthook/start-service-ip-repair-controllers ok
[+]poststarthook/rbac/bootstrap-roles ok
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/bootstrap-controller ok
[+]poststarthook/start-kubernetes-service-cidr-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-status-local-available-controller ok
[+]poststarthook/apiservice-status-remote-available-controller ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-discovery-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
I0920 04:59:36.750763 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 04:59:38.759861 467239 api_server.go:279] https://192.168.76.2:8443/healthz returned 500:
[+]ping ok
[+]log ok
[-]etcd failed: reason withheld
[+]poststarthook/start-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/storage-object-count-tracker-hook ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/start-system-namespaces-controller ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok
[+]poststarthook/start-legacy-token-tracking-controller ok
[+]poststarthook/start-service-ip-repair-controllers ok
[+]poststarthook/rbac/bootstrap-roles ok
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/bootstrap-controller ok
[+]poststarthook/start-kubernetes-service-cidr-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-status-local-available-controller ok
[+]poststarthook/apiservice-status-remote-available-controller ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-discovery-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
W0920 04:59:38.759901 467239 api_server.go:103] status: https://192.168.76.2:8443/healthz returned error 500:
[+]ping ok
[+]log ok
[-]etcd failed: reason withheld
[+]poststarthook/start-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/storage-object-count-tracker-hook ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/start-system-namespaces-controller ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok
[+]poststarthook/start-legacy-token-tracking-controller ok
[+]poststarthook/start-service-ip-repair-controllers ok
[+]poststarthook/rbac/bootstrap-roles ok
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/bootstrap-controller ok
[+]poststarthook/start-kubernetes-service-cidr-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-status-local-available-controller ok
[+]poststarthook/apiservice-status-remote-available-controller ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-discovery-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
I0920 04:59:38.759916 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 04:59:40.769181 467239 api_server.go:279] https://192.168.76.2:8443/healthz returned 500:
[+]ping ok
[+]log ok
[-]etcd failed: reason withheld
[+]poststarthook/start-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/storage-object-count-tracker-hook ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/start-system-namespaces-controller ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok
[+]poststarthook/start-legacy-token-tracking-controller ok
[+]poststarthook/start-service-ip-repair-controllers ok
[+]poststarthook/rbac/bootstrap-roles ok
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/bootstrap-controller ok
[+]poststarthook/start-kubernetes-service-cidr-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-status-local-available-controller ok
[+]poststarthook/apiservice-status-remote-available-controller ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-discovery-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
W0920 04:59:40.769218 467239 api_server.go:103] status: https://192.168.76.2:8443/healthz returned error 500:
[+]ping ok
[+]log ok
[-]etcd failed: reason withheld
[+]poststarthook/start-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/storage-object-count-tracker-hook ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/start-system-namespaces-controller ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok
[+]poststarthook/start-legacy-token-tracking-controller ok
[+]poststarthook/start-service-ip-repair-controllers ok
[+]poststarthook/rbac/bootstrap-roles ok
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/bootstrap-controller ok
[+]poststarthook/start-kubernetes-service-cidr-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-status-local-available-controller ok
[+]poststarthook/apiservice-status-remote-available-controller ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-discovery-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
healthz check failed
I0920 04:59:40.769235 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 04:59:45.769581 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 04:59:45.769625 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 04:59:50.773631 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 04:59:50.773672 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 04:59:55.774301 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 04:59:55.774338 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 05:00:00.777825 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 05:00:00.777871 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 05:00:05.778347 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 05:00:05.778403 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 05:00:10.781548 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 05:00:10.781583 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 05:00:15.782246 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 05:00:15.782280 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 05:00:20.783162 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 05:00:20.783195 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 05:00:25.784109 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 05:00:25.784141 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 05:00:30.787192 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 05:00:30.787228 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 05:00:35.291814 467239 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: (1m0.533390045s)
W0920 05:00:35.291847 467239 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "storage.k8s.io/v1, Resource=storageclasses", GroupVersionKind: "storage.k8s.io/v1, Kind=StorageClass"
Name: "standard", Namespace: ""
from server for: "/etc/kubernetes/addons/storageclass.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get storageclasses.storage.k8s.io standard)
I0920 05:00:35.291870 467239 retry.go:31] will retry after 225.462782ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "storage.k8s.io/v1, Resource=storageclasses", GroupVersionKind: "storage.k8s.io/v1, Kind=StorageClass"
Name: "standard", Namespace: ""
from server for: "/etc/kubernetes/addons/storageclass.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get storageclasses.storage.k8s.io standard)
I0920 05:00:35.518299 467239 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0920 05:00:35.787761 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 05:00:35.787819 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0920 05:00:35.787884 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0920 05:05:35.540876 467239 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (6m0.777186086s)
W0920 05:05:35.540920 467239 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "storage-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get serviceaccounts storage-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding"
Name: "storage-provisioner", Namespace: ""
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get clusterrolebindings.rbac.authorization.k8s.io storage-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=roles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=Role"
Name: "system:persistent-volume-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get roles.rbac.authorization.k8s.io system:persistent-volume-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=rolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=RoleBinding"
Name: "system:persistent-volume-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get rolebindings.rbac.authorization.k8s.io system:persistent-volume-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=endpoints", GroupVersionKind: "/v1, Kind=Endpoints"
Name: "k8s.io-minikube-hostpath", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=pods", GroupVersionKind: "/v1, Kind=Pod"
Name: "storage-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get pods storage-provisioner)
I0920 05:05:35.540926 467239 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (5m0.022575299s)
W0920 05:05:35.540953 467239 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "storage.k8s.io/v1, Resource=storageclasses", GroupVersionKind: "storage.k8s.io/v1, Kind=StorageClass"
Name: "standard", Namespace: ""
from server for: "/etc/kubernetes/addons/storageclass.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get storageclasses.storage.k8s.io standard)
W0920 05:05:35.541015 467239 out.go:285] ! Enabling 'default-storageclass' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "storage.k8s.io/v1, Resource=storageclasses", GroupVersionKind: "storage.k8s.io/v1, Kind=StorageClass"
Name: "standard", Namespace: ""
from server for: "/etc/kubernetes/addons/storageclass.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get storageclasses.storage.k8s.io standard)
]
! Enabling 'default-storageclass' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "storage.k8s.io/v1, Resource=storageclasses", GroupVersionKind: "storage.k8s.io/v1, Kind=StorageClass"
Name: "standard", Namespace: ""
from server for: "/etc/kubernetes/addons/storageclass.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get storageclasses.storage.k8s.io standard)
]
W0920 05:05:35.541015 467239 out.go:285] ! Enabling 'storage-provisioner' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "storage-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get serviceaccounts storage-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding"
Name: "storage-provisioner", Namespace: ""
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get clusterrolebindings.rbac.authorization.k8s.io storage-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=roles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=Role"
Name: "system:persistent-volume-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get roles.rbac.authorization.k8s.io system:persistent-volume-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=rolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=RoleBinding"
Name: "system:persistent-volume-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get rolebindings.rbac.authorization.k8s.io system:persistent-volume-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=endpoints", GroupVersionKind: "/v1, Kind=Endpoints"
Name: "k8s.io-minikube-hostpath", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=pods", GroupVersionKind: "/v1, Kind=Pod"
Name: "storage-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get pods storage-provisioner)
]
! Enabling 'storage-provisioner' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "storage-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get serviceaccounts storage-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding"
Name: "storage-provisioner", Namespace: ""
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get clusterrolebindings.rbac.authorization.k8s.io storage-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=roles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=Role"
Name: "system:persistent-volume-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get roles.rbac.authorization.k8s.io system:persistent-volume-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=rolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=RoleBinding"
Name: "system:persistent-volume-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get rolebindings.rbac.authorization.k8s.io system:persistent-volume-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=endpoints", GroupVersionKind: "/v1, Kind=Endpoints"
Name: "k8s.io-minikube-hostpath", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=pods", GroupVersionKind: "/v1, Kind=Pod"
Name: "storage-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get pods storage-provisioner)
]
I0920 05:05:35.541413 467239 ssh_runner.go:235] Completed: sudo crictl ps -a --quiet --name=kube-apiserver: (4m59.753506736s)
I0920 05:05:35.541438 467239 cri.go:89] found id: "7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921"
I0920 05:05:35.541445 467239 cri.go:89] found id: ""
I0920 05:05:35.541452 467239 logs.go:282] 1 containers: [7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921]
I0920 05:05:35.541522 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.545232 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0920 05:05:35.545311 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0920 05:05:35.546030 467239 out.go:179] * Enabled addons:
I0920 05:05:35.549039 467239 addons.go:514] duration metric: took 6m1.091166443s for enable addons: enabled=[]
I0920 05:05:35.581548 467239 cri.go:89] found id: "af627e9f5bff0f7a5c022832e92c94112890fbf9d88d0f3c977d84b13ddfa93e"
I0920 05:05:35.581570 467239 cri.go:89] found id: ""
I0920 05:05:35.581578 467239 logs.go:282] 1 containers: [af627e9f5bff0f7a5c022832e92c94112890fbf9d88d0f3c977d84b13ddfa93e]
I0920 05:05:35.581635 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.585239 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0920 05:05:35.585317 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0920 05:05:35.625494 467239 cri.go:89] found id: "c431baba84177c8f678eee3d7d1a75e48913abadcc4afd030f92b029ee9ca136"
I0920 05:05:35.625517 467239 cri.go:89] found id: ""
I0920 05:05:35.625526 467239 logs.go:282] 1 containers: [c431baba84177c8f678eee3d7d1a75e48913abadcc4afd030f92b029ee9ca136]
I0920 05:05:35.625582 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.629130 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0920 05:05:35.629207 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0920 05:05:35.666922 467239 cri.go:89] found id: "9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714"
I0920 05:05:35.666944 467239 cri.go:89] found id: ""
I0920 05:05:35.666952 467239 logs.go:282] 1 containers: [9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714]
I0920 05:05:35.667008 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.670518 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0920 05:05:35.670599 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0920 05:05:35.714743 467239 cri.go:89] found id: "fdbb732bced3687cb23143983d2939993e962365e155bd9edf617feb4b44cd8b"
I0920 05:05:35.714822 467239 cri.go:89] found id: ""
I0920 05:05:35.714862 467239 logs.go:282] 1 containers: [fdbb732bced3687cb23143983d2939993e962365e155bd9edf617feb4b44cd8b]
I0920 05:05:35.714941 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.718912 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0920 05:05:35.718987 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0920 05:05:35.757811 467239 cri.go:89] found id: "0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726"
I0920 05:05:35.757889 467239 cri.go:89] found id: ""
I0920 05:05:35.757912 467239 logs.go:282] 1 containers: [0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726]
I0920 05:05:35.757996 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.761687 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0920 05:05:35.761766 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0920 05:05:35.802323 467239 cri.go:89] found id: "331ce690dc1105bc226ee21c46805afe1dcfb4affa37785dca1bb35bc43e178f"
I0920 05:05:35.802388 467239 cri.go:89] found id: ""
I0920 05:05:35.802411 467239 logs.go:282] 1 containers: [331ce690dc1105bc226ee21c46805afe1dcfb4affa37785dca1bb35bc43e178f]
I0920 05:05:35.802504 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.806260 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0920 05:05:35.806357 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0920 05:05:35.846466 467239 cri.go:89] found id: "da1c4fe4a0b3a07a3ea6e372ba61c81eaff9776ad30d0b28e04686d21757f663"
I0920 05:05:35.846493 467239 cri.go:89] found id: ""
I0920 05:05:35.846502 467239 logs.go:282] 1 containers: [da1c4fe4a0b3a07a3ea6e372ba61c81eaff9776ad30d0b28e04686d21757f663]
I0920 05:05:35.846592 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.850828 467239 logs.go:123] Gathering logs for kube-apiserver [7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921] ...
I0920 05:05:35.850857 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921"
W0920 05:05:35.896131 467239 logs.go:130] failed kube-apiserver [7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921]: command: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921" /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921": Process exited with status 1
stdout:
stderr:
E0920 05:05:35.893210 4823 remote_runtime.go:625] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921\": not found" containerID="7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921"
time="2025-09-20T05:05:35Z" level=fatal msg="rpc error: code = NotFound desc = an error occurred when try to find container \"7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921\": not found"
output:
** stderr **
E0920 05:05:35.893210 4823 remote_runtime.go:625] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921\": not found" containerID="7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921"
time="2025-09-20T05:05:35Z" level=fatal msg="rpc error: code = NotFound desc = an error occurred when try to find container \"7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921\": not found"
** /stderr **
I0920 05:05:35.896172 467239 logs.go:123] Gathering logs for etcd [af627e9f5bff0f7a5c022832e92c94112890fbf9d88d0f3c977d84b13ddfa93e] ...
I0920 05:05:35.896202 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 af627e9f5bff0f7a5c022832e92c94112890fbf9d88d0f3c977d84b13ddfa93e"
I0920 05:05:35.961120 467239 logs.go:123] Gathering logs for kube-proxy [fdbb732bced3687cb23143983d2939993e962365e155bd9edf617feb4b44cd8b] ...
I0920 05:05:35.961159 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fdbb732bced3687cb23143983d2939993e962365e155bd9edf617feb4b44cd8b"
I0920 05:05:36.015874 467239 logs.go:123] Gathering logs for containerd ...
I0920 05:05:36.015906 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0920 05:05:36.092346 467239 logs.go:123] Gathering logs for dmesg ...
I0920 05:05:36.092382 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0920 05:05:36.115263 467239 logs.go:123] Gathering logs for coredns [c431baba84177c8f678eee3d7d1a75e48913abadcc4afd030f92b029ee9ca136] ...
I0920 05:05:36.115296 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c431baba84177c8f678eee3d7d1a75e48913abadcc4afd030f92b029ee9ca136"
I0920 05:05:36.155323 467239 logs.go:123] Gathering logs for kube-scheduler [9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714] ...
I0920 05:05:36.155352 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714"
W0920 05:05:36.194013 467239 logs.go:138] Found kube-scheduler [9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714] problem: E0920 05:02:57.234215 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10259: listen tcp 127.0.0.1:10259: bind: address already in use"
I0920 05:05:36.194042 467239 logs.go:123] Gathering logs for kube-controller-manager [0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726] ...
I0920 05:05:36.194054 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726"
W0920 05:05:36.235628 467239 logs.go:138] Found kube-controller-manager [0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726] problem: E0920 05:03:00.247948 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10257: listen tcp 127.0.0.1:10257: bind: address already in use"
I0920 05:05:36.235653 467239 logs.go:123] Gathering logs for kindnet [331ce690dc1105bc226ee21c46805afe1dcfb4affa37785dca1bb35bc43e178f] ...
I0920 05:05:36.235669 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 331ce690dc1105bc226ee21c46805afe1dcfb4affa37785dca1bb35bc43e178f"
I0920 05:05:36.281086 467239 logs.go:123] Gathering logs for storage-provisioner [da1c4fe4a0b3a07a3ea6e372ba61c81eaff9776ad30d0b28e04686d21757f663] ...
I0920 05:05:36.281117 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 da1c4fe4a0b3a07a3ea6e372ba61c81eaff9776ad30d0b28e04686d21757f663"
I0920 05:05:36.321993 467239 logs.go:123] Gathering logs for container status ...
I0920 05:05:36.322024 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0920 05:05:36.364652 467239 logs.go:123] Gathering logs for kubelet ...
I0920 05:05:36.364683 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0920 05:05:36.490563 467239 logs.go:123] Gathering logs for describe nodes ...
I0920 05:05:36.490602 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0920 05:06:36.593529 467239 ssh_runner.go:235] Completed: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": (1m0.102902199s)
W0920 05:06:36.593569 467239 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
Error from server (Timeout): the server was unable to return a response in the time allotted, but may still be processing the request (get nodes)
output:
** stderr **
Error from server (Timeout): the server was unable to return a response in the time allotted, but may still be processing the request (get nodes)
** /stderr **
I0920 05:06:36.593589 467239 out.go:374] Setting ErrFile to fd 2...
I0920 05:06:36.593600 467239 out.go:408] TERM=,COLORTERM=, which probably does not support color
W0920 05:06:36.593658 467239 out.go:285] X Problems detected in kube-scheduler [9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714]:
X Problems detected in kube-scheduler [9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714]:
W0920 05:06:36.593669 467239 out.go:285] E0920 05:02:57.234215 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10259: listen tcp 127.0.0.1:10259: bind: address already in use"
E0920 05:02:57.234215 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10259: listen tcp 127.0.0.1:10259: bind: address already in use"
W0920 05:06:36.593688 467239 out.go:285] X Problems detected in kube-controller-manager [0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726]:
X Problems detected in kube-controller-manager [0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726]:
W0920 05:06:36.593696 467239 out.go:285] E0920 05:03:00.247948 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10257: listen tcp 127.0.0.1:10257: bind: address already in use"
E0920 05:03:00.247948 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10257: listen tcp 127.0.0.1:10257: bind: address already in use"
I0920 05:06:36.593700 467239 out.go:374] Setting ErrFile to fd 2...
I0920 05:06:36.593705 467239 out.go:408] TERM=,COLORTERM=, which probably does not support color
I0920 05:06:46.597081 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 05:06:51.601616 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 05:06:51.604903 467239 out.go:203]
W0920 05:06:51.608026 467239 out.go:285] X Exiting due to GUEST_START: failed to start node: wait 6m0s for node: wait for healthy API server: apiserver healthz never reported healthy: context deadline exceeded
X Exiting due to GUEST_START: failed to start node: wait 6m0s for node: wait for healthy API server: apiserver healthz never reported healthy: context deadline exceeded
W0920 05:06:51.608053 467239 out.go:285] *
*
W0920 05:06:51.610872 467239 out.go:308] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0920 05:06:51.613768 467239 out.go:203]
** /stderr **
version_upgrade_test.go:277: start after failed upgrade: out/minikube-linux-arm64 start -p kubernetes-upgrade-068009 --memory=3072 --kubernetes-version=v1.34.0 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd: exit status 80
version_upgrade_test.go:279: *** TestKubernetesUpgrade FAILED at 2025-09-20 05:06:51.673174389 +0000 UTC m=+2746.634926239
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:223: ======> post-mortem[TestKubernetesUpgrade]: network settings <======
helpers_test.go:230: HOST ENV snapshots: PROXY env: HTTP_PROXY="<empty>" HTTPS_PROXY="<empty>" NO_PROXY="<empty>"
helpers_test.go:238: ======> post-mortem[TestKubernetesUpgrade]: docker inspect <======
helpers_test.go:239: (dbg) Run: docker inspect kubernetes-upgrade-068009
helpers_test.go:243: (dbg) docker inspect kubernetes-upgrade-068009:
-- stdout --
[
{
"Id": "58530bbe33cca1a26a11cff4e9770ade761484cb7ec7d328f9a97eda30905c31",
"Created": "2025-09-20T04:58:16.287574964Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 463889,
"ExitCode": 0,
"Error": "",
"StartedAt": "2025-09-20T04:58:45.695315721Z",
"FinishedAt": "2025-09-20T04:58:44.455210147Z"
},
"Image": "sha256:3d6f74760dfc17060da5abc5d463d3d45b4ceea05955c9cc42b3ec56cb38cc48",
"ResolvConfPath": "/var/lib/docker/containers/58530bbe33cca1a26a11cff4e9770ade761484cb7ec7d328f9a97eda30905c31/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/58530bbe33cca1a26a11cff4e9770ade761484cb7ec7d328f9a97eda30905c31/hostname",
"HostsPath": "/var/lib/docker/containers/58530bbe33cca1a26a11cff4e9770ade761484cb7ec7d328f9a97eda30905c31/hosts",
"LogPath": "/var/lib/docker/containers/58530bbe33cca1a26a11cff4e9770ade761484cb7ec7d328f9a97eda30905c31/58530bbe33cca1a26a11cff4e9770ade761484cb7ec7d328f9a97eda30905c31-json.log",
"Name": "/kubernetes-upgrade-068009",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"kubernetes-upgrade-068009:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "kubernetes-upgrade-068009",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 3221225472,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 6442450944,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"ID": "58530bbe33cca1a26a11cff4e9770ade761484cb7ec7d328f9a97eda30905c31",
"LowerDir": "/var/lib/docker/overlay2/d3ca6edcde381ba08f7eb2efba74804b77e3f3f656d7961382b893e90e988249-init/diff:/var/lib/docker/overlay2/fa53b2541d376d7dcf2aa9050a2fe731887c53aee5fa52f9de21f872b59a853e/diff",
"MergedDir": "/var/lib/docker/overlay2/d3ca6edcde381ba08f7eb2efba74804b77e3f3f656d7961382b893e90e988249/merged",
"UpperDir": "/var/lib/docker/overlay2/d3ca6edcde381ba08f7eb2efba74804b77e3f3f656d7961382b893e90e988249/diff",
"WorkDir": "/var/lib/docker/overlay2/d3ca6edcde381ba08f7eb2efba74804b77e3f3f656d7961382b893e90e988249/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "kubernetes-upgrade-068009",
"Source": "/var/lib/docker/volumes/kubernetes-upgrade-068009/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "kubernetes-upgrade-068009",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "kubernetes-upgrade-068009",
"name.minikube.sigs.k8s.io": "kubernetes-upgrade-068009",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "db079dcba34291d32592e50cbfa7366971fbbb64883597181403d336c4fd6403",
"SandboxKey": "/var/run/docker/netns/db079dcba342",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33378"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33379"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33382"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33380"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33381"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"kubernetes-upgrade-068009": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "52:2c:a0:64:05:c0",
"DriverOpts": null,
"GwPriority": 0,
"NetworkID": "353319a0c2dd802ba917a0952eb033a72e026d195897aff6b8be694ec770a227",
"EndpointID": "72ca4fe8d592bd82410791866ec02cd37d462ac313b69b1e0b966bcad971fc23",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"kubernetes-upgrade-068009",
"58530bbe33cc"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p kubernetes-upgrade-068009 -n kubernetes-upgrade-068009
helpers_test.go:247: (dbg) Non-zero exit: out/minikube-linux-arm64 status --format={{.Host}} -p kubernetes-upgrade-068009 -n kubernetes-upgrade-068009: exit status 2 (15.891846397s)
-- stdout --
Running
-- /stdout --
helpers_test.go:247: status error: exit status 2 (may be ok)
helpers_test.go:252: <<< TestKubernetesUpgrade FAILED: start of post-mortem logs <<<
helpers_test.go:253: ======> post-mortem[TestKubernetesUpgrade]: minikube logs <======
helpers_test.go:255: (dbg) Run: out/minikube-linux-arm64 -p kubernetes-upgrade-068009 logs -n 25
helpers_test.go:255: (dbg) Done: out/minikube-linux-arm64 -p kubernetes-upgrade-068009 logs -n 25: (1m1.595141823s)
helpers_test.go:260: TestKubernetesUpgrade logs:
-- stdout --
==> Audit <==
┌─────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬──────────────────────────┬─────────┬─────────┬─────────────────────┬─────────────────────┐
│ COMMAND │ ARGS │ PROFILE │ USER │ VERSION │ START TIME │ END TIME │
├─────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼──────────────────────────┼─────────┼─────────┼─────────────────────┼─────────────────────┤
│ ssh │ -p cilium-825371 sudo cat /etc/kubernetes/kubelet.conf │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo cat /var/lib/kubelet/config.yaml │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo systemctl status docker --all --full --no-pager │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo systemctl cat docker --no-pager │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo cat /etc/docker/daemon.json │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo docker system info │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo systemctl status cri-docker --all --full --no-pager │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo systemctl cat cri-docker --no-pager │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo cat /etc/systemd/system/cri-docker.service.d/10-cni.conf │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo cat /usr/lib/systemd/system/cri-docker.service │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo cri-dockerd --version │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo systemctl status containerd --all --full --no-pager │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo systemctl cat containerd --no-pager │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo cat /lib/systemd/system/containerd.service │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo cat /etc/containerd/config.toml │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo containerd config dump │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo systemctl status crio --all --full --no-pager │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo systemctl cat crio --no-pager │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo find /etc/crio -type f -exec sh -c 'echo {}; cat {}' \; │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ ssh │ -p cilium-825371 sudo crio config │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ │
│ delete │ -p cilium-825371 │ cilium-825371 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ 20 Sep 25 05:03 UTC │
│ start │ -p force-systemd-env-747224 --memory=3072 --alsologtostderr -v=5 --driver=docker --container-runtime=containerd │ force-systemd-env-747224 │ jenkins │ v1.37.0 │ 20 Sep 25 05:03 UTC │ 20 Sep 25 05:04 UTC │
│ ssh │ force-systemd-env-747224 ssh cat /etc/containerd/config.toml │ force-systemd-env-747224 │ jenkins │ v1.37.0 │ 20 Sep 25 05:04 UTC │ 20 Sep 25 05:04 UTC │
│ delete │ -p force-systemd-env-747224 │ force-systemd-env-747224 │ jenkins │ v1.37.0 │ 20 Sep 25 05:04 UTC │ 20 Sep 25 05:04 UTC │
│ start │ -p cert-expiration-920996 --memory=3072 --cert-expiration=3m --driver=docker --container-runtime=containerd │ cert-expiration-920996 │ jenkins │ v1.37.0 │ 20 Sep 25 05:04 UTC │ 20 Sep 25 05:04 UTC │
└─────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────────────────┴─────────┴─────────┴─────────────────────┴─────────────────────┘
==> Last Start <==
Log file created at: 2025/09/20 05:04:20
Running on machine: ip-172-31-24-2
Binary: Built with gc go1.24.6 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0920 05:04:20.995048 489259 out.go:360] Setting OutFile to fd 1 ...
I0920 05:04:20.995175 489259 out.go:408] TERM=,COLORTERM=, which probably does not support color
I0920 05:04:20.995179 489259 out.go:374] Setting ErrFile to fd 2...
I0920 05:04:20.995182 489259 out.go:408] TERM=,COLORTERM=, which probably does not support color
I0920 05:04:20.995460 489259 root.go:338] Updating PATH: /home/jenkins/minikube-integration/21599-293644/.minikube/bin
I0920 05:04:20.995881 489259 out.go:368] Setting JSON to false
I0920 05:04:20.996974 489259 start.go:130] hostinfo: {"hostname":"ip-172-31-24-2","uptime":6413,"bootTime":1758338248,"procs":218,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1084-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"6d436adf-771e-4269-b9a3-c25fd4fca4f5"}
I0920 05:04:20.997034 489259 start.go:140] virtualization:
I0920 05:04:21.005292 489259 out.go:179] * [cert-expiration-920996] minikube v1.37.0 on Ubuntu 20.04 (arm64)
I0920 05:04:21.009650 489259 out.go:179] - MINIKUBE_LOCATION=21599
I0920 05:04:21.009812 489259 notify.go:220] Checking for updates...
I0920 05:04:21.016187 489259 out.go:179] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0920 05:04:21.019559 489259 out.go:179] - KUBECONFIG=/home/jenkins/minikube-integration/21599-293644/kubeconfig
I0920 05:04:21.022766 489259 out.go:179] - MINIKUBE_HOME=/home/jenkins/minikube-integration/21599-293644/.minikube
I0920 05:04:21.025832 489259 out.go:179] - MINIKUBE_BIN=out/minikube-linux-arm64
I0920 05:04:21.028850 489259 out.go:179] - MINIKUBE_FORCE_SYSTEMD=
I0920 05:04:21.032432 489259 config.go:182] Loaded profile config "kubernetes-upgrade-068009": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.34.0
I0920 05:04:21.032520 489259 driver.go:421] Setting default libvirt URI to qemu:///system
I0920 05:04:21.059029 489259 docker.go:123] docker version: linux-28.1.1:Docker Engine - Community
I0920 05:04:21.059150 489259 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0920 05:04:21.122596 489259 info.go:266] docker info: {ID:J4M5:W6MX:GOX4:4LAQ:VI7E:VJNF:J3OP:OPBH:GF7G:PPY4:WQWD:7N4L Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:37 OomKillDisable:true NGoroutines:52 SystemTime:2025-09-20 05:04:21.112564136 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1084-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[::1/128 127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-24-2 Labels:[] ExperimentalBuild:false ServerVersion:28.1.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:05044ec0a9a75232cad458027ca83437aae3f4da Expected:} RuncCommit:{ID:v1.2.5-0-g59923ef Expected:} InitCommit:{ID:de40ad0 Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Pat
h:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.23.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.35.1]] Warnings:<nil>}}
I0920 05:04:21.122688 489259 docker.go:318] overlay module found
I0920 05:04:21.126037 489259 out.go:179] * Using the docker driver based on user configuration
I0920 05:04:21.128855 489259 start.go:304] selected driver: docker
I0920 05:04:21.128878 489259 start.go:924] validating driver "docker" against <nil>
I0920 05:04:21.128889 489259 start.go:935] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0920 05:04:21.129676 489259 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0920 05:04:21.186862 489259 info.go:266] docker info: {ID:J4M5:W6MX:GOX4:4LAQ:VI7E:VJNF:J3OP:OPBH:GF7G:PPY4:WQWD:7N4L Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:37 OomKillDisable:true NGoroutines:52 SystemTime:2025-09-20 05:04:21.177793455 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1084-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[::1/128 127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-24-2 Labels:[] ExperimentalBuild:false ServerVersion:28.1.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:05044ec0a9a75232cad458027ca83437aae3f4da Expected:} RuncCommit:{ID:v1.2.5-0-g59923ef Expected:} InitCommit:{ID:de40ad0 Expected:} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Pat
h:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.23.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.35.1]] Warnings:<nil>}}
I0920 05:04:21.187007 489259 start_flags.go:327] no existing cluster config was found, will generate one from the flags
I0920 05:04:21.187213 489259 start_flags.go:974] Wait components to verify : map[apiserver:true system_pods:true]
I0920 05:04:21.190027 489259 out.go:179] * Using Docker driver with root privileges
I0920 05:04:21.193047 489259 cni.go:84] Creating CNI manager for ""
I0920 05:04:21.193164 489259 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0920 05:04:21.193171 489259 start_flags.go:336] Found "CNI" CNI - setting NetworkPlugin=cni
I0920 05:04:21.193253 489259 start.go:348] cluster config:
{Name:cert-expiration-920996 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.0 ClusterName:cert-expiration-920996 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:conta
inerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.34.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:3m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0920 05:04:21.196541 489259 out.go:179] * Starting "cert-expiration-920996" primary control-plane node in "cert-expiration-920996" cluster
I0920 05:04:21.199329 489259 cache.go:123] Beginning downloading kic base image for docker with containerd
I0920 05:04:21.202220 489259 out.go:179] * Pulling base image v0.0.48 ...
I0920 05:04:21.205095 489259 preload.go:131] Checking if preload exists for k8s version v1.34.0 and runtime containerd
I0920 05:04:21.205146 489259 preload.go:146] Found local preload: /home/jenkins/minikube-integration/21599-293644/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.34.0-containerd-overlay2-arm64.tar.lz4
I0920 05:04:21.205171 489259 cache.go:58] Caching tarball of preloaded images
I0920 05:04:21.205176 489259 image.go:81] Checking for gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 in local docker daemon
I0920 05:04:21.205273 489259 preload.go:172] Found /home/jenkins/minikube-integration/21599-293644/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.34.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0920 05:04:21.205282 489259 cache.go:61] Finished verifying existence of preloaded tar for v1.34.0 on containerd
I0920 05:04:21.205416 489259 profile.go:143] Saving config to /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/config.json ...
I0920 05:04:21.205433 489259 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/config.json: {Name:mk033c1fb2078e4568fd252ab24555936c477e83 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 05:04:21.224647 489259 image.go:100] Found gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 in local docker daemon, skipping pull
I0920 05:04:21.224660 489259 cache.go:147] gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 exists in daemon, skipping load
I0920 05:04:21.224678 489259 cache.go:232] Successfully downloaded all kic artifacts
I0920 05:04:21.224710 489259 start.go:360] acquireMachinesLock for cert-expiration-920996: {Name:mk366b41cb38ebfabbdf920145797e327bda5baf Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0920 05:04:21.224827 489259 start.go:364] duration metric: took 103.615µs to acquireMachinesLock for "cert-expiration-920996"
I0920 05:04:21.224852 489259 start.go:93] Provisioning new machine with config: &{Name:cert-expiration-920996 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.0 ClusterName:cert-expiration-920996 Namespace:default APIServerHAVIP: APIServerName:minikubeC
A APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.34.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:3m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID
:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.34.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0920 05:04:21.224925 489259 start.go:125] createHost starting for "" (driver="docker")
I0920 05:04:21.230142 489259 out.go:252] * Creating docker container (CPUs=2, Memory=3072MB) ...
I0920 05:04:21.230397 489259 start.go:159] libmachine.API.Create for "cert-expiration-920996" (driver="docker")
I0920 05:04:21.230431 489259 client.go:168] LocalClient.Create starting
I0920 05:04:21.230511 489259 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca.pem
I0920 05:04:21.230546 489259 main.go:141] libmachine: Decoding PEM data...
I0920 05:04:21.230558 489259 main.go:141] libmachine: Parsing certificate...
I0920 05:04:21.230617 489259 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/21599-293644/.minikube/certs/cert.pem
I0920 05:04:21.230633 489259 main.go:141] libmachine: Decoding PEM data...
I0920 05:04:21.230641 489259 main.go:141] libmachine: Parsing certificate...
I0920 05:04:21.231019 489259 cli_runner.go:164] Run: docker network inspect cert-expiration-920996 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0920 05:04:21.246374 489259 cli_runner.go:211] docker network inspect cert-expiration-920996 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0920 05:04:21.246460 489259 network_create.go:284] running [docker network inspect cert-expiration-920996] to gather additional debugging logs...
I0920 05:04:21.246476 489259 cli_runner.go:164] Run: docker network inspect cert-expiration-920996
W0920 05:04:21.263139 489259 cli_runner.go:211] docker network inspect cert-expiration-920996 returned with exit code 1
I0920 05:04:21.263159 489259 network_create.go:287] error running [docker network inspect cert-expiration-920996]: docker network inspect cert-expiration-920996: exit status 1
stdout:
[]
stderr:
Error response from daemon: network cert-expiration-920996 not found
I0920 05:04:21.263171 489259 network_create.go:289] output of [docker network inspect cert-expiration-920996]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network cert-expiration-920996 not found
** /stderr **
I0920 05:04:21.263280 489259 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0920 05:04:21.279814 489259 network.go:211] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName:br-c800afb16cfb IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:6e:b6:2a:28:db:34} reservation:<nil>}
I0920 05:04:21.280192 489259 network.go:211] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName:br-a15331f91056 IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:32:57:f6:3a:25:ff} reservation:<nil>}
I0920 05:04:21.280502 489259 network.go:211] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName:br-0876584792ab IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:fe:da:35:35:52:71} reservation:<nil>}
I0920 05:04:21.280756 489259 network.go:211] skipping subnet 192.168.76.0/24 that is taken: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName:br-353319a0c2dd IfaceIPv4:192.168.76.1 IfaceMTU:1500 IfaceMAC:12:ff:f2:9f:db:09} reservation:<nil>}
I0920 05:04:21.281197 489259 network.go:206] using free private subnet 192.168.85.0/24: &{IP:192.168.85.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.85.0/24 Gateway:192.168.85.1 ClientMin:192.168.85.2 ClientMax:192.168.85.254 Broadcast:192.168.85.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0x400187bdc0}
I0920 05:04:21.281213 489259 network_create.go:124] attempt to create docker network cert-expiration-920996 192.168.85.0/24 with gateway 192.168.85.1 and MTU of 1500 ...
I0920 05:04:21.281274 489259 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.85.0/24 --gateway=192.168.85.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=cert-expiration-920996 cert-expiration-920996
I0920 05:04:21.343779 489259 network_create.go:108] docker network cert-expiration-920996 192.168.85.0/24 created
I0920 05:04:21.343817 489259 kic.go:121] calculated static IP "192.168.85.2" for the "cert-expiration-920996" container
I0920 05:04:21.343935 489259 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0920 05:04:21.360481 489259 cli_runner.go:164] Run: docker volume create cert-expiration-920996 --label name.minikube.sigs.k8s.io=cert-expiration-920996 --label created_by.minikube.sigs.k8s.io=true
I0920 05:04:21.378708 489259 oci.go:103] Successfully created a docker volume cert-expiration-920996
I0920 05:04:21.378779 489259 cli_runner.go:164] Run: docker run --rm --name cert-expiration-920996-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=cert-expiration-920996 --entrypoint /usr/bin/test -v cert-expiration-920996:/var gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 -d /var/lib
I0920 05:04:21.891102 489259 oci.go:107] Successfully prepared a docker volume cert-expiration-920996
I0920 05:04:21.891151 489259 preload.go:131] Checking if preload exists for k8s version v1.34.0 and runtime containerd
I0920 05:04:21.891169 489259 kic.go:194] Starting extracting preloaded images to volume ...
I0920 05:04:21.891242 489259 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/21599-293644/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.34.0-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v cert-expiration-920996:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 -I lz4 -xf /preloaded.tar -C /extractDir
I0920 05:04:26.136850 489259 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/21599-293644/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.34.0-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v cert-expiration-920996:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 -I lz4 -xf /preloaded.tar -C /extractDir: (4.24556669s)
I0920 05:04:26.136871 489259 kic.go:203] duration metric: took 4.245698703s to extract preloaded images to volume ...
W0920 05:04:26.137004 489259 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0920 05:04:26.137104 489259 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0920 05:04:26.200463 489259 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname cert-expiration-920996 --name cert-expiration-920996 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=cert-expiration-920996 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=cert-expiration-920996 --network cert-expiration-920996 --ip 192.168.85.2 --volume cert-expiration-920996:/var --security-opt apparmor=unconfined --memory=3072mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1
I0920 05:04:26.516422 489259 cli_runner.go:164] Run: docker container inspect cert-expiration-920996 --format={{.State.Running}}
I0920 05:04:26.539855 489259 cli_runner.go:164] Run: docker container inspect cert-expiration-920996 --format={{.State.Status}}
I0920 05:04:26.561234 489259 cli_runner.go:164] Run: docker exec cert-expiration-920996 stat /var/lib/dpkg/alternatives/iptables
I0920 05:04:26.611498 489259 oci.go:144] the created container "cert-expiration-920996" has a running status.
I0920 05:04:26.611518 489259 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/21599-293644/.minikube/machines/cert-expiration-920996/id_rsa...
I0920 05:04:27.125935 489259 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/21599-293644/.minikube/machines/cert-expiration-920996/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0920 05:04:27.155002 489259 cli_runner.go:164] Run: docker container inspect cert-expiration-920996 --format={{.State.Status}}
I0920 05:04:27.180310 489259 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0920 05:04:27.180321 489259 kic_runner.go:114] Args: [docker exec --privileged cert-expiration-920996 chown docker:docker /home/docker/.ssh/authorized_keys]
I0920 05:04:27.237138 489259 cli_runner.go:164] Run: docker container inspect cert-expiration-920996 --format={{.State.Status}}
I0920 05:04:27.257671 489259 machine.go:93] provisionDockerMachine start ...
I0920 05:04:27.257748 489259 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" cert-expiration-920996
I0920 05:04:27.284208 489259 main.go:141] libmachine: Using SSH client type: native
I0920 05:04:27.284533 489259 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ef1e0] 0x3f19a0 <nil> [] 0s} 127.0.0.1 33413 <nil> <nil>}
I0920 05:04:27.284540 489259 main.go:141] libmachine: About to run SSH command:
hostname
I0920 05:04:27.441055 489259 main.go:141] libmachine: SSH cmd err, output: <nil>: cert-expiration-920996
I0920 05:04:27.441068 489259 ubuntu.go:182] provisioning hostname "cert-expiration-920996"
I0920 05:04:27.441179 489259 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" cert-expiration-920996
I0920 05:04:27.463637 489259 main.go:141] libmachine: Using SSH client type: native
I0920 05:04:27.463945 489259 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ef1e0] 0x3f19a0 <nil> [] 0s} 127.0.0.1 33413 <nil> <nil>}
I0920 05:04:27.463959 489259 main.go:141] libmachine: About to run SSH command:
sudo hostname cert-expiration-920996 && echo "cert-expiration-920996" | sudo tee /etc/hostname
I0920 05:04:27.629524 489259 main.go:141] libmachine: SSH cmd err, output: <nil>: cert-expiration-920996
I0920 05:04:27.629608 489259 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" cert-expiration-920996
I0920 05:04:27.654624 489259 main.go:141] libmachine: Using SSH client type: native
I0920 05:04:27.654922 489259 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ef1e0] 0x3f19a0 <nil> [] 0s} 127.0.0.1 33413 <nil> <nil>}
I0920 05:04:27.654938 489259 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\scert-expiration-920996' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 cert-expiration-920996/g' /etc/hosts;
else
echo '127.0.1.1 cert-expiration-920996' | sudo tee -a /etc/hosts;
fi
fi
I0920 05:04:27.806530 489259 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0920 05:04:27.806547 489259 ubuntu.go:188] set auth options {CertDir:/home/jenkins/minikube-integration/21599-293644/.minikube CaCertPath:/home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/21599-293644/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/21599-293644/.minikube}
I0920 05:04:27.806575 489259 ubuntu.go:190] setting up certificates
I0920 05:04:27.806583 489259 provision.go:84] configureAuth start
I0920 05:04:27.806662 489259 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" cert-expiration-920996
I0920 05:04:27.828854 489259 provision.go:143] copyHostCerts
I0920 05:04:27.828915 489259 exec_runner.go:144] found /home/jenkins/minikube-integration/21599-293644/.minikube/key.pem, removing ...
I0920 05:04:27.828922 489259 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21599-293644/.minikube/key.pem
I0920 05:04:27.829002 489259 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/21599-293644/.minikube/key.pem (1675 bytes)
I0920 05:04:27.829101 489259 exec_runner.go:144] found /home/jenkins/minikube-integration/21599-293644/.minikube/ca.pem, removing ...
I0920 05:04:27.829105 489259 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21599-293644/.minikube/ca.pem
I0920 05:04:27.829129 489259 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/21599-293644/.minikube/ca.pem (1078 bytes)
I0920 05:04:27.829189 489259 exec_runner.go:144] found /home/jenkins/minikube-integration/21599-293644/.minikube/cert.pem, removing ...
I0920 05:04:27.829192 489259 exec_runner.go:203] rm: /home/jenkins/minikube-integration/21599-293644/.minikube/cert.pem
I0920 05:04:27.829215 489259 exec_runner.go:151] cp: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/21599-293644/.minikube/cert.pem (1123 bytes)
I0920 05:04:27.829274 489259 provision.go:117] generating server cert: /home/jenkins/minikube-integration/21599-293644/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca-key.pem org=jenkins.cert-expiration-920996 san=[127.0.0.1 192.168.85.2 cert-expiration-920996 localhost minikube]
I0920 05:04:28.036850 489259 provision.go:177] copyRemoteCerts
I0920 05:04:28.036909 489259 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0920 05:04:28.036949 489259 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" cert-expiration-920996
I0920 05:04:28.059015 489259 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33413 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/cert-expiration-920996/id_rsa Username:docker}
I0920 05:04:28.158965 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0920 05:04:28.186981 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0920 05:04:28.214810 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0920 05:04:28.238822 489259 provision.go:87] duration metric: took 432.216208ms to configureAuth
I0920 05:04:28.238838 489259 ubuntu.go:206] setting minikube options for container-runtime
I0920 05:04:28.239019 489259 config.go:182] Loaded profile config "cert-expiration-920996": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.34.0
I0920 05:04:28.239025 489259 machine.go:96] duration metric: took 981.344674ms to provisionDockerMachine
I0920 05:04:28.239030 489259 client.go:171] duration metric: took 7.008594886s to LocalClient.Create
I0920 05:04:28.239050 489259 start.go:167] duration metric: took 7.00865471s to libmachine.API.Create "cert-expiration-920996"
I0920 05:04:28.239056 489259 start.go:293] postStartSetup for "cert-expiration-920996" (driver="docker")
I0920 05:04:28.239064 489259 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0920 05:04:28.239122 489259 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0920 05:04:28.239160 489259 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" cert-expiration-920996
I0920 05:04:28.258271 489259 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33413 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/cert-expiration-920996/id_rsa Username:docker}
I0920 05:04:28.359110 489259 ssh_runner.go:195] Run: cat /etc/os-release
I0920 05:04:28.362693 489259 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0920 05:04:28.362715 489259 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0920 05:04:28.362724 489259 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0920 05:04:28.362730 489259 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I0920 05:04:28.362744 489259 filesync.go:126] Scanning /home/jenkins/minikube-integration/21599-293644/.minikube/addons for local assets ...
I0920 05:04:28.362801 489259 filesync.go:126] Scanning /home/jenkins/minikube-integration/21599-293644/.minikube/files for local assets ...
I0920 05:04:28.362882 489259 filesync.go:149] local asset: /home/jenkins/minikube-integration/21599-293644/.minikube/files/etc/ssl/certs/2955092.pem -> 2955092.pem in /etc/ssl/certs
I0920 05:04:28.362981 489259 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0920 05:04:28.371969 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/files/etc/ssl/certs/2955092.pem --> /etc/ssl/certs/2955092.pem (1708 bytes)
I0920 05:04:28.402002 489259 start.go:296] duration metric: took 162.930567ms for postStartSetup
I0920 05:04:28.402466 489259 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" cert-expiration-920996
I0920 05:04:28.421604 489259 profile.go:143] Saving config to /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/config.json ...
I0920 05:04:28.421995 489259 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0920 05:04:28.422057 489259 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" cert-expiration-920996
I0920 05:04:28.443854 489259 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33413 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/cert-expiration-920996/id_rsa Username:docker}
I0920 05:04:28.538479 489259 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0920 05:04:28.543443 489259 start.go:128] duration metric: took 7.318495944s to createHost
I0920 05:04:28.543457 489259 start.go:83] releasing machines lock for "cert-expiration-920996", held for 7.318623418s
I0920 05:04:28.543555 489259 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" cert-expiration-920996
I0920 05:04:28.561591 489259 ssh_runner.go:195] Run: cat /version.json
I0920 05:04:28.561636 489259 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" cert-expiration-920996
I0920 05:04:28.561649 489259 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0920 05:04:28.561723 489259 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" cert-expiration-920996
I0920 05:04:28.579749 489259 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33413 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/cert-expiration-920996/id_rsa Username:docker}
I0920 05:04:28.581116 489259 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33413 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/cert-expiration-920996/id_rsa Username:docker}
I0920 05:04:28.814305 489259 ssh_runner.go:195] Run: systemctl --version
I0920 05:04:28.818854 489259 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0920 05:04:28.823167 489259 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0920 05:04:28.850761 489259 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0920 05:04:28.850841 489259 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0920 05:04:28.882954 489259 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0920 05:04:28.882969 489259 start.go:495] detecting cgroup driver to use...
I0920 05:04:28.883009 489259 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0920 05:04:28.883102 489259 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0920 05:04:28.895953 489259 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0920 05:04:28.908678 489259 docker.go:218] disabling cri-docker service (if available) ...
I0920 05:04:28.908730 489259 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0920 05:04:28.923187 489259 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0920 05:04:28.938396 489259 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0920 05:04:29.023108 489259 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0920 05:04:29.120905 489259 docker.go:234] disabling docker service ...
I0920 05:04:29.120958 489259 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0920 05:04:29.142333 489259 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0920 05:04:29.154823 489259 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0920 05:04:29.238310 489259 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0920 05:04:29.330387 489259 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0920 05:04:29.342340 489259 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0920 05:04:29.359218 489259 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10.1"|' /etc/containerd/config.toml"
I0920 05:04:29.370630 489259 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0920 05:04:29.381391 489259 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0920 05:04:29.381451 489259 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0920 05:04:29.392077 489259 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0920 05:04:29.403146 489259 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0920 05:04:29.413490 489259 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0920 05:04:29.423951 489259 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0920 05:04:29.433519 489259 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0920 05:04:29.444417 489259 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0920 05:04:29.454867 489259 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0920 05:04:29.464827 489259 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0920 05:04:29.473680 489259 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0920 05:04:29.482295 489259 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0920 05:04:29.573392 489259 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0920 05:04:29.714764 489259 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0920 05:04:29.714823 489259 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0920 05:04:29.718926 489259 start.go:563] Will wait 60s for crictl version
I0920 05:04:29.718983 489259 ssh_runner.go:195] Run: which crictl
I0920 05:04:29.722506 489259 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0920 05:04:29.765599 489259 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.27
RuntimeApiVersion: v1
I0920 05:04:29.765659 489259 ssh_runner.go:195] Run: containerd --version
I0920 05:04:29.796005 489259 ssh_runner.go:195] Run: containerd --version
I0920 05:04:29.823349 489259 out.go:179] * Preparing Kubernetes v1.34.0 on containerd 1.7.27 ...
I0920 05:04:29.826286 489259 cli_runner.go:164] Run: docker network inspect cert-expiration-920996 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0920 05:04:29.842241 489259 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0920 05:04:29.846167 489259 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0920 05:04:29.858310 489259 kubeadm.go:875] updating cluster {Name:cert-expiration-920996 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.0 ClusterName:cert-expiration-920996 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[]
APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.34.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:3m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs
: AutoPauseInterval:1m0s} ...
I0920 05:04:29.858417 489259 preload.go:131] Checking if preload exists for k8s version v1.34.0 and runtime containerd
I0920 05:04:29.858472 489259 ssh_runner.go:195] Run: sudo crictl images --output json
I0920 05:04:29.894948 489259 containerd.go:627] all images are preloaded for containerd runtime.
I0920 05:04:29.894959 489259 containerd.go:534] Images already preloaded, skipping extraction
I0920 05:04:29.895019 489259 ssh_runner.go:195] Run: sudo crictl images --output json
I0920 05:04:29.936126 489259 containerd.go:627] all images are preloaded for containerd runtime.
I0920 05:04:29.936138 489259 cache_images.go:85] Images are preloaded, skipping loading
I0920 05:04:29.936144 489259 kubeadm.go:926] updating node { 192.168.85.2 8443 v1.34.0 containerd true true} ...
I0920 05:04:29.936277 489259 kubeadm.go:938] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.34.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=cert-expiration-920996 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.34.0 ClusterName:cert-expiration-920996 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0920 05:04:29.936341 489259 ssh_runner.go:195] Run: sudo crictl info
I0920 05:04:29.980763 489259 cni.go:84] Creating CNI manager for ""
I0920 05:04:29.980773 489259 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0920 05:04:29.980782 489259 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0920 05:04:29.980802 489259 kubeadm.go:189] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.34.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:cert-expiration-920996 NodeName:cert-expiration-920996 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0920 05:04:29.980918 489259 kubeadm.go:195] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "cert-expiration-920996"
kubeletExtraArgs:
- name: "node-ip"
value: "192.168.85.2"
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
- name: "enable-admission-plugins"
value: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
- name: "allocate-node-cidrs"
value: "true"
- name: "leader-elect"
value: "false"
scheduler:
extraArgs:
- name: "leader-elect"
value: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
kubernetesVersion: v1.34.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0920 05:04:29.980985 489259 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.34.0
I0920 05:04:29.990614 489259 binaries.go:44] Found k8s binaries, skipping transfer
I0920 05:04:29.990676 489259 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0920 05:04:29.999884 489259 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (326 bytes)
I0920 05:04:30.053925 489259 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0920 05:04:30.080585 489259 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2235 bytes)
I0920 05:04:30.104752 489259 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0920 05:04:30.109077 489259 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0920 05:04:30.121747 489259 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0920 05:04:30.211364 489259 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0920 05:04:30.227435 489259 certs.go:68] Setting up /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996 for IP: 192.168.85.2
I0920 05:04:30.227446 489259 certs.go:194] generating shared ca certs ...
I0920 05:04:30.227460 489259 certs.go:226] acquiring lock for ca certs: {Name:mk5aa50fa7f0dce0a90669167c944409dfea47c0 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 05:04:30.227605 489259 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/21599-293644/.minikube/ca.key
I0920 05:04:30.227644 489259 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/21599-293644/.minikube/proxy-client-ca.key
I0920 05:04:30.227650 489259 certs.go:256] generating profile certs ...
I0920 05:04:30.227704 489259 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/client.key
I0920 05:04:30.227723 489259 crypto.go:68] Generating cert /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/client.crt with IP's: []
I0920 05:04:30.732890 489259 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/client.crt ...
I0920 05:04:30.732906 489259 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/client.crt: {Name:mkb04ed18be2fdb011c0dad3aed3e10a6d4f28b2 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 05:04:30.733112 489259 crypto.go:164] Writing key to /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/client.key ...
I0920 05:04:30.733121 489259 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/client.key: {Name:mk26ab3271c659998188cff65b62d87d7bb10a58 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 05:04:30.733891 489259 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.key.f8e75100
I0920 05:04:30.733908 489259 crypto.go:68] Generating cert /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.crt.f8e75100 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.85.2]
I0920 05:04:31.575400 489259 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.crt.f8e75100 ...
I0920 05:04:31.575416 489259 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.crt.f8e75100: {Name:mkc6b836fac765fdf37c9787290dedf9f31dc0cd Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 05:04:31.575617 489259 crypto.go:164] Writing key to /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.key.f8e75100 ...
I0920 05:04:31.575633 489259 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.key.f8e75100: {Name:mkc71e8b0d92e725b093ad0a0e6ce38c41617c89 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 05:04:31.575719 489259 certs.go:381] copying /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.crt.f8e75100 -> /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.crt
I0920 05:04:31.575806 489259 certs.go:385] copying /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.key.f8e75100 -> /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.key
I0920 05:04:31.575860 489259 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/proxy-client.key
I0920 05:04:31.575872 489259 crypto.go:68] Generating cert /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/proxy-client.crt with IP's: []
I0920 05:04:31.778921 489259 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/proxy-client.crt ...
I0920 05:04:31.778935 489259 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/proxy-client.crt: {Name:mke489516214e902bc11c8fadcdbc61b03e88e5f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 05:04:31.779115 489259 crypto.go:164] Writing key to /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/proxy-client.key ...
I0920 05:04:31.779123 489259 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/proxy-client.key: {Name:mk57613d8f2b7baf708a15498b27bdb2720f6c93 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 05:04:31.779314 489259 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/295509.pem (1338 bytes)
W0920 05:04:31.779349 489259 certs.go:480] ignoring /home/jenkins/minikube-integration/21599-293644/.minikube/certs/295509_empty.pem, impossibly tiny 0 bytes
I0920 05:04:31.779358 489259 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca-key.pem (1679 bytes)
I0920 05:04:31.779381 489259 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/ca.pem (1078 bytes)
I0920 05:04:31.779400 489259 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/cert.pem (1123 bytes)
I0920 05:04:31.779426 489259 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/certs/key.pem (1675 bytes)
I0920 05:04:31.779465 489259 certs.go:484] found cert: /home/jenkins/minikube-integration/21599-293644/.minikube/files/etc/ssl/certs/2955092.pem (1708 bytes)
I0920 05:04:31.780135 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0920 05:04:31.808365 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0920 05:04:31.835985 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0920 05:04:31.862087 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0920 05:04:31.886785 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0920 05:04:31.912508 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0920 05:04:31.936578 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0920 05:04:31.960874 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/profiles/cert-expiration-920996/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0920 05:04:31.986342 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0920 05:04:32.013894 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/certs/295509.pem --> /usr/share/ca-certificates/295509.pem (1338 bytes)
I0920 05:04:32.040331 489259 ssh_runner.go:362] scp /home/jenkins/minikube-integration/21599-293644/.minikube/files/etc/ssl/certs/2955092.pem --> /usr/share/ca-certificates/2955092.pem (1708 bytes)
I0920 05:04:32.065935 489259 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0920 05:04:32.085440 489259 ssh_runner.go:195] Run: openssl version
I0920 05:04:32.091591 489259 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0920 05:04:32.101813 489259 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0920 05:04:32.105616 489259 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 20 04:21 /usr/share/ca-certificates/minikubeCA.pem
I0920 05:04:32.105669 489259 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0920 05:04:32.113290 489259 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0920 05:04:32.123282 489259 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/295509.pem && ln -fs /usr/share/ca-certificates/295509.pem /etc/ssl/certs/295509.pem"
I0920 05:04:32.132933 489259 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/295509.pem
I0920 05:04:32.136671 489259 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 20 04:28 /usr/share/ca-certificates/295509.pem
I0920 05:04:32.136729 489259 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/295509.pem
I0920 05:04:32.143720 489259 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/295509.pem /etc/ssl/certs/51391683.0"
I0920 05:04:32.154053 489259 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/2955092.pem && ln -fs /usr/share/ca-certificates/2955092.pem /etc/ssl/certs/2955092.pem"
I0920 05:04:32.164141 489259 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/2955092.pem
I0920 05:04:32.167790 489259 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 20 04:28 /usr/share/ca-certificates/2955092.pem
I0920 05:04:32.167846 489259 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/2955092.pem
I0920 05:04:32.175244 489259 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/2955092.pem /etc/ssl/certs/3ec20f2e.0"
I0920 05:04:32.185318 489259 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0920 05:04:32.189015 489259 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0920 05:04:32.189058 489259 kubeadm.go:392] StartCluster: {Name:cert-expiration-920996 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.48@sha256:7171c97a51623558720f8e5878e4f4637da093e2f2ed589997bedc6c1549b2b1 Memory:3072 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.34.0 ClusterName:cert-expiration-920996 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] AP
IServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.34.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:3m0s MountString: Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false DisableCoreDNSLog:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: A
utoPauseInterval:1m0s}
I0920 05:04:32.189121 489259 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0920 05:04:32.189187 489259 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0920 05:04:32.225966 489259 cri.go:89] found id: ""
I0920 05:04:32.226037 489259 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0920 05:04:32.234970 489259 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0920 05:04:32.246155 489259 kubeadm.go:214] ignoring SystemVerification for kubeadm because of docker driver
I0920 05:04:32.246209 489259 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0920 05:04:32.256577 489259 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0920 05:04:32.256587 489259 kubeadm.go:157] found existing configuration files:
I0920 05:04:32.256639 489259 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0920 05:04:32.265497 489259 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0920 05:04:32.265566 489259 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0920 05:04:32.275108 489259 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0920 05:04:32.284189 489259 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0920 05:04:32.284255 489259 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0920 05:04:32.293039 489259 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0920 05:04:32.302288 489259 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0920 05:04:32.302342 489259 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0920 05:04:32.311187 489259 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0920 05:04:32.320209 489259 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0920 05:04:32.320290 489259 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0920 05:04:32.329490 489259 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.34.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0920 05:04:32.373517 489259 kubeadm.go:310] [init] Using Kubernetes version: v1.34.0
I0920 05:04:32.373594 489259 kubeadm.go:310] [preflight] Running pre-flight checks
I0920 05:04:32.392953 489259 kubeadm.go:310] [preflight] The system verification failed. Printing the output from the verification:
I0920 05:04:32.393036 489259 kubeadm.go:310] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1084-aws[0m
I0920 05:04:32.393072 489259 kubeadm.go:310] [0;37mOS[0m: [0;32mLinux[0m
I0920 05:04:32.393129 489259 kubeadm.go:310] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0920 05:04:32.393183 489259 kubeadm.go:310] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0920 05:04:32.393234 489259 kubeadm.go:310] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0920 05:04:32.393287 489259 kubeadm.go:310] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0920 05:04:32.393339 489259 kubeadm.go:310] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0920 05:04:32.393425 489259 kubeadm.go:310] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0920 05:04:32.393479 489259 kubeadm.go:310] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0920 05:04:32.393531 489259 kubeadm.go:310] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0920 05:04:32.393582 489259 kubeadm.go:310] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0920 05:04:32.470437 489259 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0920 05:04:32.470545 489259 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0920 05:04:32.470639 489259 kubeadm.go:310] [preflight] You can also perform this action beforehand using 'kubeadm config images pull'
I0920 05:04:32.476384 489259 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0920 05:04:32.482449 489259 out.go:252] - Generating certificates and keys ...
I0920 05:04:32.482541 489259 kubeadm.go:310] [certs] Using existing ca certificate authority
I0920 05:04:32.482608 489259 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0920 05:04:32.629221 489259 kubeadm.go:310] [certs] Generating "apiserver-kubelet-client" certificate and key
I0920 05:04:33.385500 489259 kubeadm.go:310] [certs] Generating "front-proxy-ca" certificate and key
I0920 05:04:35.178819 489259 kubeadm.go:310] [certs] Generating "front-proxy-client" certificate and key
I0920 05:04:35.657374 489259 kubeadm.go:310] [certs] Generating "etcd/ca" certificate and key
I0920 05:04:36.523777 489259 kubeadm.go:310] [certs] Generating "etcd/server" certificate and key
I0920 05:04:36.524228 489259 kubeadm.go:310] [certs] etcd/server serving cert is signed for DNS names [cert-expiration-920996 localhost] and IPs [192.168.85.2 127.0.0.1 ::1]
I0920 05:04:36.732074 489259 kubeadm.go:310] [certs] Generating "etcd/peer" certificate and key
I0920 05:04:36.732354 489259 kubeadm.go:310] [certs] etcd/peer serving cert is signed for DNS names [cert-expiration-920996 localhost] and IPs [192.168.85.2 127.0.0.1 ::1]
I0920 05:04:36.888499 489259 kubeadm.go:310] [certs] Generating "etcd/healthcheck-client" certificate and key
I0920 05:04:37.601559 489259 kubeadm.go:310] [certs] Generating "apiserver-etcd-client" certificate and key
I0920 05:04:38.031587 489259 kubeadm.go:310] [certs] Generating "sa" key and public key
I0920 05:04:38.031854 489259 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0920 05:04:38.350165 489259 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0920 05:04:38.721203 489259 kubeadm.go:310] [kubeconfig] Writing "super-admin.conf" kubeconfig file
I0920 05:04:39.256280 489259 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0920 05:04:39.693470 489259 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0920 05:04:40.318697 489259 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0920 05:04:40.319593 489259 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0920 05:04:40.322329 489259 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0920 05:04:40.326214 489259 out.go:252] - Booting up control plane ...
I0920 05:04:40.326330 489259 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0920 05:04:40.326422 489259 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0920 05:04:40.326507 489259 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0920 05:04:40.337441 489259 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0920 05:04:40.337764 489259 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
I0920 05:04:40.347132 489259 kubeadm.go:310] [patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
I0920 05:04:40.349609 489259 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0920 05:04:40.349665 489259 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0920 05:04:40.474213 489259 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
I0920 05:04:40.474330 489259 kubeadm.go:310] [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
I0920 05:04:40.989127 489259 kubeadm.go:310] [kubelet-check] The kubelet is healthy after 514.970349ms
I0920 05:04:40.993525 489259 kubeadm.go:310] [control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
I0920 05:04:40.993631 489259 kubeadm.go:310] [control-plane-check] Checking kube-apiserver at https://192.168.85.2:8443/livez
I0920 05:04:40.993724 489259 kubeadm.go:310] [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
I0920 05:04:40.993807 489259 kubeadm.go:310] [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
I0920 05:04:45.477808 489259 kubeadm.go:310] [control-plane-check] kube-controller-manager is healthy after 4.483652633s
I0920 05:04:47.012326 489259 kubeadm.go:310] [control-plane-check] kube-scheduler is healthy after 6.018804408s
I0920 05:04:48.995926 489259 kubeadm.go:310] [control-plane-check] kube-apiserver is healthy after 8.00231773s
I0920 05:04:49.019096 489259 kubeadm.go:310] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0920 05:04:49.035342 489259 kubeadm.go:310] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0920 05:04:49.049386 489259 kubeadm.go:310] [upload-certs] Skipping phase. Please see --upload-certs
I0920 05:04:49.049604 489259 kubeadm.go:310] [mark-control-plane] Marking the node cert-expiration-920996 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
I0920 05:04:49.062361 489259 kubeadm.go:310] [bootstrap-token] Using token: 52m51f.2xxnk5wiv4jjcr4b
I0920 05:04:49.065432 489259 out.go:252] - Configuring RBAC rules ...
I0920 05:04:49.065576 489259 kubeadm.go:310] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
I0920 05:04:49.070668 489259 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
I0920 05:04:49.080437 489259 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I0920 05:04:49.084887 489259 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
I0920 05:04:49.091467 489259 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I0920 05:04:49.096097 489259 kubeadm.go:310] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I0920 05:04:49.405877 489259 kubeadm.go:310] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0920 05:04:49.830331 489259 kubeadm.go:310] [addons] Applied essential addon: CoreDNS
I0920 05:04:50.403855 489259 kubeadm.go:310] [addons] Applied essential addon: kube-proxy
I0920 05:04:50.405095 489259 kubeadm.go:310]
I0920 05:04:50.405164 489259 kubeadm.go:310] Your Kubernetes control-plane has initialized successfully!
I0920 05:04:50.405168 489259 kubeadm.go:310]
I0920 05:04:50.405248 489259 kubeadm.go:310] To start using your cluster, you need to run the following as a regular user:
I0920 05:04:50.405252 489259 kubeadm.go:310]
I0920 05:04:50.405277 489259 kubeadm.go:310] mkdir -p $HOME/.kube
I0920 05:04:50.405338 489259 kubeadm.go:310] sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
I0920 05:04:50.405432 489259 kubeadm.go:310] sudo chown $(id -u):$(id -g) $HOME/.kube/config
I0920 05:04:50.405436 489259 kubeadm.go:310]
I0920 05:04:50.405492 489259 kubeadm.go:310] Alternatively, if you are the root user, you can run:
I0920 05:04:50.405495 489259 kubeadm.go:310]
I0920 05:04:50.405544 489259 kubeadm.go:310] export KUBECONFIG=/etc/kubernetes/admin.conf
I0920 05:04:50.405547 489259 kubeadm.go:310]
I0920 05:04:50.405600 489259 kubeadm.go:310] You should now deploy a pod network to the cluster.
I0920 05:04:50.405677 489259 kubeadm.go:310] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
I0920 05:04:50.405764 489259 kubeadm.go:310] https://kubernetes.io/docs/concepts/cluster-administration/addons/
I0920 05:04:50.405768 489259 kubeadm.go:310]
I0920 05:04:50.405855 489259 kubeadm.go:310] You can now join any number of control-plane nodes by copying certificate authorities
I0920 05:04:50.405934 489259 kubeadm.go:310] and service account keys on each node and then running the following as root:
I0920 05:04:50.405938 489259 kubeadm.go:310]
I0920 05:04:50.406035 489259 kubeadm.go:310] kubeadm join control-plane.minikube.internal:8443 --token 52m51f.2xxnk5wiv4jjcr4b \
I0920 05:04:50.406149 489259 kubeadm.go:310] --discovery-token-ca-cert-hash sha256:97ed4913290d2312e940f6907d6b2690a0aa1e9107428762234b5abfe7675247 \
I0920 05:04:50.406169 489259 kubeadm.go:310] --control-plane
I0920 05:04:50.406177 489259 kubeadm.go:310]
I0920 05:04:50.406265 489259 kubeadm.go:310] Then you can join any number of worker nodes by running the following on each as root:
I0920 05:04:50.406268 489259 kubeadm.go:310]
I0920 05:04:50.406352 489259 kubeadm.go:310] kubeadm join control-plane.minikube.internal:8443 --token 52m51f.2xxnk5wiv4jjcr4b \
I0920 05:04:50.406458 489259 kubeadm.go:310] --discovery-token-ca-cert-hash sha256:97ed4913290d2312e940f6907d6b2690a0aa1e9107428762234b5abfe7675247
I0920 05:04:50.409926 489259 kubeadm.go:310] [WARNING SystemVerification]: cgroups v1 support is in maintenance mode, please migrate to cgroups v2
I0920 05:04:50.410154 489259 kubeadm.go:310] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1084-aws\n", err: exit status 1
I0920 05:04:50.410263 489259 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0920 05:04:50.410277 489259 cni.go:84] Creating CNI manager for ""
I0920 05:04:50.410284 489259 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0920 05:04:50.415455 489259 out.go:179] * Configuring CNI (Container Networking Interface) ...
I0920 05:04:50.418436 489259 ssh_runner.go:195] Run: stat /opt/cni/bin/portmap
I0920 05:04:50.422581 489259 cni.go:182] applying CNI manifest using /var/lib/minikube/binaries/v1.34.0/kubectl ...
I0920 05:04:50.422592 489259 ssh_runner.go:362] scp memory --> /var/tmp/minikube/cni.yaml (2601 bytes)
I0920 05:04:50.443175 489259 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.34.0/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml
I0920 05:04:50.908426 489259 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"
I0920 05:04:50.908552 489259 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.34.0/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig
I0920 05:04:50.908642 489259 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.34.0/kubectl --kubeconfig=/var/lib/minikube/kubeconfig label --overwrite nodes cert-expiration-920996 minikube.k8s.io/updated_at=2025_09_20T05_04_50_0700 minikube.k8s.io/version=v1.37.0 minikube.k8s.io/commit=e00824f82cad947a0032e69b00336ba0de61731f minikube.k8s.io/name=cert-expiration-920996 minikube.k8s.io/primary=true
I0920 05:04:51.056191 489259 ops.go:34] apiserver oom_adj: -16
I0920 05:04:51.056211 489259 kubeadm.go:1105] duration metric: took 147.714074ms to wait for elevateKubeSystemPrivileges
I0920 05:04:51.056227 489259 kubeadm.go:394] duration metric: took 18.8671725s to StartCluster
I0920 05:04:51.056242 489259 settings.go:142] acquiring lock: {Name:mk0431f8e6fad2f77306cccf2f2d56ababc5b1b1 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 05:04:51.056314 489259 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/21599-293644/kubeconfig
I0920 05:04:51.057266 489259 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/21599-293644/kubeconfig: {Name:mka409452b1806eb89337f306620696a06e9e870 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 05:04:51.057538 489259 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"
I0920 05:04:51.057550 489259 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.34.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0920 05:04:51.057783 489259 config.go:182] Loaded profile config "cert-expiration-920996": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.34.0
I0920 05:04:51.057813 489259 addons.go:511] enable addons start: toEnable=map[ambassador:false amd-gpu-device-plugin:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubetail:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0920 05:04:51.057873 489259 addons.go:69] Setting storage-provisioner=true in profile "cert-expiration-920996"
I0920 05:04:51.057891 489259 addons.go:238] Setting addon storage-provisioner=true in "cert-expiration-920996"
I0920 05:04:51.057912 489259 host.go:66] Checking if "cert-expiration-920996" exists ...
I0920 05:04:51.058359 489259 cli_runner.go:164] Run: docker container inspect cert-expiration-920996 --format={{.State.Status}}
I0920 05:04:51.058808 489259 addons.go:69] Setting default-storageclass=true in profile "cert-expiration-920996"
I0920 05:04:51.058823 489259 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "cert-expiration-920996"
I0920 05:04:51.059090 489259 cli_runner.go:164] Run: docker container inspect cert-expiration-920996 --format={{.State.Status}}
I0920 05:04:51.063232 489259 out.go:179] * Verifying Kubernetes components...
I0920 05:04:51.067455 489259 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0920 05:04:51.097740 489259 out.go:179] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0920 05:04:51.103318 489259 addons.go:435] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0920 05:04:51.103330 489259 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0920 05:04:51.103411 489259 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" cert-expiration-920996
I0920 05:04:51.105758 489259 addons.go:238] Setting addon default-storageclass=true in "cert-expiration-920996"
I0920 05:04:51.105790 489259 host.go:66] Checking if "cert-expiration-920996" exists ...
I0920 05:04:51.106257 489259 cli_runner.go:164] Run: docker container inspect cert-expiration-920996 --format={{.State.Status}}
I0920 05:04:51.147958 489259 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33413 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/cert-expiration-920996/id_rsa Username:docker}
I0920 05:04:51.150489 489259 addons.go:435] installing /etc/kubernetes/addons/storageclass.yaml
I0920 05:04:51.150503 489259 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0920 05:04:51.150568 489259 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" cert-expiration-920996
I0920 05:04:51.186668 489259 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33413 SSHKeyPath:/home/jenkins/minikube-integration/21599-293644/.minikube/machines/cert-expiration-920996/id_rsa Username:docker}
I0920 05:04:51.287882 489259 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^ forward . \/etc\/resolv.conf.*/i \ hosts {\n 192.168.85.1 host.minikube.internal\n fallthrough\n }' -e '/^ errors *$/i \ log' | sudo /var/lib/minikube/binaries/v1.34.0/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"
I0920 05:04:51.287995 489259 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0920 05:04:51.424799 489259 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0920 05:04:51.442246 489259 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0920 05:04:51.687775 489259 start.go:976] {"host.minikube.internal": 192.168.85.1} host record injected into CoreDNS's ConfigMap
I0920 05:04:51.690214 489259 api_server.go:52] waiting for apiserver process to appear ...
I0920 05:04:51.690272 489259 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0920 05:04:51.955768 489259 api_server.go:72] duration metric: took 898.193802ms to wait for apiserver process to appear ...
I0920 05:04:51.955780 489259 api_server.go:88] waiting for apiserver healthz status ...
I0920 05:04:51.955797 489259 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0920 05:04:51.969454 489259 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0920 05:04:51.970627 489259 api_server.go:141] control plane version: v1.34.0
I0920 05:04:51.970642 489259 api_server.go:131] duration metric: took 14.857158ms to wait for apiserver health ...
I0920 05:04:51.970649 489259 system_pods.go:43] waiting for kube-system pods to appear ...
I0920 05:04:51.975447 489259 system_pods.go:59] 5 kube-system pods found
I0920 05:04:51.975467 489259 system_pods.go:61] "etcd-cert-expiration-920996" [c4f03f85-6b56-4189-8237-7afb07e5d82f] Running / Ready:ContainersNotReady (containers with unready status: [etcd]) / ContainersReady:ContainersNotReady (containers with unready status: [etcd])
I0920 05:04:51.975475 489259 system_pods.go:61] "kube-apiserver-cert-expiration-920996" [5a01972d-667d-4680-a876-6d02979104fe] Running / Ready:ContainersNotReady (containers with unready status: [kube-apiserver]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-apiserver])
I0920 05:04:51.975482 489259 system_pods.go:61] "kube-controller-manager-cert-expiration-920996" [f9898b8a-207b-4f81-b83d-b87ecb1f4602] Running / Ready:ContainersNotReady (containers with unready status: [kube-controller-manager]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-controller-manager])
I0920 05:04:51.975488 489259 system_pods.go:61] "kube-scheduler-cert-expiration-920996" [684dc206-d4c3-45c1-b84a-0969335746cc] Running / Ready:ContainersNotReady (containers with unready status: [kube-scheduler]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-scheduler])
I0920 05:04:51.975493 489259 system_pods.go:61] "storage-provisioner" [f3287423-1531-4e78-8f0d-b2213b2e7cb4] Pending: PodScheduled:Unschedulable (0/1 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/not-ready: }. no new claims to deallocate, preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.)
I0920 05:04:51.975499 489259 system_pods.go:74] duration metric: took 4.844581ms to wait for pod list to return data ...
I0920 05:04:51.975520 489259 kubeadm.go:578] duration metric: took 917.938897ms to wait for: map[apiserver:true system_pods:true]
I0920 05:04:51.975532 489259 node_conditions.go:102] verifying NodePressure condition ...
I0920 05:04:51.975885 489259 out.go:179] * Enabled addons: storage-provisioner, default-storageclass
I0920 05:04:51.978456 489259 node_conditions.go:122] node storage ephemeral capacity is 203034800Ki
I0920 05:04:51.978473 489259 node_conditions.go:123] node cpu capacity is 2
I0920 05:04:51.978484 489259 node_conditions.go:105] duration metric: took 2.948201ms to run NodePressure ...
I0920 05:04:51.978497 489259 start.go:241] waiting for startup goroutines ...
I0920 05:04:51.978667 489259 addons.go:514] duration metric: took 920.840609ms for enable addons: enabled=[storage-provisioner default-storageclass]
I0920 05:04:52.192334 489259 kapi.go:214] "coredns" deployment in "kube-system" namespace and "cert-expiration-920996" context rescaled to 1 replicas
I0920 05:04:52.192366 489259 start.go:246] waiting for cluster config update ...
I0920 05:04:52.192377 489259 start.go:255] writing updated cluster config ...
I0920 05:04:52.192670 489259 ssh_runner.go:195] Run: rm -f paused
I0920 05:04:52.248936 489259 start.go:623] kubectl: 1.33.2, cluster: 1.34.0 (minor skew: 1)
I0920 05:04:52.252117 489259 out.go:179] * Done! kubectl is now configured to use "cert-expiration-920996" cluster and "default" namespace by default
I0920 05:05:35.540876 467239 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (6m0.777186086s)
W0920 05:05:35.540920 467239 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "storage-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get serviceaccounts storage-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding"
Name: "storage-provisioner", Namespace: ""
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get clusterrolebindings.rbac.authorization.k8s.io storage-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=roles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=Role"
Name: "system:persistent-volume-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get roles.rbac.authorization.k8s.io system:persistent-volume-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=rolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=RoleBinding"
Name: "system:persistent-volume-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get rolebindings.rbac.authorization.k8s.io system:persistent-volume-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=endpoints", GroupVersionKind: "/v1, Kind=Endpoints"
Name: "k8s.io-minikube-hostpath", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=pods", GroupVersionKind: "/v1, Kind=Pod"
Name: "storage-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get pods storage-provisioner)
I0920 05:05:35.540926 467239 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (5m0.022575299s)
W0920 05:05:35.540953 467239 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "storage.k8s.io/v1, Resource=storageclasses", GroupVersionKind: "storage.k8s.io/v1, Kind=StorageClass"
Name: "standard", Namespace: ""
from server for: "/etc/kubernetes/addons/storageclass.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get storageclasses.storage.k8s.io standard)
W0920 05:05:35.541015 467239 out.go:285] ! Enabling 'default-storageclass' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "storage.k8s.io/v1, Resource=storageclasses", GroupVersionKind: "storage.k8s.io/v1, Kind=StorageClass"
Name: "standard", Namespace: ""
from server for: "/etc/kubernetes/addons/storageclass.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get storageclasses.storage.k8s.io standard)
]
W0920 05:05:35.541015 467239 out.go:285] ! Enabling 'storage-provisioner' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.34.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "storage-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get serviceaccounts storage-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding"
Name: "storage-provisioner", Namespace: ""
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get clusterrolebindings.rbac.authorization.k8s.io storage-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=roles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=Role"
Name: "system:persistent-volume-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get roles.rbac.authorization.k8s.io system:persistent-volume-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=rolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=RoleBinding"
Name: "system:persistent-volume-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get rolebindings.rbac.authorization.k8s.io system:persistent-volume-provisioner)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=endpoints", GroupVersionKind: "/v1, Kind=Endpoints"
Name: "k8s.io-minikube-hostpath", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
Error from server (Timeout): error when retrieving current configuration of:
Resource: "/v1, Resource=pods", GroupVersionKind: "/v1, Kind=Pod"
Name: "storage-provisioner", Namespace: "kube-system"
from server for: "/etc/kubernetes/addons/storage-provisioner.yaml": the server was unable to return a response in the time allotted, but may still be processing the request (get pods storage-provisioner)
]
I0920 05:05:35.541413 467239 ssh_runner.go:235] Completed: sudo crictl ps -a --quiet --name=kube-apiserver: (4m59.753506736s)
I0920 05:05:35.541438 467239 cri.go:89] found id: "7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921"
I0920 05:05:35.541445 467239 cri.go:89] found id: ""
I0920 05:05:35.541452 467239 logs.go:282] 1 containers: [7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921]
I0920 05:05:35.541522 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.545232 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0920 05:05:35.545311 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0920 05:05:35.546030 467239 out.go:179] * Enabled addons:
I0920 05:05:35.549039 467239 addons.go:514] duration metric: took 6m1.091166443s for enable addons: enabled=[]
I0920 05:05:35.581548 467239 cri.go:89] found id: "af627e9f5bff0f7a5c022832e92c94112890fbf9d88d0f3c977d84b13ddfa93e"
I0920 05:05:35.581570 467239 cri.go:89] found id: ""
I0920 05:05:35.581578 467239 logs.go:282] 1 containers: [af627e9f5bff0f7a5c022832e92c94112890fbf9d88d0f3c977d84b13ddfa93e]
I0920 05:05:35.581635 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.585239 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0920 05:05:35.585317 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0920 05:05:35.625494 467239 cri.go:89] found id: "c431baba84177c8f678eee3d7d1a75e48913abadcc4afd030f92b029ee9ca136"
I0920 05:05:35.625517 467239 cri.go:89] found id: ""
I0920 05:05:35.625526 467239 logs.go:282] 1 containers: [c431baba84177c8f678eee3d7d1a75e48913abadcc4afd030f92b029ee9ca136]
I0920 05:05:35.625582 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.629130 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0920 05:05:35.629207 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0920 05:05:35.666922 467239 cri.go:89] found id: "9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714"
I0920 05:05:35.666944 467239 cri.go:89] found id: ""
I0920 05:05:35.666952 467239 logs.go:282] 1 containers: [9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714]
I0920 05:05:35.667008 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.670518 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0920 05:05:35.670599 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0920 05:05:35.714743 467239 cri.go:89] found id: "fdbb732bced3687cb23143983d2939993e962365e155bd9edf617feb4b44cd8b"
I0920 05:05:35.714822 467239 cri.go:89] found id: ""
I0920 05:05:35.714862 467239 logs.go:282] 1 containers: [fdbb732bced3687cb23143983d2939993e962365e155bd9edf617feb4b44cd8b]
I0920 05:05:35.714941 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.718912 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0920 05:05:35.718987 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0920 05:05:35.757811 467239 cri.go:89] found id: "0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726"
I0920 05:05:35.757889 467239 cri.go:89] found id: ""
I0920 05:05:35.757912 467239 logs.go:282] 1 containers: [0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726]
I0920 05:05:35.757996 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.761687 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0920 05:05:35.761766 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0920 05:05:35.802323 467239 cri.go:89] found id: "331ce690dc1105bc226ee21c46805afe1dcfb4affa37785dca1bb35bc43e178f"
I0920 05:05:35.802388 467239 cri.go:89] found id: ""
I0920 05:05:35.802411 467239 logs.go:282] 1 containers: [331ce690dc1105bc226ee21c46805afe1dcfb4affa37785dca1bb35bc43e178f]
I0920 05:05:35.802504 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.806260 467239 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0920 05:05:35.806357 467239 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0920 05:05:35.846466 467239 cri.go:89] found id: "da1c4fe4a0b3a07a3ea6e372ba61c81eaff9776ad30d0b28e04686d21757f663"
I0920 05:05:35.846493 467239 cri.go:89] found id: ""
I0920 05:05:35.846502 467239 logs.go:282] 1 containers: [da1c4fe4a0b3a07a3ea6e372ba61c81eaff9776ad30d0b28e04686d21757f663]
I0920 05:05:35.846592 467239 ssh_runner.go:195] Run: which crictl
I0920 05:05:35.850828 467239 logs.go:123] Gathering logs for kube-apiserver [7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921] ...
I0920 05:05:35.850857 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921"
W0920 05:05:35.896131 467239 logs.go:130] failed kube-apiserver [7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921]: command: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921" /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921": Process exited with status 1
stdout:
stderr:
E0920 05:05:35.893210 4823 remote_runtime.go:625] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921\": not found" containerID="7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921"
time="2025-09-20T05:05:35Z" level=fatal msg="rpc error: code = NotFound desc = an error occurred when try to find container \"7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921\": not found"
output:
** stderr **
E0920 05:05:35.893210 4823 remote_runtime.go:625] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921\": not found" containerID="7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921"
time="2025-09-20T05:05:35Z" level=fatal msg="rpc error: code = NotFound desc = an error occurred when try to find container \"7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921\": not found"
** /stderr **
I0920 05:05:35.896172 467239 logs.go:123] Gathering logs for etcd [af627e9f5bff0f7a5c022832e92c94112890fbf9d88d0f3c977d84b13ddfa93e] ...
I0920 05:05:35.896202 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 af627e9f5bff0f7a5c022832e92c94112890fbf9d88d0f3c977d84b13ddfa93e"
I0920 05:05:35.961120 467239 logs.go:123] Gathering logs for kube-proxy [fdbb732bced3687cb23143983d2939993e962365e155bd9edf617feb4b44cd8b] ...
I0920 05:05:35.961159 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fdbb732bced3687cb23143983d2939993e962365e155bd9edf617feb4b44cd8b"
I0920 05:05:36.015874 467239 logs.go:123] Gathering logs for containerd ...
I0920 05:05:36.015906 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0920 05:05:36.092346 467239 logs.go:123] Gathering logs for dmesg ...
I0920 05:05:36.092382 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0920 05:05:36.115263 467239 logs.go:123] Gathering logs for coredns [c431baba84177c8f678eee3d7d1a75e48913abadcc4afd030f92b029ee9ca136] ...
I0920 05:05:36.115296 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c431baba84177c8f678eee3d7d1a75e48913abadcc4afd030f92b029ee9ca136"
I0920 05:05:36.155323 467239 logs.go:123] Gathering logs for kube-scheduler [9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714] ...
I0920 05:05:36.155352 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714"
W0920 05:05:36.194013 467239 logs.go:138] Found kube-scheduler [9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714] problem: E0920 05:02:57.234215 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10259: listen tcp 127.0.0.1:10259: bind: address already in use"
I0920 05:05:36.194042 467239 logs.go:123] Gathering logs for kube-controller-manager [0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726] ...
I0920 05:05:36.194054 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726"
W0920 05:05:36.235628 467239 logs.go:138] Found kube-controller-manager [0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726] problem: E0920 05:03:00.247948 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10257: listen tcp 127.0.0.1:10257: bind: address already in use"
I0920 05:05:36.235653 467239 logs.go:123] Gathering logs for kindnet [331ce690dc1105bc226ee21c46805afe1dcfb4affa37785dca1bb35bc43e178f] ...
I0920 05:05:36.235669 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 331ce690dc1105bc226ee21c46805afe1dcfb4affa37785dca1bb35bc43e178f"
I0920 05:05:36.281086 467239 logs.go:123] Gathering logs for storage-provisioner [da1c4fe4a0b3a07a3ea6e372ba61c81eaff9776ad30d0b28e04686d21757f663] ...
I0920 05:05:36.281117 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 da1c4fe4a0b3a07a3ea6e372ba61c81eaff9776ad30d0b28e04686d21757f663"
I0920 05:05:36.321993 467239 logs.go:123] Gathering logs for container status ...
I0920 05:05:36.322024 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0920 05:05:36.364652 467239 logs.go:123] Gathering logs for kubelet ...
I0920 05:05:36.364683 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0920 05:05:36.490563 467239 logs.go:123] Gathering logs for describe nodes ...
I0920 05:05:36.490602 467239 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0920 05:06:36.593529 467239 ssh_runner.go:235] Completed: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": (1m0.102902199s)
W0920 05:06:36.593569 467239 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
Error from server (Timeout): the server was unable to return a response in the time allotted, but may still be processing the request (get nodes)
output:
** stderr **
Error from server (Timeout): the server was unable to return a response in the time allotted, but may still be processing the request (get nodes)
** /stderr **
I0920 05:06:36.593589 467239 out.go:374] Setting ErrFile to fd 2...
I0920 05:06:36.593600 467239 out.go:408] TERM=,COLORTERM=, which probably does not support color
W0920 05:06:36.593658 467239 out.go:285] X Problems detected in kube-scheduler [9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714]:
W0920 05:06:36.593669 467239 out.go:285] E0920 05:02:57.234215 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10259: listen tcp 127.0.0.1:10259: bind: address already in use"
W0920 05:06:36.593688 467239 out.go:285] X Problems detected in kube-controller-manager [0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726]:
W0920 05:06:36.593696 467239 out.go:285] E0920 05:03:00.247948 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10257: listen tcp 127.0.0.1:10257: bind: address already in use"
I0920 05:06:36.593700 467239 out.go:374] Setting ErrFile to fd 2...
I0920 05:06:36.593705 467239 out.go:408] TERM=,COLORTERM=, which probably does not support color
I0920 05:06:46.597081 467239 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 05:06:51.601616 467239 api_server.go:269] stopped: https://192.168.76.2:8443/healthz: Get "https://192.168.76.2:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I0920 05:06:51.604903 467239 out.go:203]
W0920 05:06:51.608026 467239 out.go:285] X Exiting due to GUEST_START: failed to start node: wait 6m0s for node: wait for healthy API server: apiserver healthz never reported healthy: context deadline exceeded
W0920 05:06:51.608053 467239 out.go:285] *
W0920 05:06:51.610872 467239 out.go:308] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0920 05:06:51.613768 467239 out.go:203]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
0490dec2e72af 996be7e86d9b3 About a minute ago Exited kube-controller-manager 7 2fde8768449fd kube-controller-manager-kubernetes-upgrade-068009
4b0175c9605c0 a25f5ef9c34c3 About a minute ago Exited kube-scheduler 7 dd93267c89f79 kube-scheduler-kubernetes-upgrade-068009
9ffc7ac2177d2 d291939e99406 About a minute ago Exited kube-apiserver 7 0bf671a5b6a59 kube-apiserver-kubernetes-upgrade-068009
da1c4fe4a0b3a 66749159455b3 6 minutes ago Running storage-provisioner 1 194ea9f80cdd1 storage-provisioner
c431baba84177 138784d87c9c5 6 minutes ago Running coredns 0 c7bd7affa3a38 coredns-66bc5c9577-vr9r2
af627e9f5bff0 a1894772a478e 6 minutes ago Running etcd 0 c73a398bd88ab etcd-kubernetes-upgrade-068009
fdbb732bced36 6fc32d66c1411 7 minutes ago Running kube-proxy 0 07be619a00523 kube-proxy-d9z5s
331ce690dc110 b1a8c6f707935 7 minutes ago Running kindnet-cni 0 fe4d8f7ba609d kindnet-q85xj
==> containerd <==
Sep 20 05:05:16 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:16.787951576Z" level=warning msg="cleaning up after shim disconnected" id=9ffc7ac2177d215a296fc2b47aee806e280f63978e2d5576c63b070226db3657 namespace=k8s.io
Sep 20 05:05:16 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:16.787988204Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 20 05:05:16 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:16.841782479Z" level=info msg="RemoveContainer for \"f1f1b61c84b77b630621ab3886794534d88373d757743fac2b99c8b3ef88a3ec\""
Sep 20 05:05:16 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:16.848118483Z" level=info msg="RemoveContainer for \"f1f1b61c84b77b630621ab3886794534d88373d757743fac2b99c8b3ef88a3ec\" returns successfully"
Sep 20 05:05:35 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:35.888767707Z" level=error msg="ContainerStatus for \"7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find container \"7b840d53e1e169e0e349b3c23035f0e0cf1649546556dc4ccc3ea8f470271921\": not found"
Sep 20 05:05:41 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:41.594677046Z" level=info msg="CreateContainer within sandbox \"dd93267c89f7905d1cddf747c73610d85257d2fe9c73b0305a80e2e71904ca27\" for container &ContainerMetadata{Name:kube-scheduler,Attempt:7,}"
Sep 20 05:05:41 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:41.614360824Z" level=info msg="CreateContainer within sandbox \"dd93267c89f7905d1cddf747c73610d85257d2fe9c73b0305a80e2e71904ca27\" for &ContainerMetadata{Name:kube-scheduler,Attempt:7,} returns container id \"4b0175c9605c0f8d9be09d5992d8b324ccc8146d0ab6f703ddf0971927478f2f\""
Sep 20 05:05:41 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:41.615114125Z" level=info msg="StartContainer for \"4b0175c9605c0f8d9be09d5992d8b324ccc8146d0ab6f703ddf0971927478f2f\""
Sep 20 05:05:41 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:41.691690498Z" level=info msg="StartContainer for \"4b0175c9605c0f8d9be09d5992d8b324ccc8146d0ab6f703ddf0971927478f2f\" returns successfully"
Sep 20 05:05:43 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:43.668015205Z" level=info msg="received exit event container_id:\"4b0175c9605c0f8d9be09d5992d8b324ccc8146d0ab6f703ddf0971927478f2f\" id:\"4b0175c9605c0f8d9be09d5992d8b324ccc8146d0ab6f703ddf0971927478f2f\" pid:4927 exit_status:1 exited_at:{seconds:1758344743 nanos:667630112}"
Sep 20 05:05:43 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:43.691800424Z" level=info msg="shim disconnected" id=4b0175c9605c0f8d9be09d5992d8b324ccc8146d0ab6f703ddf0971927478f2f namespace=k8s.io
Sep 20 05:05:43 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:43.691845594Z" level=warning msg="cleaning up after shim disconnected" id=4b0175c9605c0f8d9be09d5992d8b324ccc8146d0ab6f703ddf0971927478f2f namespace=k8s.io
Sep 20 05:05:43 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:43.691882082Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 20 05:05:43 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:43.915048090Z" level=info msg="RemoveContainer for \"9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714\""
Sep 20 05:05:43 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:43.921343061Z" level=info msg="RemoveContainer for \"9a8a5122e53603607e531db5a6ca9435a4f64020cd78b953a67b2d3c6bd08714\" returns successfully"
Sep 20 05:05:44 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:44.593005162Z" level=info msg="CreateContainer within sandbox \"2fde8768449fdf70f3393ad7e3dffb896ed6684d5695ae59992766196aa7cae0\" for container &ContainerMetadata{Name:kube-controller-manager,Attempt:7,}"
Sep 20 05:05:44 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:44.612690361Z" level=info msg="CreateContainer within sandbox \"2fde8768449fdf70f3393ad7e3dffb896ed6684d5695ae59992766196aa7cae0\" for &ContainerMetadata{Name:kube-controller-manager,Attempt:7,} returns container id \"0490dec2e72afd3858b78fc47322493daa2d2645374c6028baf3b33bbe6b44b4\""
Sep 20 05:05:44 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:44.613287419Z" level=info msg="StartContainer for \"0490dec2e72afd3858b78fc47322493daa2d2645374c6028baf3b33bbe6b44b4\""
Sep 20 05:05:44 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:44.685496465Z" level=info msg="StartContainer for \"0490dec2e72afd3858b78fc47322493daa2d2645374c6028baf3b33bbe6b44b4\" returns successfully"
Sep 20 05:05:45 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:45.787072129Z" level=info msg="received exit event container_id:\"0490dec2e72afd3858b78fc47322493daa2d2645374c6028baf3b33bbe6b44b4\" id:\"0490dec2e72afd3858b78fc47322493daa2d2645374c6028baf3b33bbe6b44b4\" pid:4996 exit_status:1 exited_at:{seconds:1758344745 nanos:786776898}"
Sep 20 05:05:45 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:45.810571209Z" level=info msg="shim disconnected" id=0490dec2e72afd3858b78fc47322493daa2d2645374c6028baf3b33bbe6b44b4 namespace=k8s.io
Sep 20 05:05:45 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:45.810775748Z" level=warning msg="cleaning up after shim disconnected" id=0490dec2e72afd3858b78fc47322493daa2d2645374c6028baf3b33bbe6b44b4 namespace=k8s.io
Sep 20 05:05:45 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:45.810887084Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 20 05:05:45 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:45.928351110Z" level=info msg="RemoveContainer for \"0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726\""
Sep 20 05:05:45 kubernetes-upgrade-068009 containerd[2141]: time="2025-09-20T05:05:45.934311647Z" level=info msg="RemoveContainer for \"0bc19c2ebc6624b3e0660740866265c54f81354ddd7a4aa3fdbfac45af274726\" returns successfully"
==> coredns [c431baba84177c8f678eee3d7d1a75e48913abadcc4afd030f92b029ee9ca136] <==
maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined
.:53
[INFO] plugin/reload: Running configuration SHA512 = b7aacdf6a6aa730aafe4d018cac9b7b5ecfb346cba84a99f64521f87aef8b4958639c1cf97967716465791d05bd38f372615327b7cb1d93c850bae532744d54d
CoreDNS-1.12.1
linux/arm64, go1.24.1, 707c7c1
[INFO] 127.0.0.1:35952 - 47707 "HINFO IN 1140853464744900748.5647714234780922592. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.026246315s
==> describe nodes <==
command /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" failed with error: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.34.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
Error from server (Timeout): the server was unable to return a response in the time allotted, but may still be processing the request (get nodes)
==> dmesg <==
[Sep20 04:20] kauditd_printk_skb: 8 callbacks suppressed
[Sep20 05:00] overlayfs: lowerdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
[ +0.000010] overlayfs: lowerdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
[ +0.000009] overlayfs: lowerdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
[ +0.001451] overlayfs: lowerdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
[ +0.000008] overlayfs: lowerdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
[ +0.000004] overlayfs: lowerdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
[ +0.022501] overlayfs: lowerdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
[ +0.000008] overlayfs: lowerdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
[ +0.000003] overlayfs: lowerdir is in-use as upperdir/workdir of another mount, accessing files from both mounts will result in undefined behavior.
==> etcd [af627e9f5bff0f7a5c022832e92c94112890fbf9d88d0f3c977d84b13ddfa93e] <==
{"level":"info","ts":"2025-09-20T05:00:10.997415Z","logger":"raft","caller":"v3@v3.6.0/raft.go:988","msg":"ea7e25599daad906 is starting a new election at term 3"}
{"level":"info","ts":"2025-09-20T05:00:10.997519Z","logger":"raft","caller":"v3@v3.6.0/raft.go:930","msg":"ea7e25599daad906 became pre-candidate at term 3"}
{"level":"info","ts":"2025-09-20T05:00:10.997591Z","logger":"raft","caller":"v3@v3.6.0/raft.go:1077","msg":"ea7e25599daad906 received MsgPreVoteResp from ea7e25599daad906 at term 3"}
{"level":"info","ts":"2025-09-20T05:00:10.997650Z","logger":"raft","caller":"v3@v3.6.0/raft.go:1693","msg":"ea7e25599daad906 has received 1 MsgPreVoteResp votes and 0 vote rejections"}
{"level":"info","ts":"2025-09-20T05:00:10.997689Z","logger":"raft","caller":"v3@v3.6.0/raft.go:912","msg":"ea7e25599daad906 became candidate at term 4"}
{"level":"info","ts":"2025-09-20T05:00:11.001004Z","logger":"raft","caller":"v3@v3.6.0/raft.go:1077","msg":"ea7e25599daad906 received MsgVoteResp from ea7e25599daad906 at term 4"}
{"level":"info","ts":"2025-09-20T05:00:11.001087Z","logger":"raft","caller":"v3@v3.6.0/raft.go:1693","msg":"ea7e25599daad906 has received 1 MsgVoteResp votes and 0 vote rejections"}
{"level":"info","ts":"2025-09-20T05:00:11.001128Z","logger":"raft","caller":"v3@v3.6.0/raft.go:970","msg":"ea7e25599daad906 became leader at term 4"}
{"level":"info","ts":"2025-09-20T05:00:11.001158Z","logger":"raft","caller":"v3@v3.6.0/node.go:370","msg":"raft.node: ea7e25599daad906 elected leader ea7e25599daad906 at term 4"}
{"level":"info","ts":"2025-09-20T05:00:11.002442Z","caller":"etcdserver/server.go:1804","msg":"published local member to cluster through raft","local-member-id":"ea7e25599daad906","local-member-attributes":"{Name:kubernetes-upgrade-068009 ClientURLs:[https://192.168.76.2:2379]}","cluster-id":"6f20f2c4b2fb5f8a","publish-timeout":"7s"}
{"level":"info","ts":"2025-09-20T05:00:11.002708Z","caller":"embed/serve.go:138","msg":"ready to serve client requests"}
{"level":"info","ts":"2025-09-20T05:00:11.003071Z","caller":"etcdserver/server.go:2409","msg":"updating cluster version using v3 API","from":"3.5","to":"3.6"}
{"level":"info","ts":"2025-09-20T05:00:11.003260Z","caller":"embed/serve.go:138","msg":"ready to serve client requests"}
{"level":"info","ts":"2025-09-20T05:00:11.003439Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}
{"level":"info","ts":"2025-09-20T05:00:11.003497Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}
{"level":"info","ts":"2025-09-20T05:00:11.012981Z","caller":"membership/cluster.go:674","msg":"updated cluster version","cluster-id":"6f20f2c4b2fb5f8a","local-member-id":"ea7e25599daad906","from":"3.5","to":"3.6"}
{"level":"info","ts":"2025-09-20T05:00:11.013145Z","caller":"api/capability.go:76","msg":"enabled capabilities for version","cluster-version":"3.6"}
{"level":"info","ts":"2025-09-20T05:00:11.013226Z","caller":"etcdserver/server.go:2424","msg":"cluster version is updated","cluster-version":"3.6"}
{"level":"info","ts":"2025-09-20T05:00:11.013305Z","caller":"version/monitor.go:116","msg":"cluster version differs from storage version.","cluster-version":"3.6.0","storage-version":"3.5.0"}
{"level":"info","ts":"2025-09-20T05:00:11.017543Z","caller":"schema/migration.go:65","msg":"updated storage version","new-storage-version":"3.6.0"}
{"level":"info","ts":"2025-09-20T05:00:11.018449Z","caller":"v3rpc/health.go:63","msg":"grpc service status changed","service":"","status":"SERVING"}
{"level":"warn","ts":"2025-09-20T05:00:11.019527Z","caller":"v3rpc/grpc.go:52","msg":"etcdserver: failed to register grpc metrics","error":"duplicate metrics collector registration attempted"}
{"level":"info","ts":"2025-09-20T05:00:11.024544Z","caller":"v3rpc/health.go:63","msg":"grpc service status changed","service":"","status":"SERVING"}
{"level":"info","ts":"2025-09-20T05:00:11.055291Z","caller":"embed/serve.go:283","msg":"serving client traffic securely","traffic":"grpc+http","address":"127.0.0.1:2379"}
{"level":"info","ts":"2025-09-20T05:00:11.077964Z","caller":"embed/serve.go:283","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.76.2:2379"}
==> kernel <==
05:08:08 up 1:50, 0 users, load average: 0.75, 1.39, 2.12
Linux kubernetes-upgrade-068009 5.15.0-1084-aws #91~20.04.1-Ubuntu SMP Fri May 2 07:00:04 UTC 2025 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.5 LTS"
==> kindnet [331ce690dc1105bc226ee21c46805afe1dcfb4affa37785dca1bb35bc43e178f] <==
I0920 05:06:08.594807 1 main.go:301] handling current node
I0920 05:06:18.594786 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:06:18.594827 1 main.go:301] handling current node
I0920 05:06:28.594373 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:06:28.594408 1 main.go:301] handling current node
I0920 05:06:38.593989 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:06:38.594019 1 main.go:301] handling current node
I0920 05:06:48.594703 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:06:48.594891 1 main.go:301] handling current node
I0920 05:06:58.594399 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:06:58.594435 1 main.go:301] handling current node
I0920 05:07:08.594681 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:07:08.594717 1 main.go:301] handling current node
I0920 05:07:18.594333 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:07:18.594368 1 main.go:301] handling current node
I0920 05:07:28.594389 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:07:28.594424 1 main.go:301] handling current node
I0920 05:07:38.594417 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:07:38.594456 1 main.go:301] handling current node
I0920 05:07:48.594732 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:07:48.594771 1 main.go:301] handling current node
I0920 05:07:58.594546 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:07:58.594584 1 main.go:301] handling current node
I0920 05:08:08.593906 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0920 05:08:08.593940 1 main.go:301] handling current node
==> kube-apiserver [9ffc7ac2177d215a296fc2b47aee806e280f63978e2d5576c63b070226db3657] <==
I0920 05:05:16.745635 1 options.go:263] external host was not specified, using 192.168.76.2
I0920 05:05:16.748260 1 server.go:150] Version: v1.34.0
I0920 05:05:16.748305 1 server.go:152] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
E0920 05:05:16.748630 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 0.0.0.0:8443: listen tcp 0.0.0.0:8443: bind: address already in use"
==> kube-controller-manager [0490dec2e72afd3858b78fc47322493daa2d2645374c6028baf3b33bbe6b44b4] <==
I0920 05:05:45.780324 1 serving.go:386] Generated self-signed cert in-memory
E0920 05:05:45.782169 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10257: listen tcp 127.0.0.1:10257: bind: address already in use"
==> kube-proxy [fdbb732bced3687cb23143983d2939993e962365e155bd9edf617feb4b44cd8b] <==
I0920 04:59:45.323913 1 server_linux.go:53] "Using iptables proxy"
I0920 04:59:45.436550 1 shared_informer.go:349] "Waiting for caches to sync" controller="node informer cache"
I0920 04:59:45.540535 1 shared_informer.go:356] "Caches are synced" controller="node informer cache"
I0920 04:59:45.540575 1 server.go:219] "Successfully retrieved NodeIPs" NodeIPs=["192.168.76.2"]
E0920 04:59:45.540689 1 server.go:256] "Kube-proxy configuration may be incomplete or incorrect" err="nodePortAddresses is unset; NodePort connections will be accepted on all local IPs. Consider using `--nodeport-addresses primary`"
I0920 04:59:45.608642 1 server.go:265] "kube-proxy running in dual-stack mode" primary ipFamily="IPv4"
I0920 04:59:45.608833 1 server_linux.go:132] "Using iptables Proxier"
I0920 04:59:45.613545 1 proxier.go:242] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses" ipFamily="IPv4"
I0920 04:59:45.613858 1 server.go:527] "Version info" version="v1.34.0"
I0920 04:59:45.613875 1 server.go:529] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0920 04:59:45.615364 1 config.go:200] "Starting service config controller"
I0920 04:59:45.615375 1 shared_informer.go:349] "Waiting for caches to sync" controller="service config"
I0920 04:59:45.615393 1 config.go:106] "Starting endpoint slice config controller"
I0920 04:59:45.615398 1 shared_informer.go:349] "Waiting for caches to sync" controller="endpoint slice config"
I0920 04:59:45.615412 1 config.go:403] "Starting serviceCIDR config controller"
I0920 04:59:45.615423 1 shared_informer.go:349] "Waiting for caches to sync" controller="serviceCIDR config"
I0920 04:59:45.621536 1 config.go:309] "Starting node config controller"
I0920 04:59:45.621562 1 shared_informer.go:349] "Waiting for caches to sync" controller="node config"
I0920 04:59:45.621580 1 shared_informer.go:356] "Caches are synced" controller="node config"
I0920 04:59:45.719735 1 shared_informer.go:356] "Caches are synced" controller="serviceCIDR config"
I0920 04:59:45.719784 1 shared_informer.go:356] "Caches are synced" controller="service config"
I0920 04:59:45.719831 1 shared_informer.go:356] "Caches are synced" controller="endpoint slice config"
E0920 05:00:19.626310 1 event_broadcaster.go:270] "Server rejected event (will not retry!)" err="Timeout: request did not complete within requested timeout - context deadline exceeded" event="&Event{ObjectMeta:{kubernetes-upgrade-068009.1866e4e71161a126 default 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},EventTime:2025-09-20 04:59:45.614945613 +0000 UTC m=+0.414992788,Series:nil,ReportingController:kube-proxy,ReportingInstance:kube-proxy-kubernetes-upgrade-068009,Action:StartKubeProxy,Reason:Starting,Regarding:{Node kubernetes-upgrade-068009 kubernetes-upgrade-068009 },Related:nil,Note:,Type:Normal,DeprecatedSource:{ },DeprecatedFirstTimestamp:0001-01-01 00:00:00 +0000 UTC,DeprecatedLastTimestamp:0001-01-01 00:00:00 +0000 UTC,DeprecatedCount:0,}"
==> kube-scheduler [4b0175c9605c0f8d9be09d5992d8b324ccc8146d0ab6f703ddf0971927478f2f] <==
I0920 05:05:43.662116 1 serving.go:386] Generated self-signed cert in-memory
E0920 05:05:43.663306 1 run.go:72] "command failed" err="failed to create listener: failed to listen on 127.0.0.1:10259: listen tcp 127.0.0.1:10259: bind: address already in use"
==> kubelet <==
Sep 20 05:07:35 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:35.944225 1075 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://control-plane.minikube.internal:8443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/kubernetes-upgrade-068009?timeout=10s\": context deadline exceeded" interval="7s"
Sep 20 05:07:36 kubernetes-upgrade-068009 kubelet[1075]: I0920 05:07:36.590965 1075 scope.go:117] "RemoveContainer" containerID="0490dec2e72afd3858b78fc47322493daa2d2645374c6028baf3b33bbe6b44b4"
Sep 20 05:07:36 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:36.591167 1075 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-controller-manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=kube-controller-manager pod=kube-controller-manager-kubernetes-upgrade-068009_kube-system(68753ed2ff04b7e77be2a6bf67dea66e)\"" pod="kube-system/kube-controller-manager-kubernetes-upgrade-068009" podUID="68753ed2ff04b7e77be2a6bf67dea66e"
Sep 20 05:07:37 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:37.937454 1075 kubelet_node_status.go:486] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2025-09-20T05:07:27Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2025-09-20T05:07:27Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2025-09-20T05:07:27Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2025-09-20T05:07:27Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"registry.k8s.io/etcd@sha256:e36c081683425b5b3bc1425bc508b37e7107bb65dfa9367bf5a80125d431fa19\\\",\\\"registry.k8s.io/etcd:3.6.4-0\\\"],\\\"sizeBytes\\\":98207481},{\\\"names\\\":[\\\"docker.io/kindest/kindnetd@sha256:07a4
b3fe0077a0ae606cc0a200fc25a28fa64dcc30b8d311b461089969449f9a\\\",\\\"docker.io/kindest/kindnetd:v20250512-df8de77b\\\"],\\\"sizeBytes\\\":40636774},{\\\"names\\\":[\\\"registry.k8s.io/kube-apiserver@sha256:fe86fe2f64021df8efa1a939a290bc21c8c128c66fc00ebbb6b5dea4c7a06812\\\",\\\"registry.k8s.io/kube-apiserver:v1.34.0\\\"],\\\"sizeBytes\\\":24570751},{\\\"names\\\":[\\\"registry.k8s.io/kube-proxy@sha256:364da8a25c742d7a35e9635cb8cf42c1faf5b367760d0f9f9a75bdd1f9d52067\\\",\\\"registry.k8s.io/kube-proxy:v1.34.0\\\"],\\\"sizeBytes\\\":22788036},{\\\"names\\\":[\\\"registry.k8s.io/kube-controller-manager@sha256:f8ba6c082136e2c85ce71628c59c6574ca4b67f162911cb200c0a51a5b9bff81\\\",\\\"registry.k8s.io/kube-controller-manager:v1.34.0\\\"],\\\"sizeBytes\\\":20720494},{\\\"names\\\":[\\\"registry.k8s.io/coredns/coredns@sha256:e8c262566636e6bc340ece6473b0eed193cad045384401529721ddbe6463d31c\\\",\\\"registry.k8s.io/coredns/coredns:v1.12.1\\\"],\\\"sizeBytes\\\":20392204},{\\\"names\\\":[\\\"registry.k8s.io/kube-scheduler@s
ha256:8fbe6d18415c8af9b31e177f6e444985f3a87349e083fe6eadd36753dddb17ff\\\",\\\"registry.k8s.io/kube-scheduler:v1.34.0\\\"],\\\"sizeBytes\\\":15779792},{\\\"names\\\":[\\\"gcr.io/k8s-minikube/storage-provisioner:v5\\\"],\\\"sizeBytes\\\":8032639},{\\\"names\\\":[\\\"registry.k8s.io/pause@sha256:278fb9dbcca9518083ad1e11276933a2e96f23de604a3a08cc3c80002767d24c\\\",\\\"registry.k8s.io/pause:3.10.1\\\"],\\\"sizeBytes\\\":267939}]}}\" for node \"kubernetes-upgrade-068009\": Patch \"https://control-plane.minikube.internal:8443/api/v1/nodes/kubernetes-upgrade-068009/status?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 20 05:07:38 kubernetes-upgrade-068009 kubelet[1075]: I0920 05:07:38.590565 1075 scope.go:117] "RemoveContainer" containerID="4b0175c9605c0f8d9be09d5992d8b324ccc8146d0ab6f703ddf0971927478f2f"
Sep 20 05:07:38 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:38.590760 1075 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-scheduler\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=kube-scheduler pod=kube-scheduler-kubernetes-upgrade-068009_kube-system(d2deb90f3760525a93376c0733b4e440)\"" pod="kube-system/kube-scheduler-kubernetes-upgrade-068009" podUID="d2deb90f3760525a93376c0733b4e440"
Sep 20 05:07:41 kubernetes-upgrade-068009 kubelet[1075]: I0920 05:07:41.590685 1075 scope.go:117] "RemoveContainer" containerID="9ffc7ac2177d215a296fc2b47aee806e280f63978e2d5576c63b070226db3657"
Sep 20 05:07:41 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:41.591352 1075 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-apiserver\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-kubernetes-upgrade-068009_kube-system(687bffc11a702480173094555b606021)\"" pod="kube-system/kube-apiserver-kubernetes-upgrade-068009" podUID="687bffc11a702480173094555b606021"
Sep 20 05:07:47 kubernetes-upgrade-068009 kubelet[1075]: I0920 05:07:47.590996 1075 scope.go:117] "RemoveContainer" containerID="0490dec2e72afd3858b78fc47322493daa2d2645374c6028baf3b33bbe6b44b4"
Sep 20 05:07:47 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:47.591286 1075 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-controller-manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=kube-controller-manager pod=kube-controller-manager-kubernetes-upgrade-068009_kube-system(68753ed2ff04b7e77be2a6bf67dea66e)\"" pod="kube-system/kube-controller-manager-kubernetes-upgrade-068009" podUID="68753ed2ff04b7e77be2a6bf67dea66e"
Sep 20 05:07:47 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:47.938728 1075 kubelet_node_status.go:486] "Error updating node status, will retry" err="error getting node \"kubernetes-upgrade-068009\": Get \"https://control-plane.minikube.internal:8443/api/v1/nodes/kubernetes-upgrade-068009?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 20 05:07:52 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:52.947762 1075 controller.go:145] "Failed to ensure lease exists, will retry" err="the server was unable to return a response in the time allotted, but may still be processing the request (get leases.coordination.k8s.io kubernetes-upgrade-068009)" interval="7s"
Sep 20 05:07:53 kubernetes-upgrade-068009 kubelet[1075]: I0920 05:07:53.591036 1075 scope.go:117] "RemoveContainer" containerID="4b0175c9605c0f8d9be09d5992d8b324ccc8146d0ab6f703ddf0971927478f2f"
Sep 20 05:07:53 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:53.591389 1075 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-scheduler\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=kube-scheduler pod=kube-scheduler-kubernetes-upgrade-068009_kube-system(d2deb90f3760525a93376c0733b4e440)\"" pod="kube-system/kube-scheduler-kubernetes-upgrade-068009" podUID="d2deb90f3760525a93376c0733b4e440"
Sep 20 05:07:53 kubernetes-upgrade-068009 kubelet[1075]: I0920 05:07:53.591913 1075 scope.go:117] "RemoveContainer" containerID="9ffc7ac2177d215a296fc2b47aee806e280f63978e2d5576c63b070226db3657"
Sep 20 05:07:53 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:53.592198 1075 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-apiserver\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-kubernetes-upgrade-068009_kube-system(687bffc11a702480173094555b606021)\"" pod="kube-system/kube-apiserver-kubernetes-upgrade-068009" podUID="687bffc11a702480173094555b606021"
Sep 20 05:07:54 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:54.979078 1075 event.go:359] "Server rejected event (will not retry!)" err="Timeout: request did not complete within requested timeout - context deadline exceeded" event="&Event{ObjectMeta:{kube-apiserver-kubernetes-upgrade-068009.1866e4df43e601b6 kube-system 400 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-kubernetes-upgrade-068009,UID:687bffc11a702480173094555b606021,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-apiserver},},Reason:Started,Message:Started container kube-apiserver,Source:EventSource{Component:kubelet,Host:kubernetes-upgrade-068009,},FirstTimestamp:2025-09-20 04:59:12 +0000 UTC,LastTimestamp:2025-09-20 04:59:27.205722022 +0000 UTC m=+23.866132565,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:kubernetes-u
pgrade-068009,}"
Sep 20 05:07:57 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:07:57.939854 1075 kubelet_node_status.go:486] "Error updating node status, will retry" err="error getting node \"kubernetes-upgrade-068009\": Get \"https://control-plane.minikube.internal:8443/api/v1/nodes/kubernetes-upgrade-068009?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 20 05:08:00 kubernetes-upgrade-068009 kubelet[1075]: I0920 05:08:00.591184 1075 scope.go:117] "RemoveContainer" containerID="0490dec2e72afd3858b78fc47322493daa2d2645374c6028baf3b33bbe6b44b4"
Sep 20 05:08:00 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:08:00.591372 1075 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-controller-manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=kube-controller-manager pod=kube-controller-manager-kubernetes-upgrade-068009_kube-system(68753ed2ff04b7e77be2a6bf67dea66e)\"" pod="kube-system/kube-controller-manager-kubernetes-upgrade-068009" podUID="68753ed2ff04b7e77be2a6bf67dea66e"
Sep 20 05:08:04 kubernetes-upgrade-068009 kubelet[1075]: I0920 05:08:04.590941 1075 scope.go:117] "RemoveContainer" containerID="4b0175c9605c0f8d9be09d5992d8b324ccc8146d0ab6f703ddf0971927478f2f"
Sep 20 05:08:04 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:08:04.591571 1075 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-scheduler\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=kube-scheduler pod=kube-scheduler-kubernetes-upgrade-068009_kube-system(d2deb90f3760525a93376c0733b4e440)\"" pod="kube-system/kube-scheduler-kubernetes-upgrade-068009" podUID="d2deb90f3760525a93376c0733b4e440"
Sep 20 05:08:07 kubernetes-upgrade-068009 kubelet[1075]: I0920 05:08:07.590305 1075 scope.go:117] "RemoveContainer" containerID="9ffc7ac2177d215a296fc2b47aee806e280f63978e2d5576c63b070226db3657"
Sep 20 05:08:07 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:08:07.590905 1075 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-apiserver\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=kube-apiserver pod=kube-apiserver-kubernetes-upgrade-068009_kube-system(687bffc11a702480173094555b606021)\"" pod="kube-system/kube-apiserver-kubernetes-upgrade-068009" podUID="687bffc11a702480173094555b606021"
Sep 20 05:08:07 kubernetes-upgrade-068009 kubelet[1075]: E0920 05:08:07.941248 1075 kubelet_node_status.go:486] "Error updating node status, will retry" err="error getting node \"kubernetes-upgrade-068009\": the server was unable to return a response in the time allotted, but may still be processing the request (get nodes kubernetes-upgrade-068009)"
==> storage-provisioner [da1c4fe4a0b3a07a3ea6e372ba61c81eaff9776ad30d0b28e04686d21757f663] <==
I0920 05:00:30.848970 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0920 05:00:30.863276 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0920 05:00:30.863557 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
E0920 05:01:30.864884 1 leaderelection.go:325] error retrieving resource lock kube-system/k8s.io-minikube-hostpath: the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
E0920 05:02:34.318633 1 leaderelection.go:325] error retrieving resource lock kube-system/k8s.io-minikube-hostpath: the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
E0920 05:03:38.577612 1 leaderelection.go:325] error retrieving resource lock kube-system/k8s.io-minikube-hostpath: the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
E0920 05:04:42.174776 1 leaderelection.go:325] error retrieving resource lock kube-system/k8s.io-minikube-hostpath: the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
E0920 05:05:45.228097 1 leaderelection.go:325] error retrieving resource lock kube-system/k8s.io-minikube-hostpath: the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
E0920 05:06:48.248951 1 leaderelection.go:325] error retrieving resource lock kube-system/k8s.io-minikube-hostpath: the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
E0920 05:07:51.899338 1 leaderelection.go:325] error retrieving resource lock kube-system/k8s.io-minikube-hostpath: the server was unable to return a response in the time allotted, but may still be processing the request (get endpoints k8s.io-minikube-hostpath)
-- /stdout --
helpers_test.go:262: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p kubernetes-upgrade-068009 -n kubernetes-upgrade-068009
helpers_test.go:262: (dbg) Non-zero exit: out/minikube-linux-arm64 status --format={{.APIServer}} -p kubernetes-upgrade-068009 -n kubernetes-upgrade-068009: exit status 2 (15.933049074s)
-- stdout --
Stopped
-- /stdout --
helpers_test.go:262: status error: exit status 2 (may be ok)
helpers_test.go:264: "kubernetes-upgrade-068009" apiserver is not running, skipping kubectl commands (state="Stopped")
helpers_test.go:175: Cleaning up "kubernetes-upgrade-068009" profile ...
helpers_test.go:178: (dbg) Run: out/minikube-linux-arm64 delete -p kubernetes-upgrade-068009
helpers_test.go:178: (dbg) Done: out/minikube-linux-arm64 delete -p kubernetes-upgrade-068009: (3.018591691s)
--- FAIL: TestKubernetesUpgrade (619.44s)