=== RUN TestAddons/serial/Volcano
addons_test.go:843: volcano-admission stabilized in 50.884822ms
addons_test.go:851: volcano-controller stabilized in 51.340401ms
addons_test.go:835: volcano-scheduler stabilized in 51.551502ms
addons_test.go:857: (dbg) TestAddons/serial/Volcano: waiting 6m0s for pods matching "app=volcano-scheduler" in namespace "volcano-system" ...
helpers_test.go:344: "volcano-scheduler-6c9778cbdf-c8k87" [be14af5a-fb69-46c5-8517-bbf9e05a120e] Running
addons_test.go:857: (dbg) TestAddons/serial/Volcano: app=volcano-scheduler healthy within 6.004457987s
addons_test.go:861: (dbg) TestAddons/serial/Volcano: waiting 6m0s for pods matching "app=volcano-admission" in namespace "volcano-system" ...
helpers_test.go:344: "volcano-admission-5874dfdd79-c5bgp" [e617ad33-0ec5-4966-af21-d49ef4ac8f40] Running
addons_test.go:861: (dbg) TestAddons/serial/Volcano: app=volcano-admission healthy within 5.003928267s
addons_test.go:865: (dbg) TestAddons/serial/Volcano: waiting 6m0s for pods matching "app=volcano-controller" in namespace "volcano-system" ...
helpers_test.go:344: "volcano-controllers-789ffc5785-bm7lv" [e94847f6-586e-4562-b434-b217b69d61bf] Running
addons_test.go:865: (dbg) TestAddons/serial/Volcano: app=volcano-controller healthy within 5.003380374s
addons_test.go:870: (dbg) Run: kubectl --context addons-376302 delete -n volcano-system job volcano-admission-init
addons_test.go:876: (dbg) Run: kubectl --context addons-376302 create -f testdata/vcjob.yaml
addons_test.go:884: (dbg) Run: kubectl --context addons-376302 get vcjob -n my-volcano
addons_test.go:902: (dbg) TestAddons/serial/Volcano: waiting 3m0s for pods matching "volcano.sh/job-name=test-job" in namespace "my-volcano" ...
helpers_test.go:344: "test-job-nginx-0" [c97720c9-fdc7-4270-b650-23398f92e360] Pending
helpers_test.go:344: "test-job-nginx-0" [c97720c9-fdc7-4270-b650-23398f92e360] Pending: PodScheduled:Unschedulable (0/1 nodes are unavailable: 1 Insufficient cpu.)
helpers_test.go:329: TestAddons/serial/Volcano: WARNING: pod list for "my-volcano" "volcano.sh/job-name=test-job" returned: client rate limiter Wait returned an error: context deadline exceeded
addons_test.go:902: ***** TestAddons/serial/Volcano: pod "volcano.sh/job-name=test-job" failed to start within 3m0s: context deadline exceeded ****
addons_test.go:902: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p addons-376302 -n addons-376302
addons_test.go:902: TestAddons/serial/Volcano: showing logs for failed pods as of 2024-09-27 00:30:56.243299106 +0000 UTC m=+429.686123473
addons_test.go:902: (dbg) Run: kubectl --context addons-376302 describe po test-job-nginx-0 -n my-volcano
addons_test.go:902: (dbg) kubectl --context addons-376302 describe po test-job-nginx-0 -n my-volcano:
Name: test-job-nginx-0
Namespace: my-volcano
Priority: 0
Service Account: default
Node: <none>
Labels: volcano.sh/job-name=test-job
volcano.sh/job-namespace=my-volcano
volcano.sh/queue-name=test
volcano.sh/task-index=0
volcano.sh/task-spec=nginx
Annotations: scheduling.k8s.io/group-name: test-job-584a85e4-8e56-4885-8569-103e2d00b5e8
volcano.sh/job-name: test-job
volcano.sh/job-retry-count: 0
volcano.sh/job-version: 0
volcano.sh/queue-name: test
volcano.sh/task-index: 0
volcano.sh/task-spec: nginx
volcano.sh/template-uid: test-job-nginx
Status: Pending
IP:
IPs: <none>
Controlled By: Job/test-job
Containers:
nginx:
Image: nginx:latest
Port: <none>
Host Port: <none>
Command:
sleep
10m
Limits:
cpu: 1
Requests:
cpu: 1
Environment:
GOOGLE_APPLICATION_CREDENTIALS: /google-app-creds.json
PROJECT_ID: this_is_fake
GCP_PROJECT: this_is_fake
GCLOUD_PROJECT: this_is_fake
GOOGLE_CLOUD_PROJECT: this_is_fake
CLOUDSDK_CORE_PROJECT: this_is_fake
Mounts:
/google-app-creds.json from gcp-creds (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9v2vs (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
kube-api-access-9v2vs:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
gcp-creds:
Type: HostPath (bare host directory volume)
Path: /var/lib/minikube/google_application_credentials.json
HostPathType: File
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 2m59s volcano 0/1 nodes are unavailable: 1 Insufficient cpu.
addons_test.go:902: (dbg) Run: kubectl --context addons-376302 logs test-job-nginx-0 -n my-volcano
addons_test.go:902: (dbg) kubectl --context addons-376302 logs test-job-nginx-0 -n my-volcano:
addons_test.go:903: failed waiting for test-local-path pod: volcano.sh/job-name=test-job within 3m0s: context deadline exceeded
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestAddons/serial/Volcano]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect addons-376302
helpers_test.go:235: (dbg) docker inspect addons-376302:
-- stdout --
[
{
"Id": "56b458f3a125ce0d3e35c1efb5ab21b0d0ee44ad655897875a0b4981679bb1ab",
"Created": "2024-09-27T00:24:28.459709474Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 590339,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-09-27T00:24:28.607197623Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:62002f6a97ad1f6cd4117c29b1c488a6bf3b6255c8231f0d600b1bc7ba1bcfd6",
"ResolvConfPath": "/var/lib/docker/containers/56b458f3a125ce0d3e35c1efb5ab21b0d0ee44ad655897875a0b4981679bb1ab/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/56b458f3a125ce0d3e35c1efb5ab21b0d0ee44ad655897875a0b4981679bb1ab/hostname",
"HostsPath": "/var/lib/docker/containers/56b458f3a125ce0d3e35c1efb5ab21b0d0ee44ad655897875a0b4981679bb1ab/hosts",
"LogPath": "/var/lib/docker/containers/56b458f3a125ce0d3e35c1efb5ab21b0d0ee44ad655897875a0b4981679bb1ab/56b458f3a125ce0d3e35c1efb5ab21b0d0ee44ad655897875a0b4981679bb1ab-json.log",
"Name": "/addons-376302",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"addons-376302:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "addons-376302",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 4194304000,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 8388608000,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/d1ea8a27628d81624b2c8bc14224aff6021d12768ed9c84d0189ad54e0919a3b-init/diff:/var/lib/docker/overlay2/bde64bdaf549207dbef5d6ae31e43a20e66f52572944c42fb69017a1243b58d5/diff",
"MergedDir": "/var/lib/docker/overlay2/d1ea8a27628d81624b2c8bc14224aff6021d12768ed9c84d0189ad54e0919a3b/merged",
"UpperDir": "/var/lib/docker/overlay2/d1ea8a27628d81624b2c8bc14224aff6021d12768ed9c84d0189ad54e0919a3b/diff",
"WorkDir": "/var/lib/docker/overlay2/d1ea8a27628d81624b2c8bc14224aff6021d12768ed9c84d0189ad54e0919a3b/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "addons-376302",
"Source": "/var/lib/docker/volumes/addons-376302/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
},
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "addons-376302",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "addons-376302",
"name.minikube.sigs.k8s.io": "addons-376302",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "bfa7256fd418af487413ae43dd6d5164a7571da1ef38026131fb98696ce6b118",
"SandboxKey": "/var/run/docker/netns/bfa7256fd418",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33509"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33510"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33513"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33511"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33512"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"addons-376302": {
"IPAMConfig": {
"IPv4Address": "192.168.49.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:31:02",
"DriverOpts": null,
"NetworkID": "dacb264def115d400235cc41d43a7c977a20b010ce495fba058fa2a190af267d",
"EndpointID": "1a224adb092e2ac52787a73bd2a4c21c2e65d31449c54c91ebd9c638d7ded400",
"Gateway": "192.168.49.1",
"IPAddress": "192.168.49.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"addons-376302",
"56b458f3a125"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p addons-376302 -n addons-376302
helpers_test.go:244: <<< TestAddons/serial/Volcano FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestAddons/serial/Volcano]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p addons-376302 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p addons-376302 logs -n 25: (1.491527329s)
helpers_test.go:252: TestAddons/serial/Volcano logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------|------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------|------------------------|---------|---------|---------------------|---------------------|
| start | -o=json --download-only | download-only-607949 | jenkins | v1.34.0 | 27 Sep 24 00:23 UTC | |
| | -p download-only-607949 | | | | | |
| | --force --alsologtostderr | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| | --container-runtime=containerd | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | --all | minikube | jenkins | v1.34.0 | 27 Sep 24 00:23 UTC | 27 Sep 24 00:23 UTC |
| delete | -p download-only-607949 | download-only-607949 | jenkins | v1.34.0 | 27 Sep 24 00:23 UTC | 27 Sep 24 00:23 UTC |
| start | -o=json --download-only | download-only-981151 | jenkins | v1.34.0 | 27 Sep 24 00:23 UTC | |
| | -p download-only-981151 | | | | | |
| | --force --alsologtostderr | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
| | --container-runtime=containerd | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | --all | minikube | jenkins | v1.34.0 | 27 Sep 24 00:24 UTC | 27 Sep 24 00:24 UTC |
| delete | -p download-only-981151 | download-only-981151 | jenkins | v1.34.0 | 27 Sep 24 00:24 UTC | 27 Sep 24 00:24 UTC |
| delete | -p download-only-607949 | download-only-607949 | jenkins | v1.34.0 | 27 Sep 24 00:24 UTC | 27 Sep 24 00:24 UTC |
| delete | -p download-only-981151 | download-only-981151 | jenkins | v1.34.0 | 27 Sep 24 00:24 UTC | 27 Sep 24 00:24 UTC |
| start | --download-only -p | download-docker-397635 | jenkins | v1.34.0 | 27 Sep 24 00:24 UTC | |
| | download-docker-397635 | | | | | |
| | --alsologtostderr | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p download-docker-397635 | download-docker-397635 | jenkins | v1.34.0 | 27 Sep 24 00:24 UTC | 27 Sep 24 00:24 UTC |
| start | --download-only -p | binary-mirror-548949 | jenkins | v1.34.0 | 27 Sep 24 00:24 UTC | |
| | binary-mirror-548949 | | | | | |
| | --alsologtostderr | | | | | |
| | --binary-mirror | | | | | |
| | http://127.0.0.1:37853 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p binary-mirror-548949 | binary-mirror-548949 | jenkins | v1.34.0 | 27 Sep 24 00:24 UTC | 27 Sep 24 00:24 UTC |
| addons | disable dashboard -p | addons-376302 | jenkins | v1.34.0 | 27 Sep 24 00:24 UTC | |
| | addons-376302 | | | | | |
| addons | enable dashboard -p | addons-376302 | jenkins | v1.34.0 | 27 Sep 24 00:24 UTC | |
| | addons-376302 | | | | | |
| start | -p addons-376302 --wait=true | addons-376302 | jenkins | v1.34.0 | 27 Sep 24 00:24 UTC | 27 Sep 24 00:27 UTC |
| | --memory=4000 --alsologtostderr | | | | | |
| | --addons=registry | | | | | |
| | --addons=metrics-server | | | | | |
| | --addons=volumesnapshots | | | | | |
| | --addons=csi-hostpath-driver | | | | | |
| | --addons=gcp-auth | | | | | |
| | --addons=cloud-spanner | | | | | |
| | --addons=inspektor-gadget | | | | | |
| | --addons=storage-provisioner-rancher | | | | | |
| | --addons=nvidia-device-plugin | | | | | |
| | --addons=yakd --addons=volcano | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --addons=ingress | | | | | |
| | --addons=ingress-dns | | | | | |
|---------|--------------------------------------|------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/09/27 00:24:04
Running on machine: ip-172-31-31-251
Binary: Built with gc go1.23.0 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0927 00:24:04.088958 589846 out.go:345] Setting OutFile to fd 1 ...
I0927 00:24:04.089115 589846 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0927 00:24:04.089139 589846 out.go:358] Setting ErrFile to fd 2...
I0927 00:24:04.089146 589846 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0927 00:24:04.089433 589846 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19711-583677/.minikube/bin
I0927 00:24:04.089922 589846 out.go:352] Setting JSON to false
I0927 00:24:04.090899 589846 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":14779,"bootTime":1727381865,"procs":166,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1070-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0927 00:24:04.090980 589846 start.go:139] virtualization:
I0927 00:24:04.093496 589846 out.go:177] * [addons-376302] minikube v1.34.0 on Ubuntu 20.04 (arm64)
I0927 00:24:04.095671 589846 out.go:177] - MINIKUBE_LOCATION=19711
I0927 00:24:04.095776 589846 notify.go:220] Checking for updates...
I0927 00:24:04.099156 589846 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0927 00:24:04.101327 589846 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19711-583677/kubeconfig
I0927 00:24:04.103591 589846 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19711-583677/.minikube
I0927 00:24:04.105626 589846 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0927 00:24:04.107458 589846 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0927 00:24:04.109275 589846 driver.go:394] Setting default libvirt URI to qemu:///system
I0927 00:24:04.129018 589846 docker.go:123] docker version: linux-27.3.1:Docker Engine - Community
I0927 00:24:04.129153 589846 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0927 00:24:04.198776 589846 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:25 OomKillDisable:true NGoroutines:44 SystemTime:2024-09-27 00:24:04.189660439 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214839296 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0927 00:24:04.198896 589846 docker.go:318] overlay module found
I0927 00:24:04.200973 589846 out.go:177] * Using the docker driver based on user configuration
I0927 00:24:04.203360 589846 start.go:297] selected driver: docker
I0927 00:24:04.203383 589846 start.go:901] validating driver "docker" against <nil>
I0927 00:24:04.203397 589846 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0927 00:24:04.204158 589846 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0927 00:24:04.250384 589846 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:25 OomKillDisable:true NGoroutines:44 SystemTime:2024-09-27 00:24:04.241196729 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214839296 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0927 00:24:04.250628 589846 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0927 00:24:04.250863 589846 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0927 00:24:04.253578 589846 out.go:177] * Using Docker driver with root privileges
I0927 00:24:04.255753 589846 cni.go:84] Creating CNI manager for ""
I0927 00:24:04.255826 589846 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0927 00:24:04.255840 589846 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni
I0927 00:24:04.255927 589846 start.go:340] cluster config:
{Name:addons-376302 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:4000 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:addons-376302 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime
:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHA
uthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0927 00:24:04.258228 589846 out.go:177] * Starting "addons-376302" primary control-plane node in "addons-376302" cluster
I0927 00:24:04.260025 589846 cache.go:121] Beginning downloading kic base image for docker with containerd
I0927 00:24:04.261660 589846 out.go:177] * Pulling base image v0.0.45-1727108449-19696 ...
I0927 00:24:04.263337 589846 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0927 00:24:04.263388 589846 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19711-583677/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4
I0927 00:24:04.263400 589846 cache.go:56] Caching tarball of preloaded images
I0927 00:24:04.263450 589846 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 in local docker daemon
I0927 00:24:04.263492 589846 preload.go:172] Found /home/jenkins/minikube-integration/19711-583677/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0927 00:24:04.263504 589846 cache.go:59] Finished verifying existence of preloaded tar for v1.31.1 on containerd
I0927 00:24:04.263862 589846 profile.go:143] Saving config to /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/config.json ...
I0927 00:24:04.263895 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/config.json: {Name:mk969fce461c4c8f5125b14256912cc924363dc5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:04.277121 589846 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 to local cache
I0927 00:24:04.277301 589846 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 in local cache directory
I0927 00:24:04.277323 589846 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 in local cache directory, skipping pull
I0927 00:24:04.277328 589846 image.go:135] gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 exists in cache, skipping pull
I0927 00:24:04.277336 589846 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 as a tarball
I0927 00:24:04.277342 589846 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 from local cache
I0927 00:24:21.217095 589846 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 from cached tarball
I0927 00:24:21.217134 589846 cache.go:194] Successfully downloaded all kic artifacts
I0927 00:24:21.217163 589846 start.go:360] acquireMachinesLock for addons-376302: {Name:mkd2ecedf1b6ed0010f5a71780cd08374b5d43a4 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0927 00:24:21.218129 589846 start.go:364] duration metric: took 924.468µs to acquireMachinesLock for "addons-376302"
I0927 00:24:21.218171 589846 start.go:93] Provisioning new machine with config: &{Name:addons-376302 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:4000 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:addons-376302 Namespace:default APIServerHAVIP: APIServerName:min
ikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:fa
lse CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0927 00:24:21.218296 589846 start.go:125] createHost starting for "" (driver="docker")
I0927 00:24:21.220597 589846 out.go:235] * Creating docker container (CPUs=2, Memory=4000MB) ...
I0927 00:24:21.220845 589846 start.go:159] libmachine.API.Create for "addons-376302" (driver="docker")
I0927 00:24:21.220886 589846 client.go:168] LocalClient.Create starting
I0927 00:24:21.221005 589846 main.go:141] libmachine: Creating CA: /home/jenkins/minikube-integration/19711-583677/.minikube/certs/ca.pem
I0927 00:24:21.879775 589846 main.go:141] libmachine: Creating client certificate: /home/jenkins/minikube-integration/19711-583677/.minikube/certs/cert.pem
I0927 00:24:22.155675 589846 cli_runner.go:164] Run: docker network inspect addons-376302 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0927 00:24:22.172552 589846 cli_runner.go:211] docker network inspect addons-376302 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0927 00:24:22.172636 589846 network_create.go:284] running [docker network inspect addons-376302] to gather additional debugging logs...
I0927 00:24:22.172660 589846 cli_runner.go:164] Run: docker network inspect addons-376302
W0927 00:24:22.188178 589846 cli_runner.go:211] docker network inspect addons-376302 returned with exit code 1
I0927 00:24:22.188211 589846 network_create.go:287] error running [docker network inspect addons-376302]: docker network inspect addons-376302: exit status 1
stdout:
[]
stderr:
Error response from daemon: network addons-376302 not found
I0927 00:24:22.188224 589846 network_create.go:289] output of [docker network inspect addons-376302]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network addons-376302 not found
** /stderr **
I0927 00:24:22.188315 589846 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0927 00:24:22.204264 589846 network.go:206] using free private subnet 192.168.49.0/24: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0x4001884f10}
I0927 00:24:22.204312 589846 network_create.go:124] attempt to create docker network addons-376302 192.168.49.0/24 with gateway 192.168.49.1 and MTU of 1500 ...
I0927 00:24:22.204368 589846 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=addons-376302 addons-376302
I0927 00:24:22.271327 589846 network_create.go:108] docker network addons-376302 192.168.49.0/24 created
I0927 00:24:22.271358 589846 kic.go:121] calculated static IP "192.168.49.2" for the "addons-376302" container
I0927 00:24:22.271443 589846 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0927 00:24:22.285689 589846 cli_runner.go:164] Run: docker volume create addons-376302 --label name.minikube.sigs.k8s.io=addons-376302 --label created_by.minikube.sigs.k8s.io=true
I0927 00:24:22.302100 589846 oci.go:103] Successfully created a docker volume addons-376302
I0927 00:24:22.302189 589846 cli_runner.go:164] Run: docker run --rm --name addons-376302-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=addons-376302 --entrypoint /usr/bin/test -v addons-376302:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 -d /var/lib
I0927 00:24:24.358800 589846 cli_runner.go:217] Completed: docker run --rm --name addons-376302-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=addons-376302 --entrypoint /usr/bin/test -v addons-376302:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 -d /var/lib: (2.05657143s)
I0927 00:24:24.358856 589846 oci.go:107] Successfully prepared a docker volume addons-376302
I0927 00:24:24.358879 589846 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0927 00:24:24.358898 589846 kic.go:194] Starting extracting preloaded images to volume ...
I0927 00:24:24.358987 589846 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/19711-583677/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v addons-376302:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 -I lz4 -xf /preloaded.tar -C /extractDir
I0927 00:24:28.400004 589846 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/19711-583677/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v addons-376302:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 -I lz4 -xf /preloaded.tar -C /extractDir: (4.040977163s)
I0927 00:24:28.400037 589846 kic.go:203] duration metric: took 4.041136276s to extract preloaded images to volume ...
W0927 00:24:28.400176 589846 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0927 00:24:28.400308 589846 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0927 00:24:28.447116 589846 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname addons-376302 --name addons-376302 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=addons-376302 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=addons-376302 --network addons-376302 --ip 192.168.49.2 --volume addons-376302:/var --security-opt apparmor=unconfined --memory=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21
I0927 00:24:28.762749 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Running}}
I0927 00:24:28.783901 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:28.816079 589846 cli_runner.go:164] Run: docker exec addons-376302 stat /var/lib/dpkg/alternatives/iptables
I0927 00:24:28.899314 589846 oci.go:144] the created container "addons-376302" has a running status.
I0927 00:24:28.899345 589846 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa...
I0927 00:24:30.002509 589846 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0927 00:24:30.056110 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:30.080259 589846 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0927 00:24:30.080289 589846 kic_runner.go:114] Args: [docker exec --privileged addons-376302 chown docker:docker /home/docker/.ssh/authorized_keys]
I0927 00:24:30.154020 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:30.197535 589846 machine.go:93] provisionDockerMachine start ...
I0927 00:24:30.197655 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:30.217924 589846 main.go:141] libmachine: Using SSH client type: native
I0927 00:24:30.218212 589846 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33509 <nil> <nil>}
I0927 00:24:30.218229 589846 main.go:141] libmachine: About to run SSH command:
hostname
I0927 00:24:30.346093 589846 main.go:141] libmachine: SSH cmd err, output: <nil>: addons-376302
I0927 00:24:30.346115 589846 ubuntu.go:169] provisioning hostname "addons-376302"
I0927 00:24:30.346182 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:30.363949 589846 main.go:141] libmachine: Using SSH client type: native
I0927 00:24:30.364198 589846 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33509 <nil> <nil>}
I0927 00:24:30.364214 589846 main.go:141] libmachine: About to run SSH command:
sudo hostname addons-376302 && echo "addons-376302" | sudo tee /etc/hostname
I0927 00:24:30.506040 589846 main.go:141] libmachine: SSH cmd err, output: <nil>: addons-376302
I0927 00:24:30.506118 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:30.524205 589846 main.go:141] libmachine: Using SSH client type: native
I0927 00:24:30.524456 589846 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33509 <nil> <nil>}
I0927 00:24:30.524478 589846 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\saddons-376302' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 addons-376302/g' /etc/hosts;
else
echo '127.0.1.1 addons-376302' | sudo tee -a /etc/hosts;
fi
fi
I0927 00:24:30.654606 589846 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0927 00:24:30.654636 589846 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19711-583677/.minikube CaCertPath:/home/jenkins/minikube-integration/19711-583677/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19711-583677/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19711-583677/.minikube}
I0927 00:24:30.654655 589846 ubuntu.go:177] setting up certificates
I0927 00:24:30.654665 589846 provision.go:84] configureAuth start
I0927 00:24:30.654727 589846 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" addons-376302
I0927 00:24:30.674537 589846 provision.go:143] copyHostCerts
I0927 00:24:30.674628 589846 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19711-583677/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19711-583677/.minikube/cert.pem (1123 bytes)
I0927 00:24:30.674761 589846 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19711-583677/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19711-583677/.minikube/key.pem (1679 bytes)
I0927 00:24:30.674836 589846 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19711-583677/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19711-583677/.minikube/ca.pem (1082 bytes)
I0927 00:24:30.674895 589846 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19711-583677/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19711-583677/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19711-583677/.minikube/certs/ca-key.pem org=jenkins.addons-376302 san=[127.0.0.1 192.168.49.2 addons-376302 localhost minikube]
I0927 00:24:31.002446 589846 provision.go:177] copyRemoteCerts
I0927 00:24:31.002542 589846 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0927 00:24:31.002589 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:31.022489 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:31.115824 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0927 00:24:31.140493 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes)
I0927 00:24:31.164619 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0927 00:24:31.188308 589846 provision.go:87] duration metric: took 533.629255ms to configureAuth
I0927 00:24:31.188333 589846 ubuntu.go:193] setting minikube options for container-runtime
I0927 00:24:31.188533 589846 config.go:182] Loaded profile config "addons-376302": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0927 00:24:31.188546 589846 machine.go:96] duration metric: took 990.982059ms to provisionDockerMachine
I0927 00:24:31.188553 589846 client.go:171] duration metric: took 9.967657679s to LocalClient.Create
I0927 00:24:31.188578 589846 start.go:167] duration metric: took 9.967733871s to libmachine.API.Create "addons-376302"
I0927 00:24:31.188590 589846 start.go:293] postStartSetup for "addons-376302" (driver="docker")
I0927 00:24:31.188600 589846 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0927 00:24:31.188652 589846 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0927 00:24:31.188695 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:31.205711 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:31.303140 589846 ssh_runner.go:195] Run: cat /etc/os-release
I0927 00:24:31.306104 589846 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0927 00:24:31.306142 589846 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0927 00:24:31.306156 589846 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0927 00:24:31.306163 589846 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I0927 00:24:31.306173 589846 filesync.go:126] Scanning /home/jenkins/minikube-integration/19711-583677/.minikube/addons for local assets ...
I0927 00:24:31.306239 589846 filesync.go:126] Scanning /home/jenkins/minikube-integration/19711-583677/.minikube/files for local assets ...
I0927 00:24:31.306272 589846 start.go:296] duration metric: took 117.675618ms for postStartSetup
I0927 00:24:31.306615 589846 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" addons-376302
I0927 00:24:31.322584 589846 profile.go:143] Saving config to /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/config.json ...
I0927 00:24:31.322884 589846 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0927 00:24:31.323101 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:31.340929 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:31.431039 589846 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0927 00:24:31.435433 589846 start.go:128] duration metric: took 10.217119414s to createHost
I0927 00:24:31.435458 589846 start.go:83] releasing machines lock for "addons-376302", held for 10.217310469s
I0927 00:24:31.435528 589846 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" addons-376302
I0927 00:24:31.451708 589846 ssh_runner.go:195] Run: cat /version.json
I0927 00:24:31.451763 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:31.451768 589846 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0927 00:24:31.451844 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:31.469547 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:31.470062 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:31.557802 589846 ssh_runner.go:195] Run: systemctl --version
I0927 00:24:31.686214 589846 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0927 00:24:31.690678 589846 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0927 00:24:31.715880 589846 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0927 00:24:31.715981 589846 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0927 00:24:31.745592 589846 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0927 00:24:31.745618 589846 start.go:495] detecting cgroup driver to use...
I0927 00:24:31.745674 589846 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0927 00:24:31.745749 589846 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0927 00:24:31.758335 589846 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0927 00:24:31.770102 589846 docker.go:217] disabling cri-docker service (if available) ...
I0927 00:24:31.770184 589846 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0927 00:24:31.784364 589846 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0927 00:24:31.799116 589846 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0927 00:24:31.886538 589846 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0927 00:24:31.973823 589846 docker.go:233] disabling docker service ...
I0927 00:24:31.973887 589846 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0927 00:24:31.993171 589846 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0927 00:24:32.009445 589846 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0927 00:24:32.101954 589846 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0927 00:24:32.189957 589846 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0927 00:24:32.201623 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0927 00:24:32.218791 589846 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0927 00:24:32.228882 589846 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0927 00:24:32.238873 589846 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0927 00:24:32.238985 589846 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0927 00:24:32.248967 589846 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0927 00:24:32.259003 589846 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0927 00:24:32.269376 589846 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0927 00:24:32.279518 589846 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0927 00:24:32.289222 589846 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0927 00:24:32.299042 589846 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0927 00:24:32.308920 589846 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0927 00:24:32.318849 589846 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0927 00:24:32.327785 589846 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0927 00:24:32.336582 589846 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0927 00:24:32.424434 589846 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0927 00:24:32.564039 589846 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0927 00:24:32.564163 589846 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0927 00:24:32.568000 589846 start.go:563] Will wait 60s for crictl version
I0927 00:24:32.568112 589846 ssh_runner.go:195] Run: which crictl
I0927 00:24:32.571716 589846 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0927 00:24:32.606261 589846 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.22
RuntimeApiVersion: v1
I0927 00:24:32.606386 589846 ssh_runner.go:195] Run: containerd --version
I0927 00:24:32.630876 589846 ssh_runner.go:195] Run: containerd --version
I0927 00:24:32.654226 589846 out.go:177] * Preparing Kubernetes v1.31.1 on containerd 1.7.22 ...
I0927 00:24:32.656458 589846 cli_runner.go:164] Run: docker network inspect addons-376302 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0927 00:24:32.670721 589846 ssh_runner.go:195] Run: grep 192.168.49.1 host.minikube.internal$ /etc/hosts
I0927 00:24:32.674101 589846 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.49.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0927 00:24:32.684375 589846 kubeadm.go:883] updating cluster {Name:addons-376302 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:4000 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:addons-376302 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNa
mes:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false Cus
tomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0927 00:24:32.684494 589846 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0927 00:24:32.684560 589846 ssh_runner.go:195] Run: sudo crictl images --output json
I0927 00:24:32.720341 589846 containerd.go:627] all images are preloaded for containerd runtime.
I0927 00:24:32.720365 589846 containerd.go:534] Images already preloaded, skipping extraction
I0927 00:24:32.720426 589846 ssh_runner.go:195] Run: sudo crictl images --output json
I0927 00:24:32.755293 589846 containerd.go:627] all images are preloaded for containerd runtime.
I0927 00:24:32.755318 589846 cache_images.go:84] Images are preloaded, skipping loading
I0927 00:24:32.755326 589846 kubeadm.go:934] updating node { 192.168.49.2 8443 v1.31.1 containerd true true} ...
I0927 00:24:32.755418 589846 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=addons-376302 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.49.2
[Install]
config:
{KubernetesVersion:v1.31.1 ClusterName:addons-376302 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0927 00:24:32.755493 589846 ssh_runner.go:195] Run: sudo crictl info
I0927 00:24:32.792512 589846 cni.go:84] Creating CNI manager for ""
I0927 00:24:32.792536 589846 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0927 00:24:32.792547 589846 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0927 00:24:32.792569 589846 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.49.2 APIServerPort:8443 KubernetesVersion:v1.31.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:addons-376302 NodeName:addons-376302 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.49.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.49.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc
/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0927 00:24:32.792697 589846 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.49.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "addons-376302"
kubeletExtraArgs:
node-ip: 192.168.49.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.49.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.31.1
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0927 00:24:32.792771 589846 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.1
I0927 00:24:32.801746 589846 binaries.go:44] Found k8s binaries, skipping transfer
I0927 00:24:32.801863 589846 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0927 00:24:32.810562 589846 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (317 bytes)
I0927 00:24:32.828403 589846 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0927 00:24:32.847137 589846 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2167 bytes)
I0927 00:24:32.864855 589846 ssh_runner.go:195] Run: grep 192.168.49.2 control-plane.minikube.internal$ /etc/hosts
I0927 00:24:32.868551 589846 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.49.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0927 00:24:32.879662 589846 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0927 00:24:32.966188 589846 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0927 00:24:32.981857 589846 certs.go:68] Setting up /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302 for IP: 192.168.49.2
I0927 00:24:32.981880 589846 certs.go:194] generating shared ca certs ...
I0927 00:24:32.981897 589846 certs.go:226] acquiring lock for ca certs: {Name:mk008a6c957a7b891b6a534ee8dfae7b680b060c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:32.982110 589846 certs.go:240] generating "minikubeCA" ca cert: /home/jenkins/minikube-integration/19711-583677/.minikube/ca.key
I0927 00:24:33.181137 589846 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19711-583677/.minikube/ca.crt ...
I0927 00:24:33.181170 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/.minikube/ca.crt: {Name:mkc45fb74234d083d260abca12aab7acb9eb21fb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:33.181936 589846 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19711-583677/.minikube/ca.key ...
I0927 00:24:33.181953 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/.minikube/ca.key: {Name:mk93c8a50a2653fc9664c4633c48656e87ac595f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:33.182052 589846 certs.go:240] generating "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19711-583677/.minikube/proxy-client-ca.key
I0927 00:24:33.407364 589846 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19711-583677/.minikube/proxy-client-ca.crt ...
I0927 00:24:33.407394 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/.minikube/proxy-client-ca.crt: {Name:mk9be3228cc003fe659fbb559b9f1f34fbe74bba Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:33.408098 589846 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19711-583677/.minikube/proxy-client-ca.key ...
I0927 00:24:33.408116 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/.minikube/proxy-client-ca.key: {Name:mkdff29ea1a8b181ec44fcf84d2b49373856e7e4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:33.408199 589846 certs.go:256] generating profile certs ...
I0927 00:24:33.408260 589846 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/client.key
I0927 00:24:33.408276 589846 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/client.crt with IP's: []
I0927 00:24:33.618843 589846 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/client.crt ...
I0927 00:24:33.618893 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/client.crt: {Name:mkbf03d2274ce60910f238a804ace6b75bcc30cf Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:33.619738 589846 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/client.key ...
I0927 00:24:33.619757 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/client.key: {Name:mk7f23e5ea4a6531cc05c95a3e810674f429c521 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:33.620287 589846 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.key.64290ad9
I0927 00:24:33.620313 589846 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.crt.64290ad9 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.49.2]
I0927 00:24:33.825788 589846 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.crt.64290ad9 ...
I0927 00:24:33.825818 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.crt.64290ad9: {Name:mkf5cc25ae0116d5336f6fbc08f2cf4e9ebf9c98 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:33.826002 589846 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.key.64290ad9 ...
I0927 00:24:33.826016 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.key.64290ad9: {Name:mkaa541ada476a4753fb43f2968bc5f94ce4b002 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:33.826099 589846 certs.go:381] copying /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.crt.64290ad9 -> /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.crt
I0927 00:24:33.826181 589846 certs.go:385] copying /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.key.64290ad9 -> /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.key
I0927 00:24:33.826234 589846 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/proxy-client.key
I0927 00:24:33.826254 589846 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/proxy-client.crt with IP's: []
I0927 00:24:34.824279 589846 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/proxy-client.crt ...
I0927 00:24:34.824312 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/proxy-client.crt: {Name:mka388dc129fbea48897e69519f3f021b4e4aae4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:34.824502 589846 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/proxy-client.key ...
I0927 00:24:34.824520 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/proxy-client.key: {Name:mkbb7155b1d8755fe903e6c09513c260f231d70d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:34.824718 589846 certs.go:484] found cert: /home/jenkins/minikube-integration/19711-583677/.minikube/certs/ca-key.pem (1675 bytes)
I0927 00:24:34.824764 589846 certs.go:484] found cert: /home/jenkins/minikube-integration/19711-583677/.minikube/certs/ca.pem (1082 bytes)
I0927 00:24:34.824792 589846 certs.go:484] found cert: /home/jenkins/minikube-integration/19711-583677/.minikube/certs/cert.pem (1123 bytes)
I0927 00:24:34.824820 589846 certs.go:484] found cert: /home/jenkins/minikube-integration/19711-583677/.minikube/certs/key.pem (1679 bytes)
I0927 00:24:34.825669 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0927 00:24:34.849753 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0927 00:24:34.876069 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0927 00:24:34.902068 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0927 00:24:34.926177 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1419 bytes)
I0927 00:24:34.950876 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0927 00:24:34.977198 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0927 00:24:35.010601 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/profiles/addons-376302/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0927 00:24:35.050265 589846 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19711-583677/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0927 00:24:35.076671 589846 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0927 00:24:35.097048 589846 ssh_runner.go:195] Run: openssl version
I0927 00:24:35.103276 589846 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0927 00:24:35.113905 589846 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0927 00:24:35.117781 589846 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 27 00:24 /usr/share/ca-certificates/minikubeCA.pem
I0927 00:24:35.117848 589846 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0927 00:24:35.125173 589846 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0927 00:24:35.134687 589846 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0927 00:24:35.138308 589846 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0927 00:24:35.138359 589846 kubeadm.go:392] StartCluster: {Name:addons-376302 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:4000 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:addons-376302 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames
:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false Custom
QemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0927 00:24:35.138441 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0927 00:24:35.138541 589846 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0927 00:24:35.177547 589846 cri.go:89] found id: ""
I0927 00:24:35.177630 589846 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0927 00:24:35.187747 589846 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0927 00:24:35.196640 589846 kubeadm.go:214] ignoring SystemVerification for kubeadm because of docker driver
I0927 00:24:35.196736 589846 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0927 00:24:35.205825 589846 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0927 00:24:35.205847 589846 kubeadm.go:157] found existing configuration files:
I0927 00:24:35.205921 589846 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0927 00:24:35.214762 589846 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0927 00:24:35.214894 589846 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0927 00:24:35.223679 589846 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0927 00:24:35.233054 589846 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0927 00:24:35.233146 589846 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0927 00:24:35.241438 589846 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0927 00:24:35.250076 589846 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0927 00:24:35.250168 589846 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0927 00:24:35.258785 589846 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0927 00:24:35.267836 589846 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0927 00:24:35.267899 589846 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0927 00:24:35.276162 589846 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.31.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0927 00:24:35.317149 589846 kubeadm.go:310] [init] Using Kubernetes version: v1.31.1
I0927 00:24:35.317213 589846 kubeadm.go:310] [preflight] Running pre-flight checks
I0927 00:24:35.350366 589846 kubeadm.go:310] [preflight] The system verification failed. Printing the output from the verification:
I0927 00:24:35.350440 589846 kubeadm.go:310] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1070-aws[0m
I0927 00:24:35.350495 589846 kubeadm.go:310] [0;37mOS[0m: [0;32mLinux[0m
I0927 00:24:35.350541 589846 kubeadm.go:310] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0927 00:24:35.350596 589846 kubeadm.go:310] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0927 00:24:35.350649 589846 kubeadm.go:310] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0927 00:24:35.350702 589846 kubeadm.go:310] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0927 00:24:35.350754 589846 kubeadm.go:310] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0927 00:24:35.350808 589846 kubeadm.go:310] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0927 00:24:35.350863 589846 kubeadm.go:310] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0927 00:24:35.350915 589846 kubeadm.go:310] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0927 00:24:35.350964 589846 kubeadm.go:310] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0927 00:24:35.422722 589846 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0927 00:24:35.422861 589846 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0927 00:24:35.422957 589846 kubeadm.go:310] [preflight] You can also perform this action beforehand using 'kubeadm config images pull'
I0927 00:24:35.428399 589846 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0927 00:24:35.431091 589846 out.go:235] - Generating certificates and keys ...
I0927 00:24:35.431188 589846 kubeadm.go:310] [certs] Using existing ca certificate authority
I0927 00:24:35.431255 589846 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0927 00:24:35.685122 589846 kubeadm.go:310] [certs] Generating "apiserver-kubelet-client" certificate and key
I0927 00:24:36.102118 589846 kubeadm.go:310] [certs] Generating "front-proxy-ca" certificate and key
I0927 00:24:36.636667 589846 kubeadm.go:310] [certs] Generating "front-proxy-client" certificate and key
I0927 00:24:36.813018 589846 kubeadm.go:310] [certs] Generating "etcd/ca" certificate and key
I0927 00:24:37.185872 589846 kubeadm.go:310] [certs] Generating "etcd/server" certificate and key
I0927 00:24:37.186094 589846 kubeadm.go:310] [certs] etcd/server serving cert is signed for DNS names [addons-376302 localhost] and IPs [192.168.49.2 127.0.0.1 ::1]
I0927 00:24:37.754036 589846 kubeadm.go:310] [certs] Generating "etcd/peer" certificate and key
I0927 00:24:37.754180 589846 kubeadm.go:310] [certs] etcd/peer serving cert is signed for DNS names [addons-376302 localhost] and IPs [192.168.49.2 127.0.0.1 ::1]
I0927 00:24:38.054081 589846 kubeadm.go:310] [certs] Generating "etcd/healthcheck-client" certificate and key
I0927 00:24:38.664032 589846 kubeadm.go:310] [certs] Generating "apiserver-etcd-client" certificate and key
I0927 00:24:39.076433 589846 kubeadm.go:310] [certs] Generating "sa" key and public key
I0927 00:24:39.076599 589846 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0927 00:24:39.624527 589846 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0927 00:24:40.520184 589846 kubeadm.go:310] [kubeconfig] Writing "super-admin.conf" kubeconfig file
I0927 00:24:41.137987 589846 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0927 00:24:41.733601 589846 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0927 00:24:42.675518 589846 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0927 00:24:42.676394 589846 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0927 00:24:42.679371 589846 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0927 00:24:42.681663 589846 out.go:235] - Booting up control plane ...
I0927 00:24:42.681761 589846 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0927 00:24:42.681838 589846 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0927 00:24:42.682393 589846 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0927 00:24:42.692942 589846 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0927 00:24:42.699317 589846 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0927 00:24:42.699677 589846 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0927 00:24:42.800282 589846 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
I0927 00:24:42.800406 589846 kubeadm.go:310] [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
I0927 00:24:43.801776 589846 kubeadm.go:310] [kubelet-check] The kubelet is healthy after 1.001595411s
I0927 00:24:43.801873 589846 kubeadm.go:310] [api-check] Waiting for a healthy API server. This can take up to 4m0s
I0927 00:24:49.803415 589846 kubeadm.go:310] [api-check] The API server is healthy after 6.001624452s
I0927 00:24:49.822822 589846 kubeadm.go:310] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0927 00:24:49.836045 589846 kubeadm.go:310] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0927 00:24:49.857934 589846 kubeadm.go:310] [upload-certs] Skipping phase. Please see --upload-certs
I0927 00:24:49.858449 589846 kubeadm.go:310] [mark-control-plane] Marking the node addons-376302 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
I0927 00:24:49.870002 589846 kubeadm.go:310] [bootstrap-token] Using token: 29k8v1.wlnmbgivp7440xc5
I0927 00:24:49.872024 589846 out.go:235] - Configuring RBAC rules ...
I0927 00:24:49.872151 589846 kubeadm.go:310] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
I0927 00:24:49.878694 589846 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
I0927 00:24:49.886670 589846 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I0927 00:24:49.890130 589846 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
I0927 00:24:49.893850 589846 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I0927 00:24:49.897913 589846 kubeadm.go:310] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I0927 00:24:50.210157 589846 kubeadm.go:310] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0927 00:24:50.635693 589846 kubeadm.go:310] [addons] Applied essential addon: CoreDNS
I0927 00:24:51.210180 589846 kubeadm.go:310] [addons] Applied essential addon: kube-proxy
I0927 00:24:51.211423 589846 kubeadm.go:310]
I0927 00:24:51.211503 589846 kubeadm.go:310] Your Kubernetes control-plane has initialized successfully!
I0927 00:24:51.211515 589846 kubeadm.go:310]
I0927 00:24:51.211596 589846 kubeadm.go:310] To start using your cluster, you need to run the following as a regular user:
I0927 00:24:51.211601 589846 kubeadm.go:310]
I0927 00:24:51.211627 589846 kubeadm.go:310] mkdir -p $HOME/.kube
I0927 00:24:51.211685 589846 kubeadm.go:310] sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
I0927 00:24:51.211736 589846 kubeadm.go:310] sudo chown $(id -u):$(id -g) $HOME/.kube/config
I0927 00:24:51.211740 589846 kubeadm.go:310]
I0927 00:24:51.211793 589846 kubeadm.go:310] Alternatively, if you are the root user, you can run:
I0927 00:24:51.211798 589846 kubeadm.go:310]
I0927 00:24:51.211845 589846 kubeadm.go:310] export KUBECONFIG=/etc/kubernetes/admin.conf
I0927 00:24:51.211850 589846 kubeadm.go:310]
I0927 00:24:51.211902 589846 kubeadm.go:310] You should now deploy a pod network to the cluster.
I0927 00:24:51.211984 589846 kubeadm.go:310] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
I0927 00:24:51.212053 589846 kubeadm.go:310] https://kubernetes.io/docs/concepts/cluster-administration/addons/
I0927 00:24:51.212057 589846 kubeadm.go:310]
I0927 00:24:51.212140 589846 kubeadm.go:310] You can now join any number of control-plane nodes by copying certificate authorities
I0927 00:24:51.212215 589846 kubeadm.go:310] and service account keys on each node and then running the following as root:
I0927 00:24:51.212221 589846 kubeadm.go:310]
I0927 00:24:51.212303 589846 kubeadm.go:310] kubeadm join control-plane.minikube.internal:8443 --token 29k8v1.wlnmbgivp7440xc5 \
I0927 00:24:51.212404 589846 kubeadm.go:310] --discovery-token-ca-cert-hash sha256:12f53ed8c2ac46c62d729381f4778f67102dcff5ba944c90cf10559fb62c21c5 \
I0927 00:24:51.212425 589846 kubeadm.go:310] --control-plane
I0927 00:24:51.212429 589846 kubeadm.go:310]
I0927 00:24:51.212512 589846 kubeadm.go:310] Then you can join any number of worker nodes by running the following on each as root:
I0927 00:24:51.212517 589846 kubeadm.go:310]
I0927 00:24:51.212597 589846 kubeadm.go:310] kubeadm join control-plane.minikube.internal:8443 --token 29k8v1.wlnmbgivp7440xc5 \
I0927 00:24:51.212698 589846 kubeadm.go:310] --discovery-token-ca-cert-hash sha256:12f53ed8c2ac46c62d729381f4778f67102dcff5ba944c90cf10559fb62c21c5
I0927 00:24:51.215253 589846 kubeadm.go:310] W0927 00:24:35.312017 1015 common.go:101] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "ClusterConfiguration"). Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
I0927 00:24:51.215546 589846 kubeadm.go:310] W0927 00:24:35.313490 1015 common.go:101] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "InitConfiguration"). Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
I0927 00:24:51.215758 589846 kubeadm.go:310] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1070-aws\n", err: exit status 1
I0927 00:24:51.215865 589846 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0927 00:24:51.215885 589846 cni.go:84] Creating CNI manager for ""
I0927 00:24:51.215893 589846 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0927 00:24:51.218907 589846 out.go:177] * Configuring CNI (Container Networking Interface) ...
I0927 00:24:51.220372 589846 ssh_runner.go:195] Run: stat /opt/cni/bin/portmap
I0927 00:24:51.224172 589846 cni.go:182] applying CNI manifest using /var/lib/minikube/binaries/v1.31.1/kubectl ...
I0927 00:24:51.224192 589846 ssh_runner.go:362] scp memory --> /var/tmp/minikube/cni.yaml (2601 bytes)
I0927 00:24:51.241233 589846 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.31.1/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml
I0927 00:24:51.524579 589846 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"
I0927 00:24:51.524705 589846 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.31.1/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig
I0927 00:24:51.524788 589846 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.31.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig label --overwrite nodes addons-376302 minikube.k8s.io/updated_at=2024_09_27T00_24_51_0700 minikube.k8s.io/version=v1.34.0 minikube.k8s.io/commit=eee16a295c071ed5a0e96cbbc00bcd13b2654625 minikube.k8s.io/name=addons-376302 minikube.k8s.io/primary=true
I0927 00:24:51.761861 589846 ops.go:34] apiserver oom_adj: -16
I0927 00:24:51.761994 589846 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.31.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0927 00:24:52.262980 589846 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.31.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0927 00:24:52.762482 589846 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.31.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0927 00:24:53.262665 589846 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.31.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0927 00:24:53.762114 589846 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.31.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0927 00:24:54.262776 589846 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.31.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0927 00:24:54.762572 589846 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.31.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0927 00:24:55.262902 589846 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.31.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0927 00:24:55.417882 589846 kubeadm.go:1113] duration metric: took 3.893219968s to wait for elevateKubeSystemPrivileges
I0927 00:24:55.417907 589846 kubeadm.go:394] duration metric: took 20.279552515s to StartCluster
I0927 00:24:55.417925 589846 settings.go:142] acquiring lock: {Name:mkd70c1e53f86501638b4918726bcbef07279ae5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:55.418045 589846 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19711-583677/kubeconfig
I0927 00:24:55.418428 589846 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19711-583677/kubeconfig: {Name:mk62ce40e80630f5ee25a51f6742eda23f381c14 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0927 00:24:55.418693 589846 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0927 00:24:55.418900 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"
I0927 00:24:55.419140 589846 config.go:182] Loaded profile config "addons-376302": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0927 00:24:55.419177 589846 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:true csi-hostpath-driver:true dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:true gvisor:false headlamp:false inaccel:false ingress:true ingress-dns:true inspektor-gadget:true istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:true nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:true registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:true volcano:true volumesnapshots:true yakd:true]
I0927 00:24:55.419280 589846 addons.go:69] Setting yakd=true in profile "addons-376302"
I0927 00:24:55.419293 589846 addons.go:234] Setting addon yakd=true in "addons-376302"
I0927 00:24:55.419317 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.419775 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.420068 589846 addons.go:69] Setting inspektor-gadget=true in profile "addons-376302"
I0927 00:24:55.420085 589846 addons.go:234] Setting addon inspektor-gadget=true in "addons-376302"
I0927 00:24:55.420109 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.420519 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.421058 589846 addons.go:69] Setting cloud-spanner=true in profile "addons-376302"
I0927 00:24:55.421077 589846 addons.go:234] Setting addon cloud-spanner=true in "addons-376302"
I0927 00:24:55.421100 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.421508 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.422774 589846 addons.go:69] Setting metrics-server=true in profile "addons-376302"
I0927 00:24:55.422877 589846 addons.go:234] Setting addon metrics-server=true in "addons-376302"
I0927 00:24:55.423128 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.423015 589846 addons.go:69] Setting nvidia-device-plugin=true in profile "addons-376302"
I0927 00:24:55.424940 589846 addons.go:234] Setting addon nvidia-device-plugin=true in "addons-376302"
I0927 00:24:55.425003 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.425559 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.428451 589846 addons.go:69] Setting csi-hostpath-driver=true in profile "addons-376302"
I0927 00:24:55.428516 589846 addons.go:234] Setting addon csi-hostpath-driver=true in "addons-376302"
I0927 00:24:55.428554 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.429024 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.436002 589846 addons.go:69] Setting default-storageclass=true in profile "addons-376302"
I0927 00:24:55.436039 589846 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "addons-376302"
I0927 00:24:55.436383 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.423031 589846 addons.go:69] Setting registry=true in profile "addons-376302"
I0927 00:24:55.451826 589846 addons.go:234] Setting addon registry=true in "addons-376302"
I0927 00:24:55.451872 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.452345 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.423039 589846 addons.go:69] Setting storage-provisioner=true in profile "addons-376302"
I0927 00:24:55.465506 589846 addons.go:234] Setting addon storage-provisioner=true in "addons-376302"
I0927 00:24:55.465548 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.466035 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.472872 589846 addons.go:69] Setting gcp-auth=true in profile "addons-376302"
I0927 00:24:55.472906 589846 mustload.go:65] Loading cluster: addons-376302
I0927 00:24:55.473176 589846 config.go:182] Loaded profile config "addons-376302": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0927 00:24:55.473459 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.423046 589846 addons.go:69] Setting storage-provisioner-rancher=true in profile "addons-376302"
I0927 00:24:55.482771 589846 addons_storage_classes.go:33] enableOrDisableStorageClasses storage-provisioner-rancher=true on "addons-376302"
I0927 00:24:55.483138 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.500430 589846 addons.go:69] Setting ingress=true in profile "addons-376302"
I0927 00:24:55.500466 589846 addons.go:234] Setting addon ingress=true in "addons-376302"
I0927 00:24:55.500509 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.500996 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.423054 589846 addons.go:69] Setting volcano=true in profile "addons-376302"
I0927 00:24:55.506286 589846 addons.go:234] Setting addon volcano=true in "addons-376302"
I0927 00:24:55.506327 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.506854 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.526625 589846 addons.go:69] Setting ingress-dns=true in profile "addons-376302"
I0927 00:24:55.526654 589846 addons.go:234] Setting addon ingress-dns=true in "addons-376302"
I0927 00:24:55.526700 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.527198 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.423061 589846 addons.go:69] Setting volumesnapshots=true in profile "addons-376302"
I0927 00:24:55.530960 589846 addons.go:234] Setting addon volumesnapshots=true in "addons-376302"
I0927 00:24:55.531022 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.531722 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.557440 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.578665 589846 out.go:177] * Verifying Kubernetes components...
I0927 00:24:55.585423 589846 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0927 00:24:55.605139 589846 out.go:177] - Using image registry.k8s.io/sig-storage/csi-resizer:v1.6.0
I0927 00:24:55.604847 589846 out.go:177] - Using image docker.io/marcnuri/yakd:0.0.5
I0927 00:24:55.608961 589846 out.go:177] - Using image gcr.io/cloud-spanner-emulator/emulator:1.5.24
I0927 00:24:55.616369 589846 out.go:177] - Using image gcr.io/k8s-minikube/kube-registry-proxy:0.0.7
I0927 00:24:55.616675 589846 out.go:177] - Using image ghcr.io/inspektor-gadget/inspektor-gadget:v0.32.0
I0927 00:24:55.616866 589846 addons.go:431] installing /etc/kubernetes/addons/yakd-ns.yaml
I0927 00:24:55.616879 589846 ssh_runner.go:362] scp yakd/yakd-ns.yaml --> /etc/kubernetes/addons/yakd-ns.yaml (171 bytes)
I0927 00:24:55.616943 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.617287 589846 addons.go:431] installing /etc/kubernetes/addons/deployment.yaml
I0927 00:24:55.617298 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/deployment.yaml (1004 bytes)
I0927 00:24:55.617337 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.629457 589846 addons.go:234] Setting addon default-storageclass=true in "addons-376302"
I0927 00:24:55.629548 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.630048 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.646963 589846 out.go:177] - Using image nvcr.io/nvidia/k8s-device-plugin:v0.16.2
I0927 00:24:55.653055 589846 addons.go:431] installing /etc/kubernetes/addons/nvidia-device-plugin.yaml
I0927 00:24:55.653078 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/nvidia-device-plugin.yaml (1966 bytes)
I0927 00:24:55.653144 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.670301 589846 addons.go:431] installing /etc/kubernetes/addons/ig-namespace.yaml
I0927 00:24:55.670326 589846 ssh_runner.go:362] scp inspektor-gadget/ig-namespace.yaml --> /etc/kubernetes/addons/ig-namespace.yaml (55 bytes)
I0927 00:24:55.670395 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.678056 589846 out.go:177] - Using image registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0
I0927 00:24:55.680080 589846 out.go:177] - Using image registry.k8s.io/sig-storage/csi-provisioner:v3.3.0
I0927 00:24:55.683193 589846 out.go:177] - Using image registry.k8s.io/sig-storage/csi-attacher:v4.0.0
I0927 00:24:55.687176 589846 out.go:177] - Using image registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.7.0
I0927 00:24:55.690025 589846 out.go:177] - Using image registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.6.0
I0927 00:24:55.691813 589846 out.go:177] - Using image registry.k8s.io/sig-storage/hostpathplugin:v1.9.0
I0927 00:24:55.694112 589846 out.go:177] - Using image docker.io/registry:2.8.3
I0927 00:24:55.716613 589846 addons.go:431] installing /etc/kubernetes/addons/registry-rc.yaml
I0927 00:24:55.716633 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/registry-rc.yaml (860 bytes)
I0927 00:24:55.716692 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.729211 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.732245 589846 out.go:177] - Using image registry.k8s.io/sig-storage/snapshot-controller:v6.1.0
I0927 00:24:55.732442 589846 out.go:177] - Using image registry.k8s.io/sig-storage/livenessprobe:v2.8.0
I0927 00:24:55.733914 589846 addons.go:431] installing /etc/kubernetes/addons/rbac-external-attacher.yaml
I0927 00:24:55.733933 589846 ssh_runner.go:362] scp csi-hostpath-driver/rbac/rbac-external-attacher.yaml --> /etc/kubernetes/addons/rbac-external-attacher.yaml (3073 bytes)
I0927 00:24:55.733997 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.739092 589846 out.go:177] - Using image gcr.io/k8s-minikube/minikube-ingress-dns:0.0.3
I0927 00:24:55.739826 589846 out.go:177] - Using image registry.k8s.io/metrics-server/metrics-server:v0.7.2
I0927 00:24:55.740436 589846 addons.go:431] installing /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml
I0927 00:24:55.740451 589846 ssh_runner.go:362] scp volumesnapshots/csi-hostpath-snapshotclass.yaml --> /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml (934 bytes)
I0927 00:24:55.740532 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.741843 589846 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0927 00:24:55.741868 589846 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0927 00:24:55.741923 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.761267 589846 addons.go:431] installing /etc/kubernetes/addons/ingress-dns-pod.yaml
I0927 00:24:55.761292 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ingress-dns-pod.yaml (2442 bytes)
I0927 00:24:55.761354 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.790422 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:55.792606 589846 out.go:177] - Using image registry.k8s.io/ingress-nginx/controller:v1.11.2
I0927 00:24:55.793994 589846 addons.go:234] Setting addon storage-provisioner-rancher=true in "addons-376302"
I0927 00:24:55.794077 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:24:55.794702 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:24:55.795883 589846 out.go:177] - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3
I0927 00:24:55.801152 589846 out.go:177] - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3
I0927 00:24:55.803254 589846 addons.go:431] installing /etc/kubernetes/addons/ingress-deploy.yaml
I0927 00:24:55.803278 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ingress-deploy.yaml (16078 bytes)
I0927 00:24:55.803354 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.822344 589846 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0927 00:24:55.827851 589846 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0927 00:24:55.827880 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0927 00:24:55.827946 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.860964 589846 out.go:177] - Using image docker.io/volcanosh/vc-webhook-manager:v1.10.0
I0927 00:24:55.862595 589846 out.go:177] - Using image docker.io/volcanosh/vc-controller-manager:v1.10.0
I0927 00:24:55.866051 589846 out.go:177] - Using image docker.io/volcanosh/vc-scheduler:v1.10.0
I0927 00:24:55.869106 589846 addons.go:431] installing /etc/kubernetes/addons/volcano-deployment.yaml
I0927 00:24:55.869131 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/volcano-deployment.yaml (471825 bytes)
I0927 00:24:55.869199 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.896607 589846 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0927 00:24:55.896631 589846 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0927 00:24:55.896690 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:55.916880 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:55.933278 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:55.946752 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:55.960960 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:55.988046 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:55.993978 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:55.997566 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:56.028905 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:56.032853 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:56.041199 589846 out.go:177] - Using image docker.io/rancher/local-path-provisioner:v0.0.22
I0927 00:24:56.043036 589846 out.go:177] - Using image docker.io/busybox:stable
I0927 00:24:56.047860 589846 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner-rancher.yaml
I0927 00:24:56.047895 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner-rancher.yaml (3113 bytes)
I0927 00:24:56.047963 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:24:56.049909 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:56.066019 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
W0927 00:24:56.069313 589846 sshutil.go:64] dial failure (will retry): ssh: handshake failed: EOF
I0927 00:24:56.069343 589846 retry.go:31] will retry after 362.988195ms: ssh: handshake failed: EOF
I0927 00:24:56.070430 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
W0927 00:24:56.072856 589846 sshutil.go:64] dial failure (will retry): ssh: handshake failed: EOF
I0927 00:24:56.072886 589846 retry.go:31] will retry after 370.033267ms: ssh: handshake failed: EOF
I0927 00:24:56.090783 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:24:56.311494 589846 addons.go:431] installing /etc/kubernetes/addons/registry-svc.yaml
I0927 00:24:56.311564 589846 ssh_runner.go:362] scp registry/registry-svc.yaml --> /etc/kubernetes/addons/registry-svc.yaml (398 bytes)
I0927 00:24:56.403217 589846 addons.go:431] installing /etc/kubernetes/addons/registry-proxy.yaml
I0927 00:24:56.403288 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/registry-proxy.yaml (947 bytes)
I0927 00:24:56.415850 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^ forward . \/etc\/resolv.conf.*/i \ hosts {\n 192.168.49.1 host.minikube.internal\n fallthrough\n }' -e '/^ errors *$/i \ log' | sudo /var/lib/minikube/binaries/v1.31.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"
I0927 00:24:56.416034 589846 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0927 00:24:56.513106 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml
I0927 00:24:56.529574 589846 addons.go:431] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
I0927 00:24:56.529603 589846 ssh_runner.go:362] scp volumesnapshots/snapshot.storage.k8s.io_volumesnapshotclasses.yaml --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml (6471 bytes)
I0927 00:24:56.533930 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/deployment.yaml
I0927 00:24:56.574838 589846 addons.go:431] installing /etc/kubernetes/addons/ig-serviceaccount.yaml
I0927 00:24:56.574870 589846 ssh_runner.go:362] scp inspektor-gadget/ig-serviceaccount.yaml --> /etc/kubernetes/addons/ig-serviceaccount.yaml (80 bytes)
I0927 00:24:56.591994 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/registry-rc.yaml -f /etc/kubernetes/addons/registry-svc.yaml -f /etc/kubernetes/addons/registry-proxy.yaml
I0927 00:24:56.605665 589846 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0927 00:24:56.605699 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1907 bytes)
I0927 00:24:56.608771 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/nvidia-device-plugin.yaml
I0927 00:24:56.626555 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/ingress-dns-pod.yaml
I0927 00:24:56.634743 589846 addons.go:431] installing /etc/kubernetes/addons/rbac-hostpath.yaml
I0927 00:24:56.634765 589846 ssh_runner.go:362] scp csi-hostpath-driver/rbac/rbac-hostpath.yaml --> /etc/kubernetes/addons/rbac-hostpath.yaml (4266 bytes)
I0927 00:24:56.644549 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0927 00:24:56.675960 589846 addons.go:431] installing /etc/kubernetes/addons/yakd-sa.yaml
I0927 00:24:56.676040 589846 ssh_runner.go:362] scp yakd/yakd-sa.yaml --> /etc/kubernetes/addons/yakd-sa.yaml (247 bytes)
I0927 00:24:56.679939 589846 addons.go:431] installing /etc/kubernetes/addons/ig-role.yaml
I0927 00:24:56.680011 589846 ssh_runner.go:362] scp inspektor-gadget/ig-role.yaml --> /etc/kubernetes/addons/ig-role.yaml (210 bytes)
I0927 00:24:56.690612 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner-rancher.yaml
I0927 00:24:56.757919 589846 addons.go:431] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
I0927 00:24:56.757995 589846 ssh_runner.go:362] scp volumesnapshots/snapshot.storage.k8s.io_volumesnapshotcontents.yaml --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml (23126 bytes)
I0927 00:24:56.767935 589846 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0927 00:24:56.768011 589846 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0927 00:24:56.868777 589846 addons.go:431] installing /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml
I0927 00:24:56.868833 589846 ssh_runner.go:362] scp csi-hostpath-driver/rbac/rbac-external-health-monitor-controller.yaml --> /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml (3038 bytes)
I0927 00:24:56.917143 589846 addons.go:431] installing /etc/kubernetes/addons/ig-rolebinding.yaml
I0927 00:24:56.917170 589846 ssh_runner.go:362] scp inspektor-gadget/ig-rolebinding.yaml --> /etc/kubernetes/addons/ig-rolebinding.yaml (244 bytes)
I0927 00:24:56.945686 589846 addons.go:431] installing /etc/kubernetes/addons/yakd-crb.yaml
I0927 00:24:56.945728 589846 ssh_runner.go:362] scp yakd/yakd-crb.yaml --> /etc/kubernetes/addons/yakd-crb.yaml (422 bytes)
I0927 00:24:57.012406 589846 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0927 00:24:57.012433 589846 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0927 00:24:57.041846 589846 addons.go:431] installing /etc/kubernetes/addons/rbac-external-provisioner.yaml
I0927 00:24:57.041873 589846 ssh_runner.go:362] scp csi-hostpath-driver/rbac/rbac-external-provisioner.yaml --> /etc/kubernetes/addons/rbac-external-provisioner.yaml (4442 bytes)
I0927 00:24:57.066351 589846 addons.go:431] installing /etc/kubernetes/addons/ig-clusterrole.yaml
I0927 00:24:57.066423 589846 ssh_runner.go:362] scp inspektor-gadget/ig-clusterrole.yaml --> /etc/kubernetes/addons/ig-clusterrole.yaml (1485 bytes)
I0927 00:24:57.068477 589846 addons.go:431] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml
I0927 00:24:57.068499 589846 ssh_runner.go:362] scp volumesnapshots/snapshot.storage.k8s.io_volumesnapshots.yaml --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml (19582 bytes)
I0927 00:24:57.088049 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0927 00:24:57.094177 589846 addons.go:431] installing /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml
I0927 00:24:57.094204 589846 ssh_runner.go:362] scp volumesnapshots/rbac-volume-snapshot-controller.yaml --> /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml (3545 bytes)
I0927 00:24:57.156859 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/volcano-deployment.yaml
I0927 00:24:57.215615 589846 addons.go:431] installing /etc/kubernetes/addons/yakd-svc.yaml
I0927 00:24:57.215640 589846 ssh_runner.go:362] scp yakd/yakd-svc.yaml --> /etc/kubernetes/addons/yakd-svc.yaml (412 bytes)
I0927 00:24:57.230596 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0927 00:24:57.265459 589846 addons.go:431] installing /etc/kubernetes/addons/ig-clusterrolebinding.yaml
I0927 00:24:57.265486 589846 ssh_runner.go:362] scp inspektor-gadget/ig-clusterrolebinding.yaml --> /etc/kubernetes/addons/ig-clusterrolebinding.yaml (274 bytes)
I0927 00:24:57.271482 589846 addons.go:431] installing /etc/kubernetes/addons/rbac-external-resizer.yaml
I0927 00:24:57.271508 589846 ssh_runner.go:362] scp csi-hostpath-driver/rbac/rbac-external-resizer.yaml --> /etc/kubernetes/addons/rbac-external-resizer.yaml (2943 bytes)
I0927 00:24:57.284207 589846 addons.go:431] installing /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml
I0927 00:24:57.284233 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml (1475 bytes)
I0927 00:24:57.441183 589846 addons.go:431] installing /etc/kubernetes/addons/yakd-dp.yaml
I0927 00:24:57.441207 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/yakd-dp.yaml (2017 bytes)
I0927 00:24:57.451518 589846 addons.go:431] installing /etc/kubernetes/addons/ig-crd.yaml
I0927 00:24:57.451544 589846 ssh_runner.go:362] scp inspektor-gadget/ig-crd.yaml --> /etc/kubernetes/addons/ig-crd.yaml (5216 bytes)
I0927 00:24:57.469611 589846 addons.go:431] installing /etc/kubernetes/addons/rbac-external-snapshotter.yaml
I0927 00:24:57.469637 589846 ssh_runner.go:362] scp csi-hostpath-driver/rbac/rbac-external-snapshotter.yaml --> /etc/kubernetes/addons/rbac-external-snapshotter.yaml (3149 bytes)
I0927 00:24:57.628683 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml
I0927 00:24:57.770080 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/yakd-ns.yaml -f /etc/kubernetes/addons/yakd-sa.yaml -f /etc/kubernetes/addons/yakd-crb.yaml -f /etc/kubernetes/addons/yakd-svc.yaml -f /etc/kubernetes/addons/yakd-dp.yaml
I0927 00:24:57.818125 589846 addons.go:431] installing /etc/kubernetes/addons/csi-hostpath-attacher.yaml
I0927 00:24:57.818152 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-attacher.yaml (2143 bytes)
I0927 00:24:57.830230 589846 addons.go:431] installing /etc/kubernetes/addons/ig-daemonset.yaml
I0927 00:24:57.830256 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-daemonset.yaml (7735 bytes)
I0927 00:24:58.125453 589846 addons.go:431] installing /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml
I0927 00:24:58.125479 589846 ssh_runner.go:362] scp csi-hostpath-driver/deploy/csi-hostpath-driverinfo.yaml --> /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml (1274 bytes)
I0927 00:24:58.208557 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/ig-namespace.yaml -f /etc/kubernetes/addons/ig-serviceaccount.yaml -f /etc/kubernetes/addons/ig-role.yaml -f /etc/kubernetes/addons/ig-rolebinding.yaml -f /etc/kubernetes/addons/ig-clusterrole.yaml -f /etc/kubernetes/addons/ig-clusterrolebinding.yaml -f /etc/kubernetes/addons/ig-crd.yaml -f /etc/kubernetes/addons/ig-daemonset.yaml
I0927 00:24:58.389123 589846 ssh_runner.go:235] Completed: sudo systemctl start kubelet: (1.973039473s)
I0927 00:24:58.389964 589846 node_ready.go:35] waiting up to 6m0s for node "addons-376302" to be "Ready" ...
I0927 00:24:58.390167 589846 ssh_runner.go:235] Completed: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^ forward . \/etc\/resolv.conf.*/i \ hosts {\n 192.168.49.1 host.minikube.internal\n fallthrough\n }' -e '/^ errors *$/i \ log' | sudo /var/lib/minikube/binaries/v1.31.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -": (1.974245129s)
I0927 00:24:58.390188 589846 start.go:971] {"host.minikube.internal": 192.168.49.1} host record injected into CoreDNS's ConfigMap
I0927 00:24:58.394969 589846 node_ready.go:49] node "addons-376302" has status "Ready":"True"
I0927 00:24:58.395000 589846 node_ready.go:38] duration metric: took 5.001046ms for node "addons-376302" to be "Ready" ...
I0927 00:24:58.395012 589846 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0927 00:24:58.406333 589846 pod_ready.go:79] waiting up to 6m0s for pod "coredns-7c65d6cfc9-4jzbp" in "kube-system" namespace to be "Ready" ...
I0927 00:24:58.544147 589846 addons.go:431] installing /etc/kubernetes/addons/csi-hostpath-plugin.yaml
I0927 00:24:58.544170 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-plugin.yaml (8201 bytes)
I0927 00:24:58.783630 589846 addons.go:431] installing /etc/kubernetes/addons/csi-hostpath-resizer.yaml
I0927 00:24:58.783655 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-resizer.yaml (2191 bytes)
I0927 00:24:58.910537 589846 kapi.go:214] "coredns" deployment in "kube-system" namespace and "addons-376302" context rescaled to 1 replicas
I0927 00:24:58.929556 589846 pod_ready.go:98] pod "coredns-7c65d6cfc9-4jzbp" in "kube-system" namespace has status phase "Failed" (skipping!): {Phase:Failed Conditions:[{Type:PodReadyToStartContainers Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2024-09-27 00:24:55 +0000 UTC Reason: Message:} {Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2024-09-27 00:24:55 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2024-09-27 00:24:55 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [coredns]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2024-09-27 00:24:55 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [coredns]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2024-09-27 00:24:55 +0000 UTC Reason: Message:}] Messa
ge: Reason: NominatedNodeName: HostIP:192.168.49.2 HostIPs:[{IP:192.168.49.2}] PodIP: PodIPs:[] StartTime:2024-09-27 00:24:55 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:coredns State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:137,Signal:0,Reason:ContainerStatusUnknown,Message:The container could not be located when the pod was terminated,StartedAt:0001-01-01 00:00:00 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,ContainerID:,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/coredns/coredns:v1.11.3 ImageID: ContainerID: Started:0x400049d58a AllocatedResources:map[] Resources:nil VolumeMounts:[{Name:config-volume MountPath:/etc/coredns ReadOnly:true RecursiveReadOnly:0x40015f7e40} {Name:kube-api-access-vcq9j MountPath:/var/run/secrets/kubernetes.io/serviceaccount ReadOnly:true RecursiveReadOnly:0x40015f7e50}] User:nil AllocatedResourcesStatus:[]}] QOSClass:Burstable EphemeralContainerStatuses:[] Res
ize: ResourceClaimStatuses:[]}
I0927 00:24:58.929651 589846 pod_ready.go:82] duration metric: took 523.278312ms for pod "coredns-7c65d6cfc9-4jzbp" in "kube-system" namespace to be "Ready" ...
E0927 00:24:58.929678 589846 pod_ready.go:67] WaitExtra: waitPodCondition: pod "coredns-7c65d6cfc9-4jzbp" in "kube-system" namespace has status phase "Failed" (skipping!): {Phase:Failed Conditions:[{Type:PodReadyToStartContainers Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2024-09-27 00:24:55 +0000 UTC Reason: Message:} {Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2024-09-27 00:24:55 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2024-09-27 00:24:55 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [coredns]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2024-09-27 00:24:55 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [coredns]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2024-09-27 00:24:55 +0000
UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:192.168.49.2 HostIPs:[{IP:192.168.49.2}] PodIP: PodIPs:[] StartTime:2024-09-27 00:24:55 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:coredns State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:137,Signal:0,Reason:ContainerStatusUnknown,Message:The container could not be located when the pod was terminated,StartedAt:0001-01-01 00:00:00 +0000 UTC,FinishedAt:0001-01-01 00:00:00 +0000 UTC,ContainerID:,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/coredns/coredns:v1.11.3 ImageID: ContainerID: Started:0x400049d58a AllocatedResources:map[] Resources:nil VolumeMounts:[{Name:config-volume MountPath:/etc/coredns ReadOnly:true RecursiveReadOnly:0x40015f7e40} {Name:kube-api-access-vcq9j MountPath:/var/run/secrets/kubernetes.io/serviceaccount ReadOnly:true RecursiveReadOnly:0x40015f7e50}] User:nil AllocatedResourcesStatus:[]}] QOSClass:Burstable Ephe
meralContainerStatuses:[] Resize: ResourceClaimStatuses:[]}
I0927 00:24:58.929729 589846 pod_ready.go:79] waiting up to 6m0s for pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace to be "Ready" ...
I0927 00:24:59.185428 589846 addons.go:431] installing /etc/kubernetes/addons/csi-hostpath-storageclass.yaml
I0927 00:24:59.185502 589846 ssh_runner.go:362] scp csi-hostpath-driver/deploy/csi-hostpath-storageclass.yaml --> /etc/kubernetes/addons/csi-hostpath-storageclass.yaml (846 bytes)
I0927 00:24:59.628263 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml
I0927 00:25:00.936626 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:02.940996 589846 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_application_credentials.json (162 bytes)
I0927 00:25:02.941145 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:25:02.944397 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:02.980794 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:25:03.385703 589846 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_cloud_project (12 bytes)
I0927 00:25:03.529856 589846 addons.go:234] Setting addon gcp-auth=true in "addons-376302"
I0927 00:25:03.529958 589846 host.go:66] Checking if "addons-376302" exists ...
I0927 00:25:03.530488 589846 cli_runner.go:164] Run: docker container inspect addons-376302 --format={{.State.Status}}
I0927 00:25:03.547582 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml: (7.034434784s)
I0927 00:25:03.547613 589846 addons.go:475] Verifying addon ingress=true in "addons-376302"
I0927 00:25:03.547761 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/deployment.yaml: (7.013802762s)
I0927 00:25:03.547807 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/registry-rc.yaml -f /etc/kubernetes/addons/registry-svc.yaml -f /etc/kubernetes/addons/registry-proxy.yaml: (6.95578523s)
I0927 00:25:03.547816 589846 addons.go:475] Verifying addon registry=true in "addons-376302"
I0927 00:25:03.548049 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/nvidia-device-plugin.yaml: (6.939249586s)
I0927 00:25:03.548109 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/ingress-dns-pod.yaml: (6.921532514s)
I0927 00:25:03.548147 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: (6.90357934s)
I0927 00:25:03.548373 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner-rancher.yaml: (6.857683473s)
I0927 00:25:03.548488 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (6.460414699s)
I0927 00:25:03.552029 589846 out.go:177] * Verifying ingress addon...
I0927 00:25:03.553821 589846 out.go:177] * Verifying registry addon...
I0927 00:25:03.557700 589846 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=registry" in ns "kube-system" ...
I0927 00:25:03.558030 589846 kapi.go:75] Waiting for pod with label "app.kubernetes.io/name=ingress-nginx" in ns "ingress-nginx" ...
I0927 00:25:03.565699 589846 ssh_runner.go:195] Run: cat /var/lib/minikube/google_application_credentials.json
I0927 00:25:03.565754 589846 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-376302
I0927 00:25:03.572181 589846 kapi.go:86] Found 2 Pods for label selector kubernetes.io/minikube-addons=registry
I0927 00:25:03.572205 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
W0927 00:25:03.573018 589846 out.go:270] ! Enabling 'default-storageclass' returned an error: running callbacks: [Error making standard the default storage class: Error while marking storage class local-path as non-default: Operation cannot be fulfilled on storageclasses.storage.k8s.io "local-path": the object has been modified; please apply your changes to the latest version and try again]
I0927 00:25:03.594232 589846 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33509 SSHKeyPath:/home/jenkins/minikube-integration/19711-583677/.minikube/machines/addons-376302/id_rsa Username:docker}
I0927 00:25:03.668872 589846 kapi.go:86] Found 3 Pods for label selector app.kubernetes.io/name=ingress-nginx
I0927 00:25:03.668961 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:04.064179 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:04.066411 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:04.569882 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:04.570560 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:05.114639 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:05.153066 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:05.154113 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:05.623046 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:05.624326 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:06.037351 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/volcano-deployment.yaml: (8.880455714s)
I0927 00:25:06.037483 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (8.806856256s)
I0927 00:25:06.037537 589846 addons.go:475] Verifying addon metrics-server=true in "addons-376302"
I0927 00:25:06.037567 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: (8.408838505s)
W0927 00:25:06.037600 589846 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1
stdout:
customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created
customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created
customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created
serviceaccount/snapshot-controller created
clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created
clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created
role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created
rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created
deployment.apps/snapshot-controller created
stderr:
error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"
ensure CRDs are installed first
I0927 00:25:06.037630 589846 retry.go:31] will retry after 255.489638ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1
stdout:
customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created
customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created
customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created
serviceaccount/snapshot-controller created
clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created
clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created
role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created
rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created
deployment.apps/snapshot-controller created
stderr:
error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"
ensure CRDs are installed first
I0927 00:25:06.037720 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/ig-namespace.yaml -f /etc/kubernetes/addons/ig-serviceaccount.yaml -f /etc/kubernetes/addons/ig-role.yaml -f /etc/kubernetes/addons/ig-rolebinding.yaml -f /etc/kubernetes/addons/ig-clusterrole.yaml -f /etc/kubernetes/addons/ig-clusterrolebinding.yaml -f /etc/kubernetes/addons/ig-crd.yaml -f /etc/kubernetes/addons/ig-daemonset.yaml: (7.829119786s)
I0927 00:25:06.037764 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/yakd-ns.yaml -f /etc/kubernetes/addons/yakd-sa.yaml -f /etc/kubernetes/addons/yakd-crb.yaml -f /etc/kubernetes/addons/yakd-svc.yaml -f /etc/kubernetes/addons/yakd-dp.yaml: (8.267529961s)
I0927 00:25:06.037922 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml: (6.409578663s)
I0927 00:25:06.037940 589846 addons.go:475] Verifying addon csi-hostpath-driver=true in "addons-376302"
I0927 00:25:06.038078 589846 ssh_runner.go:235] Completed: cat /var/lib/minikube/google_application_credentials.json: (2.472356548s)
I0927 00:25:06.040094 589846 out.go:177] - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3
I0927 00:25:06.040103 589846 out.go:177] * Verifying csi-hostpath-driver addon...
I0927 00:25:06.040234 589846 out.go:177] * To access YAKD - Kubernetes Dashboard, wait for Pod to be ready and run the following command:
minikube -p addons-376302 service yakd-dashboard -n yakd-dashboard
I0927 00:25:06.044310 589846 out.go:177] - Using image gcr.io/k8s-minikube/gcp-auth-webhook:v0.1.2
I0927 00:25:06.045140 589846 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=csi-hostpath-driver" in ns "kube-system" ...
I0927 00:25:06.047901 589846 addons.go:431] installing /etc/kubernetes/addons/gcp-auth-ns.yaml
I0927 00:25:06.047939 589846 ssh_runner.go:362] scp gcp-auth/gcp-auth-ns.yaml --> /etc/kubernetes/addons/gcp-auth-ns.yaml (700 bytes)
I0927 00:25:06.107559 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:06.108940 589846 kapi.go:86] Found 3 Pods for label selector kubernetes.io/minikube-addons=csi-hostpath-driver
I0927 00:25:06.109022 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:06.109552 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:06.128377 589846 addons.go:431] installing /etc/kubernetes/addons/gcp-auth-service.yaml
I0927 00:25:06.128445 589846 ssh_runner.go:362] scp gcp-auth/gcp-auth-service.yaml --> /etc/kubernetes/addons/gcp-auth-service.yaml (788 bytes)
I0927 00:25:06.213100 589846 addons.go:431] installing /etc/kubernetes/addons/gcp-auth-webhook.yaml
I0927 00:25:06.213174 589846 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-webhook.yaml (5421 bytes)
I0927 00:25:06.293460 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply --force -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml
I0927 00:25:06.340245 589846 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/gcp-auth-ns.yaml -f /etc/kubernetes/addons/gcp-auth-service.yaml -f /etc/kubernetes/addons/gcp-auth-webhook.yaml
I0927 00:25:06.554826 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:06.562429 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:06.563355 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:07.067164 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:07.076543 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:07.077342 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:07.462996 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:07.551700 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:07.563797 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:07.565320 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:07.835509 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply --force -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: (1.541957517s)
I0927 00:25:07.835616 589846 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/gcp-auth-ns.yaml -f /etc/kubernetes/addons/gcp-auth-service.yaml -f /etc/kubernetes/addons/gcp-auth-webhook.yaml: (1.495289559s)
I0927 00:25:07.839976 589846 addons.go:475] Verifying addon gcp-auth=true in "addons-376302"
I0927 00:25:07.847197 589846 out.go:177] * Verifying gcp-auth addon...
I0927 00:25:07.849936 589846 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=gcp-auth" in ns "gcp-auth" ...
I0927 00:25:07.853345 589846 kapi.go:86] Found 0 Pods for label selector kubernetes.io/minikube-addons=gcp-auth
I0927 00:25:08.051555 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:08.064244 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:08.065811 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:08.551549 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:08.565609 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:08.567142 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:09.050631 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:09.062936 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:09.063699 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:09.553188 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:09.564191 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:09.567455 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:09.936457 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:10.054641 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:10.063765 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:10.065472 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:10.550249 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:10.561730 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:10.562644 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:11.052730 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:11.062551 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:11.064352 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:11.550967 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:11.562531 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:11.565269 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:11.937009 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:12.050162 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:12.063595 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:12.063914 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:12.554862 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:12.566481 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:12.568250 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:13.050648 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:13.062702 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:13.063637 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:13.550708 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:13.563680 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:13.564777 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:14.050085 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:14.062964 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:14.063426 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:14.436433 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:14.550397 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:14.561740 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:14.562939 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:15.057347 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:15.064455 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:15.065420 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:15.549589 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:15.563714 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:15.564023 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:16.050978 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:16.063941 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:16.065364 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:16.550062 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:16.562216 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:16.563157 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:16.937942 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:17.050308 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:17.063538 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:17.064483 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:17.549822 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:17.561251 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:17.562916 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:18.052391 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:18.062828 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:18.063510 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:18.552009 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:18.567590 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:18.567763 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:19.051196 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:19.062511 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:19.063617 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:19.436957 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:19.550355 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:19.651368 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:19.653227 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:20.049789 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:20.062324 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:20.063796 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:20.550355 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:20.563498 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:20.564515 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:21.050177 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:21.062137 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:21.063166 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:21.550347 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:21.562116 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:21.562775 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:21.936138 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:22.049577 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:22.062548 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:22.063690 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:22.550583 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:22.562367 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:22.563344 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:23.050962 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:23.062273 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:23.064150 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:23.550000 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:23.562542 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:23.563126 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:23.936539 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:24.050234 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:24.063005 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:24.064048 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:24.550392 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:24.563305 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:24.564972 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:25.050114 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:25.062751 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:25.063355 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:25.550481 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:25.563058 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:25.564060 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:26.049688 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:26.061844 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:26.063216 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:26.435831 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:26.550428 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:26.562021 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:26.563399 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:27.050405 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:27.062407 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:27.063276 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:27.551411 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:27.650912 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:27.651566 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:28.049590 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:28.062649 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:28.063084 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:28.436354 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:28.550165 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:28.562285 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:28.562925 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:29.050447 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:29.064368 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:29.064994 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:29.550573 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:29.562130 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:29.563687 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:30.051814 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:30.071804 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:30.072205 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:30.550356 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:30.561339 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:30.562513 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:30.937285 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:31.050737 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:31.063826 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:31.064686 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:31.551699 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:31.651001 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:31.652767 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:32.049617 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:32.062329 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:32.062740 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:32.550772 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:32.563172 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:32.564157 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:33.049906 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:33.062740 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:33.063765 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:33.435954 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:33.551448 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:33.651819 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:33.652421 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:34.051588 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:34.065163 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:34.066160 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:34.550552 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:34.561995 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:34.563603 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:35.050902 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:35.063043 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:35.063362 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:35.436453 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:35.552528 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:35.561855 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:35.563342 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:36.050895 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:36.063145 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:36.063740 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:36.549693 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:36.561194 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:36.563705 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:37.050897 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:37.061595 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:37.063363 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:37.551101 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:37.562367 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:37.563325 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:37.937076 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:38.050498 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:38.063240 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:38.064096 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:38.550442 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:38.566594 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:38.568183 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:39.051793 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:39.062392 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:39.063483 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:39.550999 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:39.562208 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:39.563048 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:40.053168 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:40.062658 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:40.063919 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:40.436729 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:40.549694 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:40.563830 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:40.564158 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:41.051133 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:41.063148 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:41.063489 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:41.550518 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:41.563982 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:41.565542 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:42.050657 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:42.071345 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:42.071819 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:42.438473 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:42.550492 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:42.565293 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:42.566703 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:43.050103 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:43.062375 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:43.063759 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:43.550177 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:43.562628 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:43.563227 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:44.050527 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:44.062085 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:44.063103 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:44.551546 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:44.563550 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:44.565436 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:44.935486 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:45.052500 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:45.081910 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:45.097383 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:45.551834 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:45.563925 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:45.564695 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:46.056487 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:46.063403 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:46.063738 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:46.550276 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:46.562427 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:46.562642 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:46.936267 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:47.050186 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:47.062158 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:47.063216 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:47.549893 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:47.562928 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:47.565246 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:48.051322 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:48.063029 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:48.064027 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:48.550474 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:48.563200 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:48.564463 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:48.939756 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:49.049849 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:49.063182 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:49.064379 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:49.551710 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:49.562185 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:49.563144 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:50.050108 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:50.062583 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]
I0927 00:25:50.064779 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:50.551169 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:50.563209 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:50.563489 589846 kapi.go:107] duration metric: took 47.005792749s to wait for kubernetes.io/minikube-addons=registry ...
I0927 00:25:51.050296 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:51.062295 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:51.436619 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:51.554645 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:51.563108 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:52.050534 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:52.062415 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:52.549744 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:52.562659 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:53.051238 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:53.062956 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:53.552113 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:53.652149 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:53.937212 589846 pod_ready.go:103] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:54.052297 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:54.064610 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:54.553553 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:54.575357 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:54.938368 589846 pod_ready.go:93] pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace has status "Ready":"True"
I0927 00:25:54.938440 589846 pod_ready.go:82] duration metric: took 56.008687822s for pod "coredns-7c65d6cfc9-7kj2z" in "kube-system" namespace to be "Ready" ...
I0927 00:25:54.938482 589846 pod_ready.go:79] waiting up to 6m0s for pod "etcd-addons-376302" in "kube-system" namespace to be "Ready" ...
I0927 00:25:54.949941 589846 pod_ready.go:93] pod "etcd-addons-376302" in "kube-system" namespace has status "Ready":"True"
I0927 00:25:54.950012 589846 pod_ready.go:82] duration metric: took 11.503739ms for pod "etcd-addons-376302" in "kube-system" namespace to be "Ready" ...
I0927 00:25:54.950042 589846 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-addons-376302" in "kube-system" namespace to be "Ready" ...
I0927 00:25:54.962214 589846 pod_ready.go:93] pod "kube-apiserver-addons-376302" in "kube-system" namespace has status "Ready":"True"
I0927 00:25:54.962284 589846 pod_ready.go:82] duration metric: took 12.221061ms for pod "kube-apiserver-addons-376302" in "kube-system" namespace to be "Ready" ...
I0927 00:25:54.962311 589846 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-addons-376302" in "kube-system" namespace to be "Ready" ...
I0927 00:25:54.971728 589846 pod_ready.go:93] pod "kube-controller-manager-addons-376302" in "kube-system" namespace has status "Ready":"True"
I0927 00:25:54.971795 589846 pod_ready.go:82] duration metric: took 9.464051ms for pod "kube-controller-manager-addons-376302" in "kube-system" namespace to be "Ready" ...
I0927 00:25:54.971822 589846 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-m5q7w" in "kube-system" namespace to be "Ready" ...
I0927 00:25:55.002111 589846 pod_ready.go:93] pod "kube-proxy-m5q7w" in "kube-system" namespace has status "Ready":"True"
I0927 00:25:55.002199 589846 pod_ready.go:82] duration metric: took 30.354983ms for pod "kube-proxy-m5q7w" in "kube-system" namespace to be "Ready" ...
I0927 00:25:55.002227 589846 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-addons-376302" in "kube-system" namespace to be "Ready" ...
I0927 00:25:55.052355 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:55.063933 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:55.334774 589846 pod_ready.go:93] pod "kube-scheduler-addons-376302" in "kube-system" namespace has status "Ready":"True"
I0927 00:25:55.334852 589846 pod_ready.go:82] duration metric: took 332.603057ms for pod "kube-scheduler-addons-376302" in "kube-system" namespace to be "Ready" ...
I0927 00:25:55.334880 589846 pod_ready.go:79] waiting up to 6m0s for pod "nvidia-device-plugin-daemonset-wjw6j" in "kube-system" namespace to be "Ready" ...
I0927 00:25:55.552460 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:55.566627 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:56.050053 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:56.062080 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:56.549906 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:56.568115 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:57.050556 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:57.062622 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:57.340851 589846 pod_ready.go:103] pod "nvidia-device-plugin-daemonset-wjw6j" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:57.550420 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:57.562570 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:58.051482 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:58.065016 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:58.557206 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:58.564565 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:59.054405 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:59.062919 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:25:59.340905 589846 pod_ready.go:103] pod "nvidia-device-plugin-daemonset-wjw6j" in "kube-system" namespace has status "Ready":"False"
I0927 00:25:59.551764 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:25:59.651807 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:00.090662 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:00.105879 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:00.551214 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:00.562191 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:01.050044 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:01.062779 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:01.388430 589846 pod_ready.go:103] pod "nvidia-device-plugin-daemonset-wjw6j" in "kube-system" namespace has status "Ready":"False"
I0927 00:26:01.549417 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:01.562191 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:02.051939 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:02.063376 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:02.550450 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:02.563039 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:03.050668 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:03.063271 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:03.551285 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:03.565324 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:03.841679 589846 pod_ready.go:103] pod "nvidia-device-plugin-daemonset-wjw6j" in "kube-system" namespace has status "Ready":"False"
I0927 00:26:04.050519 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:04.063847 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:04.551226 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:04.651667 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:05.050944 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:05.062681 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:05.550285 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:05.563571 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:05.843344 589846 pod_ready.go:103] pod "nvidia-device-plugin-daemonset-wjw6j" in "kube-system" namespace has status "Ready":"False"
I0927 00:26:06.051018 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:06.063409 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:06.550163 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:06.563102 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:07.049753 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:07.063006 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:07.550895 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:07.563653 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:08.050663 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:08.063374 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:08.341761 589846 pod_ready.go:103] pod "nvidia-device-plugin-daemonset-wjw6j" in "kube-system" namespace has status "Ready":"False"
I0927 00:26:08.551238 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:08.563356 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:09.051649 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:09.064127 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:09.549750 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:09.562200 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:10.067695 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:10.071491 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:10.550433 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:10.650093 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:10.841454 589846 pod_ready.go:103] pod "nvidia-device-plugin-daemonset-wjw6j" in "kube-system" namespace has status "Ready":"False"
I0927 00:26:11.123313 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:11.123408 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:11.550697 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:11.562281 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:12.050650 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:12.062683 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:12.340557 589846 pod_ready.go:93] pod "nvidia-device-plugin-daemonset-wjw6j" in "kube-system" namespace has status "Ready":"True"
I0927 00:26:12.340584 589846 pod_ready.go:82] duration metric: took 17.005684427s for pod "nvidia-device-plugin-daemonset-wjw6j" in "kube-system" namespace to be "Ready" ...
I0927 00:26:12.340594 589846 pod_ready.go:39] duration metric: took 1m13.945571466s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0927 00:26:12.340609 589846 api_server.go:52] waiting for apiserver process to appear ...
I0927 00:26:12.340663 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0927 00:26:12.340726 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0927 00:26:12.392933 589846 cri.go:89] found id: "4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984"
I0927 00:26:12.392998 589846 cri.go:89] found id: ""
I0927 00:26:12.393021 589846 logs.go:276] 1 containers: [4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984]
I0927 00:26:12.393104 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:12.397628 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0927 00:26:12.397698 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0927 00:26:12.435514 589846 cri.go:89] found id: "7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611"
I0927 00:26:12.435538 589846 cri.go:89] found id: ""
I0927 00:26:12.435548 589846 logs.go:276] 1 containers: [7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611]
I0927 00:26:12.435604 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:12.439389 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0927 00:26:12.439464 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0927 00:26:12.479769 589846 cri.go:89] found id: "67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10"
I0927 00:26:12.479799 589846 cri.go:89] found id: ""
I0927 00:26:12.479813 589846 logs.go:276] 1 containers: [67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10]
I0927 00:26:12.479875 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:12.483615 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0927 00:26:12.483684 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0927 00:26:12.527296 589846 cri.go:89] found id: "9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c"
I0927 00:26:12.527320 589846 cri.go:89] found id: ""
I0927 00:26:12.527330 589846 logs.go:276] 1 containers: [9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c]
I0927 00:26:12.527390 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:12.532160 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0927 00:26:12.532268 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0927 00:26:12.549770 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:12.562957 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:12.592720 589846 cri.go:89] found id: "70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac"
I0927 00:26:12.592739 589846 cri.go:89] found id: ""
I0927 00:26:12.592748 589846 logs.go:276] 1 containers: [70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac]
I0927 00:26:12.592821 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:12.596301 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0927 00:26:12.596387 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0927 00:26:12.637696 589846 cri.go:89] found id: "4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e"
I0927 00:26:12.637715 589846 cri.go:89] found id: ""
I0927 00:26:12.637722 589846 logs.go:276] 1 containers: [4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e]
I0927 00:26:12.637776 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:12.641402 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0927 00:26:12.641468 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0927 00:26:12.688851 589846 cri.go:89] found id: "808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58"
I0927 00:26:12.688871 589846 cri.go:89] found id: ""
I0927 00:26:12.688878 589846 logs.go:276] 1 containers: [808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58]
I0927 00:26:12.688930 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:12.692493 589846 logs.go:123] Gathering logs for describe nodes ...
I0927 00:26:12.692528 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0927 00:26:12.900623 589846 logs.go:123] Gathering logs for coredns [67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10] ...
I0927 00:26:12.900649 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10"
I0927 00:26:12.952837 589846 logs.go:123] Gathering logs for kube-scheduler [9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c] ...
I0927 00:26:12.952869 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c"
I0927 00:26:13.034259 589846 logs.go:123] Gathering logs for kubelet ...
I0927 00:26:13.034288 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0927 00:26:13.049908 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:13.062534 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
W0927 00:26:13.113305 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611441 1482 reflector.go:561] object-"volcano-system"/"volcano-admission-configmap": failed to list *v1.ConfigMap: configmaps "volcano-admission-configmap" is forbidden: User "system:node:addons-376302" cannot list resource "configmaps" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:13.113619 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611497 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-configmap\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"volcano-admission-configmap\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:13.113837 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611538 1482 reflector.go:561] object-"volcano-system"/"volcano-admission-secret": failed to list *v1.Secret: secrets "volcano-admission-secret" is forbidden: User "system:node:addons-376302" cannot list resource "secrets" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:13.114092 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611549 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-secret\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"volcano-admission-secret\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"secrets\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:13.114299 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611785 1482 reflector.go:561] object-"volcano-system"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:addons-376302" cannot list resource "configmaps" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:13.114561 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611806 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
I0927 00:26:13.208515 589846 logs.go:123] Gathering logs for dmesg ...
I0927 00:26:13.208590 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0927 00:26:13.228111 589846 logs.go:123] Gathering logs for kube-apiserver [4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984] ...
I0927 00:26:13.228138 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984"
I0927 00:26:13.366365 589846 logs.go:123] Gathering logs for etcd [7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611] ...
I0927 00:26:13.366439 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611"
I0927 00:26:13.430846 589846 logs.go:123] Gathering logs for kube-proxy [70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac] ...
I0927 00:26:13.430927 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac"
I0927 00:26:13.490731 589846 logs.go:123] Gathering logs for kube-controller-manager [4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e] ...
I0927 00:26:13.490806 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e"
I0927 00:26:13.550582 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:13.563131 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:13.598845 589846 logs.go:123] Gathering logs for kindnet [808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58] ...
I0927 00:26:13.598893 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58"
I0927 00:26:13.683560 589846 logs.go:123] Gathering logs for containerd ...
I0927 00:26:13.683592 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0927 00:26:13.776309 589846 logs.go:123] Gathering logs for container status ...
I0927 00:26:13.776346 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0927 00:26:13.833432 589846 out.go:358] Setting ErrFile to fd 2...
I0927 00:26:13.833457 589846 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0927 00:26:13.833505 589846 out.go:270] X Problems detected in kubelet:
W0927 00:26:13.833521 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611497 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-configmap\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"volcano-admission-configmap\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:13.833528 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611538 1482 reflector.go:561] object-"volcano-system"/"volcano-admission-secret": failed to list *v1.Secret: secrets "volcano-admission-secret" is forbidden: User "system:node:addons-376302" cannot list resource "secrets" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:13.833538 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611549 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-secret\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"volcano-admission-secret\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"secrets\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:13.833544 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611785 1482 reflector.go:561] object-"volcano-system"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:addons-376302" cannot list resource "configmaps" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:13.833554 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611806 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
I0927 00:26:13.833559 589846 out.go:358] Setting ErrFile to fd 2...
I0927 00:26:13.833566 589846 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0927 00:26:14.050759 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:14.062242 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:14.549908 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:14.563180 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:15.061083 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:15.064996 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:15.549660 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:15.563205 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:16.050821 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:16.062291 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:16.550788 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:16.562614 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:17.051014 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:17.067469 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:17.550212 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:17.565476 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:18.051352 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:18.063866 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:18.550948 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:18.563477 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:19.050779 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:19.062820 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:19.551299 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:19.562513 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:20.060806 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:20.064458 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:20.550137 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:20.563226 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:21.051223 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:21.064166 589846 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]
I0927 00:26:21.554751 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:21.563745 589846 kapi.go:107] duration metric: took 1m18.005708842s to wait for app.kubernetes.io/name=ingress-nginx ...
I0927 00:26:22.051338 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:22.557687 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:23.050033 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:23.550251 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:23.834802 589846 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0927 00:26:23.852759 589846 api_server.go:72] duration metric: took 1m28.434033953s to wait for apiserver process to appear ...
I0927 00:26:23.852823 589846 api_server.go:88] waiting for apiserver healthz status ...
I0927 00:26:23.852875 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0927 00:26:23.852973 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0927 00:26:23.888174 589846 cri.go:89] found id: "4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984"
I0927 00:26:23.888196 589846 cri.go:89] found id: ""
I0927 00:26:23.888203 589846 logs.go:276] 1 containers: [4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984]
I0927 00:26:23.888278 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:23.892332 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0927 00:26:23.892403 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0927 00:26:23.948926 589846 cri.go:89] found id: "7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611"
I0927 00:26:23.948946 589846 cri.go:89] found id: ""
I0927 00:26:23.948955 589846 logs.go:276] 1 containers: [7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611]
I0927 00:26:23.949044 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:23.953268 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0927 00:26:23.953359 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0927 00:26:23.992571 589846 cri.go:89] found id: "67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10"
I0927 00:26:23.992593 589846 cri.go:89] found id: ""
I0927 00:26:23.992601 589846 logs.go:276] 1 containers: [67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10]
I0927 00:26:23.992690 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:23.996938 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0927 00:26:23.997055 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0927 00:26:24.050215 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:24.053929 589846 cri.go:89] found id: "9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c"
I0927 00:26:24.053953 589846 cri.go:89] found id: ""
I0927 00:26:24.053970 589846 logs.go:276] 1 containers: [9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c]
I0927 00:26:24.054049 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:24.061300 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0927 00:26:24.061411 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0927 00:26:24.111753 589846 cri.go:89] found id: "70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac"
I0927 00:26:24.111783 589846 cri.go:89] found id: ""
I0927 00:26:24.111792 589846 logs.go:276] 1 containers: [70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac]
I0927 00:26:24.111912 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:24.115741 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0927 00:26:24.115840 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0927 00:26:24.165826 589846 cri.go:89] found id: "4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e"
I0927 00:26:24.165846 589846 cri.go:89] found id: ""
I0927 00:26:24.165854 589846 logs.go:276] 1 containers: [4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e]
I0927 00:26:24.165913 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:24.172749 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0927 00:26:24.172834 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0927 00:26:24.224683 589846 cri.go:89] found id: "808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58"
I0927 00:26:24.224755 589846 cri.go:89] found id: ""
I0927 00:26:24.224771 589846 logs.go:276] 1 containers: [808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58]
I0927 00:26:24.224850 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:24.228207 589846 logs.go:123] Gathering logs for coredns [67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10] ...
I0927 00:26:24.228272 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10"
I0927 00:26:24.280447 589846 logs.go:123] Gathering logs for kube-scheduler [9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c] ...
I0927 00:26:24.280475 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c"
I0927 00:26:24.335151 589846 logs.go:123] Gathering logs for kube-controller-manager [4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e] ...
I0927 00:26:24.335180 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e"
I0927 00:26:24.429675 589846 logs.go:123] Gathering logs for kindnet [808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58] ...
I0927 00:26:24.429761 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58"
I0927 00:26:24.481326 589846 logs.go:123] Gathering logs for containerd ...
I0927 00:26:24.481499 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0927 00:26:24.568367 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:24.593361 589846 logs.go:123] Gathering logs for kubelet ...
I0927 00:26:24.593437 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0927 00:26:24.655704 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611441 1482 reflector.go:561] object-"volcano-system"/"volcano-admission-configmap": failed to list *v1.ConfigMap: configmaps "volcano-admission-configmap" is forbidden: User "system:node:addons-376302" cannot list resource "configmaps" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:24.656049 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611497 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-configmap\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"volcano-admission-configmap\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:24.656267 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611538 1482 reflector.go:561] object-"volcano-system"/"volcano-admission-secret": failed to list *v1.Secret: secrets "volcano-admission-secret" is forbidden: User "system:node:addons-376302" cannot list resource "secrets" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:24.656693 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611549 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-secret\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"volcano-admission-secret\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"secrets\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:24.656906 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611785 1482 reflector.go:561] object-"volcano-system"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:addons-376302" cannot list resource "configmaps" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:24.657349 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611806 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
I0927 00:26:24.753927 589846 logs.go:123] Gathering logs for kube-apiserver [4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984] ...
I0927 00:26:24.754007 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984"
I0927 00:26:24.862260 589846 logs.go:123] Gathering logs for etcd [7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611] ...
I0927 00:26:24.862335 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611"
I0927 00:26:24.915653 589846 logs.go:123] Gathering logs for container status ...
I0927 00:26:24.915738 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0927 00:26:25.004317 589846 logs.go:123] Gathering logs for dmesg ...
I0927 00:26:25.004407 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0927 00:26:25.037094 589846 logs.go:123] Gathering logs for describe nodes ...
I0927 00:26:25.037121 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0927 00:26:25.050725 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:25.285189 589846 logs.go:123] Gathering logs for kube-proxy [70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac] ...
I0927 00:26:25.285366 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac"
I0927 00:26:25.326077 589846 out.go:358] Setting ErrFile to fd 2...
I0927 00:26:25.326141 589846 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0927 00:26:25.326202 589846 out.go:270] X Problems detected in kubelet:
W0927 00:26:25.326247 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611497 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-configmap\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"volcano-admission-configmap\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:25.326280 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611538 1482 reflector.go:561] object-"volcano-system"/"volcano-admission-secret": failed to list *v1.Secret: secrets "volcano-admission-secret" is forbidden: User "system:node:addons-376302" cannot list resource "secrets" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:25.326317 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611549 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-secret\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"volcano-admission-secret\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"secrets\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:25.326350 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611785 1482 reflector.go:561] object-"volcano-system"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:addons-376302" cannot list resource "configmaps" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:25.326381 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611806 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
I0927 00:26:25.326410 589846 out.go:358] Setting ErrFile to fd 2...
I0927 00:26:25.326429 589846 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0927 00:26:25.549998 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:26.051432 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:26.550166 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:27.051283 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]
I0927 00:26:27.550813 589846 kapi.go:107] duration metric: took 1m21.505670609s to wait for kubernetes.io/minikube-addons=csi-hostpath-driver ...
I0927 00:26:30.856644 589846 kapi.go:86] Found 1 Pods for label selector kubernetes.io/minikube-addons=gcp-auth
I0927 00:26:30.856670 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:31.354132 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:31.853269 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:32.353801 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:32.854333 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:33.353446 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:33.853086 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:34.353802 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:34.853345 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:35.327472 589846 api_server.go:253] Checking apiserver healthz at https://192.168.49.2:8443/healthz ...
I0927 00:26:35.335826 589846 api_server.go:279] https://192.168.49.2:8443/healthz returned 200:
ok
I0927 00:26:35.336841 589846 api_server.go:141] control plane version: v1.31.1
I0927 00:26:35.336864 589846 api_server.go:131] duration metric: took 11.484020007s to wait for apiserver health ...
I0927 00:26:35.336872 589846 system_pods.go:43] waiting for kube-system pods to appear ...
I0927 00:26:35.336894 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0927 00:26:35.336958 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0927 00:26:35.353760 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:35.381569 589846 cri.go:89] found id: "4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984"
I0927 00:26:35.381591 589846 cri.go:89] found id: ""
I0927 00:26:35.381599 589846 logs.go:276] 1 containers: [4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984]
I0927 00:26:35.381659 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:35.385244 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0927 00:26:35.385313 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0927 00:26:35.431533 589846 cri.go:89] found id: "7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611"
I0927 00:26:35.431556 589846 cri.go:89] found id: ""
I0927 00:26:35.431564 589846 logs.go:276] 1 containers: [7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611]
I0927 00:26:35.431620 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:35.435185 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0927 00:26:35.435263 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0927 00:26:35.476042 589846 cri.go:89] found id: "67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10"
I0927 00:26:35.476064 589846 cri.go:89] found id: ""
I0927 00:26:35.476072 589846 logs.go:276] 1 containers: [67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10]
I0927 00:26:35.476128 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:35.479939 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0927 00:26:35.480015 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0927 00:26:35.521726 589846 cri.go:89] found id: "9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c"
I0927 00:26:35.521748 589846 cri.go:89] found id: ""
I0927 00:26:35.521756 589846 logs.go:276] 1 containers: [9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c]
I0927 00:26:35.521843 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:35.525517 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0927 00:26:35.525588 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0927 00:26:35.562815 589846 cri.go:89] found id: "70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac"
I0927 00:26:35.562836 589846 cri.go:89] found id: ""
I0927 00:26:35.562843 589846 logs.go:276] 1 containers: [70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac]
I0927 00:26:35.562923 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:35.566439 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0927 00:26:35.566546 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0927 00:26:35.603323 589846 cri.go:89] found id: "4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e"
I0927 00:26:35.603345 589846 cri.go:89] found id: ""
I0927 00:26:35.603352 589846 logs.go:276] 1 containers: [4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e]
I0927 00:26:35.603412 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:35.606859 589846 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0927 00:26:35.606939 589846 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0927 00:26:35.644360 589846 cri.go:89] found id: "808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58"
I0927 00:26:35.644382 589846 cri.go:89] found id: ""
I0927 00:26:35.644391 589846 logs.go:276] 1 containers: [808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58]
I0927 00:26:35.644449 589846 ssh_runner.go:195] Run: which crictl
I0927 00:26:35.648420 589846 logs.go:123] Gathering logs for kubelet ...
I0927 00:26:35.648558 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0927 00:26:35.690141 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611441 1482 reflector.go:561] object-"volcano-system"/"volcano-admission-configmap": failed to list *v1.ConfigMap: configmaps "volcano-admission-configmap" is forbidden: User "system:node:addons-376302" cannot list resource "configmaps" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:35.690403 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611497 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-configmap\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"volcano-admission-configmap\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:35.690631 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611538 1482 reflector.go:561] object-"volcano-system"/"volcano-admission-secret": failed to list *v1.Secret: secrets "volcano-admission-secret" is forbidden: User "system:node:addons-376302" cannot list resource "secrets" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:35.690865 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611549 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-secret\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"volcano-admission-secret\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"secrets\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:35.691052 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611785 1482 reflector.go:561] object-"volcano-system"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:addons-376302" cannot list resource "configmaps" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:35.691277 589846 logs.go:138] Found kubelet problem: Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611806 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
I0927 00:26:35.790425 589846 logs.go:123] Gathering logs for describe nodes ...
I0927 00:26:35.790463 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0927 00:26:35.855615 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:35.928547 589846 logs.go:123] Gathering logs for etcd [7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611] ...
I0927 00:26:35.928576 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611"
I0927 00:26:35.972297 589846 logs.go:123] Gathering logs for coredns [67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10] ...
I0927 00:26:35.972333 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10"
I0927 00:26:36.024269 589846 logs.go:123] Gathering logs for kube-scheduler [9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c] ...
I0927 00:26:36.024301 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c"
I0927 00:26:36.076244 589846 logs.go:123] Gathering logs for kube-proxy [70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac] ...
I0927 00:26:36.076288 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac"
I0927 00:26:36.118132 589846 logs.go:123] Gathering logs for container status ...
I0927 00:26:36.118162 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0927 00:26:36.171638 589846 logs.go:123] Gathering logs for dmesg ...
I0927 00:26:36.171668 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0927 00:26:36.188278 589846 logs.go:123] Gathering logs for kube-apiserver [4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984] ...
I0927 00:26:36.188307 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984"
I0927 00:26:36.280294 589846 logs.go:123] Gathering logs for kube-controller-manager [4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e] ...
I0927 00:26:36.280331 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e"
I0927 00:26:36.354237 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:36.357116 589846 logs.go:123] Gathering logs for kindnet [808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58] ...
I0927 00:26:36.357149 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58"
I0927 00:26:36.409635 589846 logs.go:123] Gathering logs for containerd ...
I0927 00:26:36.409664 589846 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0927 00:26:36.490359 589846 out.go:358] Setting ErrFile to fd 2...
I0927 00:26:36.490390 589846 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0927 00:26:36.490475 589846 out.go:270] X Problems detected in kubelet:
W0927 00:26:36.490488 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611497 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-configmap\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"volcano-admission-configmap\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:36.490495 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611538 1482 reflector.go:561] object-"volcano-system"/"volcano-admission-secret": failed to list *v1.Secret: secrets "volcano-admission-secret" is forbidden: User "system:node:addons-376302" cannot list resource "secrets" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:36.490502 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611549 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"volcano-admission-secret\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"volcano-admission-secret\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"secrets\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
W0927 00:26:36.490510 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: W0927 00:25:04.611785 1482 reflector.go:561] object-"volcano-system"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:addons-376302" cannot list resource "configmaps" in API group "" in the namespace "volcano-system": no relationship found between node 'addons-376302' and this object
W0927 00:26:36.490518 589846 out.go:270] Sep 27 00:25:04 addons-376302 kubelet[1482]: E0927 00:25:04.611806 1482 reflector.go:158] "Unhandled Error" err="object-\"volcano-system\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:addons-376302\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"volcano-system\": no relationship found between node 'addons-376302' and this object" logger="UnhandledError"
I0927 00:26:36.490529 589846 out.go:358] Setting ErrFile to fd 2...
I0927 00:26:36.490535 589846 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0927 00:26:36.853765 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:37.353362 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:37.852986 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:38.354117 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:38.853371 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:39.353155 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:39.854428 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:40.354139 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:40.853092 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:41.354336 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:41.853758 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:42.355190 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:42.853321 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:43.353816 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:43.853549 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:44.354339 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:44.855010 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:45.356023 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:45.853403 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:46.354526 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:46.501603 589846 system_pods.go:59] 18 kube-system pods found
I0927 00:26:46.501690 589846 system_pods.go:61] "coredns-7c65d6cfc9-7kj2z" [0e9d70eb-bd84-48cf-a69a-c107799c6ce2] Running
I0927 00:26:46.501712 589846 system_pods.go:61] "csi-hostpath-attacher-0" [658357ee-6b68-4118-99de-606d3eb09a2e] Running
I0927 00:26:46.501733 589846 system_pods.go:61] "csi-hostpath-resizer-0" [5d4c5d6a-7a0a-4587-9fd2-a54245b2b12b] Running
I0927 00:26:46.501766 589846 system_pods.go:61] "csi-hostpathplugin-4lcld" [3f23d6a5-9f44-4de4-bcf4-28939ccf2b77] Running
I0927 00:26:46.501792 589846 system_pods.go:61] "etcd-addons-376302" [f97a17e9-257a-4301-82a8-8e04feb4b16b] Running
I0927 00:26:46.501812 589846 system_pods.go:61] "kindnet-c9rgc" [30fd8606-584c-48de-88ac-d9331440a35d] Running
I0927 00:26:46.501831 589846 system_pods.go:61] "kube-apiserver-addons-376302" [87d2f921-4a69-44df-a6f3-3c0eec8a32e3] Running
I0927 00:26:46.501853 589846 system_pods.go:61] "kube-controller-manager-addons-376302" [7848f9c8-5b21-46cf-8a42-597a584755b6] Running
I0927 00:26:46.501881 589846 system_pods.go:61] "kube-ingress-dns-minikube" [5afdfdfb-5a10-4caa-b45d-473693ec26dc] Running
I0927 00:26:46.501904 589846 system_pods.go:61] "kube-proxy-m5q7w" [a58b0736-e856-43a5-93b4-38b34d8baa0b] Running
I0927 00:26:46.501923 589846 system_pods.go:61] "kube-scheduler-addons-376302" [4ed47002-2c06-406e-bc25-03dfed4f987a] Running
I0927 00:26:46.501943 589846 system_pods.go:61] "metrics-server-84c5f94fbc-vtnnr" [bbb2738a-95e1-47e4-b01e-618faac704d5] Running
I0927 00:26:46.501960 589846 system_pods.go:61] "nvidia-device-plugin-daemonset-wjw6j" [473f336e-2969-4568-9fc5-c53cc41f9fb9] Running
I0927 00:26:46.501987 589846 system_pods.go:61] "registry-66c9cd494c-pkjkw" [5f4663c9-a883-4664-8c1a-10bd6625888a] Running
I0927 00:26:46.502014 589846 system_pods.go:61] "registry-proxy-nxrj2" [33cbf2e0-38da-4675-b8c1-d7be72de0161] Running
I0927 00:26:46.502033 589846 system_pods.go:61] "snapshot-controller-56fcc65765-c96j8" [cfe711ea-3a21-4ccd-89ad-0ed1f0c3fc45] Running
I0927 00:26:46.502052 589846 system_pods.go:61] "snapshot-controller-56fcc65765-grxpg" [493813e3-31bb-4f7c-894f-5e1cb73b4138] Running
I0927 00:26:46.502072 589846 system_pods.go:61] "storage-provisioner" [35453f84-97b4-42e4-9fc7-9a1b4f42ae06] Running
I0927 00:26:46.502101 589846 system_pods.go:74] duration metric: took 11.165220177s to wait for pod list to return data ...
I0927 00:26:46.502129 589846 default_sa.go:34] waiting for default service account to be created ...
I0927 00:26:46.505042 589846 default_sa.go:45] found service account: "default"
I0927 00:26:46.505066 589846 default_sa.go:55] duration metric: took 2.917157ms for default service account to be created ...
I0927 00:26:46.505076 589846 system_pods.go:116] waiting for k8s-apps to be running ...
I0927 00:26:46.517178 589846 system_pods.go:86] 18 kube-system pods found
I0927 00:26:46.517277 589846 system_pods.go:89] "coredns-7c65d6cfc9-7kj2z" [0e9d70eb-bd84-48cf-a69a-c107799c6ce2] Running
I0927 00:26:46.517299 589846 system_pods.go:89] "csi-hostpath-attacher-0" [658357ee-6b68-4118-99de-606d3eb09a2e] Running
I0927 00:26:46.517316 589846 system_pods.go:89] "csi-hostpath-resizer-0" [5d4c5d6a-7a0a-4587-9fd2-a54245b2b12b] Running
I0927 00:26:46.517350 589846 system_pods.go:89] "csi-hostpathplugin-4lcld" [3f23d6a5-9f44-4de4-bcf4-28939ccf2b77] Running
I0927 00:26:46.517377 589846 system_pods.go:89] "etcd-addons-376302" [f97a17e9-257a-4301-82a8-8e04feb4b16b] Running
I0927 00:26:46.517398 589846 system_pods.go:89] "kindnet-c9rgc" [30fd8606-584c-48de-88ac-d9331440a35d] Running
I0927 00:26:46.517418 589846 system_pods.go:89] "kube-apiserver-addons-376302" [87d2f921-4a69-44df-a6f3-3c0eec8a32e3] Running
I0927 00:26:46.517449 589846 system_pods.go:89] "kube-controller-manager-addons-376302" [7848f9c8-5b21-46cf-8a42-597a584755b6] Running
I0927 00:26:46.517472 589846 system_pods.go:89] "kube-ingress-dns-minikube" [5afdfdfb-5a10-4caa-b45d-473693ec26dc] Running
I0927 00:26:46.517490 589846 system_pods.go:89] "kube-proxy-m5q7w" [a58b0736-e856-43a5-93b4-38b34d8baa0b] Running
I0927 00:26:46.517509 589846 system_pods.go:89] "kube-scheduler-addons-376302" [4ed47002-2c06-406e-bc25-03dfed4f987a] Running
I0927 00:26:46.517529 589846 system_pods.go:89] "metrics-server-84c5f94fbc-vtnnr" [bbb2738a-95e1-47e4-b01e-618faac704d5] Running
I0927 00:26:46.517556 589846 system_pods.go:89] "nvidia-device-plugin-daemonset-wjw6j" [473f336e-2969-4568-9fc5-c53cc41f9fb9] Running
I0927 00:26:46.517582 589846 system_pods.go:89] "registry-66c9cd494c-pkjkw" [5f4663c9-a883-4664-8c1a-10bd6625888a] Running
I0927 00:26:46.517600 589846 system_pods.go:89] "registry-proxy-nxrj2" [33cbf2e0-38da-4675-b8c1-d7be72de0161] Running
I0927 00:26:46.517619 589846 system_pods.go:89] "snapshot-controller-56fcc65765-c96j8" [cfe711ea-3a21-4ccd-89ad-0ed1f0c3fc45] Running
I0927 00:26:46.517637 589846 system_pods.go:89] "snapshot-controller-56fcc65765-grxpg" [493813e3-31bb-4f7c-894f-5e1cb73b4138] Running
I0927 00:26:46.517666 589846 system_pods.go:89] "storage-provisioner" [35453f84-97b4-42e4-9fc7-9a1b4f42ae06] Running
I0927 00:26:46.517693 589846 system_pods.go:126] duration metric: took 12.610023ms to wait for k8s-apps to be running ...
I0927 00:26:46.517714 589846 system_svc.go:44] waiting for kubelet service to be running ....
I0927 00:26:46.517803 589846 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0927 00:26:46.539453 589846 system_svc.go:56] duration metric: took 21.717185ms WaitForService to wait for kubelet
I0927 00:26:46.539529 589846 kubeadm.go:582] duration metric: took 1m51.120800059s to wait for: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0927 00:26:46.539560 589846 node_conditions.go:102] verifying NodePressure condition ...
I0927 00:26:46.542789 589846 node_conditions.go:122] node storage ephemeral capacity is 203034800Ki
I0927 00:26:46.542823 589846 node_conditions.go:123] node cpu capacity is 2
I0927 00:26:46.542835 589846 node_conditions.go:105] duration metric: took 3.253393ms to run NodePressure ...
I0927 00:26:46.542847 589846 start.go:241] waiting for startup goroutines ...
I0927 00:26:46.853182 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:47.353276 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:47.854098 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:48.353969 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:48.853908 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:49.353138 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:49.853721 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:50.354060 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:50.853181 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:51.353793 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:51.852963 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:52.354044 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:52.853333 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:53.353799 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:53.853716 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:54.353824 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:54.853365 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:55.354111 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:55.854594 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:56.353548 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:56.853181 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:57.353680 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:57.853857 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:58.354253 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:58.854298 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:59.352938 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:26:59.853082 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:00.360302 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:00.854054 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:01.353973 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:01.853672 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:02.353938 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:02.853919 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:03.353928 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:03.853124 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:04.353616 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:04.853639 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:05.353959 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:05.853668 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:06.353952 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:06.853242 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:07.353658 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:07.854542 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:08.354300 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:08.854278 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:09.353960 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:09.853659 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:10.354196 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:10.857533 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:11.353690 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:11.853341 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:12.354070 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:12.854513 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:13.354013 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:13.854126 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:14.353756 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:14.854261 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:15.353988 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:15.853737 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:16.353718 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:16.853214 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:17.353660 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:17.854254 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:18.354395 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:18.854052 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:19.353803 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:19.853545 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:20.353437 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:20.853486 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:21.354015 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:21.853909 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:22.354303 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:22.854027 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:23.353664 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:23.853582 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:24.353426 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:24.854515 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:25.353376 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:25.853535 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:26.353234 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:26.853202 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:27.353889 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:27.854196 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:28.354248 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:28.854325 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:29.353597 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:29.853573 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:30.353467 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:30.853896 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:31.353413 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:31.853770 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:32.353071 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:32.853982 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:33.353823 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:33.853781 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:34.353197 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:34.854394 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:35.353578 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:35.853595 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:36.353535 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:36.854137 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:37.353540 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:37.855404 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:38.353285 589846 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]
I0927 00:27:38.854204 589846 kapi.go:107] duration metric: took 2m31.004265308s to wait for kubernetes.io/minikube-addons=gcp-auth ...
I0927 00:27:38.858129 589846 out.go:177] * Your GCP credentials will now be mounted into every pod created in the addons-376302 cluster.
I0927 00:27:38.860580 589846 out.go:177] * If you don't want your credentials mounted into a specific pod, add a label with the `gcp-auth-skip-secret` key to your pod configuration.
I0927 00:27:38.862338 589846 out.go:177] * If you want existing pods to be mounted with credentials, either recreate them or rerun addons enable with --refresh.
I0927 00:27:38.864671 589846 out.go:177] * Enabled addons: cloud-spanner, nvidia-device-plugin, ingress-dns, storage-provisioner, storage-provisioner-rancher, volcano, metrics-server, inspektor-gadget, yakd, volumesnapshots, registry, ingress, csi-hostpath-driver, gcp-auth
I0927 00:27:38.866762 589846 addons.go:510] duration metric: took 2m43.447583101s for enable addons: enabled=[cloud-spanner nvidia-device-plugin ingress-dns storage-provisioner storage-provisioner-rancher volcano metrics-server inspektor-gadget yakd volumesnapshots registry ingress csi-hostpath-driver gcp-auth]
I0927 00:27:38.866808 589846 start.go:246] waiting for cluster config update ...
I0927 00:27:38.866831 589846 start.go:255] writing updated cluster config ...
I0927 00:27:38.867113 589846 ssh_runner.go:195] Run: rm -f paused
I0927 00:27:39.229711 589846 start.go:600] kubectl: 1.31.1, cluster: 1.31.1 (minor skew: 0)
I0927 00:27:39.233516 589846 out.go:177] * Done! kubectl is now configured to use "addons-376302" cluster and "default" namespace by default
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
df322167f54c8 4f725bf50aaa5 2 minutes ago Exited gadget 5 97bec1de5aa74 gadget-zmj5f
f1a7500408207 6ef582f3ec844 3 minutes ago Running gcp-auth 0 9c22e008b7f9c gcp-auth-89d5ffd79-tbqxs
1ab28178a137c ee6d597e62dc8 4 minutes ago Running csi-snapshotter 0 1b71574750900 csi-hostpathplugin-4lcld
8f9433aa0eaac 642ded511e141 4 minutes ago Running csi-provisioner 0 1b71574750900 csi-hostpathplugin-4lcld
f0d5ba9b31a8c 922312104da8a 4 minutes ago Running liveness-probe 0 1b71574750900 csi-hostpathplugin-4lcld
fc275cd351a01 08f6b2990811a 4 minutes ago Running hostpath 0 1b71574750900 csi-hostpathplugin-4lcld
ca9501635c8fd 1a9605c872c1d 4 minutes ago Running admission 0 35aa985b2de42 volcano-admission-5874dfdd79-c5bgp
ec84c8b9a3356 289a818c8d9c5 4 minutes ago Running controller 0 6b66cf23c5c36 ingress-nginx-controller-bc57996ff-pwskp
357b1b4c6484f 0107d56dbc0be 4 minutes ago Running node-driver-registrar 0 1b71574750900 csi-hostpathplugin-4lcld
4425b3981e646 487fa743e1e22 4 minutes ago Running csi-resizer 0 e4e20910111aa csi-hostpath-resizer-0
316c6e5d5b4fe a9bac31a5be8d 4 minutes ago Running nvidia-device-plugin-ctr 0 fd6bd30adde40 nvidia-device-plugin-daemonset-wjw6j
3886497d74009 77bdba588b953 4 minutes ago Running yakd 0 fa36fc20752b8 yakd-dashboard-67d98fc6b-xlrnc
f776700ca3132 420193b27261a 4 minutes ago Exited patch 2 637179d0321bb ingress-nginx-admission-patch-rv4dz
3bdcaeef4c6f1 1461903ec4fe9 4 minutes ago Running csi-external-health-monitor-controller 0 1b71574750900 csi-hostpathplugin-4lcld
666ff7f01667f 23cbb28ae641a 4 minutes ago Running volcano-controllers 0 f10388a7cb2ba volcano-controllers-789ffc5785-bm7lv
89abc2e18c591 6aa88c604f2b4 4 minutes ago Running volcano-scheduler 0 49c8f282961a5 volcano-scheduler-6c9778cbdf-c8k87
b2d48324f7029 420193b27261a 4 minutes ago Exited create 0 9a800868e589d ingress-nginx-admission-create-hrlmc
70e70a044ea25 9a80d518f102c 5 minutes ago Running csi-attacher 0 31ae2bdde9c27 csi-hostpath-attacher-0
01a4874daef43 5548a49bb60ba 5 minutes ago Running metrics-server 0 d026c44e5c472 metrics-server-84c5f94fbc-vtnnr
67c512afd43d2 2f6c962e7b831 5 minutes ago Running coredns 0 9bd9e3d843048 coredns-7c65d6cfc9-7kj2z
62d6e8f484e1f f7ed138f698f6 5 minutes ago Running registry-proxy 0 17d4c00649ae6 registry-proxy-nxrj2
02ed227fed73e 7ce2150c8929b 5 minutes ago Running local-path-provisioner 0 5bfe149aba53c local-path-provisioner-86d989889c-tr8ln
08b23ed384c5c c9cf76bb104e1 5 minutes ago Running registry 0 eb7dd5af77340 registry-66c9cd494c-pkjkw
c505185ce6761 be9cac3585579 5 minutes ago Running cloud-spanner-emulator 0 e32d850efee5c cloud-spanner-emulator-5b584cc74-ml7ps
9c09924997d0d 4d1e5c3e97420 5 minutes ago Running volume-snapshot-controller 0 c38b9b6f8873a snapshot-controller-56fcc65765-grxpg
90065333162a4 4d1e5c3e97420 5 minutes ago Running volume-snapshot-controller 0 dec2b21a8ebf9 snapshot-controller-56fcc65765-c96j8
d0b7fd26302b5 35508c2f890c4 5 minutes ago Running minikube-ingress-dns 0 ec2f34f33dc00 kube-ingress-dns-minikube
886eea20089a6 ba04bb24b9575 5 minutes ago Running storage-provisioner 0 a25e4c12dd763 storage-provisioner
808cd2049ab3f 6a23fa8fd2b78 6 minutes ago Running kindnet-cni 0 b700845149ee0 kindnet-c9rgc
70222c66f1e2a 24a140c548c07 6 minutes ago Running kube-proxy 0 e80ca7c05bbc7 kube-proxy-m5q7w
4b424e70c475b 279f381cb3736 6 minutes ago Running kube-controller-manager 0 60cb1d620c092 kube-controller-manager-addons-376302
4f7d4f2ea82a3 d3f53a98c0a9d 6 minutes ago Running kube-apiserver 0 5412bf86b4696 kube-apiserver-addons-376302
9705ae8c56794 7f8aa378bb47d 6 minutes ago Running kube-scheduler 0 b6e34a2ccbd11 kube-scheduler-addons-376302
7740b0ab5972c 27e3830e14027 6 minutes ago Running etcd 0 6427177332cdb etcd-addons-376302
==> containerd <==
Sep 27 00:27:50 addons-376302 containerd[813]: time="2024-09-27T00:27:50.629027676Z" level=info msg="RemovePodSandbox \"fa5337ace6ddb21d901e9945fb14401eb39821cf42abd8bb9c7b32749622a982\" returns successfully"
Sep 27 00:28:31 addons-376302 containerd[813]: time="2024-09-27T00:28:31.501987030Z" level=info msg="PullImage \"ghcr.io/inspektor-gadget/inspektor-gadget:v0.32.0@sha256:03e677e1cf9d2c9bea454e3dbcbcef20b3022e987534a2874eb1abc5bc3e73ec\""
Sep 27 00:28:31 addons-376302 containerd[813]: time="2024-09-27T00:28:31.631002206Z" level=info msg="ImageUpdate event name:\"ghcr.io/inspektor-gadget/inspektor-gadget@sha256:03e677e1cf9d2c9bea454e3dbcbcef20b3022e987534a2874eb1abc5bc3e73ec\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}"
Sep 27 00:28:31 addons-376302 containerd[813]: time="2024-09-27T00:28:31.631240392Z" level=info msg="stop pulling image ghcr.io/inspektor-gadget/inspektor-gadget@sha256:03e677e1cf9d2c9bea454e3dbcbcef20b3022e987534a2874eb1abc5bc3e73ec: active requests=0, bytes read=89"
Sep 27 00:28:31 addons-376302 containerd[813]: time="2024-09-27T00:28:31.634557736Z" level=info msg="Pulled image \"ghcr.io/inspektor-gadget/inspektor-gadget:v0.32.0@sha256:03e677e1cf9d2c9bea454e3dbcbcef20b3022e987534a2874eb1abc5bc3e73ec\" with image id \"sha256:4f725bf50aaa5c697fbb84c107e9c7a3766f0f85f514ffce712d03ee5f62e8dd\", repo tag \"\", repo digest \"ghcr.io/inspektor-gadget/inspektor-gadget@sha256:03e677e1cf9d2c9bea454e3dbcbcef20b3022e987534a2874eb1abc5bc3e73ec\", size \"72524105\" in 132.519596ms"
Sep 27 00:28:31 addons-376302 containerd[813]: time="2024-09-27T00:28:31.634737969Z" level=info msg="PullImage \"ghcr.io/inspektor-gadget/inspektor-gadget:v0.32.0@sha256:03e677e1cf9d2c9bea454e3dbcbcef20b3022e987534a2874eb1abc5bc3e73ec\" returns image reference \"sha256:4f725bf50aaa5c697fbb84c107e9c7a3766f0f85f514ffce712d03ee5f62e8dd\""
Sep 27 00:28:31 addons-376302 containerd[813]: time="2024-09-27T00:28:31.637046890Z" level=info msg="CreateContainer within sandbox \"97bec1de5aa749411171e5390070322b41d3b0f78b05a8acd6f19d1be792d70d\" for container &ContainerMetadata{Name:gadget,Attempt:5,}"
Sep 27 00:28:31 addons-376302 containerd[813]: time="2024-09-27T00:28:31.658450138Z" level=info msg="CreateContainer within sandbox \"97bec1de5aa749411171e5390070322b41d3b0f78b05a8acd6f19d1be792d70d\" for &ContainerMetadata{Name:gadget,Attempt:5,} returns container id \"df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298\""
Sep 27 00:28:31 addons-376302 containerd[813]: time="2024-09-27T00:28:31.659418544Z" level=info msg="StartContainer for \"df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298\""
Sep 27 00:28:31 addons-376302 containerd[813]: time="2024-09-27T00:28:31.719793605Z" level=info msg="StartContainer for \"df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298\" returns successfully"
Sep 27 00:28:33 addons-376302 containerd[813]: time="2024-09-27T00:28:33.364949031Z" level=info msg="shim disconnected" id=df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298 namespace=k8s.io
Sep 27 00:28:33 addons-376302 containerd[813]: time="2024-09-27T00:28:33.365014541Z" level=warning msg="cleaning up after shim disconnected" id=df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298 namespace=k8s.io
Sep 27 00:28:33 addons-376302 containerd[813]: time="2024-09-27T00:28:33.365095607Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 27 00:28:33 addons-376302 containerd[813]: time="2024-09-27T00:28:33.721745181Z" level=info msg="RemoveContainer for \"aa8119690d1e5569ca3c7e803b0a1a9508116248af9c5103917aa9de901a4895\""
Sep 27 00:28:33 addons-376302 containerd[813]: time="2024-09-27T00:28:33.728163788Z" level=info msg="RemoveContainer for \"aa8119690d1e5569ca3c7e803b0a1a9508116248af9c5103917aa9de901a4895\" returns successfully"
Sep 27 00:28:50 addons-376302 containerd[813]: time="2024-09-27T00:28:50.638404532Z" level=info msg="RemoveContainer for \"c7193561651481697aa720fec0dd25ba2e114fce43f62b02738046343a770926\""
Sep 27 00:28:50 addons-376302 containerd[813]: time="2024-09-27T00:28:50.644687960Z" level=info msg="RemoveContainer for \"c7193561651481697aa720fec0dd25ba2e114fce43f62b02738046343a770926\" returns successfully"
Sep 27 00:28:50 addons-376302 containerd[813]: time="2024-09-27T00:28:50.646749744Z" level=info msg="StopPodSandbox for \"43d77c5188398c4e404535f0000dbc956c6cc5973788ad5853a30d417121c0ae\""
Sep 27 00:28:50 addons-376302 containerd[813]: time="2024-09-27T00:28:50.655136719Z" level=info msg="TearDown network for sandbox \"43d77c5188398c4e404535f0000dbc956c6cc5973788ad5853a30d417121c0ae\" successfully"
Sep 27 00:28:50 addons-376302 containerd[813]: time="2024-09-27T00:28:50.655178065Z" level=info msg="StopPodSandbox for \"43d77c5188398c4e404535f0000dbc956c6cc5973788ad5853a30d417121c0ae\" returns successfully"
Sep 27 00:28:50 addons-376302 containerd[813]: time="2024-09-27T00:28:50.655791600Z" level=info msg="RemovePodSandbox for \"43d77c5188398c4e404535f0000dbc956c6cc5973788ad5853a30d417121c0ae\""
Sep 27 00:28:50 addons-376302 containerd[813]: time="2024-09-27T00:28:50.655836359Z" level=info msg="Forcibly stopping sandbox \"43d77c5188398c4e404535f0000dbc956c6cc5973788ad5853a30d417121c0ae\""
Sep 27 00:28:50 addons-376302 containerd[813]: time="2024-09-27T00:28:50.663898684Z" level=info msg="TearDown network for sandbox \"43d77c5188398c4e404535f0000dbc956c6cc5973788ad5853a30d417121c0ae\" successfully"
Sep 27 00:28:50 addons-376302 containerd[813]: time="2024-09-27T00:28:50.675117844Z" level=warning msg="Failed to get podSandbox status for container event for sandboxID \"43d77c5188398c4e404535f0000dbc956c6cc5973788ad5853a30d417121c0ae\": an error occurred when try to find sandbox: not found. Sending the event with nil podSandboxStatus."
Sep 27 00:28:50 addons-376302 containerd[813]: time="2024-09-27T00:28:50.675393904Z" level=info msg="RemovePodSandbox \"43d77c5188398c4e404535f0000dbc956c6cc5973788ad5853a30d417121c0ae\" returns successfully"
==> coredns [67c512afd43d2e80ba1ff931620f8eb0cd64a4edb278defa274ff39914966d10] <==
[INFO] 10.244.0.6:41006 - 42494 "AAAA IN registry.kube-system.svc.cluster.local.cluster.local. udp 81 false 1232" NXDOMAIN qr,aa,rd 163 0.000106083s
[INFO] 10.244.0.6:41006 - 26004 "A IN registry.kube-system.svc.cluster.local.us-east-2.compute.internal. udp 94 false 1232" NXDOMAIN qr,rd,ra 83 0.002021579s
[INFO] 10.244.0.6:41006 - 20492 "AAAA IN registry.kube-system.svc.cluster.local.us-east-2.compute.internal. udp 94 false 1232" NXDOMAIN qr,rd,ra 83 0.002191335s
[INFO] 10.244.0.6:41006 - 1755 "A IN registry.kube-system.svc.cluster.local. udp 67 false 1232" NOERROR qr,aa,rd 110 0.000113551s
[INFO] 10.244.0.6:41006 - 42613 "AAAA IN registry.kube-system.svc.cluster.local. udp 67 false 1232" NOERROR qr,aa,rd 149 0.00007712s
[INFO] 10.244.0.6:44650 - 22691 "A IN registry.kube-system.svc.cluster.local.kube-system.svc.cluster.local. udp 86 false 512" NXDOMAIN qr,aa,rd 179 0.000106256s
[INFO] 10.244.0.6:44650 - 22904 "AAAA IN registry.kube-system.svc.cluster.local.kube-system.svc.cluster.local. udp 86 false 512" NXDOMAIN qr,aa,rd 179 0.000150087s
[INFO] 10.244.0.6:49002 - 35784 "A IN registry.kube-system.svc.cluster.local.svc.cluster.local. udp 74 false 512" NXDOMAIN qr,aa,rd 167 0.000089026s
[INFO] 10.244.0.6:49002 - 36039 "AAAA IN registry.kube-system.svc.cluster.local.svc.cluster.local. udp 74 false 512" NXDOMAIN qr,aa,rd 167 0.000047581s
[INFO] 10.244.0.6:47598 - 60650 "AAAA IN registry.kube-system.svc.cluster.local.cluster.local. udp 70 false 512" NXDOMAIN qr,aa,rd 163 0.000073961s
[INFO] 10.244.0.6:47598 - 60205 "A IN registry.kube-system.svc.cluster.local.cluster.local. udp 70 false 512" NXDOMAIN qr,aa,rd 163 0.0000949s
[INFO] 10.244.0.6:48247 - 62778 "A IN registry.kube-system.svc.cluster.local.us-east-2.compute.internal. udp 83 false 512" NXDOMAIN qr,rd,ra 83 0.00173107s
[INFO] 10.244.0.6:48247 - 63223 "AAAA IN registry.kube-system.svc.cluster.local.us-east-2.compute.internal. udp 83 false 512" NXDOMAIN qr,rd,ra 83 0.001707382s
[INFO] 10.244.0.6:34691 - 20 "A IN registry.kube-system.svc.cluster.local. udp 56 false 512" NOERROR qr,aa,rd 110 0.000109177s
[INFO] 10.244.0.6:34691 - 456 "AAAA IN registry.kube-system.svc.cluster.local. udp 56 false 512" NOERROR qr,aa,rd 149 0.000082625s
[INFO] 10.244.0.24:52976 - 24814 "A IN storage.googleapis.com.gcp-auth.svc.cluster.local. udp 78 false 1232" NXDOMAIN qr,aa,rd 160 0.000188939s
[INFO] 10.244.0.24:38458 - 64713 "AAAA IN storage.googleapis.com.gcp-auth.svc.cluster.local. udp 78 false 1232" NXDOMAIN qr,aa,rd 160 0.000294702s
[INFO] 10.244.0.24:57465 - 18295 "AAAA IN storage.googleapis.com.svc.cluster.local. udp 69 false 1232" NXDOMAIN qr,aa,rd 151 0.000192204s
[INFO] 10.244.0.24:42232 - 32899 "A IN storage.googleapis.com.svc.cluster.local. udp 69 false 1232" NXDOMAIN qr,aa,rd 151 0.000269603s
[INFO] 10.244.0.24:59898 - 12554 "A IN storage.googleapis.com.cluster.local. udp 65 false 1232" NXDOMAIN qr,aa,rd 147 0.000207589s
[INFO] 10.244.0.24:56947 - 35148 "AAAA IN storage.googleapis.com.cluster.local. udp 65 false 1232" NXDOMAIN qr,aa,rd 147 0.000090527s
[INFO] 10.244.0.24:48929 - 53715 "A IN storage.googleapis.com.us-east-2.compute.internal. udp 78 false 1232" NXDOMAIN qr,rd,ra 67 0.00257224s
[INFO] 10.244.0.24:53347 - 12628 "AAAA IN storage.googleapis.com.us-east-2.compute.internal. udp 78 false 1232" NXDOMAIN qr,rd,ra 67 0.002084717s
[INFO] 10.244.0.24:38976 - 60360 "AAAA IN storage.googleapis.com. udp 51 false 1232" NOERROR qr,rd,ra 240 0.002161172s
[INFO] 10.244.0.24:40243 - 31776 "A IN storage.googleapis.com. udp 51 false 1232" NOERROR qr,rd,ra 610 0.002291937s
==> describe nodes <==
Name: addons-376302
Roles: control-plane
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=addons-376302
kubernetes.io/os=linux
minikube.k8s.io/commit=eee16a295c071ed5a0e96cbbc00bcd13b2654625
minikube.k8s.io/name=addons-376302
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_09_27T00_24_51_0700
minikube.k8s.io/version=v1.34.0
node-role.kubernetes.io/control-plane=
node.kubernetes.io/exclude-from-external-load-balancers=
topology.hostpath.csi/node=addons-376302
Annotations: csi.volume.kubernetes.io/nodeid: {"hostpath.csi.k8s.io":"addons-376302"}
kubeadm.alpha.kubernetes.io/cri-socket: unix:///run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Fri, 27 Sep 2024 00:24:47 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: addons-376302
AcquireTime: <unset>
RenewTime: Fri, 27 Sep 2024 00:30:48 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Fri, 27 Sep 2024 00:27:54 +0000 Fri, 27 Sep 2024 00:24:45 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Fri, 27 Sep 2024 00:27:54 +0000 Fri, 27 Sep 2024 00:24:45 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Fri, 27 Sep 2024 00:27:54 +0000 Fri, 27 Sep 2024 00:24:45 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Fri, 27 Sep 2024 00:27:54 +0000 Fri, 27 Sep 2024 00:24:48 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.49.2
Hostname: addons-376302
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022304Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022304Ki
pods: 110
System Info:
Machine ID: 3d8279570e0347b3be77cabd7e3528af
System UUID: 09148d62-bd21-4d0a-b61b-d0c0034a3256
Boot ID: da2f37bf-bf94-43b2-9935-20902b5113b2
Kernel Version: 5.15.0-1070-aws
OS Image: Ubuntu 22.04.5 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.7.22
Kubelet Version: v1.31.1
Kube-Proxy Version: v1.31.1
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (27 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
default cloud-spanner-emulator-5b584cc74-ml7ps 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m59s
gadget gadget-zmj5f 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m55s
gcp-auth gcp-auth-89d5ffd79-tbqxs 0 (0%) 0 (0%) 0 (0%) 0 (0%) 4m27s
ingress-nginx ingress-nginx-controller-bc57996ff-pwskp 100m (5%) 0 (0%) 90Mi (1%) 0 (0%) 5m54s
kube-system coredns-7c65d6cfc9-7kj2z 100m (5%) 0 (0%) 70Mi (0%) 170Mi (2%) 6m2s
kube-system csi-hostpath-attacher-0 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m53s
kube-system csi-hostpath-resizer-0 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m52s
kube-system csi-hostpathplugin-4lcld 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m53s
kube-system etcd-addons-376302 100m (5%) 0 (0%) 100Mi (1%) 0 (0%) 6m7s
kube-system kindnet-c9rgc 100m (5%) 100m (5%) 50Mi (0%) 50Mi (0%) 6m3s
kube-system kube-apiserver-addons-376302 250m (12%) 0 (0%) 0 (0%) 0 (0%) 6m7s
kube-system kube-controller-manager-addons-376302 200m (10%) 0 (0%) 0 (0%) 0 (0%) 6m7s
kube-system kube-ingress-dns-minikube 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m59s
kube-system kube-proxy-m5q7w 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m3s
kube-system kube-scheduler-addons-376302 100m (5%) 0 (0%) 0 (0%) 0 (0%) 6m8s
kube-system metrics-server-84c5f94fbc-vtnnr 100m (5%) 0 (0%) 200Mi (2%) 0 (0%) 5m57s
kube-system nvidia-device-plugin-daemonset-wjw6j 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m
kube-system registry-66c9cd494c-pkjkw 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m59s
kube-system registry-proxy-nxrj2 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m59s
kube-system snapshot-controller-56fcc65765-c96j8 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m56s
kube-system snapshot-controller-56fcc65765-grxpg 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m56s
kube-system storage-provisioner 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m57s
local-path-storage local-path-provisioner-86d989889c-tr8ln 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m57s
volcano-system volcano-admission-5874dfdd79-c5bgp 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m53s
volcano-system volcano-controllers-789ffc5785-bm7lv 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m52s
volcano-system volcano-scheduler-6c9778cbdf-c8k87 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m52s
yakd-dashboard yakd-dashboard-67d98fc6b-xlrnc 0 (0%) 0 (0%) 128Mi (1%) 256Mi (3%) 5m56s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 1050m (52%) 100m (5%)
memory 638Mi (8%) 476Mi (6%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
hugepages-32Mi 0 (0%) 0 (0%)
hugepages-64Ki 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 6m1s kube-proxy
Normal Starting 6m7s kubelet Starting kubelet.
Warning CgroupV1 6m7s kubelet Cgroup v1 support is in maintenance mode, please migrate to Cgroup v2.
Normal NodeAllocatableEnforced 6m7s kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 6m7s kubelet Node addons-376302 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 6m7s kubelet Node addons-376302 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 6m7s kubelet Node addons-376302 status is now: NodeHasSufficientPID
Normal RegisteredNode 6m3s node-controller Node addons-376302 event: Registered Node addons-376302 in Controller
==> dmesg <==
==> etcd [7740b0ab5972c339d1019f9d139514b291ad347c45d7e78e263cf6e91dcf2611] <==
{"level":"info","ts":"2024-09-27T00:24:44.354481Z","caller":"embed/etcd.go:728","msg":"starting with client TLS","tls-info":"cert = /var/lib/minikube/certs/etcd/server.crt, key = /var/lib/minikube/certs/etcd/server.key, client-cert=, client-key=, trusted-ca = /var/lib/minikube/certs/etcd/ca.crt, client-cert-auth = true, crl-file = ","cipher-suites":[]}
{"level":"info","ts":"2024-09-27T00:24:44.354673Z","caller":"embed/etcd.go:279","msg":"now serving peer/client/metrics","local-member-id":"aec36adc501070cc","initial-advertise-peer-urls":["https://192.168.49.2:2380"],"listen-peer-urls":["https://192.168.49.2:2380"],"advertise-client-urls":["https://192.168.49.2:2379"],"listen-client-urls":["https://127.0.0.1:2379","https://192.168.49.2:2379"],"listen-metrics-urls":["http://127.0.0.1:2381"]}
{"level":"info","ts":"2024-09-27T00:24:44.354693Z","caller":"embed/etcd.go:870","msg":"serving metrics","address":"http://127.0.0.1:2381"}
{"level":"info","ts":"2024-09-27T00:24:44.354750Z","caller":"embed/etcd.go:599","msg":"serving peer traffic","address":"192.168.49.2:2380"}
{"level":"info","ts":"2024-09-27T00:24:44.354762Z","caller":"embed/etcd.go:571","msg":"cmux::serve","address":"192.168.49.2:2380"}
{"level":"info","ts":"2024-09-27T00:24:44.539678Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"aec36adc501070cc is starting a new election at term 1"}
{"level":"info","ts":"2024-09-27T00:24:44.539887Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"aec36adc501070cc became pre-candidate at term 1"}
{"level":"info","ts":"2024-09-27T00:24:44.540024Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"aec36adc501070cc received MsgPreVoteResp from aec36adc501070cc at term 1"}
{"level":"info","ts":"2024-09-27T00:24:44.540105Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"aec36adc501070cc became candidate at term 2"}
{"level":"info","ts":"2024-09-27T00:24:44.540187Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"aec36adc501070cc received MsgVoteResp from aec36adc501070cc at term 2"}
{"level":"info","ts":"2024-09-27T00:24:44.540269Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"aec36adc501070cc became leader at term 2"}
{"level":"info","ts":"2024-09-27T00:24:44.540343Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: aec36adc501070cc elected leader aec36adc501070cc at term 2"}
{"level":"info","ts":"2024-09-27T00:24:44.541889Z","caller":"etcdserver/server.go:2629","msg":"setting up initial cluster version using v2 API","cluster-version":"3.5"}
{"level":"info","ts":"2024-09-27T00:24:44.543380Z","caller":"etcdserver/server.go:2118","msg":"published local member to cluster through raft","local-member-id":"aec36adc501070cc","local-member-attributes":"{Name:addons-376302 ClientURLs:[https://192.168.49.2:2379]}","request-path":"/0/members/aec36adc501070cc/attributes","cluster-id":"fa54960ea34d58be","publish-timeout":"7s"}
{"level":"info","ts":"2024-09-27T00:24:44.543678Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}
{"level":"info","ts":"2024-09-27T00:24:44.543868Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"fa54960ea34d58be","local-member-id":"aec36adc501070cc","cluster-version":"3.5"}
{"level":"info","ts":"2024-09-27T00:24:44.544165Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}
{"level":"info","ts":"2024-09-27T00:24:44.544264Z","caller":"etcdserver/server.go:2653","msg":"cluster version is updated","cluster-version":"3.5"}
{"level":"info","ts":"2024-09-27T00:24:44.543894Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}
{"level":"info","ts":"2024-09-27T00:24:44.543919Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}
{"level":"info","ts":"2024-09-27T00:24:44.545891Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}
{"level":"info","ts":"2024-09-27T00:24:44.548229Z","caller":"v3rpc/health.go:61","msg":"grpc service status changed","service":"","status":"SERVING"}
{"level":"info","ts":"2024-09-27T00:24:44.549938Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"127.0.0.1:2379"}
{"level":"info","ts":"2024-09-27T00:24:44.587252Z","caller":"v3rpc/health.go:61","msg":"grpc service status changed","service":"","status":"SERVING"}
{"level":"info","ts":"2024-09-27T00:24:44.588340Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.49.2:2379"}
==> gcp-auth [f1a7500408207d74353ed35be7ffee7a0066ed9d03c3042e8043b467d2c5a12c] <==
2024/09/27 00:27:37 GCP Auth Webhook started!
2024/09/27 00:27:55 Ready to marshal response ...
2024/09/27 00:27:55 Ready to write response ...
2024/09/27 00:27:56 Ready to marshal response ...
2024/09/27 00:27:56 Ready to write response ...
==> kernel <==
00:30:57 up 4:13, 0 users, load average: 0.94, 1.68, 2.24
Linux addons-376302 5.15.0-1070-aws #76~20.04.1-Ubuntu SMP Mon Sep 2 12:20:48 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.5 LTS"
==> kindnet [808cd2049ab3fb612ff58d4ce254cf76df8188497a44e3a97faa28c6ed3e6f58] <==
I0927 00:28:56.359926 1 main.go:299] handling current node
I0927 00:29:06.360032 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:29:06.360068 1 main.go:299] handling current node
I0927 00:29:16.365246 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:29:16.365293 1 main.go:299] handling current node
I0927 00:29:26.362756 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:29:26.362887 1 main.go:299] handling current node
I0927 00:29:36.359547 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:29:36.359580 1 main.go:299] handling current node
I0927 00:29:46.368603 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:29:46.368640 1 main.go:299] handling current node
I0927 00:29:56.359139 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:29:56.359177 1 main.go:299] handling current node
I0927 00:30:06.362570 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:30:06.362605 1 main.go:299] handling current node
I0927 00:30:16.368136 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:30:16.368174 1 main.go:299] handling current node
I0927 00:30:26.367275 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:30:26.367312 1 main.go:299] handling current node
I0927 00:30:36.359655 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:30:36.359746 1 main.go:299] handling current node
I0927 00:30:46.368266 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:30:46.368300 1 main.go:299] handling current node
I0927 00:30:56.359173 1 main.go:295] Handling node with IPs: map[192.168.49.2:{}]
I0927 00:30:56.359267 1 main.go:299] handling current node
==> kube-apiserver [4f7d4f2ea82a3ba0369f64f959c96362a69a47e7aa17b2b833064d0de2690984] <==
W0927 00:26:10.546831 1 dispatcher.go:225] Failed calling webhook, failing closed mutatepod.volcano.sh: failed calling webhook "mutatepod.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/pods/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:11.072045 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:12.093798 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:13.145309 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:14.187134 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:15.226365 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:16.326299 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:17.431956 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:18.458763 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:19.474191 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:20.576808 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:21.668439 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:22.699934 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:23.783211 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:24.871087 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:25.963695 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:27.032206 1 dispatcher.go:225] Failed calling webhook, failing closed mutatequeue.volcano.sh: failed calling webhook "mutatequeue.volcano.sh": failed to call webhook: Post "https://volcano-admission-service.volcano-system.svc:443/queues/mutate?timeout=10s": dial tcp 10.97.35.108:443: connect: connection refused
W0927 00:26:30.450631 1 dispatcher.go:210] Failed calling webhook, failing open gcp-auth-mutate.k8s.io: failed calling webhook "gcp-auth-mutate.k8s.io": failed to call webhook: Post "https://gcp-auth.gcp-auth.svc:443/mutate?timeout=10s": dial tcp 10.111.99.251:443: connect: connection refused
E0927 00:26:30.450671 1 dispatcher.go:214] "Unhandled Error" err="failed calling webhook \"gcp-auth-mutate.k8s.io\": failed to call webhook: Post \"https://gcp-auth.gcp-auth.svc:443/mutate?timeout=10s\": dial tcp 10.111.99.251:443: connect: connection refused" logger="UnhandledError"
W0927 00:27:10.491537 1 dispatcher.go:210] Failed calling webhook, failing open gcp-auth-mutate.k8s.io: failed calling webhook "gcp-auth-mutate.k8s.io": failed to call webhook: Post "https://gcp-auth.gcp-auth.svc:443/mutate?timeout=10s": dial tcp 10.111.99.251:443: connect: connection refused
E0927 00:27:10.491822 1 dispatcher.go:214] "Unhandled Error" err="failed calling webhook \"gcp-auth-mutate.k8s.io\": failed to call webhook: Post \"https://gcp-auth.gcp-auth.svc:443/mutate?timeout=10s\": dial tcp 10.111.99.251:443: connect: connection refused" logger="UnhandledError"
W0927 00:27:10.554317 1 dispatcher.go:210] Failed calling webhook, failing open gcp-auth-mutate.k8s.io: failed calling webhook "gcp-auth-mutate.k8s.io": failed to call webhook: Post "https://gcp-auth.gcp-auth.svc:443/mutate?timeout=10s": dial tcp 10.111.99.251:443: connect: connection refused
E0927 00:27:10.554355 1 dispatcher.go:214] "Unhandled Error" err="failed calling webhook \"gcp-auth-mutate.k8s.io\": failed to call webhook: Post \"https://gcp-auth.gcp-auth.svc:443/mutate?timeout=10s\": dial tcp 10.111.99.251:443: connect: connection refused" logger="UnhandledError"
I0927 00:27:55.778873 1 controller.go:615] quota admission added evaluator for: jobs.batch.volcano.sh
I0927 00:27:55.814578 1 controller.go:615] quota admission added evaluator for: podgroups.scheduling.volcano.sh
==> kube-controller-manager [4b424e70c475b5fd67848be176bc70ecc24418788697034a3eda05ed24ef077e] <==
I0927 00:27:10.525346 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-create" delay="1s"
I0927 00:27:10.529271 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-create" delay="1s"
I0927 00:27:10.539181 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-create" delay="1s"
I0927 00:27:10.562862 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-patch" delay="1s"
I0927 00:27:10.576423 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-patch" delay="1s"
I0927 00:27:10.576727 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-patch" delay="1s"
I0927 00:27:10.588140 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-patch" delay="1s"
I0927 00:27:11.489024 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-create" delay="1s"
I0927 00:27:11.509180 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-patch" delay="1s"
I0927 00:27:12.619056 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-create" delay="1s"
I0927 00:27:12.642616 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-patch" delay="1s"
I0927 00:27:13.625583 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-create" delay="1s"
I0927 00:27:13.637103 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-create" delay="1s"
I0927 00:27:13.640358 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-create" delay="1s"
I0927 00:27:13.646992 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-patch" delay="1s"
I0927 00:27:13.655823 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-patch" delay="1s"
I0927 00:27:13.665174 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-patch" delay="1s"
I0927 00:27:38.586274 1 replica_set.go:679] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="gcp-auth/gcp-auth-89d5ffd79" duration="12.682859ms"
I0927 00:27:38.586799 1 replica_set.go:679] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="gcp-auth/gcp-auth-89d5ffd79" duration="57.641µs"
I0927 00:27:43.033179 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-create" delay="0s"
I0927 00:27:43.038776 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-patch" delay="0s"
I0927 00:27:43.131366 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-create" delay="0s"
I0927 00:27:43.133401 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="gcp-auth/gcp-auth-certs-patch" delay="0s"
I0927 00:27:54.707140 1 range_allocator.go:241] "Successfully synced" logger="node-ipam-controller" key="addons-376302"
I0927 00:27:55.494127 1 job_controller.go:568] "enqueueing job" logger="job-controller" key="volcano-system/volcano-admission-init" delay="0s"
==> kube-proxy [70222c66f1e2ab845e54fe52f713c4cb23e920f4022cfc3188f7e96d585a68ac] <==
I0927 00:24:55.721719 1 server_linux.go:66] "Using iptables proxy"
I0927 00:24:56.078964 1 server.go:677] "Successfully retrieved node IP(s)" IPs=["192.168.49.2"]
E0927 00:24:56.079042 1 server.go:234] "Kube-proxy configuration may be incomplete or incorrect" err="nodePortAddresses is unset; NodePort connections will be accepted on all local IPs. Consider using `--nodeport-addresses primary`"
I0927 00:24:56.207901 1 server.go:243] "kube-proxy running in dual-stack mode" primary ipFamily="IPv4"
I0927 00:24:56.210095 1 server_linux.go:169] "Using iptables Proxier"
I0927 00:24:56.212268 1 proxier.go:255] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses" ipFamily="IPv4"
I0927 00:24:56.212604 1 server.go:483] "Version info" version="v1.31.1"
I0927 00:24:56.212618 1 server.go:485] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0927 00:24:56.221069 1 config.go:199] "Starting service config controller"
I0927 00:24:56.221096 1 shared_informer.go:313] Waiting for caches to sync for service config
I0927 00:24:56.221127 1 config.go:105] "Starting endpoint slice config controller"
I0927 00:24:56.221132 1 shared_informer.go:313] Waiting for caches to sync for endpoint slice config
I0927 00:24:56.221514 1 config.go:328] "Starting node config controller"
I0927 00:24:56.221521 1 shared_informer.go:313] Waiting for caches to sync for node config
I0927 00:24:56.326230 1 shared_informer.go:320] Caches are synced for node config
I0927 00:24:56.326269 1 shared_informer.go:320] Caches are synced for service config
I0927 00:24:56.326303 1 shared_informer.go:320] Caches are synced for endpoint slice config
==> kube-scheduler [9705ae8c567943563ca9c096643beef9dc8914c2b3480205f7fbb9c24622df4c] <==
W0927 00:24:48.367157 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0927 00:24:48.367313 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User \"system:kube-scheduler\" cannot list resource \"pods\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0927 00:24:48.367411 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0927 00:24:48.367433 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User \"system:kube-scheduler\" cannot list resource \"storageclasses\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError"
W0927 00:24:48.367504 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0927 00:24:48.371225 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User \"system:kube-scheduler\" cannot list resource \"replicationcontrollers\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0927 00:24:48.367540 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0927 00:24:48.371286 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User \"system:kube-scheduler\" cannot list resource \"csinodes\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError"
W0927 00:24:48.367897 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0927 00:24:48.371320 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User \"system:kube-scheduler\" cannot list resource \"statefulsets\" in API group \"apps\" at the cluster scope" logger="UnhandledError"
W0927 00:24:48.367952 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope
E0927 00:24:48.371348 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User \"system:kube-scheduler\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0927 00:24:48.371188 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0927 00:24:48.371371 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User \"system:kube-scheduler\" cannot list resource \"persistentvolumes\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0927 00:24:48.371663 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0927 00:24:48.371695 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User \"system:kube-scheduler\" cannot list resource \"replicasets\" in API group \"apps\" at the cluster scope" logger="UnhandledError"
W0927 00:24:48.371774 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0927 00:24:48.371793 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User \"system:kube-scheduler\" cannot list resource \"poddisruptionbudgets\" in API group \"policy\" at the cluster scope" logger="UnhandledError"
W0927 00:24:48.371922 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0927 00:24:48.371944 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User \"system:kube-scheduler\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0927 00:24:48.371991 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0927 00:24:48.372009 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User \"system:kube-scheduler\" cannot list resource \"persistentvolumeclaims\" in API group \"\" at the cluster scope" logger="UnhandledError"
W0927 00:24:48.372057 1 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0927 00:24:48.372074 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User \"system:kube-scheduler\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError"
I0927 00:24:49.956416 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Sep 27 00:28:51 addons-376302 kubelet[1482]: I0927 00:28:51.499957 1482 scope.go:117] "RemoveContainer" containerID="df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298"
Sep 27 00:28:51 addons-376302 kubelet[1482]: E0927 00:28:51.500161 1482 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"gadget\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=gadget pod=gadget-zmj5f_gadget(45661758-86d8-449d-bd7c-64e643ed664a)\"" pod="gadget/gadget-zmj5f" podUID="45661758-86d8-449d-bd7c-64e643ed664a"
Sep 27 00:29:06 addons-376302 kubelet[1482]: I0927 00:29:06.499467 1482 scope.go:117] "RemoveContainer" containerID="df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298"
Sep 27 00:29:06 addons-376302 kubelet[1482]: E0927 00:29:06.500245 1482 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"gadget\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=gadget pod=gadget-zmj5f_gadget(45661758-86d8-449d-bd7c-64e643ed664a)\"" pod="gadget/gadget-zmj5f" podUID="45661758-86d8-449d-bd7c-64e643ed664a"
Sep 27 00:29:21 addons-376302 kubelet[1482]: I0927 00:29:21.500170 1482 scope.go:117] "RemoveContainer" containerID="df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298"
Sep 27 00:29:21 addons-376302 kubelet[1482]: E0927 00:29:21.500368 1482 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"gadget\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=gadget pod=gadget-zmj5f_gadget(45661758-86d8-449d-bd7c-64e643ed664a)\"" pod="gadget/gadget-zmj5f" podUID="45661758-86d8-449d-bd7c-64e643ed664a"
Sep 27 00:29:23 addons-376302 kubelet[1482]: I0927 00:29:23.500079 1482 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="kube-system/registry-66c9cd494c-pkjkw" secret="" err="secret \"gcp-auth\" not found"
Sep 27 00:29:33 addons-376302 kubelet[1482]: I0927 00:29:33.499876 1482 scope.go:117] "RemoveContainer" containerID="df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298"
Sep 27 00:29:33 addons-376302 kubelet[1482]: E0927 00:29:33.500493 1482 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"gadget\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=gadget pod=gadget-zmj5f_gadget(45661758-86d8-449d-bd7c-64e643ed664a)\"" pod="gadget/gadget-zmj5f" podUID="45661758-86d8-449d-bd7c-64e643ed664a"
Sep 27 00:29:37 addons-376302 kubelet[1482]: I0927 00:29:37.499788 1482 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="kube-system/registry-proxy-nxrj2" secret="" err="secret \"gcp-auth\" not found"
Sep 27 00:29:46 addons-376302 kubelet[1482]: I0927 00:29:46.500588 1482 scope.go:117] "RemoveContainer" containerID="df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298"
Sep 27 00:29:46 addons-376302 kubelet[1482]: E0927 00:29:46.501266 1482 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"gadget\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=gadget pod=gadget-zmj5f_gadget(45661758-86d8-449d-bd7c-64e643ed664a)\"" pod="gadget/gadget-zmj5f" podUID="45661758-86d8-449d-bd7c-64e643ed664a"
Sep 27 00:29:54 addons-376302 kubelet[1482]: I0927 00:29:54.499472 1482 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="kube-system/nvidia-device-plugin-daemonset-wjw6j" secret="" err="secret \"gcp-auth\" not found"
Sep 27 00:29:58 addons-376302 kubelet[1482]: I0927 00:29:58.500400 1482 scope.go:117] "RemoveContainer" containerID="df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298"
Sep 27 00:29:58 addons-376302 kubelet[1482]: E0927 00:29:58.501026 1482 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"gadget\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=gadget pod=gadget-zmj5f_gadget(45661758-86d8-449d-bd7c-64e643ed664a)\"" pod="gadget/gadget-zmj5f" podUID="45661758-86d8-449d-bd7c-64e643ed664a"
Sep 27 00:30:11 addons-376302 kubelet[1482]: I0927 00:30:11.499826 1482 scope.go:117] "RemoveContainer" containerID="df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298"
Sep 27 00:30:11 addons-376302 kubelet[1482]: E0927 00:30:11.500043 1482 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"gadget\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=gadget pod=gadget-zmj5f_gadget(45661758-86d8-449d-bd7c-64e643ed664a)\"" pod="gadget/gadget-zmj5f" podUID="45661758-86d8-449d-bd7c-64e643ed664a"
Sep 27 00:30:24 addons-376302 kubelet[1482]: I0927 00:30:24.499340 1482 scope.go:117] "RemoveContainer" containerID="df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298"
Sep 27 00:30:24 addons-376302 kubelet[1482]: E0927 00:30:24.499975 1482 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"gadget\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=gadget pod=gadget-zmj5f_gadget(45661758-86d8-449d-bd7c-64e643ed664a)\"" pod="gadget/gadget-zmj5f" podUID="45661758-86d8-449d-bd7c-64e643ed664a"
Sep 27 00:30:25 addons-376302 kubelet[1482]: I0927 00:30:25.499643 1482 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="kube-system/registry-66c9cd494c-pkjkw" secret="" err="secret \"gcp-auth\" not found"
Sep 27 00:30:39 addons-376302 kubelet[1482]: I0927 00:30:39.499507 1482 scope.go:117] "RemoveContainer" containerID="df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298"
Sep 27 00:30:39 addons-376302 kubelet[1482]: E0927 00:30:39.499702 1482 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"gadget\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=gadget pod=gadget-zmj5f_gadget(45661758-86d8-449d-bd7c-64e643ed664a)\"" pod="gadget/gadget-zmj5f" podUID="45661758-86d8-449d-bd7c-64e643ed664a"
Sep 27 00:30:49 addons-376302 kubelet[1482]: I0927 00:30:49.499310 1482 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="kube-system/registry-proxy-nxrj2" secret="" err="secret \"gcp-auth\" not found"
Sep 27 00:30:54 addons-376302 kubelet[1482]: I0927 00:30:54.500385 1482 scope.go:117] "RemoveContainer" containerID="df322167f54c8f6fdc7975c55ef6710315ecf1fe2a3330f47a9249d243df2298"
Sep 27 00:30:54 addons-376302 kubelet[1482]: E0927 00:30:54.500578 1482 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"gadget\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=gadget pod=gadget-zmj5f_gadget(45661758-86d8-449d-bd7c-64e643ed664a)\"" pod="gadget/gadget-zmj5f" podUID="45661758-86d8-449d-bd7c-64e643ed664a"
==> storage-provisioner [886eea20089a6051829661f7e8c9393ffc3db7c998aa37bb63252786d665a098] <==
I0927 00:25:01.229767 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0927 00:25:01.257267 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0927 00:25:01.257355 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0927 00:25:01.273284 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0927 00:25:01.273493 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_addons-376302_5b24d069-04ae-4ef3-a6f2-a1c0937b1b29!
I0927 00:25:01.274328 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"7907ac98-ddd6-45db-84b2-74828e2a9989", APIVersion:"v1", ResourceVersion:"554", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' addons-376302_5b24d069-04ae-4ef3-a6f2-a1c0937b1b29 became leader
I0927 00:25:01.374027 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_addons-376302_5b24d069-04ae-4ef3-a6f2-a1c0937b1b29!
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p addons-376302 -n addons-376302
helpers_test.go:261: (dbg) Run: kubectl --context addons-376302 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: ingress-nginx-admission-create-hrlmc ingress-nginx-admission-patch-rv4dz test-job-nginx-0
helpers_test.go:274: ======> post-mortem[TestAddons/serial/Volcano]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context addons-376302 describe pod ingress-nginx-admission-create-hrlmc ingress-nginx-admission-patch-rv4dz test-job-nginx-0
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context addons-376302 describe pod ingress-nginx-admission-create-hrlmc ingress-nginx-admission-patch-rv4dz test-job-nginx-0: exit status 1 (93.42242ms)
** stderr **
Error from server (NotFound): pods "ingress-nginx-admission-create-hrlmc" not found
Error from server (NotFound): pods "ingress-nginx-admission-patch-rv4dz" not found
Error from server (NotFound): pods "test-job-nginx-0" not found
** /stderr **
helpers_test.go:279: kubectl --context addons-376302 describe pod ingress-nginx-admission-create-hrlmc ingress-nginx-admission-patch-rv4dz test-job-nginx-0: exit status 1
--- FAIL: TestAddons/serial/Volcano (199.76s)