=== RUN   TestAddons/parallel/Ingress

=== PAUSE TestAddons/parallel/Ingress

=== CONT  TestAddons/parallel/Ingress

addons_test.go:207: (dbg) Run:  kubectl --context addons-649326 wait --for=condition=ready --namespace=ingress-nginx pod --selector=app.kubernetes.io/component=controller --timeout=90s

addons_test.go:232: (dbg) Run:  kubectl --context addons-649326 replace --force -f testdata/nginx-ingress-v1.yaml

addons_test.go:245: (dbg) Run:  kubectl --context addons-649326 replace --force -f testdata/nginx-pod-svc.yaml

addons_test.go:250: (dbg) TestAddons/parallel/Ingress: waiting 8m0s for pods matching "run=nginx" in namespace "default" ...

helpers_test.go:344: "nginx" [45797b3a-fb8a-4322-9a75-f26c021acec0] Pending / Ready:ContainersNotReady (containers with unready status: [nginx]) / ContainersReady:ContainersNotReady (containers with unready status: [nginx])

helpers_test.go:344: "nginx" [45797b3a-fb8a-4322-9a75-f26c021acec0] Running

addons_test.go:250: (dbg) TestAddons/parallel/Ingress: run=nginx healthy within 10.003549539s

addons_test.go:262: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 ssh "curl -s http://127.0.0.1/ -H 'Host: nginx.example.com'"

addons_test.go:262: (dbg) Non-zero exit: out/minikube-linux-arm64 -p addons-649326 ssh "curl -s http://127.0.0.1/ -H 'Host: nginx.example.com'": exit status 1 (2m10.047184721s)

** stderr ** 

	ssh: Process exited with status 28

** /stderr **

addons_test.go:278: failed to get expected response from http://127.0.0.1/ within minikube: exit status 1

addons_test.go:286: (dbg) Run:  kubectl --context addons-649326 replace --force -f testdata/ingress-dns-example-v1.yaml

addons_test.go:291: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 ip

addons_test.go:297: (dbg) Run:  nslookup hello-john.test 192.168.49.2

addons_test.go:297: (dbg) Non-zero exit: nslookup hello-john.test 192.168.49.2: exit status 1 (15.064036374s)

-- stdout --

	;; connection timed out; no servers could be reached

-- /stdout --

addons_test.go:299: failed to nslookup hello-john.test host. args "nslookup hello-john.test 192.168.49.2" : exit status 1

addons_test.go:303: unexpected output from nslookup. stdout: ;; connection timed out; no servers could be reached

stderr: 

addons_test.go:306: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 addons disable ingress-dns --alsologtostderr -v=1

addons_test.go:311: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 addons disable ingress --alsologtostderr -v=1

addons_test.go:311: (dbg) Done: out/minikube-linux-arm64 -p addons-649326 addons disable ingress --alsologtostderr -v=1: (7.741074396s)

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestAddons/parallel/Ingress]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect addons-649326

helpers_test.go:235: (dbg) docker inspect addons-649326:

-- stdout --

	[

	    {

	        "Id": "4dc8c8b83276c0cde7ffbb37a3e316869b974b9effc09d9d3041c30b11b083e1",

	        "Created": "2024-03-05T03:58:03.629620434Z",

	        "Path": "/usr/local/bin/entrypoint",

	        "Args": [

	            "/sbin/init"

	        ],

	        "State": {

	            "Status": "running",

	            "Running": true,

	            "Paused": false,

	            "Restarting": false,

	            "OOMKilled": false,

	            "Dead": false,

	            "Pid": 2132965,

	            "ExitCode": 0,

	            "Error": "",

	            "StartedAt": "2024-03-05T03:58:03.928634639Z",

	            "FinishedAt": "0001-01-01T00:00:00Z"

	        },

	        "Image": "sha256:4a9b65157dd7fb2ddb7cb7afe975b3dc288e9877c60d13613a69dd41a70e2e4e",

	        "ResolvConfPath": "/var/lib/docker/containers/4dc8c8b83276c0cde7ffbb37a3e316869b974b9effc09d9d3041c30b11b083e1/resolv.conf",

	        "HostnamePath": "/var/lib/docker/containers/4dc8c8b83276c0cde7ffbb37a3e316869b974b9effc09d9d3041c30b11b083e1/hostname",

	        "HostsPath": "/var/lib/docker/containers/4dc8c8b83276c0cde7ffbb37a3e316869b974b9effc09d9d3041c30b11b083e1/hosts",

	        "LogPath": "/var/lib/docker/containers/4dc8c8b83276c0cde7ffbb37a3e316869b974b9effc09d9d3041c30b11b083e1/4dc8c8b83276c0cde7ffbb37a3e316869b974b9effc09d9d3041c30b11b083e1-json.log",

	        "Name": "/addons-649326",

	        "RestartCount": 0,

	        "Driver": "overlay2",

	        "Platform": "linux",

	        "MountLabel": "",

	        "ProcessLabel": "",

	        "AppArmorProfile": "unconfined",

	        "ExecIDs": null,

	        "HostConfig": {

	            "Binds": [

	                "/lib/modules:/lib/modules:ro",

	                "addons-649326:/var"

	            ],

	            "ContainerIDFile": "",

	            "LogConfig": {

	                "Type": "json-file",

	                "Config": {}

	            },

	            "NetworkMode": "addons-649326",

	            "PortBindings": {

	                "22/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": ""

	                    }

	                ],

	                "2376/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": ""

	                    }

	                ],

	                "32443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": ""

	                    }

	                ],

	                "5000/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": ""

	                    }

	                ],

	                "8443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": ""

	                    }

	                ]

	            },

	            "RestartPolicy": {

	                "Name": "no",

	                "MaximumRetryCount": 0

	            },

	            "AutoRemove": false,

	            "VolumeDriver": "",

	            "VolumesFrom": null,

	            "ConsoleSize": [

	                0,

	                0

	            ],

	            "CapAdd": null,

	            "CapDrop": null,

	            "CgroupnsMode": "host",

	            "Dns": [],

	            "DnsOptions": [],

	            "DnsSearch": [],

	            "ExtraHosts": null,

	            "GroupAdd": null,

	            "IpcMode": "private",

	            "Cgroup": "",

	            "Links": null,

	            "OomScoreAdj": 0,

	            "PidMode": "",

	            "Privileged": true,

	            "PublishAllPorts": false,

	            "ReadonlyRootfs": false,

	            "SecurityOpt": [

	                "seccomp=unconfined",

	                "apparmor=unconfined",

	                "label=disable"

	            ],

	            "Tmpfs": {

	                "/run": "",

	                "/tmp": ""

	            },

	            "UTSMode": "",

	            "UsernsMode": "",

	            "ShmSize": 67108864,

	            "Runtime": "runc",

	            "Isolation": "",

	            "CpuShares": 0,

	            "Memory": 4194304000,

	            "NanoCpus": 2000000000,

	            "CgroupParent": "",

	            "BlkioWeight": 0,

	            "BlkioWeightDevice": [],

	            "BlkioDeviceReadBps": [],

	            "BlkioDeviceWriteBps": [],

	            "BlkioDeviceReadIOps": [],

	            "BlkioDeviceWriteIOps": [],

	            "CpuPeriod": 0,

	            "CpuQuota": 0,

	            "CpuRealtimePeriod": 0,

	            "CpuRealtimeRuntime": 0,

	            "CpusetCpus": "",

	            "CpusetMems": "",

	            "Devices": [],

	            "DeviceCgroupRules": null,

	            "DeviceRequests": null,

	            "MemoryReservation": 0,

	            "MemorySwap": 8388608000,

	            "MemorySwappiness": null,

	            "OomKillDisable": false,

	            "PidsLimit": null,

	            "Ulimits": [],

	            "CpuCount": 0,

	            "CpuPercent": 0,

	            "IOMaximumIOps": 0,

	            "IOMaximumBandwidth": 0,

	            "MaskedPaths": null,

	            "ReadonlyPaths": null

	        },

	        "GraphDriver": {

	            "Data": {

	                "LowerDir": "/var/lib/docker/overlay2/ce32064fcf8413278daa3731ebf294c38498cd1ba642bb0a6639ee0d0b013b46-init/diff:/var/lib/docker/overlay2/423fb181de42bf2fff4f2436233a3066e6b64d14a90221913df20465d28fdb3b/diff",

	                "MergedDir": "/var/lib/docker/overlay2/ce32064fcf8413278daa3731ebf294c38498cd1ba642bb0a6639ee0d0b013b46/merged",

	                "UpperDir": "/var/lib/docker/overlay2/ce32064fcf8413278daa3731ebf294c38498cd1ba642bb0a6639ee0d0b013b46/diff",

	                "WorkDir": "/var/lib/docker/overlay2/ce32064fcf8413278daa3731ebf294c38498cd1ba642bb0a6639ee0d0b013b46/work"

	            },

	            "Name": "overlay2"

	        },

	        "Mounts": [

	            {

	                "Type": "bind",

	                "Source": "/lib/modules",

	                "Destination": "/lib/modules",

	                "Mode": "ro",

	                "RW": false,

	                "Propagation": "rprivate"

	            },

	            {

	                "Type": "volume",

	                "Name": "addons-649326",

	                "Source": "/var/lib/docker/volumes/addons-649326/_data",

	                "Destination": "/var",

	                "Driver": "local",

	                "Mode": "z",

	                "RW": true,

	                "Propagation": ""

	            }

	        ],

	        "Config": {

	            "Hostname": "addons-649326",

	            "Domainname": "",

	            "User": "",

	            "AttachStdin": false,

	            "AttachStdout": false,

	            "AttachStderr": false,

	            "ExposedPorts": {

	                "22/tcp": {},

	                "2376/tcp": {},

	                "32443/tcp": {},

	                "5000/tcp": {},

	                "8443/tcp": {}

	            },

	            "Tty": true,

	            "OpenStdin": false,

	            "StdinOnce": false,

	            "Env": [

	                "container=docker",

	                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

	            ],

	            "Cmd": null,

	            "Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08",

	            "Volumes": null,

	            "WorkingDir": "/",

	            "Entrypoint": [

	                "/usr/local/bin/entrypoint",

	                "/sbin/init"

	            ],

	            "OnBuild": null,

	            "Labels": {

	                "created_by.minikube.sigs.k8s.io": "true",

	                "mode.minikube.sigs.k8s.io": "addons-649326",

	                "name.minikube.sigs.k8s.io": "addons-649326",

	                "role.minikube.sigs.k8s.io": ""

	            },

	            "StopSignal": "SIGRTMIN+3"

	        },

	        "NetworkSettings": {

	            "Bridge": "",

	            "SandboxID": "c2fe3b98d7ec13ce4c86635fe5ab342a0e0e6f95926a831d9a6cfa6fb9b1800c",

	            "SandboxKey": "/var/run/docker/netns/c2fe3b98d7ec",

	            "Ports": {

	                "22/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "35049"

	                    }

	                ],

	                "2376/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "35048"

	                    }

	                ],

	                "32443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "35045"

	                    }

	                ],

	                "5000/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "35047"

	                    }

	                ],

	                "8443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "35046"

	                    }

	                ]

	            },

	            "HairpinMode": false,

	            "LinkLocalIPv6Address": "",

	            "LinkLocalIPv6PrefixLen": 0,

	            "SecondaryIPAddresses": null,

	            "SecondaryIPv6Addresses": null,

	            "EndpointID": "",

	            "Gateway": "",

	            "GlobalIPv6Address": "",

	            "GlobalIPv6PrefixLen": 0,

	            "IPAddress": "",

	            "IPPrefixLen": 0,

	            "IPv6Gateway": "",

	            "MacAddress": "",

	            "Networks": {

	                "addons-649326": {

	                    "IPAMConfig": {

	                        "IPv4Address": "192.168.49.2"

	                    },

	                    "Links": null,

	                    "Aliases": [

	                        "4dc8c8b83276",

	                        "addons-649326"

	                    ],

	                    "MacAddress": "02:42:c0:a8:31:02",

	                    "NetworkID": "484b5a20cea39300325064af208b14dadeeda7b57a5b626fd80835b27a4d24f5",

	                    "EndpointID": "4d8dbf60e79ed4f338774a48404b336c0913c3a58efa8da98a695f597b0fbae6",

	                    "Gateway": "192.168.49.1",

	                    "IPAddress": "192.168.49.2",

	                    "IPPrefixLen": 24,

	                    "IPv6Gateway": "",

	                    "GlobalIPv6Address": "",

	                    "GlobalIPv6PrefixLen": 0,

	                    "DriverOpts": null,

	                    "DNSNames": [

	                        "addons-649326",

	                        "4dc8c8b83276"

	                    ]

	                }

	            }

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-linux-arm64 status --format={{.Host}} -p addons-649326 -n addons-649326

helpers_test.go:244: <<< TestAddons/parallel/Ingress FAILED: start of post-mortem logs <<<

helpers_test.go:245: ======>  post-mortem[TestAddons/parallel/Ingress]: minikube logs <======

helpers_test.go:247: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 logs -n 25

helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p addons-649326 logs -n 25: (1.449831745s)

helpers_test.go:252: TestAddons/parallel/Ingress logs: 

-- stdout --

	==> Audit <==

	|---------|---------------------------------------------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|

	| Command |                                            Args                                             |        Profile         |  User   | Version |     Start Time      |      End Time       |

	|---------|---------------------------------------------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|

	| delete  | -p download-only-565557                                                                     | download-only-565557   | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 03:57 UTC |

	| delete  | -p download-only-407140                                                                     | download-only-407140   | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 03:57 UTC |

	| delete  | -p download-only-574412                                                                     | download-only-574412   | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 03:57 UTC |

	| start   | --download-only -p                                                                          | download-docker-685329 | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC |                     |

	|         | download-docker-685329                                                                      |                        |         |         |                     |                     |

	|         | --alsologtostderr                                                                           |                        |         |         |                     |                     |

	|         | --driver=docker                                                                             |                        |         |         |                     |                     |

	|         | --container-runtime=crio                                                                    |                        |         |         |                     |                     |

	| delete  | -p download-docker-685329                                                                   | download-docker-685329 | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 03:57 UTC |

	| start   | --download-only -p                                                                          | binary-mirror-734085   | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC |                     |

	|         | binary-mirror-734085                                                                        |                        |         |         |                     |                     |

	|         | --alsologtostderr                                                                           |                        |         |         |                     |                     |

	|         | --binary-mirror                                                                             |                        |         |         |                     |                     |

	|         | http://127.0.0.1:35021                                                                      |                        |         |         |                     |                     |

	|         | --driver=docker                                                                             |                        |         |         |                     |                     |

	|         | --container-runtime=crio                                                                    |                        |         |         |                     |                     |

	| delete  | -p binary-mirror-734085                                                                     | binary-mirror-734085   | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 03:57 UTC |

	| addons  | disable dashboard -p                                                                        | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC |                     |

	|         | addons-649326                                                                               |                        |         |         |                     |                     |

	| addons  | enable dashboard -p                                                                         | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC |                     |

	|         | addons-649326                                                                               |                        |         |         |                     |                     |

	| start   | -p addons-649326 --wait=true                                                                | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 04:00 UTC |

	|         | --memory=4000 --alsologtostderr                                                             |                        |         |         |                     |                     |

	|         | --addons=registry                                                                           |                        |         |         |                     |                     |

	|         | --addons=metrics-server                                                                     |                        |         |         |                     |                     |

	|         | --addons=volumesnapshots                                                                    |                        |         |         |                     |                     |

	|         | --addons=csi-hostpath-driver                                                                |                        |         |         |                     |                     |

	|         | --addons=gcp-auth                                                                           |                        |         |         |                     |                     |

	|         | --addons=cloud-spanner                                                                      |                        |         |         |                     |                     |

	|         | --addons=inspektor-gadget                                                                   |                        |         |         |                     |                     |

	|         | --addons=storage-provisioner-rancher                                                        |                        |         |         |                     |                     |

	|         | --addons=nvidia-device-plugin                                                               |                        |         |         |                     |                     |

	|         | --addons=yakd --driver=docker                                                               |                        |         |         |                     |                     |

	|         |  --container-runtime=crio                                                                   |                        |         |         |                     |                     |

	|         | --addons=ingress                                                                            |                        |         |         |                     |                     |

	|         | --addons=ingress-dns                                                                        |                        |         |         |                     |                     |

	| ip      | addons-649326 ip                                                                            | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:00 UTC | 05 Mar 24 04:00 UTC |

	| addons  | addons-649326 addons disable                                                                | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:00 UTC | 05 Mar 24 04:00 UTC |

	|         | registry --alsologtostderr                                                                  |                        |         |         |                     |                     |

	|         | -v=1                                                                                        |                        |         |         |                     |                     |

	| addons  | disable nvidia-device-plugin                                                                | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:00 UTC | 05 Mar 24 04:00 UTC |

	|         | -p addons-649326                                                                            |                        |         |         |                     |                     |

	| ssh     | addons-649326 ssh cat                                                                       | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:01 UTC | 05 Mar 24 04:01 UTC |

	|         | /opt/local-path-provisioner/pvc-07b1e0a4-0914-45b3-a2bf-4ae142a30731_default_test-pvc/file1 |                        |         |         |                     |                     |

	| addons  | addons-649326 addons disable                                                                | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:01 UTC | 05 Mar 24 04:01 UTC |

	|         | storage-provisioner-rancher                                                                 |                        |         |         |                     |                     |

	|         | --alsologtostderr -v=1                                                                      |                        |         |         |                     |                     |

	| addons  | disable cloud-spanner -p                                                                    | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:01 UTC | 05 Mar 24 04:01 UTC |

	|         | addons-649326                                                                               |                        |         |         |                     |                     |

	| addons  | enable headlamp                                                                             | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:01 UTC | 05 Mar 24 04:01 UTC |

	|         | -p addons-649326                                                                            |                        |         |         |                     |                     |

	|         | --alsologtostderr -v=1                                                                      |                        |         |         |                     |                     |

	| addons  | addons-649326 addons                                                                        | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:01 UTC | 05 Mar 24 04:01 UTC |

	|         | disable csi-hostpath-driver                                                                 |                        |         |         |                     |                     |

	|         | --alsologtostderr -v=1                                                                      |                        |         |         |                     |                     |

	| addons  | addons-649326 addons                                                                        | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:01 UTC | 05 Mar 24 04:01 UTC |

	|         | disable volumesnapshots                                                                     |                        |         |         |                     |                     |

	|         | --alsologtostderr -v=1                                                                      |                        |         |         |                     |                     |

	| addons  | addons-649326 addons                                                                        | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:01 UTC | 05 Mar 24 04:01 UTC |

	|         | disable metrics-server                                                                      |                        |         |         |                     |                     |

	|         | --alsologtostderr -v=1                                                                      |                        |         |         |                     |                     |

	| addons  | disable inspektor-gadget -p                                                                 | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:01 UTC | 05 Mar 24 04:01 UTC |

	|         | addons-649326                                                                               |                        |         |         |                     |                     |

	| ssh     | addons-649326 ssh curl -s                                                                   | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:01 UTC |                     |

	|         | http://127.0.0.1/ -H 'Host:                                                                 |                        |         |         |                     |                     |

	|         | nginx.example.com'                                                                          |                        |         |         |                     |                     |

	| ip      | addons-649326 ip                                                                            | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:03 UTC | 05 Mar 24 04:03 UTC |

	| addons  | addons-649326 addons disable                                                                | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:04 UTC | 05 Mar 24 04:04 UTC |

	|         | ingress-dns --alsologtostderr                                                               |                        |         |         |                     |                     |

	|         | -v=1                                                                                        |                        |         |         |                     |                     |

	| addons  | addons-649326 addons disable                                                                | addons-649326          | jenkins | v1.32.0 | 05 Mar 24 04:04 UTC | 05 Mar 24 04:04 UTC |

	|         | ingress --alsologtostderr -v=1                                                              |                        |         |         |                     |                     |

	|---------|---------------------------------------------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|

	==> Last Start <==

	Log file created at: 2024/03/05 03:57:40

	Running on machine: ip-172-31-31-251

	Binary: Built with gc go1.22.0 for linux/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0305 03:57:40.305215 2132503 out.go:291] Setting OutFile to fd 1 ...

	I0305 03:57:40.305373 2132503 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 03:57:40.305399 2132503 out.go:304] Setting ErrFile to fd 2...

	I0305 03:57:40.305416 2132503 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 03:57:40.305693 2132503 root.go:338] Updating PATH: /home/jenkins/minikube-integration/17909-2126334/.minikube/bin

	I0305 03:57:40.306139 2132503 out.go:298] Setting JSON to false

	I0305 03:57:40.307048 2132503 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":34807,"bootTime":1709576254,"procs":158,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1055-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}

	I0305 03:57:40.307119 2132503 start.go:139] virtualization:  

	I0305 03:57:40.309355 2132503 out.go:177] * [addons-649326] minikube v1.32.0 on Ubuntu 20.04 (arm64)

	I0305 03:57:40.311971 2132503 out.go:177]   - MINIKUBE_LOCATION=17909

	I0305 03:57:40.313694 2132503 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0305 03:57:40.312126 2132503 notify.go:220] Checking for updates...

	I0305 03:57:40.317724 2132503 out.go:177]   - KUBECONFIG=/home/jenkins/minikube-integration/17909-2126334/kubeconfig

	I0305 03:57:40.319655 2132503 out.go:177]   - MINIKUBE_HOME=/home/jenkins/minikube-integration/17909-2126334/.minikube

	I0305 03:57:40.321519 2132503 out.go:177]   - MINIKUBE_BIN=out/minikube-linux-arm64

	I0305 03:57:40.323408 2132503 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0305 03:57:40.325310 2132503 driver.go:392] Setting default libvirt URI to qemu:///system

	I0305 03:57:40.346922 2132503 docker.go:122] docker version: linux-25.0.3:Docker Engine - Community

	I0305 03:57:40.347041 2132503 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0305 03:57:40.421191 2132503 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:28 OomKillDisable:true NGoroutines:49 SystemTime:2024-03-05 03:57:40.411174823 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar

ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro

rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.6]] Warnings:<nil>}}

	I0305 03:57:40.421296 2132503 docker.go:295] overlay module found

	I0305 03:57:40.423659 2132503 out.go:177] * Using the docker driver based on user configuration

	I0305 03:57:40.425394 2132503 start.go:297] selected driver: docker

	I0305 03:57:40.425412 2132503 start.go:901] validating driver "docker" against <nil>

	I0305 03:57:40.425427 2132503 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0305 03:57:40.426071 2132503 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0305 03:57:40.485325 2132503 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:28 OomKillDisable:true NGoroutines:49 SystemTime:2024-03-05 03:57:40.476945213 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar

ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro

rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.6]] Warnings:<nil>}}

	I0305 03:57:40.485504 2132503 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0305 03:57:40.485739 2132503 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0305 03:57:40.487597 2132503 out.go:177] * Using Docker driver with root privileges

	I0305 03:57:40.489596 2132503 cni.go:84] Creating CNI manager for ""

	I0305 03:57:40.489615 2132503 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet

	I0305 03:57:40.489624 2132503 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni

	I0305 03:57:40.489714 2132503 start.go:340] cluster config:

	{Name:addons-649326 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:4000 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:addons-649326 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime

:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSH

AgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0305 03:57:40.491865 2132503 out.go:177] * Starting "addons-649326" primary control-plane node in "addons-649326" cluster

	I0305 03:57:40.493476 2132503 cache.go:121] Beginning downloading kic base image for docker with crio

	I0305 03:57:40.495643 2132503 out.go:177] * Pulling base image v0.0.42-1708944392-18244 ...

	I0305 03:57:40.497202 2132503 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime crio

	I0305 03:57:40.497255 2132503 preload.go:147] Found local preload: /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4

	I0305 03:57:40.497267 2132503 cache.go:56] Caching tarball of preloaded images

	I0305 03:57:40.497296 2132503 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local docker daemon

	I0305 03:57:40.497345 2132503 preload.go:173] Found /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4 in cache, skipping download

	I0305 03:57:40.497355 2132503 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on crio

	I0305 03:57:40.497708 2132503 profile.go:142] Saving config to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/config.json ...

	I0305 03:57:40.497774 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/config.json: {Name:mk5e533b09f1d449bd6f29a7c760897afc8c818d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:57:40.513737 2132503 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 to local cache

	I0305 03:57:40.513893 2132503 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local cache directory

	I0305 03:57:40.513916 2132503 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local cache directory, skipping pull

	I0305 03:57:40.513922 2132503 image.go:105] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 exists in cache, skipping pull

	I0305 03:57:40.513934 2132503 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 as a tarball

	I0305 03:57:40.513944 2132503 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 from local cache

	I0305 03:57:56.404464 2132503 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 from cached tarball

	I0305 03:57:56.404505 2132503 cache.go:194] Successfully downloaded all kic artifacts

	I0305 03:57:56.404534 2132503 start.go:360] acquireMachinesLock for addons-649326: {Name:mk06a004380ceaf28edda0edd0baafab22686de7 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0305 03:57:56.404654 2132503 start.go:364] duration metric: took 96.09µs to acquireMachinesLock for "addons-649326"

	I0305 03:57:56.404688 2132503 start.go:93] Provisioning new machine with config: &{Name:addons-649326 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:4000 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:addons-649326 Namespace:default APIServerHAVIP: APIServerName:min

ikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQe

muFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}

	I0305 03:57:56.404776 2132503 start.go:125] createHost starting for "" (driver="docker")

	I0305 03:57:56.407371 2132503 out.go:204] * Creating docker container (CPUs=2, Memory=4000MB) ...

	I0305 03:57:56.407631 2132503 start.go:159] libmachine.API.Create for "addons-649326" (driver="docker")

	I0305 03:57:56.407667 2132503 client.go:168] LocalClient.Create starting

	I0305 03:57:56.407786 2132503 main.go:141] libmachine: Creating CA: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem

	I0305 03:57:56.735486 2132503 main.go:141] libmachine: Creating client certificate: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/cert.pem

	I0305 03:57:57.020754 2132503 cli_runner.go:164] Run: docker network inspect addons-649326 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	W0305 03:57:57.038002 2132503 cli_runner.go:211] docker network inspect addons-649326 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1

	I0305 03:57:57.038102 2132503 network_create.go:281] running [docker network inspect addons-649326] to gather additional debugging logs...

	I0305 03:57:57.038124 2132503 cli_runner.go:164] Run: docker network inspect addons-649326

	W0305 03:57:57.052904 2132503 cli_runner.go:211] docker network inspect addons-649326 returned with exit code 1

	I0305 03:57:57.052938 2132503 network_create.go:284] error running [docker network inspect addons-649326]: docker network inspect addons-649326: exit status 1

	stdout:

	[]

	stderr:

	Error response from daemon: network addons-649326 not found

	I0305 03:57:57.052953 2132503 network_create.go:286] output of [docker network inspect addons-649326]: -- stdout --

	[]

	-- /stdout --

	** stderr ** 

	Error response from daemon: network addons-649326 not found

	** /stderr **

	I0305 03:57:57.053076 2132503 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0305 03:57:57.072869 2132503 network.go:206] using free private subnet 192.168.49.0/24: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0x40024df050}

	I0305 03:57:57.072908 2132503 network_create.go:124] attempt to create docker network addons-649326 192.168.49.0/24 with gateway 192.168.49.1 and MTU of 1500 ...

	I0305 03:57:57.072965 2132503 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=addons-649326 addons-649326

	I0305 03:57:57.136544 2132503 network_create.go:108] docker network addons-649326 192.168.49.0/24 created

	I0305 03:57:57.136579 2132503 kic.go:121] calculated static IP "192.168.49.2" for the "addons-649326" container

	I0305 03:57:57.136668 2132503 cli_runner.go:164] Run: docker ps -a --format {{.Names}}

	I0305 03:57:57.150762 2132503 cli_runner.go:164] Run: docker volume create addons-649326 --label name.minikube.sigs.k8s.io=addons-649326 --label created_by.minikube.sigs.k8s.io=true

	I0305 03:57:57.167198 2132503 oci.go:103] Successfully created a docker volume addons-649326

	I0305 03:57:57.167306 2132503 cli_runner.go:164] Run: docker run --rm --name addons-649326-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=addons-649326 --entrypoint /usr/bin/test -v addons-649326:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 -d /var/lib

	I0305 03:57:59.031190 2132503 cli_runner.go:217] Completed: docker run --rm --name addons-649326-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=addons-649326 --entrypoint /usr/bin/test -v addons-649326:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 -d /var/lib: (1.863833804s)

	I0305 03:57:59.031219 2132503 oci.go:107] Successfully prepared a docker volume addons-649326

	I0305 03:57:59.031251 2132503 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime crio

	I0305 03:57:59.031269 2132503 kic.go:194] Starting extracting preloaded images to volume ...

	I0305 03:57:59.031357 2132503 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4:/preloaded.tar:ro -v addons-649326:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 -I lz4 -xf /preloaded.tar -C /extractDir

	I0305 03:58:03.559880 2132503 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4:/preloaded.tar:ro -v addons-649326:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 -I lz4 -xf /preloaded.tar -C /extractDir: (4.528471682s)

	I0305 03:58:03.559915 2132503 kic.go:203] duration metric: took 4.528641593s to extract preloaded images to volume ...

	W0305 03:58:03.560082 2132503 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.

	I0305 03:58:03.560195 2132503 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"

	I0305 03:58:03.616029 2132503 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname addons-649326 --name addons-649326 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=addons-649326 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=addons-649326 --network addons-649326 --ip 192.168.49.2 --volume addons-649326:/var --security-opt apparmor=unconfined --memory=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08

	I0305 03:58:03.939655 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Running}}

	I0305 03:58:03.960769 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:03.983273 2132503 cli_runner.go:164] Run: docker exec addons-649326 stat /var/lib/dpkg/alternatives/iptables

	I0305 03:58:04.084868 2132503 oci.go:144] the created container "addons-649326" has a running status.

	I0305 03:58:04.084895 2132503 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa...

	I0305 03:58:05.265228 2132503 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)

	I0305 03:58:05.283375 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:05.298882 2132503 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys

	I0305 03:58:05.298920 2132503 kic_runner.go:114] Args: [docker exec --privileged addons-649326 chown docker:docker /home/docker/.ssh/authorized_keys]

	I0305 03:58:05.343916 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:05.362557 2132503 machine.go:94] provisionDockerMachine start ...

	I0305 03:58:05.362655 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:05.378345 2132503 main.go:141] libmachine: Using SSH client type: native

	I0305 03:58:05.378630 2132503 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil>  [] 0s} 127.0.0.1 35049 <nil> <nil>}

	I0305 03:58:05.378639 2132503 main.go:141] libmachine: About to run SSH command:

	hostname

	I0305 03:58:05.507437 2132503 main.go:141] libmachine: SSH cmd err, output: <nil>: addons-649326

	I0305 03:58:05.507463 2132503 ubuntu.go:169] provisioning hostname "addons-649326"

	I0305 03:58:05.507541 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:05.524553 2132503 main.go:141] libmachine: Using SSH client type: native

	I0305 03:58:05.524803 2132503 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil>  [] 0s} 127.0.0.1 35049 <nil> <nil>}

	I0305 03:58:05.524821 2132503 main.go:141] libmachine: About to run SSH command:

	sudo hostname addons-649326 && echo "addons-649326" | sudo tee /etc/hostname

	I0305 03:58:05.668318 2132503 main.go:141] libmachine: SSH cmd err, output: <nil>: addons-649326

	I0305 03:58:05.668411 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:05.685552 2132503 main.go:141] libmachine: Using SSH client type: native

	I0305 03:58:05.685816 2132503 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil>  [] 0s} 127.0.0.1 35049 <nil> <nil>}

	I0305 03:58:05.685844 2132503 main.go:141] libmachine: About to run SSH command:

			if ! grep -xq '.*\saddons-649326' /etc/hosts; then

				if grep -xq '127.0.1.1\s.*' /etc/hosts; then

					sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 addons-649326/g' /etc/hosts;

				else 

					echo '127.0.1.1 addons-649326' | sudo tee -a /etc/hosts; 

				fi

			fi

	I0305 03:58:05.815865 2132503 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	I0305 03:58:05.815890 2132503 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/17909-2126334/.minikube CaCertPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/17909-2126334/.minikube}

	I0305 03:58:05.815908 2132503 ubuntu.go:177] setting up certificates

	I0305 03:58:05.815918 2132503 provision.go:84] configureAuth start

	I0305 03:58:05.815995 2132503 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" addons-649326

	I0305 03:58:05.834614 2132503 provision.go:143] copyHostCerts

	I0305 03:58:05.834722 2132503 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.pem (1082 bytes)

	I0305 03:58:05.834887 2132503 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/17909-2126334/.minikube/cert.pem (1123 bytes)

	I0305 03:58:05.834972 2132503 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/17909-2126334/.minikube/key.pem (1679 bytes)

	I0305 03:58:05.835058 2132503 provision.go:117] generating server cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca-key.pem org=jenkins.addons-649326 san=[127.0.0.1 192.168.49.2 addons-649326 localhost minikube]

	I0305 03:58:06.348236 2132503 provision.go:177] copyRemoteCerts

	I0305 03:58:06.348308 2132503 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker

	I0305 03:58:06.348355 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:06.370388 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:06.468733 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)

	I0305 03:58:06.493538 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/machines/server.pem --> /etc/docker/server.pem (1208 bytes)

	I0305 03:58:06.517275 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)

	I0305 03:58:06.541164 2132503 provision.go:87] duration metric: took 725.223423ms to configureAuth

	I0305 03:58:06.541198 2132503 ubuntu.go:193] setting minikube options for container-runtime

	I0305 03:58:06.541387 2132503 config.go:182] Loaded profile config "addons-649326": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.28.4

	I0305 03:58:06.541497 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:06.557246 2132503 main.go:141] libmachine: Using SSH client type: native

	I0305 03:58:06.557488 2132503 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil>  [] 0s} 127.0.0.1 35049 <nil> <nil>}

	I0305 03:58:06.557507 2132503 main.go:141] libmachine: About to run SSH command:

	sudo mkdir -p /etc/sysconfig && printf %!s(MISSING) "

	CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '

	" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio

	I0305 03:58:06.782960 2132503 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '

	I0305 03:58:06.782987 2132503 machine.go:97] duration metric: took 1.420410377s to provisionDockerMachine

	I0305 03:58:06.782998 2132503 client.go:171] duration metric: took 10.375320161s to LocalClient.Create

	I0305 03:58:06.783012 2132503 start.go:167] duration metric: took 10.375380895s to libmachine.API.Create "addons-649326"

	I0305 03:58:06.783019 2132503 start.go:293] postStartSetup for "addons-649326" (driver="docker")

	I0305 03:58:06.783030 2132503 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]

	I0305 03:58:06.783093 2132503 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs

	I0305 03:58:06.783157 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:06.799023 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:06.893866 2132503 ssh_runner.go:195] Run: cat /etc/os-release

	I0305 03:58:06.896900 2132503 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found

	I0305 03:58:06.896935 2132503 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found

	I0305 03:58:06.896946 2132503 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found

	I0305 03:58:06.896954 2132503 info.go:137] Remote host: Ubuntu 22.04.3 LTS

	I0305 03:58:06.896968 2132503 filesync.go:126] Scanning /home/jenkins/minikube-integration/17909-2126334/.minikube/addons for local assets ...

	I0305 03:58:06.897033 2132503 filesync.go:126] Scanning /home/jenkins/minikube-integration/17909-2126334/.minikube/files for local assets ...

	I0305 03:58:06.897058 2132503 start.go:296] duration metric: took 114.033438ms for postStartSetup

	I0305 03:58:06.897366 2132503 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" addons-649326

	I0305 03:58:06.912589 2132503 profile.go:142] Saving config to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/config.json ...

	I0305 03:58:06.912874 2132503 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0305 03:58:06.912938 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:06.928299 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:07.016952 2132503 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0305 03:58:07.021510 2132503 start.go:128] duration metric: took 10.616719868s to createHost

	I0305 03:58:07.021577 2132503 start.go:83] releasing machines lock for "addons-649326", held for 10.616905861s

	I0305 03:58:07.021681 2132503 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" addons-649326

	I0305 03:58:07.037912 2132503 ssh_runner.go:195] Run: cat /version.json

	I0305 03:58:07.037951 2132503 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/

	I0305 03:58:07.037967 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:07.038028 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:07.055637 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:07.073511 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:07.159412 2132503 ssh_runner.go:195] Run: systemctl --version

	I0305 03:58:07.273770 2132503 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"

	I0305 03:58:07.414574 2132503 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"

	I0305 03:58:07.418825 2132503 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "sudo mv {} {}.mk_disabled" ;

	I0305 03:58:07.439837 2132503 cni.go:221] loopback cni configuration disabled: "/etc/cni/net.d/*loopback.conf*" found

	I0305 03:58:07.439928 2132503 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;

	I0305 03:58:07.473708 2132503 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)

	I0305 03:58:07.473728 2132503 start.go:494] detecting cgroup driver to use...

	I0305 03:58:07.473765 2132503 detect.go:196] detected "cgroupfs" cgroup driver on host os

	I0305 03:58:07.473812 2132503 ssh_runner.go:195] Run: sudo systemctl stop -f containerd

	I0305 03:58:07.490040 2132503 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0305 03:58:07.501386 2132503 docker.go:217] disabling cri-docker service (if available) ...

	I0305 03:58:07.501445 2132503 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket

	I0305 03:58:07.516053 2132503 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service

	I0305 03:58:07.530328 2132503 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket

	I0305 03:58:07.630530 2132503 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service

	I0305 03:58:07.727324 2132503 docker.go:233] disabling docker service ...

	I0305 03:58:07.727402 2132503 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket

	I0305 03:58:07.747372 2132503 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service

	I0305 03:58:07.759210 2132503 ssh_runner.go:195] Run: sudo systemctl disable docker.socket

	I0305 03:58:07.855356 2132503 ssh_runner.go:195] Run: sudo systemctl mask docker.service

	I0305 03:58:07.957660 2132503 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker

	I0305 03:58:07.969802 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/crio/crio.sock

	" | sudo tee /etc/crictl.yaml"

	I0305 03:58:07.986193 2132503 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.9" pause image...

	I0305 03:58:07.986260 2132503 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.9"|' /etc/crio/crio.conf.d/02-crio.conf"

	I0305 03:58:07.996525 2132503 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...

	I0305 03:58:07.996658 2132503 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"

	I0305 03:58:08.010851 2132503 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"

	I0305 03:58:08.021707 2132503 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"

	I0305 03:58:08.032281 2132503 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"

	I0305 03:58:08.042159 2132503 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables

	I0305 03:58:08.051396 2132503 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

	I0305 03:58:08.060169 2132503 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0305 03:58:08.154666 2132503 ssh_runner.go:195] Run: sudo systemctl restart crio

	I0305 03:58:08.266501 2132503 start.go:541] Will wait 60s for socket path /var/run/crio/crio.sock

	I0305 03:58:08.266590 2132503 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock

	I0305 03:58:08.270700 2132503 start.go:562] Will wait 60s for crictl version

	I0305 03:58:08.270788 2132503 ssh_runner.go:195] Run: which crictl

	I0305 03:58:08.274314 2132503 ssh_runner.go:195] Run: sudo /usr/bin/crictl version

	I0305 03:58:08.311352 2132503 start.go:578] Version:  0.1.0

	RuntimeName:  cri-o

	RuntimeVersion:  1.24.6

	RuntimeApiVersion:  v1

	I0305 03:58:08.311522 2132503 ssh_runner.go:195] Run: crio --version

	I0305 03:58:08.348489 2132503 ssh_runner.go:195] Run: crio --version

	I0305 03:58:08.393356 2132503 out.go:177] * Preparing Kubernetes v1.28.4 on CRI-O 1.24.6 ...

	I0305 03:58:08.395075 2132503 cli_runner.go:164] Run: docker network inspect addons-649326 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0305 03:58:08.410254 2132503 ssh_runner.go:195] Run: grep 192.168.49.1	host.minikube.internal$ /etc/hosts

	I0305 03:58:08.413822 2132503 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.49.1	host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0305 03:58:08.424726 2132503 kubeadm.go:877] updating cluster {Name:addons-649326 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:4000 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:addons-649326 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNa

mes:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmw

arePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...

	I0305 03:58:08.424845 2132503 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime crio

	I0305 03:58:08.424907 2132503 ssh_runner.go:195] Run: sudo crictl images --output json

	I0305 03:58:08.495514 2132503 crio.go:496] all images are preloaded for cri-o runtime.

	I0305 03:58:08.495547 2132503 crio.go:415] Images already preloaded, skipping extraction

	I0305 03:58:08.495616 2132503 ssh_runner.go:195] Run: sudo crictl images --output json

	I0305 03:58:08.532318 2132503 crio.go:496] all images are preloaded for cri-o runtime.

	I0305 03:58:08.532341 2132503 cache_images.go:84] Images are preloaded, skipping loading

	I0305 03:58:08.532349 2132503 kubeadm.go:928] updating node { 192.168.49.2 8443 v1.28.4 crio true true} ...

	I0305 03:58:08.532450 2132503 kubeadm.go:940] kubelet [Unit]

	Wants=crio.service

	[Service]

	ExecStart=

	ExecStart=/var/lib/minikube/binaries/v1.28.4/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroups-per-qos=false --config=/var/lib/kubelet/config.yaml --enforce-node-allocatable= --hostname-override=addons-649326 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.49.2

	[Install]

	 config:

	{KubernetesVersion:v1.28.4 ClusterName:addons-649326 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}

	I0305 03:58:08.532545 2132503 ssh_runner.go:195] Run: crio config

	I0305 03:58:08.580523 2132503 cni.go:84] Creating CNI manager for ""

	I0305 03:58:08.580548 2132503 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet

	I0305 03:58:08.580559 2132503 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16

	I0305 03:58:08.580584 2132503 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.49.2 APIServerPort:8443 KubernetesVersion:v1.28.4 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:addons-649326 NodeName:addons-649326 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.49.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.49.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kuberne

tes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///var/run/crio/crio.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}

	I0305 03:58:08.580742 2132503 kubeadm.go:187] kubeadm config:

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: InitConfiguration

	localAPIEndpoint:

	  advertiseAddress: 192.168.49.2

	  bindPort: 8443

	bootstrapTokens:

	  - groups:

	      - system:bootstrappers:kubeadm:default-node-token

	    ttl: 24h0m0s

	    usages:

	      - signing

	      - authentication

	nodeRegistration:

	  criSocket: unix:///var/run/crio/crio.sock

	  name: "addons-649326"

	  kubeletExtraArgs:

	    node-ip: 192.168.49.2

	  taints: []

	---

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: ClusterConfiguration

	apiServer:

	  certSANs: ["127.0.0.1", "localhost", "192.168.49.2"]

	  extraArgs:

	    enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"

	controllerManager:

	  extraArgs:

	    allocate-node-cidrs: "true"

	    leader-elect: "false"

	scheduler:

	  extraArgs:

	    leader-elect: "false"

	certificatesDir: /var/lib/minikube/certs

	clusterName: mk

	controlPlaneEndpoint: control-plane.minikube.internal:8443

	etcd:

	  local:

	    dataDir: /var/lib/minikube/etcd

	    extraArgs:

	      proxy-refresh-interval: "70000"

	kubernetesVersion: v1.28.4

	networking:

	  dnsDomain: cluster.local

	  podSubnet: "10.244.0.0/16"

	  serviceSubnet: 10.96.0.0/12

	---

	apiVersion: kubelet.config.k8s.io/v1beta1

	kind: KubeletConfiguration

	authentication:

	  x509:

	    clientCAFile: /var/lib/minikube/certs/ca.crt

	cgroupDriver: cgroupfs

	containerRuntimeEndpoint: unix:///var/run/crio/crio.sock

	hairpinMode: hairpin-veth

	runtimeRequestTimeout: 15m

	clusterDomain: "cluster.local"

	# disable disk resource management by default

	imageGCHighThresholdPercent: 100

	evictionHard:

	  nodefs.available: "0%!"(MISSING)

	  nodefs.inodesFree: "0%!"(MISSING)

	  imagefs.available: "0%!"(MISSING)

	failSwapOn: false

	staticPodPath: /etc/kubernetes/manifests

	---

	apiVersion: kubeproxy.config.k8s.io/v1alpha1

	kind: KubeProxyConfiguration

	clusterCIDR: "10.244.0.0/16"

	metricsBindAddress: 0.0.0.0:10249

	conntrack:

	  maxPerCore: 0

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"

	  tcpEstablishedTimeout: 0s

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"

	  tcpCloseWaitTimeout: 0s

	I0305 03:58:08.580827 2132503 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.4

	I0305 03:58:08.590095 2132503 binaries.go:44] Found k8s binaries, skipping transfer

	I0305 03:58:08.590167 2132503 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube

	I0305 03:58:08.599133 2132503 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (363 bytes)

	I0305 03:58:08.616669 2132503 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)

	I0305 03:58:08.634430 2132503 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2151 bytes)

	I0305 03:58:08.653175 2132503 ssh_runner.go:195] Run: grep 192.168.49.2	control-plane.minikube.internal$ /etc/hosts

	I0305 03:58:08.656628 2132503 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.49.2	control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0305 03:58:08.667531 2132503 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0305 03:58:08.760355 2132503 ssh_runner.go:195] Run: sudo systemctl start kubelet

	I0305 03:58:08.774894 2132503 certs.go:68] Setting up /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326 for IP: 192.168.49.2

	I0305 03:58:08.774964 2132503 certs.go:194] generating shared ca certs ...

	I0305 03:58:08.774994 2132503 certs.go:226] acquiring lock for ca certs: {Name:mkbb1962e5c55bf3351f2c6e6b8a52de2404f4b8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:08.775660 2132503 certs.go:240] generating "minikubeCA" ca cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.key

	I0305 03:58:09.869513 2132503 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.crt ...

	I0305 03:58:09.869550 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.crt: {Name:mkb05f22805d949da980747f992ee5cfe53776c0 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:09.870239 2132503 crypto.go:164] Writing key to /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.key ...

	I0305 03:58:09.870257 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.key: {Name:mkabc3475ddae44eb8c547a6e398aedb3fa56b6d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:09.870850 2132503 certs.go:240] generating "proxyClientCA" ca cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/proxy-client-ca.key

	I0305 03:58:10.219113 2132503 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/17909-2126334/.minikube/proxy-client-ca.crt ...

	I0305 03:58:10.219146 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/proxy-client-ca.crt: {Name:mk2bbd66aa0f6b1612460ebb4db01d32556123c4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:10.219336 2132503 crypto.go:164] Writing key to /home/jenkins/minikube-integration/17909-2126334/.minikube/proxy-client-ca.key ...

	I0305 03:58:10.219349 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/proxy-client-ca.key: {Name:mka9bb69412313a1cc948c34b3e8a0a4b807f181 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:10.219442 2132503 certs.go:256] generating profile certs ...

	I0305 03:58:10.219509 2132503 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.key

	I0305 03:58:10.219528 2132503 crypto.go:68] Generating cert /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt with IP's: []

	I0305 03:58:10.426213 2132503 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt ...

	I0305 03:58:10.426247 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: {Name:mk0a9b2224fa27b14038c7d37b0cd520f1d6a57f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:10.426934 2132503 crypto.go:164] Writing key to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.key ...

	I0305 03:58:10.426951 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.key: {Name:mkcf6dab1186eedf150362f55f1336dcb0764ab7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:10.427535 2132503 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.key.22dcc85f

	I0305 03:58:10.427560 2132503 crypto.go:68] Generating cert /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.crt.22dcc85f with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.49.2]

	I0305 03:58:12.091347 2132503 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.crt.22dcc85f ...

	I0305 03:58:12.091384 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.crt.22dcc85f: {Name:mk668038d683d88d7b690dd888d901dd39323e9f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:12.091592 2132503 crypto.go:164] Writing key to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.key.22dcc85f ...

	I0305 03:58:12.091607 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.key.22dcc85f: {Name:mk9cd35ee5166ee3ad8a8524e9d03df0f3eb69e4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:12.091694 2132503 certs.go:381] copying /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.crt.22dcc85f -> /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.crt

	I0305 03:58:12.091794 2132503 certs.go:385] copying /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.key.22dcc85f -> /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.key

	I0305 03:58:12.091849 2132503 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/proxy-client.key

	I0305 03:58:12.091867 2132503 crypto.go:68] Generating cert /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/proxy-client.crt with IP's: []

	I0305 03:58:12.645687 2132503 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/proxy-client.crt ...

	I0305 03:58:12.645724 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/proxy-client.crt: {Name:mk38c5ebd8b6c668f000fd43996abb6080fa6565 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:12.645925 2132503 crypto.go:164] Writing key to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/proxy-client.key ...

	I0305 03:58:12.645949 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/proxy-client.key: {Name:mk3dec2be0c547170fa508abbdc450877072a3a7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:12.646164 2132503 certs.go:484] found cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca-key.pem (1679 bytes)

	I0305 03:58:12.646213 2132503 certs.go:484] found cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem (1082 bytes)

	I0305 03:58:12.646245 2132503 certs.go:484] found cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/cert.pem (1123 bytes)

	I0305 03:58:12.646276 2132503 certs.go:484] found cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/key.pem (1679 bytes)

	I0305 03:58:12.646947 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)

	I0305 03:58:12.672051 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)

	I0305 03:58:12.695727 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)

	I0305 03:58:12.720046 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)

	I0305 03:58:12.743641 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1419 bytes)

	I0305 03:58:12.767669 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)

	I0305 03:58:12.791881 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)

	I0305 03:58:12.815845 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)

	I0305 03:58:12.840540 2132503 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)

	I0305 03:58:12.864765 2132503 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)

	I0305 03:58:12.883057 2132503 ssh_runner.go:195] Run: openssl version

	I0305 03:58:12.889025 2132503 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"

	I0305 03:58:12.898781 2132503 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem

	I0305 03:58:12.902297 2132503 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar  5 03:58 /usr/share/ca-certificates/minikubeCA.pem

	I0305 03:58:12.902389 2132503 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem

	I0305 03:58:12.909483 2132503 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"

	I0305 03:58:12.919097 2132503 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt

	I0305 03:58:12.922336 2132503 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1

	stdout:

	stderr:

	stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory

	I0305 03:58:12.922381 2132503 kubeadm.go:391] StartCluster: {Name:addons-649326 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:4000 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:addons-649326 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames

:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmware

Path: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0305 03:58:12.922485 2132503 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}

	I0305 03:58:12.922549 2132503 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"

	I0305 03:58:12.959796 2132503 cri.go:89] found id: ""

	I0305 03:58:12.959919 2132503 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd

	I0305 03:58:12.969235 2132503 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml

	I0305 03:58:12.978381 2132503 kubeadm.go:213] ignoring SystemVerification for kubeadm because of docker driver

	I0305 03:58:12.978500 2132503 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf

	I0305 03:58:12.987542 2132503 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory

	I0305 03:58:12.987562 2132503 kubeadm.go:156] found existing configuration files:

	I0305 03:58:12.987634 2132503 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf

	I0305 03:58:12.996506 2132503 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/admin.conf: No such file or directory

	I0305 03:58:12.996577 2132503 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf

	I0305 03:58:13.007251 2132503 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf

	I0305 03:58:13.016861 2132503 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/kubelet.conf: No such file or directory

	I0305 03:58:13.016958 2132503 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf

	I0305 03:58:13.025645 2132503 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf

	I0305 03:58:13.035094 2132503 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/controller-manager.conf: No such file or directory

	I0305 03:58:13.035161 2132503 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf

	I0305 03:58:13.044178 2132503 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf

	I0305 03:58:13.052900 2132503 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/scheduler.conf: No such file or directory

	I0305 03:58:13.052993 2132503 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf

	I0305 03:58:13.061823 2132503 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.4:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"

	I0305 03:58:13.109360 2132503 kubeadm.go:309] [init] Using Kubernetes version: v1.28.4

	I0305 03:58:13.109623 2132503 kubeadm.go:309] [preflight] Running pre-flight checks

	I0305 03:58:13.150087 2132503 kubeadm.go:309] [preflight] The system verification failed. Printing the output from the verification:

	I0305 03:58:13.150159 2132503 kubeadm.go:309] KERNEL_VERSION: 5.15.0-1055-aws

	I0305 03:58:13.150200 2132503 kubeadm.go:309] OS: Linux

	I0305 03:58:13.150252 2132503 kubeadm.go:309] CGROUPS_CPU: enabled

	I0305 03:58:13.150303 2132503 kubeadm.go:309] CGROUPS_CPUACCT: enabled

	I0305 03:58:13.150353 2132503 kubeadm.go:309] CGROUPS_CPUSET: enabled

	I0305 03:58:13.150404 2132503 kubeadm.go:309] CGROUPS_DEVICES: enabled

	I0305 03:58:13.150452 2132503 kubeadm.go:309] CGROUPS_FREEZER: enabled

	I0305 03:58:13.150503 2132503 kubeadm.go:309] CGROUPS_MEMORY: enabled

	I0305 03:58:13.150548 2132503 kubeadm.go:309] CGROUPS_PIDS: enabled

	I0305 03:58:13.150599 2132503 kubeadm.go:309] CGROUPS_HUGETLB: enabled

	I0305 03:58:13.150654 2132503 kubeadm.go:309] CGROUPS_BLKIO: enabled

	I0305 03:58:13.226750 2132503 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster

	I0305 03:58:13.226879 2132503 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection

	I0305 03:58:13.226975 2132503 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	I0305 03:58:13.473975 2132503 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"

	I0305 03:58:13.478753 2132503 out.go:204]   - Generating certificates and keys ...

	I0305 03:58:13.478896 2132503 kubeadm.go:309] [certs] Using existing ca certificate authority

	I0305 03:58:13.478984 2132503 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk

	I0305 03:58:13.748316 2132503 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key

	I0305 03:58:14.438617 2132503 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key

	I0305 03:58:15.161512 2132503 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key

	I0305 03:58:16.125610 2132503 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key

	I0305 03:58:16.822290 2132503 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key

	I0305 03:58:16.822613 2132503 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [addons-649326 localhost] and IPs [192.168.49.2 127.0.0.1 ::1]

	I0305 03:58:18.754345 2132503 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key

	I0305 03:58:18.754753 2132503 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [addons-649326 localhost] and IPs [192.168.49.2 127.0.0.1 ::1]

	I0305 03:58:19.037791 2132503 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key

	I0305 03:58:19.845199 2132503 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key

	I0305 03:58:20.312251 2132503 kubeadm.go:309] [certs] Generating "sa" key and public key

	I0305 03:58:20.312560 2132503 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	I0305 03:58:20.688917 2132503 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file

	I0305 03:58:21.047176 2132503 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file

	I0305 03:58:21.310368 2132503 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file

	I0305 03:58:21.720435 2132503 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file

	I0305 03:58:21.721086 2132503 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	I0305 03:58:21.725132 2132503 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"

	I0305 03:58:21.728165 2132503 out.go:204]   - Booting up control plane ...

	I0305 03:58:21.728271 2132503 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"

	I0305 03:58:21.729278 2132503 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"

	I0305 03:58:21.730428 2132503 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"

	I0305 03:58:21.740982 2132503 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	I0305 03:58:21.741984 2132503 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	I0305 03:58:21.742039 2132503 kubeadm.go:309] [kubelet-start] Starting the kubelet

	I0305 03:58:21.828262 2132503 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	I0305 03:58:28.830532 2132503 kubeadm.go:309] [apiclient] All control plane components are healthy after 7.002403 seconds

	I0305 03:58:28.830658 2132503 kubeadm.go:309] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace

	I0305 03:58:28.844169 2132503 kubeadm.go:309] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster

	I0305 03:58:29.370052 2132503 kubeadm.go:309] [upload-certs] Skipping phase. Please see --upload-certs

	I0305 03:58:29.370252 2132503 kubeadm.go:309] [mark-control-plane] Marking the node addons-649326 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]

	I0305 03:58:29.885308 2132503 kubeadm.go:309] [bootstrap-token] Using token: y1bgh8.wrxmwhn9dgvpxvuh

	I0305 03:58:29.887077 2132503 out.go:204]   - Configuring RBAC rules ...

	I0305 03:58:29.887201 2132503 kubeadm.go:309] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles

	I0305 03:58:29.893876 2132503 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes

	I0305 03:58:29.904326 2132503 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials

	I0305 03:58:29.908684 2132503 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token

	I0305 03:58:29.912826 2132503 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster

	I0305 03:58:29.916652 2132503 kubeadm.go:309] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace

	I0305 03:58:29.930534 2132503 kubeadm.go:309] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key

	I0305 03:58:30.155846 2132503 kubeadm.go:309] [addons] Applied essential addon: CoreDNS

	I0305 03:58:30.312289 2132503 kubeadm.go:309] [addons] Applied essential addon: kube-proxy

	I0305 03:58:30.313416 2132503 kubeadm.go:309] 

	I0305 03:58:30.313495 2132503 kubeadm.go:309] Your Kubernetes control-plane has initialized successfully!

	I0305 03:58:30.313507 2132503 kubeadm.go:309] 

	I0305 03:58:30.313582 2132503 kubeadm.go:309] To start using your cluster, you need to run the following as a regular user:

	I0305 03:58:30.313591 2132503 kubeadm.go:309] 

	I0305 03:58:30.313616 2132503 kubeadm.go:309]   mkdir -p $HOME/.kube

	I0305 03:58:30.313676 2132503 kubeadm.go:309]   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

	I0305 03:58:30.313744 2132503 kubeadm.go:309]   sudo chown $(id -u):$(id -g) $HOME/.kube/config

	I0305 03:58:30.313753 2132503 kubeadm.go:309] 

	I0305 03:58:30.313805 2132503 kubeadm.go:309] Alternatively, if you are the root user, you can run:

	I0305 03:58:30.313813 2132503 kubeadm.go:309] 

	I0305 03:58:30.313859 2132503 kubeadm.go:309]   export KUBECONFIG=/etc/kubernetes/admin.conf

	I0305 03:58:30.313867 2132503 kubeadm.go:309] 

	I0305 03:58:30.313918 2132503 kubeadm.go:309] You should now deploy a pod network to the cluster.

	I0305 03:58:30.313994 2132503 kubeadm.go:309] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

	I0305 03:58:30.314063 2132503 kubeadm.go:309]   https://kubernetes.io/docs/concepts/cluster-administration/addons/

	I0305 03:58:30.314071 2132503 kubeadm.go:309] 

	I0305 03:58:30.314152 2132503 kubeadm.go:309] You can now join any number of control-plane nodes by copying certificate authorities

	I0305 03:58:30.314229 2132503 kubeadm.go:309] and service account keys on each node and then running the following as root:

	I0305 03:58:30.314238 2132503 kubeadm.go:309] 

	I0305 03:58:30.314318 2132503 kubeadm.go:309]   kubeadm join control-plane.minikube.internal:8443 --token y1bgh8.wrxmwhn9dgvpxvuh \

	I0305 03:58:30.314421 2132503 kubeadm.go:309] 	--discovery-token-ca-cert-hash sha256:cefcd7c0bfd2c452822fa36506aa2c27229f3d47a807a2142de836bb6edf18e7 \

	I0305 03:58:30.314443 2132503 kubeadm.go:309] 	--control-plane 

	I0305 03:58:30.314451 2132503 kubeadm.go:309] 

	I0305 03:58:30.314532 2132503 kubeadm.go:309] Then you can join any number of worker nodes by running the following on each as root:

	I0305 03:58:30.314541 2132503 kubeadm.go:309] 

	I0305 03:58:30.314630 2132503 kubeadm.go:309] kubeadm join control-plane.minikube.internal:8443 --token y1bgh8.wrxmwhn9dgvpxvuh \

	I0305 03:58:30.314739 2132503 kubeadm.go:309] 	--discovery-token-ca-cert-hash sha256:cefcd7c0bfd2c452822fa36506aa2c27229f3d47a807a2142de836bb6edf18e7 

	I0305 03:58:30.318119 2132503 kubeadm.go:309] 	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1055-aws\n", err: exit status 1

	I0305 03:58:30.318234 2132503 kubeadm.go:309] 	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	I0305 03:58:30.318254 2132503 cni.go:84] Creating CNI manager for ""

	I0305 03:58:30.318263 2132503 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet

	I0305 03:58:30.320639 2132503 out.go:177] * Configuring CNI (Container Networking Interface) ...

	I0305 03:58:30.323742 2132503 ssh_runner.go:195] Run: stat /opt/cni/bin/portmap

	I0305 03:58:30.345920 2132503 cni.go:182] applying CNI manifest using /var/lib/minikube/binaries/v1.28.4/kubectl ...

	I0305 03:58:30.345940 2132503 ssh_runner.go:362] scp memory --> /var/tmp/minikube/cni.yaml (2438 bytes)

	I0305 03:58:30.388631 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml

	I0305 03:58:31.277569 2132503 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"

	I0305 03:58:31.277703 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:31.277783 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig label --overwrite nodes addons-649326 minikube.k8s.io/updated_at=2024_03_05T03_58_31_0700 minikube.k8s.io/version=v1.32.0 minikube.k8s.io/commit=c10843c2362d401e82b228acf1f340a428845477 minikube.k8s.io/name=addons-649326 minikube.k8s.io/primary=true

	I0305 03:58:31.460088 2132503 ops.go:34] apiserver oom_adj: -16

	I0305 03:58:31.460229 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:31.960367 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:32.460871 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:32.961178 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:33.460609 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:33.961371 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:34.461032 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:34.960322 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:35.460635 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:35.960384 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:36.460396 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:36.960338 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:37.460361 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:37.960875 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:38.460401 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:38.961288 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:39.460851 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:39.960751 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:40.461187 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:40.960615 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:41.460996 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:41.960378 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:42.460677 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:42.960353 2132503 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 03:58:43.064908 2132503 kubeadm.go:1106] duration metric: took 11.7872504s to wait for elevateKubeSystemPrivileges

	W0305 03:58:43.064948 2132503 kubeadm.go:286] apiserver tunnel failed: apiserver port not set

	I0305 03:58:43.064957 2132503 kubeadm.go:393] duration metric: took 30.142579574s to StartCluster

	I0305 03:58:43.064976 2132503 settings.go:142] acquiring lock: {Name:mkcb5c641f18ebe4ce7c5a24601fb23663fb648c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:43.065106 2132503 settings.go:150] Updating kubeconfig:  /home/jenkins/minikube-integration/17909-2126334/kubeconfig

	I0305 03:58:43.065529 2132503 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/kubeconfig: {Name:mkc7f94e2bde916bb2b18dc0ff5a1ab6ea679ee4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 03:58:43.065743 2132503 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}

	I0305 03:58:43.068735 2132503 out.go:177] * Verifying Kubernetes components...

	I0305 03:58:43.065903 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"

	I0305 03:58:43.066093 2132503 config.go:182] Loaded profile config "addons-649326": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.28.4

	I0305 03:58:43.066104 2132503 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:true csi-hostpath-driver:true dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:true gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:true ingress-dns:true inspektor-gadget:true istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:true nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:true registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:true volumesnapshots:true yakd:true]

	I0305 03:58:43.070856 2132503 addons.go:69] Setting yakd=true in profile "addons-649326"

	I0305 03:58:43.070883 2132503 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0305 03:58:43.070895 2132503 addons.go:234] Setting addon yakd=true in "addons-649326"

	I0305 03:58:43.070946 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.070983 2132503 addons.go:69] Setting ingress-dns=true in profile "addons-649326"

	I0305 03:58:43.071006 2132503 addons.go:234] Setting addon ingress-dns=true in "addons-649326"

	I0305 03:58:43.071040 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.071506 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.071514 2132503 addons.go:69] Setting inspektor-gadget=true in profile "addons-649326"

	I0305 03:58:43.071540 2132503 addons.go:234] Setting addon inspektor-gadget=true in "addons-649326"

	I0305 03:58:43.071565 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.071994 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.072822 2132503 addons.go:69] Setting cloud-spanner=true in profile "addons-649326"

	I0305 03:58:43.072862 2132503 addons.go:234] Setting addon cloud-spanner=true in "addons-649326"

	I0305 03:58:43.072763 2132503 addons.go:69] Setting metrics-server=true in profile "addons-649326"

	I0305 03:58:43.072888 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.072893 2132503 addons.go:234] Setting addon metrics-server=true in "addons-649326"

	I0305 03:58:43.072920 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.073296 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.073398 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.077985 2132503 addons.go:69] Setting csi-hostpath-driver=true in profile "addons-649326"

	I0305 03:58:43.078063 2132503 addons.go:234] Setting addon csi-hostpath-driver=true in "addons-649326"

	I0305 03:58:43.078095 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.078537 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.082989 2132503 addons.go:69] Setting default-storageclass=true in profile "addons-649326"

	I0305 03:58:43.083049 2132503 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "addons-649326"

	I0305 03:58:43.083399 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.086155 2132503 addons.go:69] Setting nvidia-device-plugin=true in profile "addons-649326"

	I0305 03:58:43.086228 2132503 addons.go:234] Setting addon nvidia-device-plugin=true in "addons-649326"

	I0305 03:58:43.086296 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.086952 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.095396 2132503 addons.go:69] Setting gcp-auth=true in profile "addons-649326"

	I0305 03:58:43.095507 2132503 mustload.go:65] Loading cluster: addons-649326

	I0305 03:58:43.095766 2132503 config.go:182] Loaded profile config "addons-649326": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.28.4

	I0305 03:58:43.096245 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.100760 2132503 addons.go:69] Setting registry=true in profile "addons-649326"

	I0305 03:58:43.100812 2132503 addons.go:234] Setting addon registry=true in "addons-649326"

	I0305 03:58:43.100852 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.101320 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.110908 2132503 addons.go:69] Setting ingress=true in profile "addons-649326"

	I0305 03:58:43.111024 2132503 addons.go:234] Setting addon ingress=true in "addons-649326"

	I0305 03:58:43.111105 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.111704 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.114422 2132503 addons.go:69] Setting storage-provisioner=true in profile "addons-649326"

	I0305 03:58:43.114465 2132503 addons.go:234] Setting addon storage-provisioner=true in "addons-649326"

	I0305 03:58:43.114502 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.114956 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.071508 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.132700 2132503 addons.go:69] Setting storage-provisioner-rancher=true in profile "addons-649326"

	I0305 03:58:43.132745 2132503 addons_storage_classes.go:33] enableOrDisableStorageClasses storage-provisioner-rancher=true on "addons-649326"

	I0305 03:58:43.133071 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.182498 2132503 out.go:177]   - Using image gcr.io/k8s-minikube/minikube-ingress-dns:0.0.2

	I0305 03:58:43.189568 2132503 addons.go:426] installing /etc/kubernetes/addons/ingress-dns-pod.yaml

	I0305 03:58:43.189639 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ingress-dns-pod.yaml (2442 bytes)

	I0305 03:58:43.189745 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.175059 2132503 addons.go:69] Setting volumesnapshots=true in profile "addons-649326"

	I0305 03:58:43.248907 2132503 addons.go:234] Setting addon volumesnapshots=true in "addons-649326"

	I0305 03:58:43.249007 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.249660 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.266110 2132503 addons.go:234] Setting addon default-storageclass=true in "addons-649326"

	I0305 03:58:43.266156 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.266648 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.310137 2132503 out.go:177]   - Using image gcr.io/cloud-spanner-emulator/emulator:1.5.14

	I0305 03:58:43.334181 2132503 out.go:177]   - Using image registry.k8s.io/metrics-server/metrics-server:v0.7.0

	I0305 03:58:43.343549 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.343566 2132503 out.go:177]   - Using image ghcr.io/inspektor-gadget/inspektor-gadget:v0.25.1

	I0305 03:58:43.343728 2132503 addons.go:426] installing /etc/kubernetes/addons/deployment.yaml

	I0305 03:58:43.343735 2132503 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.7.0

	I0305 03:58:43.343740 2132503 out.go:177]   - Using image nvcr.io/nvidia/k8s-device-plugin:v0.14.5

	I0305 03:58:43.354532 2132503 addons.go:426] installing /etc/kubernetes/addons/metrics-apiservice.yaml

	I0305 03:58:43.356182 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)

	I0305 03:58:43.356252 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.364784 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/deployment.yaml (1004 bytes)

	I0305 03:58:43.364982 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.392188 2132503 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.6.0

	I0305 03:58:43.394211 2132503 out.go:177]   - Using image registry.k8s.io/sig-storage/hostpathplugin:v1.9.0

	I0305 03:58:43.396675 2132503 out.go:177]   - Using image registry.k8s.io/sig-storage/livenessprobe:v2.8.0

	I0305 03:58:43.400055 2132503 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-resizer:v1.6.0

	I0305 03:58:43.402503 2132503 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0

	I0305 03:58:43.414921 2132503 out.go:177]   - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0

	I0305 03:58:43.417257 2132503 out.go:177]   - Using image registry.k8s.io/ingress-nginx/controller:v1.10.0

	I0305 03:58:43.422197 2132503 out.go:177]   - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0

	I0305 03:58:43.415349 2132503 addons.go:426] installing /etc/kubernetes/addons/nvidia-device-plugin.yaml

	I0305 03:58:43.417185 2132503 addons.go:426] installing /etc/kubernetes/addons/ig-namespace.yaml

	I0305 03:58:43.417198 2132503 out.go:177]   - Using image gcr.io/k8s-minikube/kube-registry-proxy:0.0.5

	I0305 03:58:43.424951 2132503 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-provisioner:v3.3.0

	I0305 03:58:43.425019 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/nvidia-device-plugin.yaml (1966 bytes)

	I0305 03:58:43.425317 2132503 addons.go:426] installing /etc/kubernetes/addons/ingress-deploy.yaml

	I0305 03:58:43.436057 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ingress-deploy.yaml (16078 bytes)

	I0305 03:58:43.436157 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.442284 2132503 addons.go:234] Setting addon storage-provisioner-rancher=true in "addons-649326"

	I0305 03:58:43.442501 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:43.443104 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:43.451799 2132503 out.go:177]   - Using image docker.io/marcnuri/yakd:0.0.4

	I0305 03:58:43.454557 2132503 addons.go:426] installing /etc/kubernetes/addons/yakd-ns.yaml

	I0305 03:58:43.454578 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/yakd-ns.yaml (171 bytes)

	I0305 03:58:43.454646 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.452889 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.457789 2132503 out.go:177]   - Using image registry.k8s.io/sig-storage/csi-attacher:v4.0.0

	I0305 03:58:43.452931 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-namespace.yaml (55 bytes)

	I0305 03:58:43.453002 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.468234 2132503 addons.go:426] installing /etc/kubernetes/addons/rbac-external-attacher.yaml

	I0305 03:58:43.468320 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.496086 2132503 out.go:177]   - Using image registry.k8s.io/sig-storage/snapshot-controller:v6.1.0

	I0305 03:58:43.498012 2132503 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml

	I0305 03:58:43.498034 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml (934 bytes)

	I0305 03:58:43.498102 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.536162 2132503 out.go:177]   - Using image docker.io/registry:2.8.3

	I0305 03:58:43.492622 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-attacher.yaml (3073 bytes)

	I0305 03:58:43.536144 2132503 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml

	I0305 03:58:43.542951 2132503 addons.go:426] installing /etc/kubernetes/addons/registry-rc.yaml

	I0305 03:58:43.543043 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.559775 2132503 out.go:177]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5

	I0305 03:58:43.574246 2132503 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml

	I0305 03:58:43.574326 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)

	I0305 03:58:43.574441 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.560383 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/registry-rc.yaml (798 bytes)

	I0305 03:58:43.560404 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)

	I0305 03:58:43.606156 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.612164 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.560775 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.620471 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.624525 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.627643 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^        forward . \/etc\/resolv.conf.*/i \        hosts {\n           192.168.49.1 host.minikube.internal\n           fallthrough\n        }' -e '/^        errors *$/i \        log' | sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"

	I0305 03:58:43.628435 2132503 ssh_runner.go:195] Run: sudo systemctl start kubelet

	I0305 03:58:43.658237 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.667187 2132503 out.go:177]   - Using image docker.io/busybox:stable

	I0305 03:58:43.669871 2132503 out.go:177]   - Using image docker.io/rancher/local-path-provisioner:v0.0.22

	I0305 03:58:43.673868 2132503 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner-rancher.yaml

	I0305 03:58:43.673902 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner-rancher.yaml (3113 bytes)

	I0305 03:58:43.673984 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:43.667298 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.701715 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.730981 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.765770 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.770971 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.781189 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.805058 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.829083 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:43.909899 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/ingress-dns-pod.yaml

	I0305 03:58:44.108362 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml

	I0305 03:58:44.149653 2132503 addons.go:426] installing /etc/kubernetes/addons/metrics-server-deployment.yaml

	I0305 03:58:44.149676 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1907 bytes)

	I0305 03:58:44.233998 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/nvidia-device-plugin.yaml

	I0305 03:58:44.234701 2132503 addons.go:426] installing /etc/kubernetes/addons/yakd-sa.yaml

	I0305 03:58:44.234758 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/yakd-sa.yaml (247 bytes)

	I0305 03:58:44.260830 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/deployment.yaml

	I0305 03:58:44.339457 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml

	I0305 03:58:44.351548 2132503 addons.go:426] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml

	I0305 03:58:44.351620 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml (6471 bytes)

	I0305 03:58:44.353993 2132503 addons.go:426] installing /etc/kubernetes/addons/rbac-hostpath.yaml

	I0305 03:58:44.354058 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-hostpath.yaml (4266 bytes)

	I0305 03:58:44.363669 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml

	I0305 03:58:44.379153 2132503 addons.go:426] installing /etc/kubernetes/addons/metrics-server-rbac.yaml

	I0305 03:58:44.379224 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)

	I0305 03:58:44.421573 2132503 addons.go:426] installing /etc/kubernetes/addons/registry-svc.yaml

	I0305 03:58:44.421650 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/registry-svc.yaml (398 bytes)

	I0305 03:58:44.422100 2132503 addons.go:426] installing /etc/kubernetes/addons/ig-serviceaccount.yaml

	I0305 03:58:44.422158 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-serviceaccount.yaml (80 bytes)

	I0305 03:58:44.424920 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storage-provisioner-rancher.yaml

	I0305 03:58:44.428283 2132503 addons.go:426] installing /etc/kubernetes/addons/yakd-crb.yaml

	I0305 03:58:44.428342 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/yakd-crb.yaml (422 bytes)

	I0305 03:58:44.520257 2132503 addons.go:426] installing /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml

	I0305 03:58:44.520332 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml (3038 bytes)

	I0305 03:58:44.535538 2132503 addons.go:426] installing /etc/kubernetes/addons/registry-proxy.yaml

	I0305 03:58:44.535610 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/registry-proxy.yaml (947 bytes)

	I0305 03:58:44.559710 2132503 addons.go:426] installing /etc/kubernetes/addons/metrics-server-service.yaml

	I0305 03:58:44.559783 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)

	I0305 03:58:44.581377 2132503 addons.go:426] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml

	I0305 03:58:44.581449 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml (23126 bytes)

	I0305 03:58:44.676094 2132503 addons.go:426] installing /etc/kubernetes/addons/ig-role.yaml

	I0305 03:58:44.676166 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-role.yaml (210 bytes)

	I0305 03:58:44.689783 2132503 addons.go:426] installing /etc/kubernetes/addons/yakd-svc.yaml

	I0305 03:58:44.689860 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/yakd-svc.yaml (412 bytes)

	I0305 03:58:44.759710 2132503 addons.go:426] installing /etc/kubernetes/addons/rbac-external-provisioner.yaml

	I0305 03:58:44.759782 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-provisioner.yaml (4442 bytes)

	I0305 03:58:44.799035 2132503 addons.go:426] installing /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml

	I0305 03:58:44.799109 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml (19582 bytes)

	I0305 03:58:44.813707 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/registry-rc.yaml -f /etc/kubernetes/addons/registry-svc.yaml -f /etc/kubernetes/addons/registry-proxy.yaml

	I0305 03:58:44.824746 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml

	I0305 03:58:44.854846 2132503 addons.go:426] installing /etc/kubernetes/addons/ig-rolebinding.yaml

	I0305 03:58:44.854918 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-rolebinding.yaml (244 bytes)

	I0305 03:58:44.921469 2132503 addons.go:426] installing /etc/kubernetes/addons/yakd-dp.yaml

	I0305 03:58:44.921542 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/yakd-dp.yaml (2017 bytes)

	I0305 03:58:44.946496 2132503 addons.go:426] installing /etc/kubernetes/addons/rbac-external-resizer.yaml

	I0305 03:58:44.946570 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-resizer.yaml (2943 bytes)

	I0305 03:58:45.036899 2132503 addons.go:426] installing /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml

	I0305 03:58:45.036981 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml (3545 bytes)

	I0305 03:58:45.056757 2132503 addons.go:426] installing /etc/kubernetes/addons/ig-clusterrole.yaml

	I0305 03:58:45.056855 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-clusterrole.yaml (1485 bytes)

	I0305 03:58:45.130644 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/yakd-ns.yaml -f /etc/kubernetes/addons/yakd-sa.yaml -f /etc/kubernetes/addons/yakd-crb.yaml -f /etc/kubernetes/addons/yakd-svc.yaml -f /etc/kubernetes/addons/yakd-dp.yaml

	I0305 03:58:45.182908 2132503 addons.go:426] installing /etc/kubernetes/addons/rbac-external-snapshotter.yaml

	I0305 03:58:45.182992 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/rbac-external-snapshotter.yaml (3149 bytes)

	I0305 03:58:45.282626 2132503 addons.go:426] installing /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0305 03:58:45.282717 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml (1475 bytes)

	I0305 03:58:45.346207 2132503 addons.go:426] installing /etc/kubernetes/addons/ig-clusterrolebinding.yaml

	I0305 03:58:45.346293 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-clusterrolebinding.yaml (274 bytes)

	I0305 03:58:45.377544 2132503 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-attacher.yaml

	I0305 03:58:45.377623 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-attacher.yaml (2143 bytes)

	I0305 03:58:45.443587 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0305 03:58:45.490165 2132503 addons.go:426] installing /etc/kubernetes/addons/ig-crd.yaml

	I0305 03:58:45.490240 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-crd.yaml (5216 bytes)

	I0305 03:58:45.516940 2132503 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml

	I0305 03:58:45.517021 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml (1274 bytes)

	I0305 03:58:45.640951 2132503 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-plugin.yaml

	I0305 03:58:45.641025 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-plugin.yaml (8201 bytes)

	I0305 03:58:45.663058 2132503 addons.go:426] installing /etc/kubernetes/addons/ig-daemonset.yaml

	I0305 03:58:45.663127 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/ig-daemonset.yaml (7735 bytes)

	I0305 03:58:45.742926 2132503 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-resizer.yaml

	I0305 03:58:45.742997 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-resizer.yaml (2191 bytes)

	I0305 03:58:45.862783 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/ig-namespace.yaml -f /etc/kubernetes/addons/ig-serviceaccount.yaml -f /etc/kubernetes/addons/ig-role.yaml -f /etc/kubernetes/addons/ig-rolebinding.yaml -f /etc/kubernetes/addons/ig-clusterrole.yaml -f /etc/kubernetes/addons/ig-clusterrolebinding.yaml -f /etc/kubernetes/addons/ig-crd.yaml -f /etc/kubernetes/addons/ig-daemonset.yaml

	I0305 03:58:45.899286 2132503 addons.go:426] installing /etc/kubernetes/addons/csi-hostpath-storageclass.yaml

	I0305 03:58:45.899359 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/csi-hostpath-storageclass.yaml (846 bytes)

	I0305 03:58:46.054604 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml

	I0305 03:58:46.753508 2132503 ssh_runner.go:235] Completed: sudo systemctl start kubelet: (3.125041474s)

	I0305 03:58:46.754360 2132503 node_ready.go:35] waiting up to 6m0s for node "addons-649326" to be "Ready" ...

	I0305 03:58:46.754535 2132503 ssh_runner.go:235] Completed: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^        forward . \/etc\/resolv.conf.*/i \        hosts {\n           192.168.49.1 host.minikube.internal\n           fallthrough\n        }' -e '/^        errors *$/i \        log' | sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -": (3.126384292s)

	I0305 03:58:46.754550 2132503 start.go:948] {"host.minikube.internal": 192.168.49.1} host record injected into CoreDNS's ConfigMap

	I0305 03:58:47.972960 2132503 kapi.go:248] "coredns" deployment in "kube-system" namespace and "addons-649326" context rescaled to 1 replicas

	I0305 03:58:48.806931 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:58:49.219131 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/ingress-dns-pod.yaml: (5.309190804s)

	I0305 03:58:50.539034 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/deployment.yaml: (6.278132309s)

	I0305 03:58:50.539153 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (6.199628065s)

	I0305 03:58:50.539210 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: (6.175472081s)

	I0305 03:58:50.538993 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/nvidia-device-plugin.yaml: (6.304913187s)

	I0305 03:58:50.539542 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storage-provisioner-rancher.yaml: (6.114574542s)

	I0305 03:58:50.539768 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (5.714948534s)

	I0305 03:58:50.540194 2132503 addons.go:470] Verifying addon metrics-server=true in "addons-649326"

	I0305 03:58:50.539581 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/ingress-deploy.yaml: (6.431185338s)

	I0305 03:58:50.540276 2132503 addons.go:470] Verifying addon ingress=true in "addons-649326"

	I0305 03:58:50.544671 2132503 out.go:177] * Verifying ingress addon...

	I0305 03:58:50.539607 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/registry-rc.yaml -f /etc/kubernetes/addons/registry-svc.yaml -f /etc/kubernetes/addons/registry-proxy.yaml: (5.725826005s)

	I0305 03:58:50.539870 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/yakd-ns.yaml -f /etc/kubernetes/addons/yakd-sa.yaml -f /etc/kubernetes/addons/yakd-crb.yaml -f /etc/kubernetes/addons/yakd-svc.yaml -f /etc/kubernetes/addons/yakd-dp.yaml: (5.409126832s)

	I0305 03:58:50.539948 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: (5.0962857s)

	I0305 03:58:50.540004 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/ig-namespace.yaml -f /etc/kubernetes/addons/ig-serviceaccount.yaml -f /etc/kubernetes/addons/ig-role.yaml -f /etc/kubernetes/addons/ig-rolebinding.yaml -f /etc/kubernetes/addons/ig-clusterrole.yaml -f /etc/kubernetes/addons/ig-clusterrolebinding.yaml -f /etc/kubernetes/addons/ig-crd.yaml -f /etc/kubernetes/addons/ig-daemonset.yaml: (4.67714594s)

	I0305 03:58:50.544824 2132503 addons.go:470] Verifying addon registry=true in "addons-649326"

	W0305 03:58:50.544957 2132503 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1

	stdout:

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created

	serviceaccount/snapshot-controller created

	clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created

	clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created

	role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	deployment.apps/snapshot-controller created

	stderr:

	error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"

	ensure CRDs are installed first

	I0305 03:58:50.548861 2132503 kapi.go:75] Waiting for pod with label "app.kubernetes.io/name=ingress-nginx" in ns "ingress-nginx" ...

	I0305 03:58:50.550553 2132503 out.go:177] * To access YAKD - Kubernetes Dashboard, wait for Pod to be ready and run the following command:

		minikube -p addons-649326 service yakd-dashboard -n yakd-dashboard

	I0305 03:58:50.550560 2132503 out.go:177] * Verifying registry addon...

	I0305 03:58:50.550599 2132503 retry.go:31] will retry after 360.366052ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: Process exited with status 1

	stdout:

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created

	customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created

	serviceaccount/snapshot-controller created

	clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created

	clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created

	role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created

	deployment.apps/snapshot-controller created

	stderr:

	error: resource mapping not found for name: "csi-hostpath-snapclass" namespace: "" from "/etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml": no matches for kind "VolumeSnapshotClass" in version "snapshot.storage.k8s.io/v1"

	ensure CRDs are installed first

	I0305 03:58:50.555299 2132503 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=registry" in ns "kube-system" ...

	W0305 03:58:50.565010 2132503 out.go:239] ! Enabling 'storage-provisioner-rancher' returned an error: running callbacks: [Error making local-path the default storage class: Error while marking storage class local-path as default: Operation cannot be fulfilled on storageclasses.storage.k8s.io "local-path": the object has been modified; please apply your changes to the latest version and try again]

	I0305 03:58:50.568007 2132503 kapi.go:86] Found 1 Pods for label selector kubernetes.io/minikube-addons=registry

	I0305 03:58:50.568076 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:50.568555 2132503 kapi.go:86] Found 3 Pods for label selector app.kubernetes.io/name=ingress-nginx

	I0305 03:58:50.568597 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:50.822135 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/rbac-external-attacher.yaml -f /etc/kubernetes/addons/rbac-hostpath.yaml -f /etc/kubernetes/addons/rbac-external-health-monitor-controller.yaml -f /etc/kubernetes/addons/rbac-external-provisioner.yaml -f /etc/kubernetes/addons/rbac-external-resizer.yaml -f /etc/kubernetes/addons/rbac-external-snapshotter.yaml -f /etc/kubernetes/addons/csi-hostpath-attacher.yaml -f /etc/kubernetes/addons/csi-hostpath-driverinfo.yaml -f /etc/kubernetes/addons/csi-hostpath-plugin.yaml -f /etc/kubernetes/addons/csi-hostpath-resizer.yaml -f /etc/kubernetes/addons/csi-hostpath-storageclass.yaml: (4.767431437s)

	I0305 03:58:50.822167 2132503 addons.go:470] Verifying addon csi-hostpath-driver=true in "addons-649326"

	I0305 03:58:50.829302 2132503 out.go:177] * Verifying csi-hostpath-driver addon...

	I0305 03:58:50.834211 2132503 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=csi-hostpath-driver" in ns "kube-system" ...

	I0305 03:58:50.835719 2132503 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_application_credentials.json (162 bytes)

	I0305 03:58:50.835825 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:50.848185 2132503 kapi.go:86] Found 2 Pods for label selector kubernetes.io/minikube-addons=csi-hostpath-driver

	I0305 03:58:50.848207 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:50.858776 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:50.915875 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply --force -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml

	I0305 03:58:50.973106 2132503 ssh_runner.go:362] scp memory --> /var/lib/minikube/google_cloud_project (12 bytes)

	I0305 03:58:50.994160 2132503 addons.go:234] Setting addon gcp-auth=true in "addons-649326"

	I0305 03:58:50.994215 2132503 host.go:66] Checking if "addons-649326" exists ...

	I0305 03:58:50.994686 2132503 cli_runner.go:164] Run: docker container inspect addons-649326 --format={{.State.Status}}

	I0305 03:58:51.020268 2132503 ssh_runner.go:195] Run: cat /var/lib/minikube/google_application_credentials.json

	I0305 03:58:51.020331 2132503 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" addons-649326

	I0305 03:58:51.057257 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:51.060646 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:51.062508 2132503 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35049 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/addons-649326/id_rsa Username:docker}

	I0305 03:58:51.296816 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:58:51.346218 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:51.556345 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:51.578866 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:51.858077 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:52.055643 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:52.064432 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:52.339671 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:52.560307 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:52.567358 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:52.826191 2132503 ssh_runner.go:235] Completed: cat /var/lib/minikube/google_application_credentials.json: (1.805890456s)

	I0305 03:58:52.828841 2132503 out.go:177]   - Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0

	I0305 03:58:52.826384 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply --force -f /etc/kubernetes/addons/csi-hostpath-snapshotclass.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotclasses.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshotcontents.yaml -f /etc/kubernetes/addons/snapshot.storage.k8s.io_volumesnapshots.yaml -f /etc/kubernetes/addons/rbac-volume-snapshot-controller.yaml -f /etc/kubernetes/addons/volume-snapshot-controller-deployment.yaml: (1.910471148s)

	I0305 03:58:52.830905 2132503 out.go:177]   - Using image gcr.io/k8s-minikube/gcp-auth-webhook:v0.1.1

	I0305 03:58:52.833375 2132503 addons.go:426] installing /etc/kubernetes/addons/gcp-auth-ns.yaml

	I0305 03:58:52.833438 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-ns.yaml (700 bytes)

	I0305 03:58:52.838616 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:52.870810 2132503 addons.go:426] installing /etc/kubernetes/addons/gcp-auth-service.yaml

	I0305 03:58:52.870878 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-service.yaml (788 bytes)

	I0305 03:58:52.891117 2132503 addons.go:426] installing /etc/kubernetes/addons/gcp-auth-webhook.yaml

	I0305 03:58:52.891182 2132503 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/gcp-auth-webhook.yaml (5421 bytes)

	I0305 03:58:52.910848 2132503 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/gcp-auth-ns.yaml -f /etc/kubernetes/addons/gcp-auth-service.yaml -f /etc/kubernetes/addons/gcp-auth-webhook.yaml

	I0305 03:58:53.055052 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:53.061595 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:53.339167 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:53.555591 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:53.559320 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:53.758184 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:58:53.846034 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:54.044274 2132503 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/gcp-auth-ns.yaml -f /etc/kubernetes/addons/gcp-auth-service.yaml -f /etc/kubernetes/addons/gcp-auth-webhook.yaml: (1.131781619s)

	I0305 03:58:54.047933 2132503 addons.go:470] Verifying addon gcp-auth=true in "addons-649326"

	I0305 03:58:54.051476 2132503 out.go:177] * Verifying gcp-auth addon...

	I0305 03:58:54.054134 2132503 kapi.go:75] Waiting for pod with label "kubernetes.io/minikube-addons=gcp-auth" in ns "gcp-auth" ...

	I0305 03:58:54.073526 2132503 kapi.go:86] Found 0 Pods for label selector kubernetes.io/minikube-addons=gcp-auth

	I0305 03:58:54.074565 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:54.076173 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:54.338875 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:54.555475 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:54.564335 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:54.564681 2132503 kapi.go:86] Found 1 Pods for label selector kubernetes.io/minikube-addons=gcp-auth

	I0305 03:58:54.564714 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:58:54.838897 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:55.056638 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:55.059375 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:58:55.061299 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:55.339483 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:55.554858 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:55.558657 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:58:55.568589 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:55.758397 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:58:55.838888 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:56.055387 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:56.066459 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:58:56.069007 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:56.338352 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:56.559475 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:56.563338 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:56.563865 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:58:56.839164 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:57.055059 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:57.059548 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:58:57.060214 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:57.338253 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:57.560452 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:57.564126 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:57.566242 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:58:57.758500 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:58:57.840297 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:58.058576 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:58.062394 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:58.062668 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:58:58.339319 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:58.565497 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:58.569406 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:58.570329 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:58:58.839327 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:59.054976 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:59.058481 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:58:59.060242 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:59.338557 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:58:59.569760 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:58:59.575241 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:58:59.576298 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:58:59.758619 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:58:59.840094 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:00.056925 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:00.072571 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:00.076446 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:00.339942 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:00.557661 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:00.562589 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:00.563779 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:00.838893 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:01.055224 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:01.058806 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:01.059706 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:01.339383 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:01.556439 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:01.563789 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:01.564184 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:01.838615 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:02.055879 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:02.059726 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:02.060563 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:02.257960 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:59:02.338736 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:02.556956 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:02.558123 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:02.561558 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:02.839227 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:03.056790 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:03.061823 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:03.063366 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:03.338230 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:03.555893 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:03.558679 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:03.560865 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:03.838862 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:04.055460 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:04.057871 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:04.059745 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:04.258466 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:59:04.338691 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:04.554828 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:04.559278 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:04.560681 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:04.838712 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:05.055378 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:05.059693 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:05.060067 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:05.338700 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:05.556767 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:05.559142 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:05.560758 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:05.838403 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:06.055031 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:06.060317 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:06.060316 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:06.338687 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:06.555883 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:06.557911 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:06.560267 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:06.758824 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:59:06.840227 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:07.055827 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:07.058626 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:07.059758 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:07.339074 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:07.555213 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:07.559556 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:07.559838 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:07.839120 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:08.055663 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:08.060303 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:08.060694 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:08.338438 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:08.555283 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:08.560015 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:08.560918 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:08.839537 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:09.056175 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:09.060932 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:09.061864 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:09.258303 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:59:09.339206 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:09.555387 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:09.559408 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:09.559689 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:09.839506 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:10.055313 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:10.059210 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:10.059232 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:10.338521 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:10.555855 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:10.558788 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:10.560420 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:10.839190 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:11.055350 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:11.057694 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:11.060418 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:11.258747 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:59:11.338932 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:11.554695 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:11.559199 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:11.559552 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:11.838556 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:12.055524 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:12.060153 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:12.060153 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:12.339494 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:12.555396 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:12.558610 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:12.559090 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:12.840171 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:13.056839 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:13.060620 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:13.062036 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:13.258850 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:59:13.341842 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:13.555392 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:13.558070 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:13.559173 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:13.839523 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:14.055447 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:14.059170 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:14.059191 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:14.338676 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:14.555312 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:14.558359 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:14.559723 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:14.838663 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:15.055744 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:15.059156 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:15.059660 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:15.348458 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:15.555299 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:15.559425 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:15.565133 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:15.757504 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:59:15.838707 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:16.058232 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:16.059647 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:16.060503 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:16.338926 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:16.555889 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:16.559187 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:16.560979 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:16.838577 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:17.054877 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:17.058983 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:17.060300 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:17.338604 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:17.554899 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:17.557874 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:17.559639 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:17.757547 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:59:17.838898 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:18.054943 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:18.058626 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:18.060757 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:18.338935 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:18.555317 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:18.557422 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:18.559334 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:18.838565 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:19.055518 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:19.059347 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:19.059743 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:19.338965 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:19.555232 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:19.558961 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:19.560313 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:19.758351 2132503 node_ready.go:53] node "addons-649326" has status "Ready":"False"

	I0305 03:59:19.841181 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:20.062292 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:20.121405 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:20.124886 2132503 kapi.go:86] Found 2 Pods for label selector kubernetes.io/minikube-addons=registry

	I0305 03:59:20.124919 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:20.275079 2132503 node_ready.go:49] node "addons-649326" has status "Ready":"True"

	I0305 03:59:20.275105 2132503 node_ready.go:38] duration metric: took 33.520727005s for node "addons-649326" to be "Ready" ...

	I0305 03:59:20.275116 2132503 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I0305 03:59:20.458922 2132503 pod_ready.go:78] waiting up to 6m0s for pod "coredns-5dd5756b68-4mk85" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:20.460867 2132503 kapi.go:86] Found 3 Pods for label selector kubernetes.io/minikube-addons=csi-hostpath-driver

	I0305 03:59:20.460892 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:20.571354 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:20.574374 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:20.602857 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:20.875510 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:21.056367 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:21.060151 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:21.061793 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:21.340569 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:21.565052 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:21.579749 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:21.580749 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:21.886898 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:21.977181 2132503 pod_ready.go:92] pod "coredns-5dd5756b68-4mk85" in "kube-system" namespace has status "Ready":"True"

	I0305 03:59:21.977226 2132503 pod_ready.go:81] duration metric: took 1.518270118s for pod "coredns-5dd5756b68-4mk85" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:21.977291 2132503 pod_ready.go:78] waiting up to 6m0s for pod "etcd-addons-649326" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:22.003603 2132503 pod_ready.go:92] pod "etcd-addons-649326" in "kube-system" namespace has status "Ready":"True"

	I0305 03:59:22.003641 2132503 pod_ready.go:81] duration metric: took 26.319023ms for pod "etcd-addons-649326" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:22.003659 2132503 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-addons-649326" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:22.011769 2132503 pod_ready.go:92] pod "kube-apiserver-addons-649326" in "kube-system" namespace has status "Ready":"True"

	I0305 03:59:22.011803 2132503 pod_ready.go:81] duration metric: took 8.114126ms for pod "kube-apiserver-addons-649326" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:22.011817 2132503 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-addons-649326" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:22.018453 2132503 pod_ready.go:92] pod "kube-controller-manager-addons-649326" in "kube-system" namespace has status "Ready":"True"

	I0305 03:59:22.018479 2132503 pod_ready.go:81] duration metric: took 6.636795ms for pod "kube-controller-manager-addons-649326" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:22.018493 2132503 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-vsfnz" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:22.027155 2132503 pod_ready.go:92] pod "kube-proxy-vsfnz" in "kube-system" namespace has status "Ready":"True"

	I0305 03:59:22.027181 2132503 pod_ready.go:81] duration metric: took 8.679448ms for pod "kube-proxy-vsfnz" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:22.027193 2132503 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-addons-649326" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:22.066936 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:22.073561 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:22.075777 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:22.340740 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:22.363901 2132503 pod_ready.go:92] pod "kube-scheduler-addons-649326" in "kube-system" namespace has status "Ready":"True"

	I0305 03:59:22.364025 2132503 pod_ready.go:81] duration metric: took 336.791033ms for pod "kube-scheduler-addons-649326" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:22.364068 2132503 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-69cf46c98-7gfhq" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:22.557390 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:22.595050 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:22.596135 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:22.841264 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:23.054928 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:23.058845 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:23.060609 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:23.340315 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:23.556216 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:23.557737 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:23.561074 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:23.840379 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:24.058455 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:24.069594 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:24.069785 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:24.340670 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:24.372016 2132503 pod_ready.go:102] pod "metrics-server-69cf46c98-7gfhq" in "kube-system" namespace has status "Ready":"False"

	I0305 03:59:24.555606 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:24.567737 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:24.569613 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:24.841572 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:25.058189 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:25.071203 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:25.072135 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:25.340084 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:25.555456 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:25.558157 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:25.561246 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:25.840056 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:26.062032 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:26.069279 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:26.070341 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:26.340757 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:26.556668 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:26.567051 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:26.572958 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:26.840431 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:26.884954 2132503 pod_ready.go:102] pod "metrics-server-69cf46c98-7gfhq" in "kube-system" namespace has status "Ready":"False"

	I0305 03:59:27.055541 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:27.061446 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:27.061717 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:27.343340 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:27.555071 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:27.559241 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:27.561792 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:27.840508 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:28.054949 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:28.058662 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:28.071257 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:28.345816 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:28.566919 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:28.567814 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:28.568162 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:28.840699 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:29.065507 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:29.070951 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:29.072977 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:29.342291 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:29.370674 2132503 pod_ready.go:102] pod "metrics-server-69cf46c98-7gfhq" in "kube-system" namespace has status "Ready":"False"

	I0305 03:59:29.555055 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:29.559342 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:29.561578 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:29.842029 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:30.068996 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:30.069879 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:30.076335 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:30.341458 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:30.560235 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:30.586565 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:30.594103 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:30.841460 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:31.066931 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:31.068372 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:31.069864 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:31.341344 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:31.393217 2132503 pod_ready.go:102] pod "metrics-server-69cf46c98-7gfhq" in "kube-system" namespace has status "Ready":"False"

	I0305 03:59:31.558570 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:31.568514 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:31.575874 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:31.844025 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:32.058729 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:32.067043 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:32.068891 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:32.342224 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:32.566424 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:32.572002 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:32.572956 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:32.874639 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:33.088351 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:33.088808 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:33.113508 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:33.342248 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:33.556465 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:33.560758 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:33.563238 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:33.840340 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:33.871298 2132503 pod_ready.go:102] pod "metrics-server-69cf46c98-7gfhq" in "kube-system" namespace has status "Ready":"False"

	I0305 03:59:34.056007 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:34.058598 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:34.060765 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:34.341060 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:34.556169 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:34.562748 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:34.563387 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:34.840706 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:35.070730 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:35.071514 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:35.075229 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:35.342339 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:35.557069 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:35.562209 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:35.566550 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:35.843036 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:35.872241 2132503 pod_ready.go:102] pod "metrics-server-69cf46c98-7gfhq" in "kube-system" namespace has status "Ready":"False"

	I0305 03:59:36.057032 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:36.064333 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:36.066419 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:36.342683 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:36.556723 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:36.565698 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:36.566153 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:36.841670 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:37.063037 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:37.069247 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:37.073200 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:37.345972 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:37.555161 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:37.560349 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:37.560864 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:37.841006 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:38.060834 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:38.062133 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:38.063205 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:38.346003 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:38.383797 2132503 pod_ready.go:102] pod "metrics-server-69cf46c98-7gfhq" in "kube-system" namespace has status "Ready":"False"

	I0305 03:59:38.574240 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:38.575210 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:38.576446 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:38.848121 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:39.056921 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:39.071327 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:39.072118 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:39.361971 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:39.555936 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:39.561178 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:39.565821 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:39.840468 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:40.057829 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:40.063290 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:40.070844 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:40.344296 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:40.376162 2132503 pod_ready.go:92] pod "metrics-server-69cf46c98-7gfhq" in "kube-system" namespace has status "Ready":"True"

	I0305 03:59:40.376186 2132503 pod_ready.go:81] duration metric: took 18.012065487s for pod "metrics-server-69cf46c98-7gfhq" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:40.376199 2132503 pod_ready.go:78] waiting up to 6m0s for pod "nvidia-device-plugin-daemonset-mjt8p" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:40.556547 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:40.561928 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:40.566084 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:40.841193 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:41.056393 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:41.063530 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:41.066668 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:41.340680 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:41.555900 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:41.561530 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:41.561945 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:41.840404 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:42.071720 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:42.073452 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:42.074699 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:42.341860 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:42.384140 2132503 pod_ready.go:102] pod "nvidia-device-plugin-daemonset-mjt8p" in "kube-system" namespace has status "Ready":"False"

	I0305 03:59:42.556109 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:42.561482 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:42.562629 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:42.845491 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:43.055750 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:43.064865 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:43.065645 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:43.348999 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:43.561671 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:43.561856 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:43.563300 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:43.840572 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:44.056028 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:44.061625 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:44.062178 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:44.340061 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:44.555316 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:44.559050 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:44.560572 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:44.840485 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:44.883178 2132503 pod_ready.go:102] pod "nvidia-device-plugin-daemonset-mjt8p" in "kube-system" namespace has status "Ready":"False"

	I0305 03:59:45.056543 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:45.063002 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:45.063512 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:45.340214 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:45.555373 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:45.557645 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:45.560886 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:45.843967 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:46.056477 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:46.076337 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:46.082013 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:46.340199 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:46.561342 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:46.565995 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:46.573149 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:46.841018 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:46.888551 2132503 pod_ready.go:102] pod "nvidia-device-plugin-daemonset-mjt8p" in "kube-system" namespace has status "Ready":"False"

	I0305 03:59:47.086203 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:47.092998 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:47.108717 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:47.342666 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:47.556376 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:47.563105 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:47.564333 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:47.841221 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:48.061064 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:48.064976 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:48.071756 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:48.341239 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:48.566233 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:48.567220 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:48.568217 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:48.840868 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:49.056728 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:49.062418 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:49.067178 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:49.340443 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:49.382729 2132503 pod_ready.go:92] pod "nvidia-device-plugin-daemonset-mjt8p" in "kube-system" namespace has status "Ready":"True"

	I0305 03:59:49.382754 2132503 pod_ready.go:81] duration metric: took 9.006547057s for pod "nvidia-device-plugin-daemonset-mjt8p" in "kube-system" namespace to be "Ready" ...

	I0305 03:59:49.382775 2132503 pod_ready.go:38] duration metric: took 29.107638869s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...

	I0305 03:59:49.382792 2132503 api_server.go:52] waiting for apiserver process to appear ...

	I0305 03:59:49.382824 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}

	I0305 03:59:49.382889 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver

	I0305 03:59:49.421820 2132503 cri.go:89] found id: "c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e"

	I0305 03:59:49.421851 2132503 cri.go:89] found id: ""

	I0305 03:59:49.421861 2132503 logs.go:276] 1 containers: [c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e]

	I0305 03:59:49.421933 2132503 ssh_runner.go:195] Run: which crictl

	I0305 03:59:49.425509 2132503 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}

	I0305 03:59:49.425587 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd

	I0305 03:59:49.477020 2132503 cri.go:89] found id: "0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1"

	I0305 03:59:49.477081 2132503 cri.go:89] found id: ""

	I0305 03:59:49.477110 2132503 logs.go:276] 1 containers: [0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1]

	I0305 03:59:49.477194 2132503 ssh_runner.go:195] Run: which crictl

	I0305 03:59:49.481356 2132503 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}

	I0305 03:59:49.481491 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns

	I0305 03:59:49.526981 2132503 cri.go:89] found id: "77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630"

	I0305 03:59:49.527044 2132503 cri.go:89] found id: ""

	I0305 03:59:49.527074 2132503 logs.go:276] 1 containers: [77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630]

	I0305 03:59:49.527163 2132503 ssh_runner.go:195] Run: which crictl

	I0305 03:59:49.530715 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}

	I0305 03:59:49.530842 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler

	I0305 03:59:49.556685 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:49.562513 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:49.563195 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:49.576092 2132503 cri.go:89] found id: "d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39"

	I0305 03:59:49.576168 2132503 cri.go:89] found id: ""

	I0305 03:59:49.576190 2132503 logs.go:276] 1 containers: [d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39]

	I0305 03:59:49.576274 2132503 ssh_runner.go:195] Run: which crictl

	I0305 03:59:49.579776 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}

	I0305 03:59:49.579891 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy

	I0305 03:59:49.617276 2132503 cri.go:89] found id: "586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335"

	I0305 03:59:49.617297 2132503 cri.go:89] found id: ""

	I0305 03:59:49.617305 2132503 logs.go:276] 1 containers: [586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335]

	I0305 03:59:49.617358 2132503 ssh_runner.go:195] Run: which crictl

	I0305 03:59:49.620877 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}

	I0305 03:59:49.620953 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager

	I0305 03:59:49.661377 2132503 cri.go:89] found id: "0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52"

	I0305 03:59:49.661401 2132503 cri.go:89] found id: ""

	I0305 03:59:49.661408 2132503 logs.go:276] 1 containers: [0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52]

	I0305 03:59:49.661476 2132503 ssh_runner.go:195] Run: which crictl

	I0305 03:59:49.665389 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}

	I0305 03:59:49.665486 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet

	I0305 03:59:49.712756 2132503 cri.go:89] found id: "0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09"

	I0305 03:59:49.712779 2132503 cri.go:89] found id: ""

	I0305 03:59:49.712788 2132503 logs.go:276] 1 containers: [0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09]

	I0305 03:59:49.712858 2132503 ssh_runner.go:195] Run: which crictl

	I0305 03:59:49.716901 2132503 logs.go:123] Gathering logs for kube-controller-manager [0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52] ...

	I0305 03:59:49.716927 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52"

	I0305 03:59:49.811773 2132503 logs.go:123] Gathering logs for kindnet [0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09] ...

	I0305 03:59:49.811811 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09"

	I0305 03:59:49.841566 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:49.855239 2132503 logs.go:123] Gathering logs for kubelet ...

	I0305 03:59:49.855270 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"

	W0305 03:59:49.919903 2132503 logs.go:138] Found kubelet problem: Mar 05 03:59:19 addons-649326 kubelet[1508]: W0305 03:59:19.929363    1508 reflector.go:535] object-"ingress-nginx"/"ingress-nginx-admission": failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	W0305 03:59:49.920182 2132503 logs.go:138] Found kubelet problem: Mar 05 03:59:19 addons-649326 kubelet[1508]: E0305 03:59:19.929398    1508 reflector.go:147] object-"ingress-nginx"/"ingress-nginx-admission": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	I0305 03:59:49.949133 2132503 logs.go:123] Gathering logs for describe nodes ...

	I0305 03:59:49.949163 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"

	I0305 03:59:50.056566 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:50.062059 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:50.063816 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:50.142357 2132503 logs.go:123] Gathering logs for kube-apiserver [c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e] ...

	I0305 03:59:50.142392 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e"

	I0305 03:59:50.204452 2132503 logs.go:123] Gathering logs for etcd [0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1] ...

	I0305 03:59:50.204489 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1"

	I0305 03:59:50.274538 2132503 logs.go:123] Gathering logs for coredns [77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630] ...

	I0305 03:59:50.274572 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630"

	I0305 03:59:50.319434 2132503 logs.go:123] Gathering logs for kube-scheduler [d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39] ...

	I0305 03:59:50.319508 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39"

	I0305 03:59:50.342736 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:50.392192 2132503 logs.go:123] Gathering logs for container status ...

	I0305 03:59:50.392227 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"

	I0305 03:59:50.451143 2132503 logs.go:123] Gathering logs for dmesg ...

	I0305 03:59:50.451233 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"

	I0305 03:59:50.495720 2132503 logs.go:123] Gathering logs for kube-proxy [586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335] ...

	I0305 03:59:50.495814 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335"

	I0305 03:59:50.566441 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:50.567055 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:50.567632 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:50.589872 2132503 logs.go:123] Gathering logs for CRI-O ...

	I0305 03:59:50.589901 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"

	I0305 03:59:50.751695 2132503 out.go:304] Setting ErrFile to fd 2...

	I0305 03:59:50.751749 2132503 out.go:338] TERM=,COLORTERM=, which probably does not support color

	W0305 03:59:50.751825 2132503 out.go:239] X Problems detected in kubelet:

	W0305 03:59:50.751847 2132503 out.go:239]   Mar 05 03:59:19 addons-649326 kubelet[1508]: W0305 03:59:19.929363    1508 reflector.go:535] object-"ingress-nginx"/"ingress-nginx-admission": failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	W0305 03:59:50.751859 2132503 out.go:239]   Mar 05 03:59:19 addons-649326 kubelet[1508]: E0305 03:59:19.929398    1508 reflector.go:147] object-"ingress-nginx"/"ingress-nginx-admission": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	I0305 03:59:50.751874 2132503 out.go:304] Setting ErrFile to fd 2...

	I0305 03:59:50.751886 2132503 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 03:59:50.840586 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:51.074567 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:51.075540 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:51.076247 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:51.340505 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:51.557034 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:51.564424 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:51.565163 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:51.840213 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:52.055447 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:52.059401 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:52.061533 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:52.342108 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:52.559661 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:52.569325 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:52.570109 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:52.850604 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:53.060019 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:53.060468 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:53.062742 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:53.349148 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:53.561035 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:53.567024 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:53.568231 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:53.839834 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:54.060661 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:54.062570 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:54.064640 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:54.343575 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:54.557205 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:54.562821 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:54.564929 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:54.841319 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:55.056307 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:55.062486 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:55.063177 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:55.348491 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:55.555296 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:55.558900 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:55.561335 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:55.840351 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:56.055817 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:56.059900 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:56.060858 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:56.348571 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:56.556071 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:56.563021 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:56.563987 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:56.847234 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:57.059000 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:57.076602 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:57.079185 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:57.340231 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:57.555912 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:57.559381 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:57.560893 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:57.851357 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:58.055276 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:58.058697 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:58.061046 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:58.346137 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:58.556181 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:58.570718 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:58.573208 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:58.841808 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:59.062153 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:59.074199 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:59.074797 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:59.343724 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 03:59:59.555814 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 03:59:59.562744 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 03:59:59.564978 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 03:59:59.840321 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:00.074550 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:00.091426 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:00.092479 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 04:00:00.389063 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:00.570291 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 04:00:00.601933 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:00.602366 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:00.763844 2132503 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I0305 04:00:00.811788 2132503 api_server.go:72] duration metric: took 1m17.746003644s to wait for apiserver process to appear ...

	I0305 04:00:00.811876 2132503 api_server.go:88] waiting for apiserver healthz status ...

	I0305 04:00:00.811969 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}

	I0305 04:00:00.812130 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver

	I0305 04:00:00.858725 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:00.945276 2132503 cri.go:89] found id: "c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e"

	I0305 04:00:00.945353 2132503 cri.go:89] found id: ""

	I0305 04:00:00.945376 2132503 logs.go:276] 1 containers: [c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e]

	I0305 04:00:00.945471 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:00.953164 2132503 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}

	I0305 04:00:00.953320 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd

	I0305 04:00:01.025902 2132503 cri.go:89] found id: "0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1"

	I0305 04:00:01.025982 2132503 cri.go:89] found id: ""

	I0305 04:00:01.026006 2132503 logs.go:276] 1 containers: [0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1]

	I0305 04:00:01.026109 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:01.030462 2132503 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}

	I0305 04:00:01.030592 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns

	I0305 04:00:01.067115 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:01.103496 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:01.112461 2132503 cri.go:89] found id: "77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630"

	I0305 04:00:01.112485 2132503 cri.go:89] found id: ""

	I0305 04:00:01.112493 2132503 logs.go:276] 1 containers: [77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630]

	I0305 04:00:01.112566 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:01.114501 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 04:00:01.119339 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}

	I0305 04:00:01.119418 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler

	I0305 04:00:01.247233 2132503 cri.go:89] found id: "d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39"

	I0305 04:00:01.247266 2132503 cri.go:89] found id: ""

	I0305 04:00:01.247275 2132503 logs.go:276] 1 containers: [d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39]

	I0305 04:00:01.247335 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:01.259677 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}

	I0305 04:00:01.259757 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy

	I0305 04:00:01.340767 2132503 cri.go:89] found id: "586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335"

	I0305 04:00:01.340791 2132503 cri.go:89] found id: ""

	I0305 04:00:01.340799 2132503 logs.go:276] 1 containers: [586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335]

	I0305 04:00:01.340860 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:01.344363 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:01.348710 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}

	I0305 04:00:01.348793 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager

	I0305 04:00:01.424136 2132503 cri.go:89] found id: "0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52"

	I0305 04:00:01.424162 2132503 cri.go:89] found id: ""

	I0305 04:00:01.424171 2132503 logs.go:276] 1 containers: [0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52]

	I0305 04:00:01.424231 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:01.441642 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}

	I0305 04:00:01.441732 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet

	I0305 04:00:01.526450 2132503 cri.go:89] found id: "0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09"

	I0305 04:00:01.526532 2132503 cri.go:89] found id: ""

	I0305 04:00:01.526562 2132503 logs.go:276] 1 containers: [0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09]

	I0305 04:00:01.526719 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:01.532582 2132503 logs.go:123] Gathering logs for kubelet ...

	I0305 04:00:01.532607 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"

	I0305 04:00:01.557803 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:01.571835 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 04:00:01.573801 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	W0305 04:00:01.626662 2132503 logs.go:138] Found kubelet problem: Mar 05 03:59:19 addons-649326 kubelet[1508]: W0305 03:59:19.929363    1508 reflector.go:535] object-"ingress-nginx"/"ingress-nginx-admission": failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	W0305 04:00:01.626895 2132503 logs.go:138] Found kubelet problem: Mar 05 03:59:19 addons-649326 kubelet[1508]: E0305 03:59:19.929398    1508 reflector.go:147] object-"ingress-nginx"/"ingress-nginx-admission": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	I0305 04:00:01.662325 2132503 logs.go:123] Gathering logs for dmesg ...

	I0305 04:00:01.662374 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"

	I0305 04:00:01.683768 2132503 logs.go:123] Gathering logs for kube-apiserver [c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e] ...

	I0305 04:00:01.683804 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e"

	I0305 04:00:01.779172 2132503 logs.go:123] Gathering logs for etcd [0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1] ...

	I0305 04:00:01.779216 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1"

	I0305 04:00:01.843392 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:01.867740 2132503 logs.go:123] Gathering logs for kube-scheduler [d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39] ...

	I0305 04:00:01.867831 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39"

	I0305 04:00:02.034558 2132503 logs.go:123] Gathering logs for kube-proxy [586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335] ...

	I0305 04:00:02.034653 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335"

	I0305 04:00:02.056232 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:02.069286 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:02.070735 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 04:00:02.097682 2132503 logs.go:123] Gathering logs for kube-controller-manager [0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52] ...

	I0305 04:00:02.097965 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52"

	I0305 04:00:02.237984 2132503 logs.go:123] Gathering logs for CRI-O ...

	I0305 04:00:02.238069 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"

	I0305 04:00:02.339418 2132503 logs.go:123] Gathering logs for container status ...

	I0305 04:00:02.339488 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"

	I0305 04:00:02.343522 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:02.450807 2132503 logs.go:123] Gathering logs for describe nodes ...

	I0305 04:00:02.450894 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"

	I0305 04:00:02.556235 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:02.571146 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:02.571699 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 04:00:02.669990 2132503 logs.go:123] Gathering logs for coredns [77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630] ...

	I0305 04:00:02.670074 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630"

	I0305 04:00:02.727477 2132503 logs.go:123] Gathering logs for kindnet [0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09] ...

	I0305 04:00:02.727503 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09"

	I0305 04:00:02.797792 2132503 out.go:304] Setting ErrFile to fd 2...

	I0305 04:00:02.797863 2132503 out.go:338] TERM=,COLORTERM=, which probably does not support color

	W0305 04:00:02.797940 2132503 out.go:239] X Problems detected in kubelet:

	W0305 04:00:02.797989 2132503 out.go:239]   Mar 05 03:59:19 addons-649326 kubelet[1508]: W0305 03:59:19.929363    1508 reflector.go:535] object-"ingress-nginx"/"ingress-nginx-admission": failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	W0305 04:00:02.798024 2132503 out.go:239]   Mar 05 03:59:19 addons-649326 kubelet[1508]: E0305 03:59:19.929398    1508 reflector.go:147] object-"ingress-nginx"/"ingress-nginx-admission": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	I0305 04:00:02.798072 2132503 out.go:304] Setting ErrFile to fd 2...

	I0305 04:00:02.798091 2132503 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 04:00:02.840754 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:03.056563 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:03.072227 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:03.074536 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=registry", current state: Pending: [<nil>]

	I0305 04:00:03.341926 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:03.555721 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:03.558401 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:03.560530 2132503 kapi.go:107] duration metric: took 1m13.005227249s to wait for kubernetes.io/minikube-addons=registry ...

	I0305 04:00:03.841105 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:04.056081 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:04.058187 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:04.341077 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:04.555811 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:04.559429 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:04.846150 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:05.065919 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:05.077794 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:05.355475 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:05.557444 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:05.560889 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:05.844472 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:06.056735 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:06.063690 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:06.341281 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:06.557565 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:06.564758 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:06.847461 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:07.055572 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:07.059028 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:07.343295 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:07.558698 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:07.560417 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:07.840595 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:08.056313 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:08.057632 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:08.344192 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:08.556622 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:08.560428 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:08.841911 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:09.056133 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:09.059942 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:09.341423 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:09.557181 2132503 kapi.go:96] waiting for pod "app.kubernetes.io/name=ingress-nginx", current state: Pending: [<nil>]

	I0305 04:00:09.564144 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:09.840809 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:10.055847 2132503 kapi.go:107] duration metric: took 1m19.50699193s to wait for app.kubernetes.io/name=ingress-nginx ...

	I0305 04:00:10.062139 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:10.340744 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:10.567639 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:10.840904 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:11.058374 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:11.340692 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:11.558214 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:11.840063 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:12.058326 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:12.341517 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:12.558162 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:12.799532 2132503 api_server.go:253] Checking apiserver healthz at https://192.168.49.2:8443/healthz ...

	I0305 04:00:12.809614 2132503 api_server.go:279] https://192.168.49.2:8443/healthz returned 200:

	ok

	I0305 04:00:12.810952 2132503 api_server.go:141] control plane version: v1.28.4

	I0305 04:00:12.810981 2132503 api_server.go:131] duration metric: took 11.999081947s to wait for apiserver health ...

	I0305 04:00:12.810989 2132503 system_pods.go:43] waiting for kube-system pods to appear ...

	I0305 04:00:12.811010 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}

	I0305 04:00:12.811072 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver

	I0305 04:00:12.840902 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:12.857922 2132503 cri.go:89] found id: "c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e"

	I0305 04:00:12.857958 2132503 cri.go:89] found id: ""

	I0305 04:00:12.857966 2132503 logs.go:276] 1 containers: [c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e]

	I0305 04:00:12.858046 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:12.862225 2132503 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}

	I0305 04:00:12.862310 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd

	I0305 04:00:12.928051 2132503 cri.go:89] found id: "0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1"

	I0305 04:00:12.928072 2132503 cri.go:89] found id: ""

	I0305 04:00:12.928079 2132503 logs.go:276] 1 containers: [0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1]

	I0305 04:00:12.928135 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:12.935474 2132503 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}

	I0305 04:00:12.936157 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns

	I0305 04:00:13.024250 2132503 cri.go:89] found id: "77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630"

	I0305 04:00:13.024313 2132503 cri.go:89] found id: ""

	I0305 04:00:13.024335 2132503 logs.go:276] 1 containers: [77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630]

	I0305 04:00:13.024421 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:13.030397 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}

	I0305 04:00:13.030515 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler

	I0305 04:00:13.058535 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:13.081758 2132503 cri.go:89] found id: "d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39"

	I0305 04:00:13.081781 2132503 cri.go:89] found id: ""

	I0305 04:00:13.081788 2132503 logs.go:276] 1 containers: [d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39]

	I0305 04:00:13.081863 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:13.085501 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}

	I0305 04:00:13.085584 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy

	I0305 04:00:13.136531 2132503 cri.go:89] found id: "586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335"

	I0305 04:00:13.136561 2132503 cri.go:89] found id: ""

	I0305 04:00:13.136569 2132503 logs.go:276] 1 containers: [586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335]

	I0305 04:00:13.136636 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:13.140291 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}

	I0305 04:00:13.140371 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager

	I0305 04:00:13.202379 2132503 cri.go:89] found id: "0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52"

	I0305 04:00:13.202400 2132503 cri.go:89] found id: ""

	I0305 04:00:13.202408 2132503 logs.go:276] 1 containers: [0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52]

	I0305 04:00:13.202474 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:13.206064 2132503 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}

	I0305 04:00:13.206148 2132503 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet

	I0305 04:00:13.260006 2132503 cri.go:89] found id: "0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09"

	I0305 04:00:13.260035 2132503 cri.go:89] found id: ""

	I0305 04:00:13.260044 2132503 logs.go:276] 1 containers: [0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09]

	I0305 04:00:13.260111 2132503 ssh_runner.go:195] Run: which crictl

	I0305 04:00:13.263787 2132503 logs.go:123] Gathering logs for etcd [0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1] ...

	I0305 04:00:13.263857 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1"

	I0305 04:00:13.320486 2132503 logs.go:123] Gathering logs for coredns [77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630] ...

	I0305 04:00:13.320569 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630"

	I0305 04:00:13.344639 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:13.376166 2132503 logs.go:123] Gathering logs for kube-scheduler [d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39] ...

	I0305 04:00:13.376301 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39"

	I0305 04:00:13.448885 2132503 logs.go:123] Gathering logs for CRI-O ...

	I0305 04:00:13.448927 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"

	I0305 04:00:13.563536 2132503 logs.go:123] Gathering logs for kubelet ...

	I0305 04:00:13.563608 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"

	I0305 04:00:13.564535 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	W0305 04:00:13.635735 2132503 logs.go:138] Found kubelet problem: Mar 05 03:59:19 addons-649326 kubelet[1508]: W0305 03:59:19.929363    1508 reflector.go:535] object-"ingress-nginx"/"ingress-nginx-admission": failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	W0305 04:00:13.636051 2132503 logs.go:138] Found kubelet problem: Mar 05 03:59:19 addons-649326 kubelet[1508]: E0305 03:59:19.929398    1508 reflector.go:147] object-"ingress-nginx"/"ingress-nginx-admission": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	I0305 04:00:13.675020 2132503 logs.go:123] Gathering logs for dmesg ...

	I0305 04:00:13.675059 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"

	I0305 04:00:13.721587 2132503 logs.go:123] Gathering logs for kube-proxy [586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335] ...

	I0305 04:00:13.721812 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335"

	I0305 04:00:13.766412 2132503 logs.go:123] Gathering logs for kube-controller-manager [0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52] ...

	I0305 04:00:13.766486 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52"

	I0305 04:00:13.840930 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:13.868562 2132503 logs.go:123] Gathering logs for kindnet [0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09] ...

	I0305 04:00:13.868600 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09"

	I0305 04:00:13.908504 2132503 logs.go:123] Gathering logs for container status ...

	I0305 04:00:13.908534 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"

	I0305 04:00:13.974913 2132503 logs.go:123] Gathering logs for describe nodes ...

	I0305 04:00:13.974948 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"

	I0305 04:00:14.078231 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:14.246284 2132503 logs.go:123] Gathering logs for kube-apiserver [c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e] ...

	I0305 04:00:14.246322 2132503 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e"

	I0305 04:00:14.344731 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:14.353039 2132503 out.go:304] Setting ErrFile to fd 2...

	I0305 04:00:14.353073 2132503 out.go:338] TERM=,COLORTERM=, which probably does not support color

	W0305 04:00:14.355734 2132503 out.go:239] X Problems detected in kubelet:

	W0305 04:00:14.355752 2132503 out.go:239]   Mar 05 03:59:19 addons-649326 kubelet[1508]: W0305 03:59:19.929363    1508 reflector.go:535] object-"ingress-nginx"/"ingress-nginx-admission": failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	W0305 04:00:14.355760 2132503 out.go:239]   Mar 05 03:59:19 addons-649326 kubelet[1508]: E0305 03:59:19.929398    1508 reflector.go:147] object-"ingress-nginx"/"ingress-nginx-admission": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "ingress-nginx-admission" is forbidden: User "system:node:addons-649326" cannot list resource "secrets" in API group "" in the namespace "ingress-nginx": no relationship found between node 'addons-649326' and this object

	I0305 04:00:14.355769 2132503 out.go:304] Setting ErrFile to fd 2...

	I0305 04:00:14.355775 2132503 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 04:00:14.559203 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:14.840315 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:15.059802 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:15.342107 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:15.558093 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:15.840360 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:16.057899 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:16.340534 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:16.558715 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:16.842842 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:17.058385 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:17.346792 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:17.559225 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:17.840146 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:18.058863 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:18.341845 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:18.558200 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:18.841452 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:19.057864 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:19.340557 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:19.558123 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:19.841807 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:20.059051 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:20.342059 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:20.558081 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:20.839733 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:21.058903 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:21.339982 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:21.557498 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:21.866026 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:22.058397 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:22.340851 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:22.559119 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:22.839715 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=csi-hostpath-driver", current state: Pending: [<nil>]

	I0305 04:00:23.058775 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:23.346179 2132503 kapi.go:107] duration metric: took 1m32.511962521s to wait for kubernetes.io/minikube-addons=csi-hostpath-driver ...

	I0305 04:00:23.558998 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:24.058548 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:24.366874 2132503 system_pods.go:59] 18 kube-system pods found

	I0305 04:00:24.366913 2132503 system_pods.go:61] "coredns-5dd5756b68-4mk85" [fe940587-6447-475a-8859-423ef8be6332] Running

	I0305 04:00:24.366919 2132503 system_pods.go:61] "csi-hostpath-attacher-0" [ac68080b-b1b1-434f-862e-1a7698b7e4e8] Running

	I0305 04:00:24.366923 2132503 system_pods.go:61] "csi-hostpath-resizer-0" [87e0aa8f-d085-4767-b404-82b5dd63fbbd] Running

	I0305 04:00:24.366952 2132503 system_pods.go:61] "csi-hostpathplugin-dv2fl" [ac204edf-bce3-48c4-9211-797423fbe19f] Running

	I0305 04:00:24.366965 2132503 system_pods.go:61] "etcd-addons-649326" [4fdcea63-1507-4d9c-a873-e88221c85ed2] Running

	I0305 04:00:24.366970 2132503 system_pods.go:61] "kindnet-46wbp" [aab12478-4677-4cba-8191-afc2e0eccb62] Running

	I0305 04:00:24.366974 2132503 system_pods.go:61] "kube-apiserver-addons-649326" [b9a5dbef-3b25-49ed-bee6-8536a8e6aed2] Running

	I0305 04:00:24.366978 2132503 system_pods.go:61] "kube-controller-manager-addons-649326" [4fc7fcdb-dee0-4b7a-a7cd-04efd783d94c] Running

	I0305 04:00:24.366990 2132503 system_pods.go:61] "kube-ingress-dns-minikube" [bddb3a77-ef22-42d1-b1e9-8f173abd14b0] Running / Ready:ContainersNotReady (containers with unready status: [minikube-ingress-dns]) / ContainersReady:ContainersNotReady (containers with unready status: [minikube-ingress-dns])

	I0305 04:00:24.366999 2132503 system_pods.go:61] "kube-proxy-vsfnz" [fb1c1320-7925-4643-9153-f541bcac3d74] Running

	I0305 04:00:24.367008 2132503 system_pods.go:61] "kube-scheduler-addons-649326" [9c574ef4-61fd-486e-8f82-9a20d47fb631] Running

	I0305 04:00:24.367019 2132503 system_pods.go:61] "metrics-server-69cf46c98-7gfhq" [4ecac735-3006-4634-b22e-5bbbd02c53a3] Running

	I0305 04:00:24.367025 2132503 system_pods.go:61] "nvidia-device-plugin-daemonset-mjt8p" [9e26fee0-c9a4-43c8-b502-59bd1eb43478] Running

	I0305 04:00:24.367028 2132503 system_pods.go:61] "registry-g7rd8" [939c48ab-c821-49f0-ba12-25e9e9903c0c] Running

	I0305 04:00:24.367032 2132503 system_pods.go:61] "registry-proxy-8xs2x" [5cc5f670-a6f3-41a6-bfd3-90a772af9ca1] Running

	I0305 04:00:24.367035 2132503 system_pods.go:61] "snapshot-controller-58dbcc7b99-cqc4b" [6fa45b38-43e4-45d6-8730-a4bb3f3d9a6b] Running

	I0305 04:00:24.367040 2132503 system_pods.go:61] "snapshot-controller-58dbcc7b99-pr6z7" [fe2d28df-ab70-4672-a424-f3bab8a22df4] Running

	I0305 04:00:24.367043 2132503 system_pods.go:61] "storage-provisioner" [6de1f33a-4e22-4673-b7c8-8452f98e7f75] Running

	I0305 04:00:24.367050 2132503 system_pods.go:74] duration metric: took 11.556054206s to wait for pod list to return data ...

	I0305 04:00:24.367066 2132503 default_sa.go:34] waiting for default service account to be created ...

	I0305 04:00:24.369693 2132503 default_sa.go:45] found service account: "default"

	I0305 04:00:24.369720 2132503 default_sa.go:55] duration metric: took 2.647662ms for default service account to be created ...

	I0305 04:00:24.369729 2132503 system_pods.go:116] waiting for k8s-apps to be running ...

	I0305 04:00:24.379116 2132503 system_pods.go:86] 18 kube-system pods found

	I0305 04:00:24.379152 2132503 system_pods.go:89] "coredns-5dd5756b68-4mk85" [fe940587-6447-475a-8859-423ef8be6332] Running

	I0305 04:00:24.379159 2132503 system_pods.go:89] "csi-hostpath-attacher-0" [ac68080b-b1b1-434f-862e-1a7698b7e4e8] Running

	I0305 04:00:24.379163 2132503 system_pods.go:89] "csi-hostpath-resizer-0" [87e0aa8f-d085-4767-b404-82b5dd63fbbd] Running

	I0305 04:00:24.379167 2132503 system_pods.go:89] "csi-hostpathplugin-dv2fl" [ac204edf-bce3-48c4-9211-797423fbe19f] Running

	I0305 04:00:24.379171 2132503 system_pods.go:89] "etcd-addons-649326" [4fdcea63-1507-4d9c-a873-e88221c85ed2] Running

	I0305 04:00:24.379175 2132503 system_pods.go:89] "kindnet-46wbp" [aab12478-4677-4cba-8191-afc2e0eccb62] Running

	I0305 04:00:24.379179 2132503 system_pods.go:89] "kube-apiserver-addons-649326" [b9a5dbef-3b25-49ed-bee6-8536a8e6aed2] Running

	I0305 04:00:24.379184 2132503 system_pods.go:89] "kube-controller-manager-addons-649326" [4fc7fcdb-dee0-4b7a-a7cd-04efd783d94c] Running

	I0305 04:00:24.379191 2132503 system_pods.go:89] "kube-ingress-dns-minikube" [bddb3a77-ef22-42d1-b1e9-8f173abd14b0] Running / Ready:ContainersNotReady (containers with unready status: [minikube-ingress-dns]) / ContainersReady:ContainersNotReady (containers with unready status: [minikube-ingress-dns])

	I0305 04:00:24.379202 2132503 system_pods.go:89] "kube-proxy-vsfnz" [fb1c1320-7925-4643-9153-f541bcac3d74] Running

	I0305 04:00:24.379208 2132503 system_pods.go:89] "kube-scheduler-addons-649326" [9c574ef4-61fd-486e-8f82-9a20d47fb631] Running

	I0305 04:00:24.379212 2132503 system_pods.go:89] "metrics-server-69cf46c98-7gfhq" [4ecac735-3006-4634-b22e-5bbbd02c53a3] Running

	I0305 04:00:24.379220 2132503 system_pods.go:89] "nvidia-device-plugin-daemonset-mjt8p" [9e26fee0-c9a4-43c8-b502-59bd1eb43478] Running

	I0305 04:00:24.379225 2132503 system_pods.go:89] "registry-g7rd8" [939c48ab-c821-49f0-ba12-25e9e9903c0c] Running

	I0305 04:00:24.379228 2132503 system_pods.go:89] "registry-proxy-8xs2x" [5cc5f670-a6f3-41a6-bfd3-90a772af9ca1] Running

	I0305 04:00:24.379235 2132503 system_pods.go:89] "snapshot-controller-58dbcc7b99-cqc4b" [6fa45b38-43e4-45d6-8730-a4bb3f3d9a6b] Running

	I0305 04:00:24.379239 2132503 system_pods.go:89] "snapshot-controller-58dbcc7b99-pr6z7" [fe2d28df-ab70-4672-a424-f3bab8a22df4] Running

	I0305 04:00:24.379248 2132503 system_pods.go:89] "storage-provisioner" [6de1f33a-4e22-4673-b7c8-8452f98e7f75] Running

	I0305 04:00:24.379255 2132503 system_pods.go:126] duration metric: took 9.520567ms to wait for k8s-apps to be running ...

	I0305 04:00:24.379266 2132503 system_svc.go:44] waiting for kubelet service to be running ....

	I0305 04:00:24.379327 2132503 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet

	I0305 04:00:24.391080 2132503 system_svc.go:56] duration metric: took 11.804998ms WaitForService to wait for kubelet

	I0305 04:00:24.391110 2132503 kubeadm.go:576] duration metric: took 1m41.325331393s to wait for: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]

	I0305 04:00:24.391130 2132503 node_conditions.go:102] verifying NodePressure condition ...

	I0305 04:00:24.394440 2132503 node_conditions.go:122] node storage ephemeral capacity is 203034800Ki

	I0305 04:00:24.394473 2132503 node_conditions.go:123] node cpu capacity is 2

	I0305 04:00:24.394486 2132503 node_conditions.go:105] duration metric: took 3.35096ms to run NodePressure ...

	I0305 04:00:24.394530 2132503 start.go:240] waiting for startup goroutines ...

	I0305 04:00:24.558312 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:25.058241 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:25.562578 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:26.058533 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:26.558079 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:27.058568 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:27.558268 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:28.058203 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:28.559048 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:29.057662 2132503 kapi.go:96] waiting for pod "kubernetes.io/minikube-addons=gcp-auth", current state: Pending: [<nil>]

	I0305 04:00:29.557656 2132503 kapi.go:107] duration metric: took 1m35.503520834s to wait for kubernetes.io/minikube-addons=gcp-auth ...

	I0305 04:00:29.559436 2132503 out.go:177] * Your GCP credentials will now be mounted into every pod created in the addons-649326 cluster.

	I0305 04:00:29.561392 2132503 out.go:177] * If you don't want your credentials mounted into a specific pod, add a label with the `gcp-auth-skip-secret` key to your pod configuration.

	I0305 04:00:29.563362 2132503 out.go:177] * If you want existing pods to be mounted with credentials, either recreate them or rerun addons enable with --refresh.

	I0305 04:00:29.565197 2132503 out.go:177] * Enabled addons: ingress-dns, cloud-spanner, storage-provisioner, nvidia-device-plugin, metrics-server, inspektor-gadget, yakd, default-storageclass, volumesnapshots, registry, ingress, csi-hostpath-driver, gcp-auth

	I0305 04:00:29.566974 2132503 addons.go:505] duration metric: took 1m46.500866534s for enable addons: enabled=[ingress-dns cloud-spanner storage-provisioner nvidia-device-plugin metrics-server inspektor-gadget yakd default-storageclass volumesnapshots registry ingress csi-hostpath-driver gcp-auth]

	I0305 04:00:29.567012 2132503 start.go:245] waiting for cluster config update ...

	I0305 04:00:29.567030 2132503 start.go:254] writing updated cluster config ...

	I0305 04:00:29.567317 2132503 ssh_runner.go:195] Run: rm -f paused

	I0305 04:00:29.881767 2132503 start.go:600] kubectl: 1.29.2, cluster: 1.28.4 (minor skew: 1)

	I0305 04:00:29.884276 2132503 out.go:177] * Done! kubectl is now configured to use "addons-649326" cluster and "default" namespace by default

	==> CRI-O <==

	Mar 05 04:04:14 addons-649326 crio[904]: time="2024-03-05 04:04:14.249493810Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:dd1b12fcb60978ac32686ef6732d56f612c8636ef86693c09613946a54c69d79,RepoTags:[gcr.io/google-samples/hello-app:1.0],RepoDigests:[gcr.io/google-samples/hello-app@sha256:b1455e1c4fcc5ea1023c9e3b584cd84b64eb920e332feff690a2829696e379e7],Size_:28999827,Uid:nil,Username:nonroot,Spec:nil,},Info:map[string]string{},}" id=efcadbea-5bb3-4c6e-8198-2e445ae451c7 name=/runtime.v1.ImageService/ImageStatus

	Mar 05 04:04:14 addons-649326 crio[904]: time="2024-03-05 04:04:14.250455404Z" level=info msg="Checking image status: gcr.io/google-samples/hello-app:1.0" id=45284191-5623-447c-abb3-41d71cd5ebd3 name=/runtime.v1.ImageService/ImageStatus

	Mar 05 04:04:14 addons-649326 crio[904]: time="2024-03-05 04:04:14.250632141Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:dd1b12fcb60978ac32686ef6732d56f612c8636ef86693c09613946a54c69d79,RepoTags:[gcr.io/google-samples/hello-app:1.0],RepoDigests:[gcr.io/google-samples/hello-app@sha256:b1455e1c4fcc5ea1023c9e3b584cd84b64eb920e332feff690a2829696e379e7],Size_:28999827,Uid:nil,Username:nonroot,Spec:nil,},Info:map[string]string{},}" id=45284191-5623-447c-abb3-41d71cd5ebd3 name=/runtime.v1.ImageService/ImageStatus

	Mar 05 04:04:14 addons-649326 crio[904]: time="2024-03-05 04:04:14.251396025Z" level=info msg="Creating container: default/hello-world-app-5d77478584-j66vr/hello-world-app" id=3661f3dc-3924-4360-86d9-2b33ce0ee9f2 name=/runtime.v1.RuntimeService/CreateContainer

	Mar 05 04:04:14 addons-649326 crio[904]: time="2024-03-05 04:04:14.251495963Z" level=warning msg="Allowed annotations are specified for workload []"

	Mar 05 04:04:14 addons-649326 crio[904]: time="2024-03-05 04:04:14.320864269Z" level=info msg="Created container d556da5998d0e7e2f4dc4ccd3eb21e61cc7ed6701971fa80fd8860d65a93a346: default/hello-world-app-5d77478584-j66vr/hello-world-app" id=3661f3dc-3924-4360-86d9-2b33ce0ee9f2 name=/runtime.v1.RuntimeService/CreateContainer

	Mar 05 04:04:14 addons-649326 crio[904]: time="2024-03-05 04:04:14.321408791Z" level=info msg="Starting container: d556da5998d0e7e2f4dc4ccd3eb21e61cc7ed6701971fa80fd8860d65a93a346" id=51e79844-d30c-4cdb-bf62-d867475ed244 name=/runtime.v1.RuntimeService/StartContainer

	Mar 05 04:04:14 addons-649326 crio[904]: time="2024-03-05 04:04:14.328419509Z" level=info msg="Started container" PID=8507 containerID=d556da5998d0e7e2f4dc4ccd3eb21e61cc7ed6701971fa80fd8860d65a93a346 description=default/hello-world-app-5d77478584-j66vr/hello-world-app id=51e79844-d30c-4cdb-bf62-d867475ed244 name=/runtime.v1.RuntimeService/StartContainer sandboxID=5f20c4c43a5472adea901cfb85daa03e71bda3c66b82b183a141894c27d6f39e

	Mar 05 04:04:14 addons-649326 conmon[8495]: conmon d556da5998d0e7e2f4dc <ninfo>: container 8507 exited with status 1

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.038270229Z" level=info msg="Removing container: 36db5db6de4b1190203741d4035c43db3172e92a72dfd1ecd4c97b160a766d12" id=8f2dcf03-14eb-46b3-89c7-6046c5afc9d8 name=/runtime.v1.RuntimeService/RemoveContainer

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.059207041Z" level=info msg="Removed container 36db5db6de4b1190203741d4035c43db3172e92a72dfd1ecd4c97b160a766d12: default/hello-world-app-5d77478584-j66vr/hello-world-app" id=8f2dcf03-14eb-46b3-89c7-6046c5afc9d8 name=/runtime.v1.RuntimeService/RemoveContainer

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.754645779Z" level=warning msg="Stopping container 8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba with stop signal timed out: timeout reached after 2 seconds waiting for container process to exit" id=d83d7f6f-6e5d-43e1-91f3-568a783d5a03 name=/runtime.v1.RuntimeService/StopContainer

	Mar 05 04:04:15 addons-649326 conmon[5010]: conmon 8f48cb2eed45c233a552 <ninfo>: container 5022 exited with status 137

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.893986229Z" level=info msg="Stopped container 8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba: ingress-nginx/ingress-nginx-controller-76dc478dd8-qnjmj/controller" id=d83d7f6f-6e5d-43e1-91f3-568a783d5a03 name=/runtime.v1.RuntimeService/StopContainer

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.894551131Z" level=info msg="Stopping pod sandbox: 54df043ddbaab626dc2e9315efdf2912eacebb44bd5d5529533984e6880ad206" id=9c369fa5-7c09-4b2e-86ee-e230b9478471 name=/runtime.v1.RuntimeService/StopPodSandbox

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.897807060Z" level=info msg="Restoring iptables rules: *nat\n:KUBE-HOSTPORTS - [0:0]\n:KUBE-HP-GK54FMWVHAC5NWIU - [0:0]\n:KUBE-HP-72Z4HC4R6QABH2KV - [0:0]\n-X KUBE-HP-72Z4HC4R6QABH2KV\n-X KUBE-HP-GK54FMWVHAC5NWIU\nCOMMIT\n"

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.901145221Z" level=info msg="Closing host port tcp:80"

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.901192769Z" level=info msg="Closing host port tcp:443"

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.902642335Z" level=info msg="Host port tcp:80 does not have an open socket"

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.902665974Z" level=info msg="Host port tcp:443 does not have an open socket"

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.902835015Z" level=info msg="Got pod network &{Name:ingress-nginx-controller-76dc478dd8-qnjmj Namespace:ingress-nginx ID:54df043ddbaab626dc2e9315efdf2912eacebb44bd5d5529533984e6880ad206 UID:9d1d732e-85cc-4469-875e-1aa6918bf15b NetNS:/var/run/netns/1ac144ec-85e5-46c4-ba97-92f4a57bd528 Networks:[{Name:kindnet Ifname:eth0}] RuntimeConfig:map[kindnet:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.902978866Z" level=info msg="Deleting pod ingress-nginx_ingress-nginx-controller-76dc478dd8-qnjmj from CNI network \"kindnet\" (type=ptp)"

	Mar 05 04:04:15 addons-649326 crio[904]: time="2024-03-05 04:04:15.930100883Z" level=info msg="Stopped pod sandbox: 54df043ddbaab626dc2e9315efdf2912eacebb44bd5d5529533984e6880ad206" id=9c369fa5-7c09-4b2e-86ee-e230b9478471 name=/runtime.v1.RuntimeService/StopPodSandbox

	Mar 05 04:04:16 addons-649326 crio[904]: time="2024-03-05 04:04:16.042928419Z" level=info msg="Removing container: 8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba" id=71d95e58-c19a-44fa-9e05-7c3b4fbc877c name=/runtime.v1.RuntimeService/RemoveContainer

	Mar 05 04:04:16 addons-649326 crio[904]: time="2024-03-05 04:04:16.058618143Z" level=info msg="Removed container 8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba: ingress-nginx/ingress-nginx-controller-76dc478dd8-qnjmj/controller" id=71d95e58-c19a-44fa-9e05-7c3b4fbc877c name=/runtime.v1.RuntimeService/RemoveContainer

	==> container status <==

	CONTAINER           IMAGE                                                                                                                        CREATED             STATE               NAME                      ATTEMPT             POD ID              POD

	d556da5998d0e       dd1b12fcb60978ac32686ef6732d56f612c8636ef86693c09613946a54c69d79                                                             6 seconds ago       Exited              hello-world-app           2                   5f20c4c43a547       hello-world-app-5d77478584-j66vr

	f03355aa23d34       docker.io/library/nginx@sha256:34aa0a372d3220dc0448131f809c72d8085f79bdec8058ad6970fc034a395674                              2 minutes ago       Running             nginx                     0                   ec5d7a255b402       nginx

	e4baeb923da27       ghcr.io/headlamp-k8s/headlamp@sha256:0fe50c48c186b89ff3d341dba427174d8232a64c3062af5de854a3a7cb2105ce                        2 minutes ago       Running             headlamp                  0                   87edf81a2633b       headlamp-7ddfbb94ff-9nxrl

	7cf8c4c5c9dbc       gcr.io/k8s-minikube/gcp-auth-webhook@sha256:01b0de782aa30e7fc91ac5a91b5cc35e95e9679dee7ef07af06457b471f88f32                 3 minutes ago       Running             gcp-auth                  0                   db0214d700bc8       gcp-auth-5f6b4f85fd-zhwbj

	bf7625ac6c691       docker.io/marcnuri/yakd@sha256:a3f540278e4c11373e15605311851dd9c64d208f4d63e727bccc0e39f9329310                              4 minutes ago       Running             yakd                      0                   b5920209a58b2       yakd-dashboard-9947fc6bf-xjgvd

	f666149263007       docker.io/rancher/local-path-provisioner@sha256:689a2489a24e74426e4a4666e611c988202c5fa995908b0c60133aca3eb87d98             4 minutes ago       Running             local-path-provisioner    0                   f3275fd42d2e3       local-path-provisioner-78b46b4d5c-wx828

	269d59dacaede       registry.k8s.io/ingress-nginx/kube-webhook-certgen@sha256:0b1098ef00acee905f9736f98dd151af0a38d0fef0ccf9fb5ad189b20933e5f8   4 minutes ago       Exited              patch                     0                   5d7a580e5ec71       ingress-nginx-admission-patch-dv9dn

	5c26da7b6128b       registry.k8s.io/ingress-nginx/kube-webhook-certgen@sha256:0b1098ef00acee905f9736f98dd151af0a38d0fef0ccf9fb5ad189b20933e5f8   4 minutes ago       Exited              create                    0                   9def5954c8c21       ingress-nginx-admission-create-xxhnr

	dfe27ce1014ef       ba04bb24b95753201135cbc420b233c1b0b9fa2e1fd21d28319c348c33fbcde6                                                             5 minutes ago       Running             storage-provisioner       0                   323c73653f4ad       storage-provisioner

	77599f9b89443       97e04611ad43405a2e5863ae17c6f1bc9181bdefdaa78627c432ef754a4eb108                                                             5 minutes ago       Running             coredns                   0                   8b76f6de72ace       coredns-5dd5756b68-4mk85

	0ce8e20304379       docker.io/kindest/kindnetd@sha256:61f9956af8019caf6dcc4d39b31857b868aaab80521432ddcc216b805c4f7988                           5 minutes ago       Running             kindnet-cni               0                   e5a72c340d1e9       kindnet-46wbp

	586635c2af78a       3ca3ca488cf13fde14cfc4b3ffde0c53a8c161b030f4a444a797fba6aef38c39                                                             5 minutes ago       Running             kube-proxy                0                   2d3b32b2a3386       kube-proxy-vsfnz

	0dd56a796beab       9cdd6470f48c8b127530b7ce6ea4b3524137984481e48bcde619735890840ace                                                             5 minutes ago       Running             etcd                      0                   810bcea3f2de3       etcd-addons-649326

	c15fd9e9ed9c0       04b4c447bb9d4840af3bf7e836397379d65df87c86e55dcd27f31a8d11df2419                                                             5 minutes ago       Running             kube-apiserver            0                   7cf637660ba1e       kube-apiserver-addons-649326

	d2744aaadb2f0       05c284c929889d88306fdb3dd14ee2d0132543740f9e247685243214fc3d2c54                                                             5 minutes ago       Running             kube-scheduler            0                   ac13b6d3fb2cc       kube-scheduler-addons-649326

	0ee4ae23473b6       9961cbceaf234d59b7dcf8a197a024f3e3ce4b7fe2b67c2378efd3d209ca994b                                                             5 minutes ago       Running             kube-controller-manager   0                   9748ac78f8a05       kube-controller-manager-addons-649326

	==> coredns [77599f9b89443ab48b99f212ce1044c51a2d59365cb0df1431f1e57f5a314630] <==

	[INFO] 10.244.0.19:40275 - 46715 "AAAA IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000050905s

	[INFO] 10.244.0.19:46701 - 5320 "A IN hello-world-app.default.svc.cluster.local.us-east-2.compute.internal. udp 86 false 512" NXDOMAIN qr,rd,ra 86 0.002564774s

	[INFO] 10.244.0.19:40275 - 25081 "A IN hello-world-app.default.svc.cluster.local.us-east-2.compute.internal. udp 86 false 512" NXDOMAIN qr,rd,ra 86 0.001186076s

	[INFO] 10.244.0.19:46701 - 55794 "AAAA IN hello-world-app.default.svc.cluster.local.us-east-2.compute.internal. udp 86 false 512" NXDOMAIN qr,rd,ra 86 0.001797222s

	[INFO] 10.244.0.19:40275 - 40155 "AAAA IN hello-world-app.default.svc.cluster.local.us-east-2.compute.internal. udp 86 false 512" NXDOMAIN qr,rd,ra 86 0.001847241s

	[INFO] 10.244.0.19:46701 - 23380 "A IN hello-world-app.default.svc.cluster.local. udp 59 false 512" NOERROR qr,aa,rd 116 0.00013211s

	[INFO] 10.244.0.19:40275 - 12255 "A IN hello-world-app.default.svc.cluster.local. udp 59 false 512" NOERROR qr,aa,rd 116 0.000052701s

	[INFO] 10.244.0.19:50132 - 37736 "A IN hello-world-app.default.svc.cluster.local.ingress-nginx.svc.cluster.local. udp 91 false 512" NXDOMAIN qr,aa,rd 184 0.000104894s

	[INFO] 10.244.0.19:37729 - 55749 "A IN hello-world-app.default.svc.cluster.local.ingress-nginx.svc.cluster.local. udp 91 false 512" NXDOMAIN qr,aa,rd 184 0.000047016s

	[INFO] 10.244.0.19:37729 - 63283 "AAAA IN hello-world-app.default.svc.cluster.local.ingress-nginx.svc.cluster.local. udp 91 false 512" NXDOMAIN qr,aa,rd 184 0.000052521s

	[INFO] 10.244.0.19:37729 - 49656 "A IN hello-world-app.default.svc.cluster.local.svc.cluster.local. udp 77 false 512" NXDOMAIN qr,aa,rd 170 0.000054785s

	[INFO] 10.244.0.19:37729 - 25850 "AAAA IN hello-world-app.default.svc.cluster.local.svc.cluster.local. udp 77 false 512" NXDOMAIN qr,aa,rd 170 0.000043815s

	[INFO] 10.244.0.19:50132 - 480 "AAAA IN hello-world-app.default.svc.cluster.local.ingress-nginx.svc.cluster.local. udp 91 false 512" NXDOMAIN qr,aa,rd 184 0.000061858s

	[INFO] 10.244.0.19:37729 - 30290 "A IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000045554s

	[INFO] 10.244.0.19:50132 - 41880 "A IN hello-world-app.default.svc.cluster.local.svc.cluster.local. udp 77 false 512" NXDOMAIN qr,aa,rd 170 0.000036168s

	[INFO] 10.244.0.19:37729 - 13554 "AAAA IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000049485s

	[INFO] 10.244.0.19:50132 - 7453 "AAAA IN hello-world-app.default.svc.cluster.local.svc.cluster.local. udp 77 false 512" NXDOMAIN qr,aa,rd 170 0.000077743s

	[INFO] 10.244.0.19:50132 - 47810 "A IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000181529s

	[INFO] 10.244.0.19:50132 - 60537 "AAAA IN hello-world-app.default.svc.cluster.local.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000068783s

	[INFO] 10.244.0.19:37729 - 47197 "A IN hello-world-app.default.svc.cluster.local.us-east-2.compute.internal. udp 86 false 512" NXDOMAIN qr,rd,ra 86 0.00131904s

	[INFO] 10.244.0.19:50132 - 56065 "A IN hello-world-app.default.svc.cluster.local.us-east-2.compute.internal. udp 86 false 512" NXDOMAIN qr,rd,ra 86 0.001337312s

	[INFO] 10.244.0.19:37729 - 6589 "AAAA IN hello-world-app.default.svc.cluster.local.us-east-2.compute.internal. udp 86 false 512" NXDOMAIN qr,rd,ra 86 0.001197063s

	[INFO] 10.244.0.19:37729 - 12888 "A IN hello-world-app.default.svc.cluster.local. udp 59 false 512" NOERROR qr,aa,rd 116 0.000192663s

	[INFO] 10.244.0.19:50132 - 64068 "AAAA IN hello-world-app.default.svc.cluster.local.us-east-2.compute.internal. udp 86 false 512" NXDOMAIN qr,rd,ra 86 0.001200279s

	[INFO] 10.244.0.19:50132 - 8059 "A IN hello-world-app.default.svc.cluster.local. udp 59 false 512" NOERROR qr,aa,rd 116 0.000052504s

	==> describe nodes <==

	Name:               addons-649326

	Roles:              control-plane

	Labels:             beta.kubernetes.io/arch=arm64

	                    beta.kubernetes.io/os=linux

	                    kubernetes.io/arch=arm64

	                    kubernetes.io/hostname=addons-649326

	                    kubernetes.io/os=linux

	                    minikube.k8s.io/commit=c10843c2362d401e82b228acf1f340a428845477

	                    minikube.k8s.io/name=addons-649326

	                    minikube.k8s.io/primary=true

	                    minikube.k8s.io/updated_at=2024_03_05T03_58_31_0700

	                    minikube.k8s.io/version=v1.32.0

	                    node-role.kubernetes.io/control-plane=

	                    node.kubernetes.io/exclude-from-external-load-balancers=

	                    topology.hostpath.csi/node=addons-649326

	Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/crio/crio.sock

	                    node.alpha.kubernetes.io/ttl: 0

	                    volumes.kubernetes.io/controller-managed-attach-detach: true

	CreationTimestamp:  Tue, 05 Mar 2024 03:58:27 +0000

	Taints:             <none>

	Unschedulable:      false

	Lease:

	  HolderIdentity:  addons-649326

	  AcquireTime:     <unset>

	  RenewTime:       Tue, 05 Mar 2024 04:04:16 +0000

	Conditions:

	  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message

	  ----             ------  -----------------                 ------------------                ------                       -------

	  MemoryPressure   False   Tue, 05 Mar 2024 04:04:07 +0000   Tue, 05 Mar 2024 03:58:24 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available

	  DiskPressure     False   Tue, 05 Mar 2024 04:04:07 +0000   Tue, 05 Mar 2024 03:58:24 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure

	  PIDPressure      False   Tue, 05 Mar 2024 04:04:07 +0000   Tue, 05 Mar 2024 03:58:24 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available

	  Ready            True    Tue, 05 Mar 2024 04:04:07 +0000   Tue, 05 Mar 2024 03:59:19 +0000   KubeletReady                 kubelet is posting ready status

	Addresses:

	  InternalIP:  192.168.49.2

	  Hostname:    addons-649326

	Capacity:

	  cpu:                2

	  ephemeral-storage:  203034800Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             8022504Ki

	  pods:               110

	Allocatable:

	  cpu:                2

	  ephemeral-storage:  203034800Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             8022504Ki

	  pods:               110

	System Info:

	  Machine ID:                 3384c1db57734fe3ab52c1fd7852c66d

	  System UUID:                5a8a76e9-a49d-46a5-8108-1a024e592cbb

	  Boot ID:                    e693da36-93b6-4a73-8932-63823d30425e

	  Kernel Version:             5.15.0-1055-aws

	  OS Image:                   Ubuntu 22.04.3 LTS

	  Operating System:           linux

	  Architecture:               arm64

	  Container Runtime Version:  cri-o://1.24.6

	  Kubelet Version:            v1.28.4

	  Kube-Proxy Version:         v1.28.4

	PodCIDR:                      10.244.0.0/24

	PodCIDRs:                     10.244.0.0/24

	Non-terminated Pods:          (14 in total)

	  Namespace                   Name                                       CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age

	  ---------                   ----                                       ------------  ----------  ---------------  -------------  ---

	  default                     hello-world-app-5d77478584-j66vr           0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         26s

	  default                     nginx                                      0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         2m46s

	  gcp-auth                    gcp-auth-5f6b4f85fd-zhwbj                  0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         5m27s

	  headlamp                    headlamp-7ddfbb94ff-9nxrl                  0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         3m4s

	  kube-system                 coredns-5dd5756b68-4mk85                   100m (5%!)(MISSING)     0 (0%!)(MISSING)      70Mi (0%!)(MISSING)        170Mi (2%!)(MISSING)     5m38s

	  kube-system                 etcd-addons-649326                         100m (5%!)(MISSING)     0 (0%!)(MISSING)      100Mi (1%!)(MISSING)       0 (0%!)(MISSING)         5m53s

	  kube-system                 kindnet-46wbp                              100m (5%!)(MISSING)     100m (5%!)(MISSING)   50Mi (0%!)(MISSING)        50Mi (0%!)(MISSING)      5m39s

	  kube-system                 kube-apiserver-addons-649326               250m (12%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         5m51s

	  kube-system                 kube-controller-manager-addons-649326      200m (10%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         5m51s

	  kube-system                 kube-proxy-vsfnz                           0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         5m39s

	  kube-system                 kube-scheduler-addons-649326               100m (5%!)(MISSING)     0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         5m52s

	  kube-system                 storage-provisioner                        0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         5m32s

	  local-path-storage          local-path-provisioner-78b46b4d5c-wx828    0 (0%!)(MISSING)        0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         5m32s

	  yakd-dashboard              yakd-dashboard-9947fc6bf-xjgvd             0 (0%!)(MISSING)        0 (0%!)(MISSING)      128Mi (1%!)(MISSING)       256Mi (3%!)(MISSING)     5m32s

	Allocated resources:

	  (Total limits may be over 100 percent, i.e., overcommitted.)

	  Resource           Requests    Limits

	  --------           --------    ------

	  cpu                850m (42%!)(MISSING)  100m (5%!)(MISSING)

	  memory             348Mi (4%!)(MISSING)  476Mi (6%!)(MISSING)

	  ephemeral-storage  0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-1Gi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-2Mi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-32Mi     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-64Ki     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	Events:

	  Type    Reason                   Age                    From             Message

	  ----    ------                   ----                   ----             -------

	  Normal  Starting                 5m31s                  kube-proxy       

	  Normal  NodeHasSufficientMemory  5m59s (x8 over 5m59s)  kubelet          Node addons-649326 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    5m59s (x8 over 5m59s)  kubelet          Node addons-649326 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     5m59s (x8 over 5m59s)  kubelet          Node addons-649326 status is now: NodeHasSufficientPID

	  Normal  Starting                 5m51s                  kubelet          Starting kubelet.

	  Normal  NodeHasSufficientMemory  5m51s                  kubelet          Node addons-649326 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    5m51s                  kubelet          Node addons-649326 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     5m51s                  kubelet          Node addons-649326 status is now: NodeHasSufficientPID

	  Normal  RegisteredNode           5m39s                  node-controller  Node addons-649326 event: Registered Node addons-649326 in Controller

	  Normal  NodeReady                5m2s                   kubelet          Node addons-649326 status is now: NodeReady

	==> dmesg <==

	[  +0.001093] FS-Cache: O-key=[8] 'b9435c0100000000'

	[  +0.000760] FS-Cache: N-cookie c=00000078 [p=0000006f fl=2 nc=0 na=1]

	[  +0.001011] FS-Cache: N-cookie d=0000000007158bf0{9p.inode} n=000000006cba7e0b

	[  +0.001113] FS-Cache: N-key=[8] 'b9435c0100000000'

	[  +0.002979] FS-Cache: Duplicate cookie detected

	[  +0.000723] FS-Cache: O-cookie c=00000072 [p=0000006f fl=226 nc=0 na=1]

	[  +0.001047] FS-Cache: O-cookie d=0000000007158bf0{9p.inode} n=00000000eaee5be9

	[  +0.001114] FS-Cache: O-key=[8] 'b9435c0100000000'

	[  +0.000820] FS-Cache: N-cookie c=00000079 [p=0000006f fl=2 nc=0 na=1]

	[  +0.001046] FS-Cache: N-cookie d=0000000007158bf0{9p.inode} n=00000000af94a326

	[  +0.001113] FS-Cache: N-key=[8] 'b9435c0100000000'

	[  +1.870421] FS-Cache: Duplicate cookie detected

	[  +0.000747] FS-Cache: O-cookie c=00000070 [p=0000006f fl=226 nc=0 na=1]

	[  +0.001078] FS-Cache: O-cookie d=0000000007158bf0{9p.inode} n=0000000002ad72c5

	[  +0.001112] FS-Cache: O-key=[8] 'b8435c0100000000'

	[  +0.000817] FS-Cache: N-cookie c=0000007b [p=0000006f fl=2 nc=0 na=1]

	[  +0.001059] FS-Cache: N-cookie d=0000000007158bf0{9p.inode} n=000000007c8d9be1

	[  +0.001138] FS-Cache: N-key=[8] 'b8435c0100000000'

	[  +0.359494] FS-Cache: Duplicate cookie detected

	[  +0.000776] FS-Cache: O-cookie c=00000075 [p=0000006f fl=226 nc=0 na=1]

	[  +0.001022] FS-Cache: O-cookie d=0000000007158bf0{9p.inode} n=00000000cbe2cfaf

	[  +0.001170] FS-Cache: O-key=[8] 'be435c0100000000'

	[  +0.000768] FS-Cache: N-cookie c=0000007c [p=0000006f fl=2 nc=0 na=1]

	[  +0.001032] FS-Cache: N-cookie d=0000000007158bf0{9p.inode} n=0000000053e344cf

	[  +0.001165] FS-Cache: N-key=[8] 'be435c0100000000'

	==> etcd [0dd56a796beab128a82405c306d7b43edd6cedc32fadfe0bca6d3a6f03f319d1] <==

	{"level":"info","ts":"2024-03-05T03:58:23.708138Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"aec36adc501070cc became pre-candidate at term 1"}

	{"level":"info","ts":"2024-03-05T03:58:23.708182Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"aec36adc501070cc received MsgPreVoteResp from aec36adc501070cc at term 1"}

	{"level":"info","ts":"2024-03-05T03:58:23.708234Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"aec36adc501070cc became candidate at term 2"}

	{"level":"info","ts":"2024-03-05T03:58:23.708268Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"aec36adc501070cc received MsgVoteResp from aec36adc501070cc at term 2"}

	{"level":"info","ts":"2024-03-05T03:58:23.708314Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"aec36adc501070cc became leader at term 2"}

	{"level":"info","ts":"2024-03-05T03:58:23.708347Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: aec36adc501070cc elected leader aec36adc501070cc at term 2"}

	{"level":"info","ts":"2024-03-05T03:58:23.712123Z","caller":"etcdserver/server.go:2062","msg":"published local member to cluster through raft","local-member-id":"aec36adc501070cc","local-member-attributes":"{Name:addons-649326 ClientURLs:[https://192.168.49.2:2379]}","request-path":"/0/members/aec36adc501070cc/attributes","cluster-id":"fa54960ea34d58be","publish-timeout":"7s"}

	{"level":"info","ts":"2024-03-05T03:58:23.71231Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2024-03-05T03:58:23.713381Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.49.2:2379"}

	{"level":"info","ts":"2024-03-05T03:58:23.713471Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2024-03-05T03:58:23.714283Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"127.0.0.1:2379"}

	{"level":"info","ts":"2024-03-05T03:58:23.714784Z","caller":"etcdserver/server.go:2571","msg":"setting up initial cluster version using v2 API","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-05T03:58:23.71495Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}

	{"level":"info","ts":"2024-03-05T03:58:23.715004Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}

	{"level":"info","ts":"2024-03-05T03:58:23.734217Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"fa54960ea34d58be","local-member-id":"aec36adc501070cc","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-05T03:58:23.73432Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-05T03:58:23.734348Z","caller":"etcdserver/server.go:2595","msg":"cluster version is updated","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-05T03:58:44.830754Z","caller":"traceutil/trace.go:171","msg":"trace[2037040321] transaction","detail":"{read_only:false; response_revision:400; number_of_response:1; }","duration":"143.570854ms","start":"2024-03-05T03:58:44.682496Z","end":"2024-03-05T03:58:44.826067Z","steps":["trace[2037040321] 'process raft request'  (duration: 131.208478ms)"],"step_count":1}

	{"level":"info","ts":"2024-03-05T03:58:47.000124Z","caller":"traceutil/trace.go:171","msg":"trace[1327442268] transaction","detail":"{read_only:false; response_revision:409; number_of_response:1; }","duration":"132.50789ms","start":"2024-03-05T03:58:46.867593Z","end":"2024-03-05T03:58:47.000101Z","steps":["trace[1327442268] 'process raft request'  (duration: 53.486267ms)","trace[1327442268] 'compare'  (duration: 78.07763ms)"],"step_count":2}

	{"level":"info","ts":"2024-03-05T03:58:47.763385Z","caller":"traceutil/trace.go:171","msg":"trace[436243907] transaction","detail":"{read_only:false; response_revision:415; number_of_response:1; }","duration":"144.154292ms","start":"2024-03-05T03:58:47.619209Z","end":"2024-03-05T03:58:47.763363Z","steps":["trace[436243907] 'process raft request'  (duration: 68.269517ms)","trace[436243907] 'compare'  (duration: 63.887304ms)"],"step_count":2}

	{"level":"info","ts":"2024-03-05T03:58:47.773404Z","caller":"traceutil/trace.go:171","msg":"trace[1740095330] linearizableReadLoop","detail":"{readStateIndex:427; appliedIndex:425; }","duration":"154.104ms","start":"2024-03-05T03:58:47.619286Z","end":"2024-03-05T03:58:47.77339Z","steps":["trace[1740095330] 'read index received'  (duration: 1.002307ms)","trace[1740095330] 'applied index is now lower than readState.Index'  (duration: 153.10061ms)"],"step_count":2}

	{"level":"warn","ts":"2024-03-05T03:58:47.783393Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"164.10233ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/addons-649326\" ","response":"range_response_count:1 size:5743"}

	{"level":"info","ts":"2024-03-05T03:58:47.783558Z","caller":"traceutil/trace.go:171","msg":"trace[200435348] range","detail":"{range_begin:/registry/minions/addons-649326; range_end:; response_count:1; response_revision:416; }","duration":"164.276089ms","start":"2024-03-05T03:58:47.61927Z","end":"2024-03-05T03:58:47.783546Z","steps":["trace[200435348] 'agreement among raft nodes before linearized reading'  (duration: 154.393031ms)"],"step_count":1}

	{"level":"warn","ts":"2024-03-05T03:58:47.821442Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"115.220938ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/deployments/kube-system/coredns\" ","response":"range_response_count:1 size:4109"}

	{"level":"info","ts":"2024-03-05T03:58:47.823775Z","caller":"traceutil/trace.go:171","msg":"trace[2095493718] range","detail":"{range_begin:/registry/deployments/kube-system/coredns; range_end:; response_count:1; response_revision:417; }","duration":"117.557825ms","start":"2024-03-05T03:58:47.706197Z","end":"2024-03-05T03:58:47.823755Z","steps":["trace[2095493718] 'agreement among raft nodes before linearized reading'  (duration: 76.993478ms)","trace[2095493718] 'get authentication metadata'  (duration: 25.370146ms)"],"step_count":2}

	==> gcp-auth [7cf8c4c5c9dbce713322b6dfb86a8d28aa48d3ced309b4bd4f839973851f8242] <==

	2024/03/05 04:00:28 GCP Auth Webhook started!

	2024/03/05 04:00:41 Ready to marshal response ...

	2024/03/05 04:00:41 Ready to write response ...

	2024/03/05 04:00:54 Ready to marshal response ...

	2024/03/05 04:00:54 Ready to write response ...

	2024/03/05 04:00:59 Ready to marshal response ...

	2024/03/05 04:00:59 Ready to write response ...

	2024/03/05 04:00:59 Ready to marshal response ...

	2024/03/05 04:00:59 Ready to write response ...

	2024/03/05 04:01:08 Ready to marshal response ...

	2024/03/05 04:01:08 Ready to write response ...

	2024/03/05 04:01:15 Ready to marshal response ...

	2024/03/05 04:01:15 Ready to write response ...

	2024/03/05 04:01:17 Ready to marshal response ...

	2024/03/05 04:01:17 Ready to write response ...

	2024/03/05 04:01:17 Ready to marshal response ...

	2024/03/05 04:01:17 Ready to write response ...

	2024/03/05 04:01:17 Ready to marshal response ...

	2024/03/05 04:01:17 Ready to write response ...

	2024/03/05 04:01:35 Ready to marshal response ...

	2024/03/05 04:01:35 Ready to write response ...

	2024/03/05 04:03:55 Ready to marshal response ...

	2024/03/05 04:03:55 Ready to write response ...

	==> kernel <==

	 04:04:21 up  9:46,  0 users,  load average: 0.28, 1.52, 2.03

	Linux addons-649326 5.15.0-1055-aws #60~20.04.1-Ubuntu SMP Thu Feb 22 15:54:21 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

	PRETTY_NAME="Ubuntu 22.04.3 LTS"

	==> kindnet [0ce8e20304379b306229459776ef1ade801691890ac10b53f5cf933c311dac09] <==

	I0305 04:02:19.680611       1 main.go:227] handling current node

	I0305 04:02:29.692238       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:02:29.692339       1 main.go:227] handling current node

	I0305 04:02:39.699206       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:02:39.699310       1 main.go:227] handling current node

	I0305 04:02:49.704347       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:02:49.704375       1 main.go:227] handling current node

	I0305 04:02:59.717423       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:02:59.717452       1 main.go:227] handling current node

	I0305 04:03:09.730501       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:03:09.730528       1 main.go:227] handling current node

	I0305 04:03:19.735336       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:03:19.735365       1 main.go:227] handling current node

	I0305 04:03:29.746373       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:03:29.746403       1 main.go:227] handling current node

	I0305 04:03:39.755761       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:03:39.755789       1 main.go:227] handling current node

	I0305 04:03:49.760383       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:03:49.760486       1 main.go:227] handling current node

	I0305 04:03:59.768238       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:03:59.768265       1 main.go:227] handling current node

	I0305 04:04:09.774443       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:04:09.774470       1 main.go:227] handling current node

	I0305 04:04:19.779583       1 main.go:223] Handling node with IPs: map[192.168.49.2:{}]

	I0305 04:04:19.779617       1 main.go:227] handling current node

	==> kube-apiserver [c15fd9e9ed9c078dc5293ea70718aa5b9b2954e293795d77534611ac1841f09e] <==

	I0305 04:01:31.642300       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0305 04:01:31.656799       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0305 04:01:31.656853       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0305 04:01:31.678241       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0305 04:01:31.680068       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0305 04:01:31.682607       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0305 04:01:31.683002       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0305 04:01:31.690751       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0305 04:01:31.690868       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0305 04:01:31.703844       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0305 04:01:31.704936       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0305 04:01:31.722239       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0305 04:01:31.722665       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	I0305 04:01:31.789627       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1 to ResourceManager

	I0305 04:01:31.789687       1 handler.go:232] Adding GroupVersion snapshot.storage.k8s.io v1beta1 to ResourceManager

	W0305 04:01:32.683453       1 cacher.go:171] Terminating all watchers from cacher volumesnapshotclasses.snapshot.storage.k8s.io

	W0305 04:01:32.791237       1 cacher.go:171] Terminating all watchers from cacher volumesnapshotcontents.snapshot.storage.k8s.io

	W0305 04:01:32.818548       1 cacher.go:171] Terminating all watchers from cacher volumesnapshots.snapshot.storage.k8s.io

	I0305 04:01:35.336653       1 controller.go:624] quota admission added evaluator for: ingresses.networking.k8s.io

	I0305 04:01:35.603983       1 alloc.go:330] "allocated clusterIPs" service="default/nginx" clusterIPs={"IPv4":"10.106.26.10"}

	I0305 04:01:38.567689       1 handler.go:232] Adding GroupVersion gadget.kinvolk.io v1alpha1 to ResourceManager

	I0305 04:01:38.578032       1 handler.go:232] Adding GroupVersion gadget.kinvolk.io v1alpha1 to ResourceManager

	W0305 04:01:39.596536       1 cacher.go:171] Terminating all watchers from cacher traces.gadget.kinvolk.io

	I0305 04:01:41.313281       1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Nothing (removed from the queue).

	I0305 04:03:56.090206       1 alloc.go:330] "allocated clusterIPs" service="default/hello-world-app" clusterIPs={"IPv4":"10.106.41.87"}

	==> kube-controller-manager [0ee4ae23473b626d0d4fff3ca4248b668d8928fce0f7843a5679b32f4c324a52] <==

	W0305 04:03:17.789881       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0305 04:03:17.789915       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0305 04:03:43.464091       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0305 04:03:43.464127       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	W0305 04:03:49.791347       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0305 04:03:49.791379       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	I0305 04:03:55.823283       1 event.go:307] "Event occurred" object="default/hello-world-app" fieldPath="" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set hello-world-app-5d77478584 to 1"

	I0305 04:03:55.858735       1 event.go:307] "Event occurred" object="default/hello-world-app-5d77478584" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: hello-world-app-5d77478584-j66vr"

	I0305 04:03:55.873650       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="50.054811ms"

	I0305 04:03:55.887811       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="14.012313ms"

	I0305 04:03:55.907719       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="19.856055ms"

	I0305 04:03:55.907817       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="60.693µs"

	I0305 04:04:00.027279       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="73.903µs"

	W0305 04:04:00.656778       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0305 04:04:00.656813       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	I0305 04:04:01.018874       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="44.685µs"

	I0305 04:04:02.017761       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="73.033µs"

	W0305 04:04:03.972235       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0305 04:04:03.972269       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	I0305 04:04:12.716644       1 job_controller.go:562] "enqueueing job" key="ingress-nginx/ingress-nginx-admission-create"

	I0305 04:04:12.721506       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="ingress-nginx/ingress-nginx-controller-76dc478dd8" duration="7.466µs"

	I0305 04:04:12.724705       1 job_controller.go:562] "enqueueing job" key="ingress-nginx/ingress-nginx-admission-patch"

	I0305 04:04:15.057341       1 replica_set.go:676] "Finished syncing" kind="ReplicaSet" key="default/hello-world-app-5d77478584" duration="76.775µs"

	W0305 04:04:19.519479       1 reflector.go:535] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	E0305 04:04:19.519513       1 reflector.go:147] vendor/k8s.io/client-go/metadata/metadatainformer/informer.go:106: Failed to watch *v1.PartialObjectMetadata: failed to list *v1.PartialObjectMetadata: the server could not find the requested resource

	==> kube-proxy [586635c2af78a98d91ba4f594f0454fbe9820996da04f3ff24fc7e31fa2da335] <==

	I0305 03:58:49.343918       1 server_others.go:69] "Using iptables proxy"

	I0305 03:58:49.573548       1 node.go:141] Successfully retrieved node IP: 192.168.49.2

	I0305 03:58:50.132015       1 server.go:632] "kube-proxy running in dual-stack mode" primary ipFamily="IPv4"

	I0305 03:58:50.134566       1 server_others.go:152] "Using iptables Proxier"

	I0305 03:58:50.135335       1 server_others.go:421] "Detect-local-mode set to ClusterCIDR, but no cluster CIDR for family" ipFamily="IPv6"

	I0305 03:58:50.135849       1 server_others.go:438] "Defaulting to no-op detect-local"

	I0305 03:58:50.136051       1 proxier.go:251] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses"

	I0305 03:58:50.138724       1 server.go:846] "Version info" version="v1.28.4"

	I0305 03:58:50.138985       1 server.go:848] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""

	I0305 03:58:50.141918       1 config.go:188] "Starting service config controller"

	I0305 03:58:50.142603       1 shared_informer.go:311] Waiting for caches to sync for service config

	I0305 03:58:50.142713       1 config.go:97] "Starting endpoint slice config controller"

	I0305 03:58:50.142757       1 shared_informer.go:311] Waiting for caches to sync for endpoint slice config

	I0305 03:58:50.146160       1 config.go:315] "Starting node config controller"

	I0305 03:58:50.146184       1 shared_informer.go:311] Waiting for caches to sync for node config

	I0305 03:58:50.243054       1 shared_informer.go:318] Caches are synced for endpoint slice config

	I0305 03:58:50.243105       1 shared_informer.go:318] Caches are synced for service config

	I0305 03:58:50.258417       1 shared_informer.go:318] Caches are synced for node config

	==> kube-scheduler [d2744aaadb2f03f92609be80169f8a66c0b5f7e33b1ad9efdf2577e70737ac39] <==

	W0305 03:58:27.258484       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	E0305 03:58:27.258521       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	W0305 03:58:27.258568       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	E0305 03:58:27.258600       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	W0305 03:58:27.258647       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	E0305 03:58:27.258688       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	W0305 03:58:27.292581       1 reflector.go:535] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	E0305 03:58:27.292711       1 reflector.go:147] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	W0305 03:58:28.071089       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope

	E0305 03:58:28.071125       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope

	W0305 03:58:28.097122       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	E0305 03:58:28.097262       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	W0305 03:58:28.148242       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	E0305 03:58:28.148278       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope

	W0305 03:58:28.172226       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	E0305 03:58:28.172348       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope

	W0305 03:58:28.245717       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	E0305 03:58:28.245829       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	W0305 03:58:28.252475       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope

	E0305 03:58:28.252637       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope

	W0305 03:58:28.418378       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	E0305 03:58:28.418524       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	W0305 03:58:28.431619       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	E0305 03:58:28.431722       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	I0305 03:58:28.814083       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	==> kubelet <==

	Mar 05 04:04:12 addons-649326 kubelet[1508]: I0305 04:04:12.055648    1508 reconciler_common.go:300] "Volume detached for volume \"kube-api-access-btbl6\" (UniqueName: \"kubernetes.io/projected/bddb3a77-ef22-42d1-b1e9-8f173abd14b0-kube-api-access-btbl6\") on node \"addons-649326\" DevicePath \"\""

	Mar 05 04:04:12 addons-649326 kubelet[1508]: I0305 04:04:12.250043    1508 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bddb3a77-ef22-42d1-b1e9-8f173abd14b0" path="/var/lib/kubelet/pods/bddb3a77-ef22-42d1-b1e9-8f173abd14b0/volumes"

	Mar 05 04:04:14 addons-649326 kubelet[1508]: I0305 04:04:14.248841    1508 scope.go:117] "RemoveContainer" containerID="36db5db6de4b1190203741d4035c43db3172e92a72dfd1ecd4c97b160a766d12"

	Mar 05 04:04:14 addons-649326 kubelet[1508]: I0305 04:04:14.250236    1508 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63c867c3-36a9-44ca-ade2-709a2591d012" path="/var/lib/kubelet/pods/63c867c3-36a9-44ca-ade2-709a2591d012/volumes"

	Mar 05 04:04:14 addons-649326 kubelet[1508]: I0305 04:04:14.250642    1508 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf256a76-e9c1-4741-9ed6-f102e8535603" path="/var/lib/kubelet/pods/bf256a76-e9c1-4741-9ed6-f102e8535603/volumes"

	Mar 05 04:04:15 addons-649326 kubelet[1508]: I0305 04:04:15.036052    1508 scope.go:117] "RemoveContainer" containerID="36db5db6de4b1190203741d4035c43db3172e92a72dfd1ecd4c97b160a766d12"

	Mar 05 04:04:15 addons-649326 kubelet[1508]: I0305 04:04:15.036396    1508 scope.go:117] "RemoveContainer" containerID="d556da5998d0e7e2f4dc4ccd3eb21e61cc7ed6701971fa80fd8860d65a93a346"

	Mar 05 04:04:15 addons-649326 kubelet[1508]: E0305 04:04:15.036706    1508 pod_workers.go:1300] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"hello-world-app\" with CrashLoopBackOff: \"back-off 20s restarting failed container=hello-world-app pod=hello-world-app-5d77478584-j66vr_default(b0fde714-4bee-4f0a-a992-31f5f495600a)\"" pod="default/hello-world-app-5d77478584-j66vr" podUID="b0fde714-4bee-4f0a-a992-31f5f495600a"

	Mar 05 04:04:15 addons-649326 kubelet[1508]: I0305 04:04:15.980520    1508 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfml6\" (UniqueName: \"kubernetes.io/projected/9d1d732e-85cc-4469-875e-1aa6918bf15b-kube-api-access-gfml6\") pod \"9d1d732e-85cc-4469-875e-1aa6918bf15b\" (UID: \"9d1d732e-85cc-4469-875e-1aa6918bf15b\") "

	Mar 05 04:04:15 addons-649326 kubelet[1508]: I0305 04:04:15.980573    1508 reconciler_common.go:172] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d1d732e-85cc-4469-875e-1aa6918bf15b-webhook-cert\") pod \"9d1d732e-85cc-4469-875e-1aa6918bf15b\" (UID: \"9d1d732e-85cc-4469-875e-1aa6918bf15b\") "

	Mar 05 04:04:15 addons-649326 kubelet[1508]: I0305 04:04:15.987033    1508 operation_generator.go:882] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d1d732e-85cc-4469-875e-1aa6918bf15b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9d1d732e-85cc-4469-875e-1aa6918bf15b" (UID: "9d1d732e-85cc-4469-875e-1aa6918bf15b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue ""

	Mar 05 04:04:15 addons-649326 kubelet[1508]: I0305 04:04:15.990705    1508 operation_generator.go:882] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d1d732e-85cc-4469-875e-1aa6918bf15b-kube-api-access-gfml6" (OuterVolumeSpecName: "kube-api-access-gfml6") pod "9d1d732e-85cc-4469-875e-1aa6918bf15b" (UID: "9d1d732e-85cc-4469-875e-1aa6918bf15b"). InnerVolumeSpecName "kube-api-access-gfml6". PluginName "kubernetes.io/projected", VolumeGidValue ""

	Mar 05 04:04:16 addons-649326 kubelet[1508]: I0305 04:04:16.041125    1508 scope.go:117] "RemoveContainer" containerID="8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba"

	Mar 05 04:04:16 addons-649326 kubelet[1508]: I0305 04:04:16.058866    1508 scope.go:117] "RemoveContainer" containerID="8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba"

	Mar 05 04:04:16 addons-649326 kubelet[1508]: E0305 04:04:16.059291    1508 remote_runtime.go:432] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba\": container with ID starting with 8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba not found: ID does not exist" containerID="8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba"

	Mar 05 04:04:16 addons-649326 kubelet[1508]: I0305 04:04:16.059341    1508 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba"} err="failed to get container status \"8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba\": rpc error: code = NotFound desc = could not find container \"8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba\": container with ID starting with 8f48cb2eed45c233a552317053bf9c5a4cea34d59c0805214f5c4d1991a2bdba not found: ID does not exist"

	Mar 05 04:04:16 addons-649326 kubelet[1508]: I0305 04:04:16.081721    1508 reconciler_common.go:300] "Volume detached for volume \"kube-api-access-gfml6\" (UniqueName: \"kubernetes.io/projected/9d1d732e-85cc-4469-875e-1aa6918bf15b-kube-api-access-gfml6\") on node \"addons-649326\" DevicePath \"\""

	Mar 05 04:04:16 addons-649326 kubelet[1508]: I0305 04:04:16.081767    1508 reconciler_common.go:300] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d1d732e-85cc-4469-875e-1aa6918bf15b-webhook-cert\") on node \"addons-649326\" DevicePath \"\""

	Mar 05 04:04:16 addons-649326 kubelet[1508]: I0305 04:04:16.250109    1508 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d1d732e-85cc-4469-875e-1aa6918bf15b" path="/var/lib/kubelet/pods/9d1d732e-85cc-4469-875e-1aa6918bf15b/volumes"

	Mar 05 04:04:20 addons-649326 kubelet[1508]: E0305 04:04:20.254729    1508 fsHandler.go:119] failed to collect filesystem stats - rootDiskErr: could not stat "/var/lib/containers/storage/overlay/8ca2526fec392298f48e5dd51ccc88d83e9bb146fb9b8f618a3515dacbf75284/diff" to get inode usage: stat /var/lib/containers/storage/overlay/8ca2526fec392298f48e5dd51ccc88d83e9bb146fb9b8f618a3515dacbf75284/diff: no such file or directory, extraDiskErr: <nil>

	Mar 05 04:04:20 addons-649326 kubelet[1508]: E0305 04:04:20.347442    1508 fsHandler.go:119] failed to collect filesystem stats - rootDiskErr: could not stat "/var/lib/containers/storage/overlay/383bf4d8ba5072e52abc6fcfdfefaf0ea5ba6a1388e8f71141610e15e4210387/diff" to get inode usage: stat /var/lib/containers/storage/overlay/383bf4d8ba5072e52abc6fcfdfefaf0ea5ba6a1388e8f71141610e15e4210387/diff: no such file or directory, extraDiskErr: <nil>

	Mar 05 04:04:20 addons-649326 kubelet[1508]: E0305 04:04:20.712679    1508 fsHandler.go:119] failed to collect filesystem stats - rootDiskErr: could not stat "/var/lib/containers/storage/overlay/a8842da2d3a6d822535c4d8313f9655f5afa306ddc2e3c7f909f3bc41160d268/diff" to get inode usage: stat /var/lib/containers/storage/overlay/a8842da2d3a6d822535c4d8313f9655f5afa306ddc2e3c7f909f3bc41160d268/diff: no such file or directory, extraDiskErr: <nil>

	Mar 05 04:04:20 addons-649326 kubelet[1508]: E0305 04:04:20.786212    1508 fsHandler.go:119] failed to collect filesystem stats - rootDiskErr: could not stat "/var/lib/containers/storage/overlay/39a2a8120f9a6075a5e9fe0451ac0d060d3f04044834b07e73c49bbf59a045b1/diff" to get inode usage: stat /var/lib/containers/storage/overlay/39a2a8120f9a6075a5e9fe0451ac0d060d3f04044834b07e73c49bbf59a045b1/diff: no such file or directory, extraDiskErr: <nil>

	Mar 05 04:04:20 addons-649326 kubelet[1508]: E0305 04:04:20.804556    1508 fsHandler.go:119] failed to collect filesystem stats - rootDiskErr: could not stat "/var/lib/containers/storage/overlay/0ec3d2fa5cb60fb40e2f8ddcd789855401f0095cec5c75136063633a065c9d32/diff" to get inode usage: stat /var/lib/containers/storage/overlay/0ec3d2fa5cb60fb40e2f8ddcd789855401f0095cec5c75136063633a065c9d32/diff: no such file or directory, extraDiskErr: <nil>

	Mar 05 04:04:20 addons-649326 kubelet[1508]: E0305 04:04:20.852629    1508 fsHandler.go:119] failed to collect filesystem stats - rootDiskErr: could not stat "/var/lib/containers/storage/overlay/32f99b9c3e548f0853de53b9e2b26476e991a32b6269270a654d23c9bd96adca/diff" to get inode usage: stat /var/lib/containers/storage/overlay/32f99b9c3e548f0853de53b9e2b26476e991a32b6269270a654d23c9bd96adca/diff: no such file or directory, extraDiskErr: <nil>

	==> storage-provisioner [dfe27ce1014ef872e9eac1eab2b0af27147dd2c025433dd75d8b71066273003b] <==

	I0305 03:59:20.938530       1 storage_provisioner.go:116] Initializing the minikube storage provisioner...

	I0305 03:59:20.966663       1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!

	I0305 03:59:20.966738       1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...

	I0305 03:59:20.989521       1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath

	I0305 03:59:20.990126       1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_addons-649326_c3a46694-4a53-4ca5-8c77-9f79899c8914!

	I0305 03:59:20.991434       1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"234625be-085a-4ded-bc68-a87a1a3955de", APIVersion:"v1", ResourceVersion:"935", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' addons-649326_c3a46694-4a53-4ca5-8c77-9f79899c8914 became leader

	I0305 03:59:21.090574       1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_addons-649326_c3a46694-4a53-4ca5-8c77-9f79899c8914!

-- /stdout --

helpers_test.go:254: (dbg) Run:  out/minikube-linux-arm64 status --format={{.APIServer}} -p addons-649326 -n addons-649326

helpers_test.go:261: (dbg) Run:  kubectl --context addons-649326 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running

helpers_test.go:285: <<< TestAddons/parallel/Ingress FAILED: end of post-mortem logs <<<

helpers_test.go:286: ---------------------/post-mortem---------------------------------

--- FAIL: TestAddons/parallel/Ingress (167.18s)

=== RUN   TestScheduledStopUnix

scheduled_stop_test.go:128: (dbg) Run:  out/minikube-linux-arm64 start -p scheduled-stop-160289 --memory=2048 --driver=docker  --container-runtime=crio

scheduled_stop_test.go:128: (dbg) Done: out/minikube-linux-arm64 start -p scheduled-stop-160289 --memory=2048 --driver=docker  --container-runtime=crio: (32.692753222s)

scheduled_stop_test.go:137: (dbg) Run:  out/minikube-linux-arm64 stop -p scheduled-stop-160289 --schedule 5m

scheduled_stop_test.go:191: (dbg) Run:  out/minikube-linux-arm64 status --format={{.TimeToStop}} -p scheduled-stop-160289 -n scheduled-stop-160289

scheduled_stop_test.go:169: signal error was:  <nil>

scheduled_stop_test.go:137: (dbg) Run:  out/minikube-linux-arm64 stop -p scheduled-stop-160289 --schedule 15s

scheduled_stop_test.go:169: signal error was:  <nil>

scheduled_stop_test.go:98: process 2261895 running but should have been killed on reschedule of stop

panic.go:626: *** TestScheduledStopUnix FAILED at 2024-03-05 04:34:38.433503431 +0000 UTC m=+2244.827093199

helpers_test.go:222: -----------------------post-mortem--------------------------------

helpers_test.go:230: ======>  post-mortem[TestScheduledStopUnix]: docker inspect <======

helpers_test.go:231: (dbg) Run:  docker inspect scheduled-stop-160289

helpers_test.go:235: (dbg) docker inspect scheduled-stop-160289:

-- stdout --

	[

	    {

	        "Id": "c997b11a903915028f78f5d3409ac07683958c217203d4f67b42e7ba2de51eaa",

	        "Created": "2024-03-05T04:34:10.776651868Z",

	        "Path": "/usr/local/bin/entrypoint",

	        "Args": [

	            "/sbin/init"

	        ],

	        "State": {

	            "Status": "running",

	            "Running": true,

	            "Paused": false,

	            "Restarting": false,

	            "OOMKilled": false,

	            "Dead": false,

	            "Pid": 2260031,

	            "ExitCode": 0,

	            "Error": "",

	            "StartedAt": "2024-03-05T04:34:11.075396492Z",

	            "FinishedAt": "0001-01-01T00:00:00Z"

	        },

	        "Image": "sha256:4a9b65157dd7fb2ddb7cb7afe975b3dc288e9877c60d13613a69dd41a70e2e4e",

	        "ResolvConfPath": "/var/lib/docker/containers/c997b11a903915028f78f5d3409ac07683958c217203d4f67b42e7ba2de51eaa/resolv.conf",

	        "HostnamePath": "/var/lib/docker/containers/c997b11a903915028f78f5d3409ac07683958c217203d4f67b42e7ba2de51eaa/hostname",

	        "HostsPath": "/var/lib/docker/containers/c997b11a903915028f78f5d3409ac07683958c217203d4f67b42e7ba2de51eaa/hosts",

	        "LogPath": "/var/lib/docker/containers/c997b11a903915028f78f5d3409ac07683958c217203d4f67b42e7ba2de51eaa/c997b11a903915028f78f5d3409ac07683958c217203d4f67b42e7ba2de51eaa-json.log",

	        "Name": "/scheduled-stop-160289",

	        "RestartCount": 0,

	        "Driver": "overlay2",

	        "Platform": "linux",

	        "MountLabel": "",

	        "ProcessLabel": "",

	        "AppArmorProfile": "unconfined",

	        "ExecIDs": null,

	        "HostConfig": {

	            "Binds": [

	                "/lib/modules:/lib/modules:ro",

	                "scheduled-stop-160289:/var"

	            ],

	            "ContainerIDFile": "",

	            "LogConfig": {

	                "Type": "json-file",

	                "Config": {}

	            },

	            "NetworkMode": "scheduled-stop-160289",

	            "PortBindings": {

	                "22/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": ""

	                    }

	                ],

	                "2376/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": ""

	                    }

	                ],

	                "32443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": ""

	                    }

	                ],

	                "5000/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": ""

	                    }

	                ],

	                "8443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": ""

	                    }

	                ]

	            },

	            "RestartPolicy": {

	                "Name": "no",

	                "MaximumRetryCount": 0

	            },

	            "AutoRemove": false,

	            "VolumeDriver": "",

	            "VolumesFrom": null,

	            "ConsoleSize": [

	                0,

	                0

	            ],

	            "CapAdd": null,

	            "CapDrop": null,

	            "CgroupnsMode": "host",

	            "Dns": [],

	            "DnsOptions": [],

	            "DnsSearch": [],

	            "ExtraHosts": null,

	            "GroupAdd": null,

	            "IpcMode": "private",

	            "Cgroup": "",

	            "Links": null,

	            "OomScoreAdj": 0,

	            "PidMode": "",

	            "Privileged": true,

	            "PublishAllPorts": false,

	            "ReadonlyRootfs": false,

	            "SecurityOpt": [

	                "seccomp=unconfined",

	                "apparmor=unconfined",

	                "label=disable"

	            ],

	            "Tmpfs": {

	                "/run": "",

	                "/tmp": ""

	            },

	            "UTSMode": "",

	            "UsernsMode": "",

	            "ShmSize": 67108864,

	            "Runtime": "runc",

	            "Isolation": "",

	            "CpuShares": 0,

	            "Memory": 2147483648,

	            "NanoCpus": 2000000000,

	            "CgroupParent": "",

	            "BlkioWeight": 0,

	            "BlkioWeightDevice": [],

	            "BlkioDeviceReadBps": [],

	            "BlkioDeviceWriteBps": [],

	            "BlkioDeviceReadIOps": [],

	            "BlkioDeviceWriteIOps": [],

	            "CpuPeriod": 0,

	            "CpuQuota": 0,

	            "CpuRealtimePeriod": 0,

	            "CpuRealtimeRuntime": 0,

	            "CpusetCpus": "",

	            "CpusetMems": "",

	            "Devices": [],

	            "DeviceCgroupRules": null,

	            "DeviceRequests": null,

	            "MemoryReservation": 0,

	            "MemorySwap": 4294967296,

	            "MemorySwappiness": null,

	            "OomKillDisable": false,

	            "PidsLimit": null,

	            "Ulimits": [],

	            "CpuCount": 0,

	            "CpuPercent": 0,

	            "IOMaximumIOps": 0,

	            "IOMaximumBandwidth": 0,

	            "MaskedPaths": null,

	            "ReadonlyPaths": null

	        },

	        "GraphDriver": {

	            "Data": {

	                "LowerDir": "/var/lib/docker/overlay2/319c5661274d709eb7e678ea1f390a62b276665e68b8e591dd935860d21a2f55-init/diff:/var/lib/docker/overlay2/423fb181de42bf2fff4f2436233a3066e6b64d14a90221913df20465d28fdb3b/diff",

	                "MergedDir": "/var/lib/docker/overlay2/319c5661274d709eb7e678ea1f390a62b276665e68b8e591dd935860d21a2f55/merged",

	                "UpperDir": "/var/lib/docker/overlay2/319c5661274d709eb7e678ea1f390a62b276665e68b8e591dd935860d21a2f55/diff",

	                "WorkDir": "/var/lib/docker/overlay2/319c5661274d709eb7e678ea1f390a62b276665e68b8e591dd935860d21a2f55/work"

	            },

	            "Name": "overlay2"

	        },

	        "Mounts": [

	            {

	                "Type": "bind",

	                "Source": "/lib/modules",

	                "Destination": "/lib/modules",

	                "Mode": "ro",

	                "RW": false,

	                "Propagation": "rprivate"

	            },

	            {

	                "Type": "volume",

	                "Name": "scheduled-stop-160289",

	                "Source": "/var/lib/docker/volumes/scheduled-stop-160289/_data",

	                "Destination": "/var",

	                "Driver": "local",

	                "Mode": "z",

	                "RW": true,

	                "Propagation": ""

	            }

	        ],

	        "Config": {

	            "Hostname": "scheduled-stop-160289",

	            "Domainname": "",

	            "User": "",

	            "AttachStdin": false,

	            "AttachStdout": false,

	            "AttachStderr": false,

	            "ExposedPorts": {

	                "22/tcp": {},

	                "2376/tcp": {},

	                "32443/tcp": {},

	                "5000/tcp": {},

	                "8443/tcp": {}

	            },

	            "Tty": true,

	            "OpenStdin": false,

	            "StdinOnce": false,

	            "Env": [

	                "container=docker",

	                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

	            ],

	            "Cmd": null,

	            "Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08",

	            "Volumes": null,

	            "WorkingDir": "/",

	            "Entrypoint": [

	                "/usr/local/bin/entrypoint",

	                "/sbin/init"

	            ],

	            "OnBuild": null,

	            "Labels": {

	                "created_by.minikube.sigs.k8s.io": "true",

	                "mode.minikube.sigs.k8s.io": "scheduled-stop-160289",

	                "name.minikube.sigs.k8s.io": "scheduled-stop-160289",

	                "role.minikube.sigs.k8s.io": ""

	            },

	            "StopSignal": "SIGRTMIN+3"

	        },

	        "NetworkSettings": {

	            "Bridge": "",

	            "SandboxID": "3b9479ef3cb17279bfb8d78a766adf0adc573ae1ae292a8e5edc1a8dbd51ff1d",

	            "SandboxKey": "/var/run/docker/netns/3b9479ef3cb1",

	            "Ports": {

	                "22/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "35244"

	                    }

	                ],

	                "2376/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "35243"

	                    }

	                ],

	                "32443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "35240"

	                    }

	                ],

	                "5000/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "35242"

	                    }

	                ],

	                "8443/tcp": [

	                    {

	                        "HostIp": "127.0.0.1",

	                        "HostPort": "35241"

	                    }

	                ]

	            },

	            "HairpinMode": false,

	            "LinkLocalIPv6Address": "",

	            "LinkLocalIPv6PrefixLen": 0,

	            "SecondaryIPAddresses": null,

	            "SecondaryIPv6Addresses": null,

	            "EndpointID": "",

	            "Gateway": "",

	            "GlobalIPv6Address": "",

	            "GlobalIPv6PrefixLen": 0,

	            "IPAddress": "",

	            "IPPrefixLen": 0,

	            "IPv6Gateway": "",

	            "MacAddress": "",

	            "Networks": {

	                "scheduled-stop-160289": {

	                    "IPAMConfig": {

	                        "IPv4Address": "192.168.76.2"

	                    },

	                    "Links": null,

	                    "Aliases": [

	                        "c997b11a9039",

	                        "scheduled-stop-160289"

	                    ],

	                    "MacAddress": "02:42:c0:a8:4c:02",

	                    "NetworkID": "af711e3144d5269524924dd888746eacda66f265da6181be3c03daf5ef6e5684",

	                    "EndpointID": "58bfda147d1c54efdaf28f8407b789e226d741f83a7657aba34145dc9bfd7eef",

	                    "Gateway": "192.168.76.1",

	                    "IPAddress": "192.168.76.2",

	                    "IPPrefixLen": 24,

	                    "IPv6Gateway": "",

	                    "GlobalIPv6Address": "",

	                    "GlobalIPv6PrefixLen": 0,

	                    "DriverOpts": null,

	                    "DNSNames": [

	                        "scheduled-stop-160289",

	                        "c997b11a9039"

	                    ]

	                }

	            }

	        }

	    }

	]

-- /stdout --

helpers_test.go:239: (dbg) Run:  out/minikube-linux-arm64 status --format={{.Host}} -p scheduled-stop-160289 -n scheduled-stop-160289

helpers_test.go:244: <<< TestScheduledStopUnix FAILED: start of post-mortem logs <<<

helpers_test.go:245: ======>  post-mortem[TestScheduledStopUnix]: minikube logs <======

helpers_test.go:247: (dbg) Run:  out/minikube-linux-arm64 -p scheduled-stop-160289 logs -n 25

helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p scheduled-stop-160289 logs -n 25: (1.063585279s)

helpers_test.go:252: TestScheduledStopUnix logs: 

-- stdout --

	==> Audit <==

	|---------|--------------------------------|-----------------------|---------|---------|---------------------|---------------------|

	| Command |              Args              |        Profile        |  User   | Version |     Start Time      |      End Time       |

	|---------|--------------------------------|-----------------------|---------|---------|---------------------|---------------------|

	| stop    | -p multinode-974081            | multinode-974081      | jenkins | v1.32.0 | 05 Mar 24 04:28 UTC | 05 Mar 24 04:29 UTC |

	| start   | -p multinode-974081            | multinode-974081      | jenkins | v1.32.0 | 05 Mar 24 04:29 UTC | 05 Mar 24 04:30 UTC |

	|         | --wait=true -v=8               |                       |         |         |                     |                     |

	|         | --alsologtostderr              |                       |         |         |                     |                     |

	| node    | list -p multinode-974081       | multinode-974081      | jenkins | v1.32.0 | 05 Mar 24 04:30 UTC |                     |

	| node    | multinode-974081 node delete   | multinode-974081      | jenkins | v1.32.0 | 05 Mar 24 04:30 UTC | 05 Mar 24 04:30 UTC |

	|         | m03                            |                       |         |         |                     |                     |

	| stop    | multinode-974081 stop          | multinode-974081      | jenkins | v1.32.0 | 05 Mar 24 04:30 UTC | 05 Mar 24 04:30 UTC |

	| start   | -p multinode-974081            | multinode-974081      | jenkins | v1.32.0 | 05 Mar 24 04:30 UTC | 05 Mar 24 04:31 UTC |

	|         | --wait=true -v=8               |                       |         |         |                     |                     |

	|         | --alsologtostderr              |                       |         |         |                     |                     |

	|         | --driver=docker                |                       |         |         |                     |                     |

	|         | --container-runtime=crio       |                       |         |         |                     |                     |

	| node    | list -p multinode-974081       | multinode-974081      | jenkins | v1.32.0 | 05 Mar 24 04:31 UTC |                     |

	| start   | -p multinode-974081-m02        | multinode-974081-m02  | jenkins | v1.32.0 | 05 Mar 24 04:31 UTC |                     |

	|         | --driver=docker                |                       |         |         |                     |                     |

	|         | --container-runtime=crio       |                       |         |         |                     |                     |

	| start   | -p multinode-974081-m03        | multinode-974081-m03  | jenkins | v1.32.0 | 05 Mar 24 04:31 UTC | 05 Mar 24 04:32 UTC |

	|         | --driver=docker                |                       |         |         |                     |                     |

	|         | --container-runtime=crio       |                       |         |         |                     |                     |

	| node    | add -p multinode-974081        | multinode-974081      | jenkins | v1.32.0 | 05 Mar 24 04:32 UTC |                     |

	| delete  | -p multinode-974081-m03        | multinode-974081-m03  | jenkins | v1.32.0 | 05 Mar 24 04:32 UTC | 05 Mar 24 04:32 UTC |

	| delete  | -p multinode-974081            | multinode-974081      | jenkins | v1.32.0 | 05 Mar 24 04:32 UTC | 05 Mar 24 04:32 UTC |

	| start   | -p test-preload-775137         | test-preload-775137   | jenkins | v1.32.0 | 05 Mar 24 04:32 UTC | 05 Mar 24 04:33 UTC |

	|         | --memory=2200                  |                       |         |         |                     |                     |

	|         | --alsologtostderr              |                       |         |         |                     |                     |

	|         | --wait=true --preload=false    |                       |         |         |                     |                     |

	|         | --driver=docker                |                       |         |         |                     |                     |

	|         | --container-runtime=crio       |                       |         |         |                     |                     |

	|         | --kubernetes-version=v1.24.4   |                       |         |         |                     |                     |

	| image   | test-preload-775137 image pull | test-preload-775137   | jenkins | v1.32.0 | 05 Mar 24 04:33 UTC | 05 Mar 24 04:33 UTC |

	|         | gcr.io/k8s-minikube/busybox    |                       |         |         |                     |                     |

	| stop    | -p test-preload-775137         | test-preload-775137   | jenkins | v1.32.0 | 05 Mar 24 04:33 UTC | 05 Mar 24 04:33 UTC |

	| start   | -p test-preload-775137         | test-preload-775137   | jenkins | v1.32.0 | 05 Mar 24 04:33 UTC | 05 Mar 24 04:34 UTC |

	|         | --memory=2200                  |                       |         |         |                     |                     |

	|         | --alsologtostderr -v=1         |                       |         |         |                     |                     |

	|         | --wait=true --driver=docker    |                       |         |         |                     |                     |

	|         | --container-runtime=crio       |                       |         |         |                     |                     |

	| image   | test-preload-775137 image list | test-preload-775137   | jenkins | v1.32.0 | 05 Mar 24 04:34 UTC | 05 Mar 24 04:34 UTC |

	| delete  | -p test-preload-775137         | test-preload-775137   | jenkins | v1.32.0 | 05 Mar 24 04:34 UTC | 05 Mar 24 04:34 UTC |

	| start   | -p scheduled-stop-160289       | scheduled-stop-160289 | jenkins | v1.32.0 | 05 Mar 24 04:34 UTC | 05 Mar 24 04:34 UTC |

	|         | --memory=2048 --driver=docker  |                       |         |         |                     |                     |

	|         | --container-runtime=crio       |                       |         |         |                     |                     |

	| stop    | -p scheduled-stop-160289       | scheduled-stop-160289 | jenkins | v1.32.0 | 05 Mar 24 04:34 UTC |                     |

	|         | --schedule 5m                  |                       |         |         |                     |                     |

	| stop    | -p scheduled-stop-160289       | scheduled-stop-160289 | jenkins | v1.32.0 | 05 Mar 24 04:34 UTC |                     |

	|         | --schedule 5m                  |                       |         |         |                     |                     |

	| stop    | -p scheduled-stop-160289       | scheduled-stop-160289 | jenkins | v1.32.0 | 05 Mar 24 04:34 UTC |                     |

	|         | --schedule 5m                  |                       |         |         |                     |                     |

	| stop    | -p scheduled-stop-160289       | scheduled-stop-160289 | jenkins | v1.32.0 | 05 Mar 24 04:34 UTC |                     |

	|         | --schedule 15s                 |                       |         |         |                     |                     |

	| stop    | -p scheduled-stop-160289       | scheduled-stop-160289 | jenkins | v1.32.0 | 05 Mar 24 04:34 UTC |                     |

	|         | --schedule 15s                 |                       |         |         |                     |                     |

	| stop    | -p scheduled-stop-160289       | scheduled-stop-160289 | jenkins | v1.32.0 | 05 Mar 24 04:34 UTC |                     |

	|         | --schedule 15s                 |                       |         |         |                     |                     |

	|---------|--------------------------------|-----------------------|---------|---------|---------------------|---------------------|

	==> Last Start <==

	Log file created at: 2024/03/05 04:34:05

	Running on machine: ip-172-31-31-251

	Binary: Built with gc go1.22.0 for linux/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0305 04:34:05.277636 2259571 out.go:291] Setting OutFile to fd 1 ...

	I0305 04:34:05.278149 2259571 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 04:34:05.278156 2259571 out.go:304] Setting ErrFile to fd 2...

	I0305 04:34:05.278160 2259571 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 04:34:05.278435 2259571 root.go:338] Updating PATH: /home/jenkins/minikube-integration/17909-2126334/.minikube/bin

	I0305 04:34:05.278868 2259571 out.go:298] Setting JSON to false

	I0305 04:34:05.279721 2259571 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":36992,"bootTime":1709576254,"procs":169,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1055-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}

	I0305 04:34:05.279783 2259571 start.go:139] virtualization:  

	I0305 04:34:05.282969 2259571 out.go:177] * [scheduled-stop-160289] minikube v1.32.0 on Ubuntu 20.04 (arm64)

	I0305 04:34:05.284926 2259571 out.go:177]   - MINIKUBE_LOCATION=17909

	I0305 04:34:05.286878 2259571 out.go:177]   - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0305 04:34:05.285116 2259571 notify.go:220] Checking for updates...

	I0305 04:34:05.290079 2259571 out.go:177]   - KUBECONFIG=/home/jenkins/minikube-integration/17909-2126334/kubeconfig

	I0305 04:34:05.291599 2259571 out.go:177]   - MINIKUBE_HOME=/home/jenkins/minikube-integration/17909-2126334/.minikube

	I0305 04:34:05.293489 2259571 out.go:177]   - MINIKUBE_BIN=out/minikube-linux-arm64

	I0305 04:34:05.295158 2259571 out.go:177]   - MINIKUBE_FORCE_SYSTEMD=

	I0305 04:34:05.297105 2259571 driver.go:392] Setting default libvirt URI to qemu:///system

	I0305 04:34:05.317863 2259571 docker.go:122] docker version: linux-25.0.3:Docker Engine - Community

	I0305 04:34:05.317992 2259571 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0305 04:34:05.397691 2259571 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:29 OomKillDisable:true NGoroutines:46 SystemTime:2024-03-05 04:34:05.388256583 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar

ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro

rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.6]] Warnings:<nil>}}

	I0305 04:34:05.397795 2259571 docker.go:295] overlay module found

	I0305 04:34:05.401059 2259571 out.go:177] * Using the docker driver based on user configuration

	I0305 04:34:05.403099 2259571 start.go:297] selected driver: docker

	I0305 04:34:05.403107 2259571 start.go:901] validating driver "docker" against <nil>

	I0305 04:34:05.403119 2259571 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}

	I0305 04:34:05.403753 2259571 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0305 04:34:05.467443 2259571 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:3 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:29 OomKillDisable:true NGoroutines:46 SystemTime:2024-03-05 04:34:05.45840733 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aarc

h64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerError

s:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.6]] Warnings:<nil>}}

	I0305 04:34:05.467628 2259571 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0305 04:34:05.467902 2259571 start_flags.go:929] Wait components to verify : map[apiserver:true system_pods:true]

	I0305 04:34:05.470085 2259571 out.go:177] * Using Docker driver with root privileges

	I0305 04:34:05.472136 2259571 cni.go:84] Creating CNI manager for ""

	I0305 04:34:05.472149 2259571 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet

	I0305 04:34:05.472156 2259571 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni

	I0305 04:34:05.472251 2259571 start.go:340] cluster config:

	{Name:scheduled-stop-160289 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:scheduled-stop-160289 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local

ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0305 04:34:05.474267 2259571 out.go:177] * Starting "scheduled-stop-160289" primary control-plane node in "scheduled-stop-160289" cluster

	I0305 04:34:05.476484 2259571 cache.go:121] Beginning downloading kic base image for docker with crio

	I0305 04:34:05.478590 2259571 out.go:177] * Pulling base image v0.0.42-1708944392-18244 ...

	I0305 04:34:05.480383 2259571 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime crio

	I0305 04:34:05.480435 2259571 preload.go:147] Found local preload: /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4

	I0305 04:34:05.480456 2259571 cache.go:56] Caching tarball of preloaded images

	I0305 04:34:05.480464 2259571 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local docker daemon

	I0305 04:34:05.480558 2259571 preload.go:173] Found /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4 in cache, skipping download

	I0305 04:34:05.480566 2259571 cache.go:59] Finished verifying existence of preloaded tar for v1.28.4 on crio

	I0305 04:34:05.480925 2259571 profile.go:142] Saving config to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/config.json ...

	I0305 04:34:05.480943 2259571 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/config.json: {Name:mk826ef9270264740a116c7fbcd591fdca1880e9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 04:34:05.496334 2259571 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local docker daemon, skipping pull

	I0305 04:34:05.496357 2259571 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 exists in daemon, skipping load

	I0305 04:34:05.496380 2259571 cache.go:194] Successfully downloaded all kic artifacts

	I0305 04:34:05.496421 2259571 start.go:360] acquireMachinesLock for scheduled-stop-160289: {Name:mka9892fbe0f7c3bd4fd942d113468268ade2d4f Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}

	I0305 04:34:05.496540 2259571 start.go:364] duration metric: took 102.514µs to acquireMachinesLock for "scheduled-stop-160289"

	I0305 04:34:05.496565 2259571 start.go:93] Provisioning new machine with config: &{Name:scheduled-stop-160289 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:scheduled-stop-160289 Namespace:default APIServerHAVIP: A

PIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: St

aticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}

	I0305 04:34:05.496645 2259571 start.go:125] createHost starting for "" (driver="docker")

	I0305 04:34:05.499538 2259571 out.go:204] * Creating docker container (CPUs=2, Memory=2048MB) ...

	I0305 04:34:05.499787 2259571 start.go:159] libmachine.API.Create for "scheduled-stop-160289" (driver="docker")

	I0305 04:34:05.499825 2259571 client.go:168] LocalClient.Create starting

	I0305 04:34:05.499919 2259571 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem

	I0305 04:34:05.499986 2259571 main.go:141] libmachine: Decoding PEM data...

	I0305 04:34:05.500002 2259571 main.go:141] libmachine: Parsing certificate...

	I0305 04:34:05.500058 2259571 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/cert.pem

	I0305 04:34:05.500076 2259571 main.go:141] libmachine: Decoding PEM data...

	I0305 04:34:05.500085 2259571 main.go:141] libmachine: Parsing certificate...

	I0305 04:34:05.500475 2259571 cli_runner.go:164] Run: docker network inspect scheduled-stop-160289 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	W0305 04:34:05.515785 2259571 cli_runner.go:211] docker network inspect scheduled-stop-160289 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1

	I0305 04:34:05.515852 2259571 network_create.go:281] running [docker network inspect scheduled-stop-160289] to gather additional debugging logs...

	I0305 04:34:05.515869 2259571 cli_runner.go:164] Run: docker network inspect scheduled-stop-160289

	W0305 04:34:05.530388 2259571 cli_runner.go:211] docker network inspect scheduled-stop-160289 returned with exit code 1

	I0305 04:34:05.530409 2259571 network_create.go:284] error running [docker network inspect scheduled-stop-160289]: docker network inspect scheduled-stop-160289: exit status 1

	stdout:

	[]

	stderr:

	Error response from daemon: network scheduled-stop-160289 not found

	I0305 04:34:05.530420 2259571 network_create.go:286] output of [docker network inspect scheduled-stop-160289]: -- stdout --

	[]

	-- /stdout --

	** stderr ** 

	Error response from daemon: network scheduled-stop-160289 not found

	** /stderr **

	I0305 04:34:05.530517 2259571 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0305 04:34:05.546455 2259571 network.go:211] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName:br-7b94a5fa6edd IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:02:42:5a:d2:57:60} reservation:<nil>}

	I0305 04:34:05.546818 2259571 network.go:211] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName:br-9e5f69059ac2 IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:02:42:b5:9b:53:f3} reservation:<nil>}

	I0305 04:34:05.547082 2259571 network.go:211] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName:br-1f395d79e384 IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:02:42:4a:6f:c8:25} reservation:<nil>}

	I0305 04:34:05.547520 2259571 network.go:206] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0x4002552da0}

	I0305 04:34:05.547542 2259571 network_create.go:124] attempt to create docker network scheduled-stop-160289 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...

	I0305 04:34:05.547604 2259571 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=scheduled-stop-160289 scheduled-stop-160289

	I0305 04:34:05.606889 2259571 network_create.go:108] docker network scheduled-stop-160289 192.168.76.0/24 created

	I0305 04:34:05.606912 2259571 kic.go:121] calculated static IP "192.168.76.2" for the "scheduled-stop-160289" container

	I0305 04:34:05.606993 2259571 cli_runner.go:164] Run: docker ps -a --format {{.Names}}

	I0305 04:34:05.623913 2259571 cli_runner.go:164] Run: docker volume create scheduled-stop-160289 --label name.minikube.sigs.k8s.io=scheduled-stop-160289 --label created_by.minikube.sigs.k8s.io=true

	I0305 04:34:05.641494 2259571 oci.go:103] Successfully created a docker volume scheduled-stop-160289

	I0305 04:34:05.641571 2259571 cli_runner.go:164] Run: docker run --rm --name scheduled-stop-160289-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=scheduled-stop-160289 --entrypoint /usr/bin/test -v scheduled-stop-160289:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 -d /var/lib

	I0305 04:34:06.220966 2259571 oci.go:107] Successfully prepared a docker volume scheduled-stop-160289

	I0305 04:34:06.221004 2259571 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime crio

	I0305 04:34:06.221023 2259571 kic.go:194] Starting extracting preloaded images to volume ...

	I0305 04:34:06.221117 2259571 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4:/preloaded.tar:ro -v scheduled-stop-160289:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 -I lz4 -xf /preloaded.tar -C /extractDir

	I0305 04:34:10.709233 2259571 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4:/preloaded.tar:ro -v scheduled-stop-160289:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 -I lz4 -xf /preloaded.tar -C /extractDir: (4.488070648s)

	I0305 04:34:10.709261 2259571 kic.go:203] duration metric: took 4.488228892s to extract preloaded images to volume ...

	W0305 04:34:10.709395 2259571 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.

	I0305 04:34:10.709498 2259571 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"

	I0305 04:34:10.762916 2259571 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname scheduled-stop-160289 --name scheduled-stop-160289 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=scheduled-stop-160289 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=scheduled-stop-160289 --network scheduled-stop-160289 --ip 192.168.76.2 --volume scheduled-stop-160289:/var --security-opt apparmor=unconfined --memory=2048mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08

	I0305 04:34:11.085695 2259571 cli_runner.go:164] Run: docker container inspect scheduled-stop-160289 --format={{.State.Running}}

	I0305 04:34:11.105607 2259571 cli_runner.go:164] Run: docker container inspect scheduled-stop-160289 --format={{.State.Status}}

	I0305 04:34:11.129550 2259571 cli_runner.go:164] Run: docker exec scheduled-stop-160289 stat /var/lib/dpkg/alternatives/iptables

	I0305 04:34:11.203145 2259571 oci.go:144] the created container "scheduled-stop-160289" has a running status.

	I0305 04:34:11.203174 2259571 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/17909-2126334/.minikube/machines/scheduled-stop-160289/id_rsa...

	I0305 04:34:11.790077 2259571 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/17909-2126334/.minikube/machines/scheduled-stop-160289/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)

	I0305 04:34:11.822037 2259571 cli_runner.go:164] Run: docker container inspect scheduled-stop-160289 --format={{.State.Status}}

	I0305 04:34:11.840346 2259571 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys

	I0305 04:34:11.840357 2259571 kic_runner.go:114] Args: [docker exec --privileged scheduled-stop-160289 chown docker:docker /home/docker/.ssh/authorized_keys]

	I0305 04:34:11.908999 2259571 cli_runner.go:164] Run: docker container inspect scheduled-stop-160289 --format={{.State.Status}}

	I0305 04:34:11.926620 2259571 machine.go:94] provisionDockerMachine start ...

	I0305 04:34:11.926700 2259571 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" scheduled-stop-160289

	I0305 04:34:11.951377 2259571 main.go:141] libmachine: Using SSH client type: native

	I0305 04:34:11.951642 2259571 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil>  [] 0s} 127.0.0.1 35244 <nil> <nil>}

	I0305 04:34:11.951657 2259571 main.go:141] libmachine: About to run SSH command:

	hostname

	I0305 04:34:12.100934 2259571 main.go:141] libmachine: SSH cmd err, output: <nil>: scheduled-stop-160289

	I0305 04:34:12.100949 2259571 ubuntu.go:169] provisioning hostname "scheduled-stop-160289"

	I0305 04:34:12.101013 2259571 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" scheduled-stop-160289

	I0305 04:34:12.119271 2259571 main.go:141] libmachine: Using SSH client type: native

	I0305 04:34:12.119504 2259571 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil>  [] 0s} 127.0.0.1 35244 <nil> <nil>}

	I0305 04:34:12.119512 2259571 main.go:141] libmachine: About to run SSH command:

	sudo hostname scheduled-stop-160289 && echo "scheduled-stop-160289" | sudo tee /etc/hostname

	I0305 04:34:12.268805 2259571 main.go:141] libmachine: SSH cmd err, output: <nil>: scheduled-stop-160289

	I0305 04:34:12.268890 2259571 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" scheduled-stop-160289

	I0305 04:34:12.285919 2259571 main.go:141] libmachine: Using SSH client type: native

	I0305 04:34:12.286153 2259571 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil>  [] 0s} 127.0.0.1 35244 <nil> <nil>}

	I0305 04:34:12.286176 2259571 main.go:141] libmachine: About to run SSH command:

			if ! grep -xq '.*\sscheduled-stop-160289' /etc/hosts; then

				if grep -xq '127.0.1.1\s.*' /etc/hosts; then

					sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 scheduled-stop-160289/g' /etc/hosts;

				else 

					echo '127.0.1.1 scheduled-stop-160289' | sudo tee -a /etc/hosts; 

				fi

			fi

	I0305 04:34:12.416088 2259571 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	I0305 04:34:12.416102 2259571 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/17909-2126334/.minikube CaCertPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/17909-2126334/.minikube}

	I0305 04:34:12.416126 2259571 ubuntu.go:177] setting up certificates

	I0305 04:34:12.416135 2259571 provision.go:84] configureAuth start

	I0305 04:34:12.416205 2259571 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" scheduled-stop-160289

	I0305 04:34:12.432427 2259571 provision.go:143] copyHostCerts

	I0305 04:34:12.432485 2259571 exec_runner.go:144] found /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.pem, removing ...

	I0305 04:34:12.432492 2259571 exec_runner.go:203] rm: /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.pem

	I0305 04:34:12.432572 2259571 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.pem (1082 bytes)

	I0305 04:34:12.432694 2259571 exec_runner.go:144] found /home/jenkins/minikube-integration/17909-2126334/.minikube/cert.pem, removing ...

	I0305 04:34:12.432698 2259571 exec_runner.go:203] rm: /home/jenkins/minikube-integration/17909-2126334/.minikube/cert.pem

	I0305 04:34:12.432725 2259571 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/17909-2126334/.minikube/cert.pem (1123 bytes)

	I0305 04:34:12.432783 2259571 exec_runner.go:144] found /home/jenkins/minikube-integration/17909-2126334/.minikube/key.pem, removing ...

	I0305 04:34:12.432787 2259571 exec_runner.go:203] rm: /home/jenkins/minikube-integration/17909-2126334/.minikube/key.pem

	I0305 04:34:12.432809 2259571 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/17909-2126334/.minikube/key.pem (1679 bytes)

	I0305 04:34:12.432852 2259571 provision.go:117] generating server cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca-key.pem org=jenkins.scheduled-stop-160289 san=[127.0.0.1 192.168.76.2 localhost minikube scheduled-stop-160289]

	I0305 04:34:12.876289 2259571 provision.go:177] copyRemoteCerts

	I0305 04:34:12.876348 2259571 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker

	I0305 04:34:12.876403 2259571 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" scheduled-stop-160289

	I0305 04:34:12.891634 2259571 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35244 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/scheduled-stop-160289/id_rsa Username:docker}

	I0305 04:34:12.984685 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)

	I0305 04:34:13.010191 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/machines/server.pem --> /etc/docker/server.pem (1229 bytes)

	I0305 04:34:13.034444 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)

	I0305 04:34:13.058375 2259571 provision.go:87] duration metric: took 642.205782ms to configureAuth

	I0305 04:34:13.058393 2259571 ubuntu.go:193] setting minikube options for container-runtime

	I0305 04:34:13.058572 2259571 config.go:182] Loaded profile config "scheduled-stop-160289": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.28.4

	I0305 04:34:13.058678 2259571 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" scheduled-stop-160289

	I0305 04:34:13.074757 2259571 main.go:141] libmachine: Using SSH client type: native

	I0305 04:34:13.075009 2259571 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1c40] 0x3e44a0 <nil>  [] 0s} 127.0.0.1 35244 <nil> <nil>}

	I0305 04:34:13.075021 2259571 main.go:141] libmachine: About to run SSH command:

	sudo mkdir -p /etc/sysconfig && printf %!s(MISSING) "

	CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '

	" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio

	I0305 04:34:13.314922 2259571 main.go:141] libmachine: SSH cmd err, output: <nil>: 

	CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '

	I0305 04:34:13.314940 2259571 machine.go:97] duration metric: took 1.388307496s to provisionDockerMachine

	I0305 04:34:13.314949 2259571 client.go:171] duration metric: took 7.815119469s to LocalClient.Create

	I0305 04:34:13.314967 2259571 start.go:167] duration metric: took 7.815181065s to libmachine.API.Create "scheduled-stop-160289"

	I0305 04:34:13.314973 2259571 start.go:293] postStartSetup for "scheduled-stop-160289" (driver="docker")

	I0305 04:34:13.314982 2259571 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]

	I0305 04:34:13.315053 2259571 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs

	I0305 04:34:13.315094 2259571 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" scheduled-stop-160289

	I0305 04:34:13.333224 2259571 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35244 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/scheduled-stop-160289/id_rsa Username:docker}

	I0305 04:34:13.425226 2259571 ssh_runner.go:195] Run: cat /etc/os-release

	I0305 04:34:13.428340 2259571 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found

	I0305 04:34:13.428366 2259571 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found

	I0305 04:34:13.428374 2259571 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found

	I0305 04:34:13.428381 2259571 info.go:137] Remote host: Ubuntu 22.04.3 LTS

	I0305 04:34:13.428390 2259571 filesync.go:126] Scanning /home/jenkins/minikube-integration/17909-2126334/.minikube/addons for local assets ...

	I0305 04:34:13.428443 2259571 filesync.go:126] Scanning /home/jenkins/minikube-integration/17909-2126334/.minikube/files for local assets ...

	I0305 04:34:13.428518 2259571 filesync.go:149] local asset: /home/jenkins/minikube-integration/17909-2126334/.minikube/files/etc/ssl/certs/21317012.pem -> 21317012.pem in /etc/ssl/certs

	I0305 04:34:13.428622 2259571 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs

	I0305 04:34:13.437375 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/files/etc/ssl/certs/21317012.pem --> /etc/ssl/certs/21317012.pem (1708 bytes)

	I0305 04:34:13.461836 2259571 start.go:296] duration metric: took 146.850311ms for postStartSetup

	I0305 04:34:13.462203 2259571 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" scheduled-stop-160289

	I0305 04:34:13.477685 2259571 profile.go:142] Saving config to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/config.json ...

	I0305 04:34:13.477960 2259571 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"

	I0305 04:34:13.478000 2259571 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" scheduled-stop-160289

	I0305 04:34:13.493993 2259571 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35244 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/scheduled-stop-160289/id_rsa Username:docker}

	I0305 04:34:13.584939 2259571 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"

	I0305 04:34:13.589414 2259571 start.go:128] duration metric: took 8.092755437s to createHost

	I0305 04:34:13.589430 2259571 start.go:83] releasing machines lock for "scheduled-stop-160289", held for 8.092882189s

	I0305 04:34:13.589500 2259571 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" scheduled-stop-160289

	I0305 04:34:13.605176 2259571 ssh_runner.go:195] Run: cat /version.json

	I0305 04:34:13.605223 2259571 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" scheduled-stop-160289

	I0305 04:34:13.605260 2259571 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/

	I0305 04:34:13.605316 2259571 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" scheduled-stop-160289

	I0305 04:34:13.621314 2259571 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35244 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/scheduled-stop-160289/id_rsa Username:docker}

	I0305 04:34:13.629632 2259571 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35244 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/scheduled-stop-160289/id_rsa Username:docker}

	I0305 04:34:13.821490 2259571 ssh_runner.go:195] Run: systemctl --version

	I0305 04:34:13.825629 2259571 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"

	I0305 04:34:13.968312 2259571 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"

	I0305 04:34:13.972751 2259571 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "sudo mv {} {}.mk_disabled" ;

	I0305 04:34:13.996661 2259571 cni.go:221] loopback cni configuration disabled: "/etc/cni/net.d/*loopback.conf*" found

	I0305 04:34:13.996760 2259571 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;

	I0305 04:34:14.037864 2259571 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)

	I0305 04:34:14.037878 2259571 start.go:494] detecting cgroup driver to use...

	I0305 04:34:14.037911 2259571 detect.go:196] detected "cgroupfs" cgroup driver on host os

	I0305 04:34:14.037961 2259571 ssh_runner.go:195] Run: sudo systemctl stop -f containerd

	I0305 04:34:14.057311 2259571 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd

	I0305 04:34:14.075944 2259571 docker.go:217] disabling cri-docker service (if available) ...

	I0305 04:34:14.076027 2259571 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket

	I0305 04:34:14.091158 2259571 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service

	I0305 04:34:14.106916 2259571 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket

	I0305 04:34:14.200344 2259571 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service

	I0305 04:34:14.287206 2259571 docker.go:233] disabling docker service ...

	I0305 04:34:14.287265 2259571 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket

	I0305 04:34:14.308708 2259571 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service

	I0305 04:34:14.320927 2259571 ssh_runner.go:195] Run: sudo systemctl disable docker.socket

	I0305 04:34:14.405399 2259571 ssh_runner.go:195] Run: sudo systemctl mask docker.service

	I0305 04:34:14.505268 2259571 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker

	I0305 04:34:14.517188 2259571 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/crio/crio.sock

	" | sudo tee /etc/crictl.yaml"

	I0305 04:34:14.532900 2259571 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.9" pause image...

	I0305 04:34:14.532974 2259571 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.9"|' /etc/crio/crio.conf.d/02-crio.conf"

	I0305 04:34:14.542471 2259571 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...

	I0305 04:34:14.542529 2259571 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"

	I0305 04:34:14.551950 2259571 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"

	I0305 04:34:14.561569 2259571 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"

	I0305 04:34:14.571310 2259571 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"

	I0305 04:34:14.579913 2259571 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables

	I0305 04:34:14.588641 2259571 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

	I0305 04:34:14.597320 2259571 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0305 04:34:14.680280 2259571 ssh_runner.go:195] Run: sudo systemctl restart crio

	I0305 04:34:14.787350 2259571 start.go:541] Will wait 60s for socket path /var/run/crio/crio.sock

	I0305 04:34:14.787412 2259571 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock

	I0305 04:34:14.790893 2259571 start.go:562] Will wait 60s for crictl version

	I0305 04:34:14.790943 2259571 ssh_runner.go:195] Run: which crictl

	I0305 04:34:14.794001 2259571 ssh_runner.go:195] Run: sudo /usr/bin/crictl version

	I0305 04:34:14.833980 2259571 start.go:578] Version:  0.1.0

	RuntimeName:  cri-o

	RuntimeVersion:  1.24.6

	RuntimeApiVersion:  v1

	I0305 04:34:14.834080 2259571 ssh_runner.go:195] Run: crio --version

	I0305 04:34:14.873332 2259571 ssh_runner.go:195] Run: crio --version

	I0305 04:34:14.919066 2259571 out.go:177] * Preparing Kubernetes v1.28.4 on CRI-O 1.24.6 ...

	I0305 04:34:14.921113 2259571 cli_runner.go:164] Run: docker network inspect scheduled-stop-160289 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"

	I0305 04:34:14.936068 2259571 ssh_runner.go:195] Run: grep 192.168.76.1	host.minikube.internal$ /etc/hosts

	I0305 04:34:14.939637 2259571 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1	host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0305 04:34:14.950397 2259571 kubeadm.go:877] updating cluster {Name:scheduled-stop-160289 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:scheduled-stop-160289 Namespace:default APIServerHAVIP: APIServerName:miniku

beCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP:

SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...

	I0305 04:34:14.950500 2259571 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime crio

	I0305 04:34:14.950560 2259571 ssh_runner.go:195] Run: sudo crictl images --output json

	I0305 04:34:15.014773 2259571 crio.go:496] all images are preloaded for cri-o runtime.

	I0305 04:34:15.014787 2259571 crio.go:415] Images already preloaded, skipping extraction

	I0305 04:34:15.014855 2259571 ssh_runner.go:195] Run: sudo crictl images --output json

	I0305 04:34:15.062273 2259571 crio.go:496] all images are preloaded for cri-o runtime.

	I0305 04:34:15.062286 2259571 cache_images.go:84] Images are preloaded, skipping loading

	I0305 04:34:15.062293 2259571 kubeadm.go:928] updating node { 192.168.76.2 8443 v1.28.4 crio true true} ...

	I0305 04:34:15.062389 2259571 kubeadm.go:940] kubelet [Unit]

	Wants=crio.service

	[Service]

	ExecStart=

	ExecStart=/var/lib/minikube/binaries/v1.28.4/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroups-per-qos=false --config=/var/lib/kubelet/config.yaml --enforce-node-allocatable= --hostname-override=scheduled-stop-160289 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2

	[Install]

	 config:

	{KubernetesVersion:v1.28.4 ClusterName:scheduled-stop-160289 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}

	I0305 04:34:15.062467 2259571 ssh_runner.go:195] Run: crio config

	I0305 04:34:15.114146 2259571 cni.go:84] Creating CNI manager for ""

	I0305 04:34:15.114160 2259571 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet

	I0305 04:34:15.114171 2259571 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16

	I0305 04:34:15.114200 2259571 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.28.4 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:scheduled-stop-160289 NodeName:scheduled-stop-160289 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodP

ath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///var/run/crio/crio.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}

	I0305 04:34:15.114371 2259571 kubeadm.go:187] kubeadm config:

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: InitConfiguration

	localAPIEndpoint:

	  advertiseAddress: 192.168.76.2

	  bindPort: 8443

	bootstrapTokens:

	  - groups:

	      - system:bootstrappers:kubeadm:default-node-token

	    ttl: 24h0m0s

	    usages:

	      - signing

	      - authentication

	nodeRegistration:

	  criSocket: unix:///var/run/crio/crio.sock

	  name: "scheduled-stop-160289"

	  kubeletExtraArgs:

	    node-ip: 192.168.76.2

	  taints: []

	---

	apiVersion: kubeadm.k8s.io/v1beta3

	kind: ClusterConfiguration

	apiServer:

	  certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]

	  extraArgs:

	    enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"

	controllerManager:

	  extraArgs:

	    allocate-node-cidrs: "true"

	    leader-elect: "false"

	scheduler:

	  extraArgs:

	    leader-elect: "false"

	certificatesDir: /var/lib/minikube/certs

	clusterName: mk

	controlPlaneEndpoint: control-plane.minikube.internal:8443

	etcd:

	  local:

	    dataDir: /var/lib/minikube/etcd

	    extraArgs:

	      proxy-refresh-interval: "70000"

	kubernetesVersion: v1.28.4

	networking:

	  dnsDomain: cluster.local

	  podSubnet: "10.244.0.0/16"

	  serviceSubnet: 10.96.0.0/12

	---

	apiVersion: kubelet.config.k8s.io/v1beta1

	kind: KubeletConfiguration

	authentication:

	  x509:

	    clientCAFile: /var/lib/minikube/certs/ca.crt

	cgroupDriver: cgroupfs

	containerRuntimeEndpoint: unix:///var/run/crio/crio.sock

	hairpinMode: hairpin-veth

	runtimeRequestTimeout: 15m

	clusterDomain: "cluster.local"

	# disable disk resource management by default

	imageGCHighThresholdPercent: 100

	evictionHard:

	  nodefs.available: "0%!"(MISSING)

	  nodefs.inodesFree: "0%!"(MISSING)

	  imagefs.available: "0%!"(MISSING)

	failSwapOn: false

	staticPodPath: /etc/kubernetes/manifests

	---

	apiVersion: kubeproxy.config.k8s.io/v1alpha1

	kind: KubeProxyConfiguration

	clusterCIDR: "10.244.0.0/16"

	metricsBindAddress: 0.0.0.0:10249

	conntrack:

	  maxPerCore: 0

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"

	  tcpEstablishedTimeout: 0s

	# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"

	  tcpCloseWaitTimeout: 0s

	I0305 04:34:15.114444 2259571 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.28.4

	I0305 04:34:15.124627 2259571 binaries.go:44] Found k8s binaries, skipping transfer

	I0305 04:34:15.124703 2259571 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube

	I0305 04:34:15.133987 2259571 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (371 bytes)

	I0305 04:34:15.153195 2259571 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)

	I0305 04:34:15.172632 2259571 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2159 bytes)

	I0305 04:34:15.192382 2259571 ssh_runner.go:195] Run: grep 192.168.76.2	control-plane.minikube.internal$ /etc/hosts

	I0305 04:34:15.196135 2259571 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2	control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""

	I0305 04:34:15.207463 2259571 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0305 04:34:15.290944 2259571 ssh_runner.go:195] Run: sudo systemctl start kubelet

	I0305 04:34:15.304714 2259571 certs.go:68] Setting up /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289 for IP: 192.168.76.2

	I0305 04:34:15.304725 2259571 certs.go:194] generating shared ca certs ...

	I0305 04:34:15.304741 2259571 certs.go:226] acquiring lock for ca certs: {Name:mkbb1962e5c55bf3351f2c6e6b8a52de2404f4b8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 04:34:15.304881 2259571 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.key

	I0305 04:34:15.304949 2259571 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/proxy-client-ca.key

	I0305 04:34:15.304955 2259571 certs.go:256] generating profile certs ...

	I0305 04:34:15.305012 2259571 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/client.key

	I0305 04:34:15.305021 2259571 crypto.go:68] Generating cert /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/client.crt with IP's: []

	I0305 04:34:15.756596 2259571 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/client.crt ...

	I0305 04:34:15.756612 2259571 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/client.crt: {Name:mkbd64311da2507354b6350cd54b3713e5358dfe Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 04:34:15.756819 2259571 crypto.go:164] Writing key to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/client.key ...

	I0305 04:34:15.756828 2259571 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/client.key: {Name:mkf3e1f80561507e02a996b2d0fd39017b74927a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 04:34:15.756931 2259571 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.key.a2850d11

	I0305 04:34:15.756957 2259571 crypto.go:68] Generating cert /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.crt.a2850d11 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.76.2]

	I0305 04:34:16.470516 2259571 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.crt.a2850d11 ...

	I0305 04:34:16.470532 2259571 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.crt.a2850d11: {Name:mka70b2900e95c8555cb1135a0707a9fcf251f14 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 04:34:16.470746 2259571 crypto.go:164] Writing key to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.key.a2850d11 ...

	I0305 04:34:16.470756 2259571 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.key.a2850d11: {Name:mk8952fcb0fea93c5837d937e32c6539a697ab63 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 04:34:16.470853 2259571 certs.go:381] copying /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.crt.a2850d11 -> /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.crt

	I0305 04:34:16.470943 2259571 certs.go:385] copying /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.key.a2850d11 -> /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.key

	I0305 04:34:16.470998 2259571 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/proxy-client.key

	I0305 04:34:16.471011 2259571 crypto.go:68] Generating cert /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/proxy-client.crt with IP's: []

	I0305 04:34:16.608019 2259571 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/proxy-client.crt ...

	I0305 04:34:16.608034 2259571 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/proxy-client.crt: {Name:mkfa036808b8363316287571a670e757cc792aa4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 04:34:16.608223 2259571 crypto.go:164] Writing key to /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/proxy-client.key ...

	I0305 04:34:16.608233 2259571 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/proxy-client.key: {Name:mk42119e4c883534daedb4b544842a17bc097b10 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 04:34:16.608420 2259571 certs.go:484] found cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/2131701.pem (1338 bytes)

	W0305 04:34:16.608463 2259571 certs.go:480] ignoring /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/2131701_empty.pem, impossibly tiny 0 bytes

	I0305 04:34:16.608470 2259571 certs.go:484] found cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca-key.pem (1679 bytes)

	I0305 04:34:16.608495 2259571 certs.go:484] found cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/ca.pem (1082 bytes)

	I0305 04:34:16.608516 2259571 certs.go:484] found cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/cert.pem (1123 bytes)

	I0305 04:34:16.608536 2259571 certs.go:484] found cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/key.pem (1679 bytes)

	I0305 04:34:16.608583 2259571 certs.go:484] found cert: /home/jenkins/minikube-integration/17909-2126334/.minikube/files/etc/ssl/certs/21317012.pem (1708 bytes)

	I0305 04:34:16.609294 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)

	I0305 04:34:16.635345 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)

	I0305 04:34:16.659918 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)

	I0305 04:34:16.684465 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)

	I0305 04:34:16.708961 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)

	I0305 04:34:16.734284 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)

	I0305 04:34:16.758838 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)

	I0305 04:34:16.782678 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/scheduled-stop-160289/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)

	I0305 04:34:16.806048 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/files/etc/ssl/certs/21317012.pem --> /usr/share/ca-certificates/21317012.pem (1708 bytes)

	I0305 04:34:16.835243 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)

	I0305 04:34:16.860634 2259571 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17909-2126334/.minikube/certs/2131701.pem --> /usr/share/ca-certificates/2131701.pem (1338 bytes)

	I0305 04:34:16.885350 2259571 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)

	I0305 04:34:16.903575 2259571 ssh_runner.go:195] Run: openssl version

	I0305 04:34:16.909412 2259571 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"

	I0305 04:34:16.921369 2259571 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem

	I0305 04:34:16.925273 2259571 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar  5 03:58 /usr/share/ca-certificates/minikubeCA.pem

	I0305 04:34:16.925346 2259571 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem

	I0305 04:34:16.932754 2259571 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"

	I0305 04:34:16.942715 2259571 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/2131701.pem && ln -fs /usr/share/ca-certificates/2131701.pem /etc/ssl/certs/2131701.pem"

	I0305 04:34:16.952824 2259571 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/2131701.pem

	I0305 04:34:16.956414 2259571 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Mar  5 04:05 /usr/share/ca-certificates/2131701.pem

	I0305 04:34:16.956478 2259571 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/2131701.pem

	I0305 04:34:16.963765 2259571 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/2131701.pem /etc/ssl/certs/51391683.0"

	I0305 04:34:16.973416 2259571 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/21317012.pem && ln -fs /usr/share/ca-certificates/21317012.pem /etc/ssl/certs/21317012.pem"

	I0305 04:34:16.982875 2259571 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/21317012.pem

	I0305 04:34:16.986923 2259571 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Mar  5 04:05 /usr/share/ca-certificates/21317012.pem

	I0305 04:34:16.986980 2259571 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/21317012.pem

	I0305 04:34:16.994892 2259571 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/21317012.pem /etc/ssl/certs/3ec20f2e.0"

	I0305 04:34:17.006038 2259571 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt

	I0305 04:34:17.009711 2259571 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1

	stdout:

	stderr:

	stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory

	I0305 04:34:17.009753 2259571 kubeadm.go:391] StartCluster: {Name:scheduled-stop-160289 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2048 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:scheduled-stop-160289 Namespace:default APIServerHAVIP: APIServerName:minikubeC

A APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SS

HAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0305 04:34:17.009826 2259571 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}

	I0305 04:34:17.009886 2259571 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"

	I0305 04:34:17.049694 2259571 cri.go:89] found id: ""

	I0305 04:34:17.049760 2259571 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd

	I0305 04:34:17.058657 2259571 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml

	I0305 04:34:17.067428 2259571 kubeadm.go:213] ignoring SystemVerification for kubeadm because of docker driver

	I0305 04:34:17.067486 2259571 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf

	I0305 04:34:17.076511 2259571 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory

	ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory

	I0305 04:34:17.076521 2259571 kubeadm.go:156] found existing configuration files:

	I0305 04:34:17.076573 2259571 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf

	I0305 04:34:17.085485 2259571 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/admin.conf: No such file or directory

	I0305 04:34:17.085554 2259571 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf

	I0305 04:34:17.094310 2259571 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf

	I0305 04:34:17.103150 2259571 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/kubelet.conf: No such file or directory

	I0305 04:34:17.103210 2259571 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf

	I0305 04:34:17.111772 2259571 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf

	I0305 04:34:17.120473 2259571 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/controller-manager.conf: No such file or directory

	I0305 04:34:17.120530 2259571 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf

	I0305 04:34:17.129322 2259571 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf

	I0305 04:34:17.138031 2259571 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2

	stdout:

	stderr:

	grep: /etc/kubernetes/scheduler.conf: No such file or directory

	I0305 04:34:17.138097 2259571 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf

	I0305 04:34:17.146513 2259571 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.28.4:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"

	I0305 04:34:17.188960 2259571 kubeadm.go:309] [init] Using Kubernetes version: v1.28.4

	I0305 04:34:17.189168 2259571 kubeadm.go:309] [preflight] Running pre-flight checks

	I0305 04:34:17.229121 2259571 kubeadm.go:309] [preflight] The system verification failed. Printing the output from the verification:

	I0305 04:34:17.229184 2259571 kubeadm.go:309] KERNEL_VERSION: 5.15.0-1055-aws

	I0305 04:34:17.229216 2259571 kubeadm.go:309] OS: Linux

	I0305 04:34:17.229259 2259571 kubeadm.go:309] CGROUPS_CPU: enabled

	I0305 04:34:17.229305 2259571 kubeadm.go:309] CGROUPS_CPUACCT: enabled

	I0305 04:34:17.229353 2259571 kubeadm.go:309] CGROUPS_CPUSET: enabled

	I0305 04:34:17.229399 2259571 kubeadm.go:309] CGROUPS_DEVICES: enabled

	I0305 04:34:17.229445 2259571 kubeadm.go:309] CGROUPS_FREEZER: enabled

	I0305 04:34:17.229492 2259571 kubeadm.go:309] CGROUPS_MEMORY: enabled

	I0305 04:34:17.229536 2259571 kubeadm.go:309] CGROUPS_PIDS: enabled

	I0305 04:34:17.229581 2259571 kubeadm.go:309] CGROUPS_HUGETLB: enabled

	I0305 04:34:17.229625 2259571 kubeadm.go:309] CGROUPS_BLKIO: enabled

	I0305 04:34:17.306060 2259571 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster

	I0305 04:34:17.306160 2259571 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection

	I0305 04:34:17.306249 2259571 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

	I0305 04:34:17.528808 2259571 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"

	I0305 04:34:17.532384 2259571 out.go:204]   - Generating certificates and keys ...

	I0305 04:34:17.532552 2259571 kubeadm.go:309] [certs] Using existing ca certificate authority

	I0305 04:34:17.532623 2259571 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk

	I0305 04:34:17.965875 2259571 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key

	I0305 04:34:18.301440 2259571 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key

	I0305 04:34:18.720233 2259571 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key

	I0305 04:34:20.067728 2259571 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key

	I0305 04:34:20.498773 2259571 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key

	I0305 04:34:20.498968 2259571 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [localhost scheduled-stop-160289] and IPs [192.168.76.2 127.0.0.1 ::1]

	I0305 04:34:20.748721 2259571 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key

	I0305 04:34:20.749008 2259571 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [localhost scheduled-stop-160289] and IPs [192.168.76.2 127.0.0.1 ::1]

	I0305 04:34:21.472426 2259571 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key

	I0305 04:34:22.111626 2259571 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key

	I0305 04:34:22.444981 2259571 kubeadm.go:309] [certs] Generating "sa" key and public key

	I0305 04:34:22.446234 2259571 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"

	I0305 04:34:22.682410 2259571 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file

	I0305 04:34:22.965295 2259571 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file

	I0305 04:34:24.042632 2259571 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file

	I0305 04:34:25.146848 2259571 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file

	I0305 04:34:25.147615 2259571 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

	I0305 04:34:25.150411 2259571 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"

	I0305 04:34:25.152882 2259571 out.go:204]   - Booting up control plane ...

	I0305 04:34:25.152989 2259571 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"

	I0305 04:34:25.153064 2259571 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"

	I0305 04:34:25.153868 2259571 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"

	I0305 04:34:25.165566 2259571 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

	I0305 04:34:25.166576 2259571 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

	I0305 04:34:25.166792 2259571 kubeadm.go:309] [kubelet-start] Starting the kubelet

	I0305 04:34:25.259449 2259571 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s

	I0305 04:34:33.762052 2259571 kubeadm.go:309] [apiclient] All control plane components are healthy after 8.502678 seconds

	I0305 04:34:33.762176 2259571 kubeadm.go:309] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace

	I0305 04:34:33.778654 2259571 kubeadm.go:309] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster

	I0305 04:34:34.307437 2259571 kubeadm.go:309] [upload-certs] Skipping phase. Please see --upload-certs

	I0305 04:34:34.307629 2259571 kubeadm.go:309] [mark-control-plane] Marking the node scheduled-stop-160289 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]

	I0305 04:34:34.820349 2259571 kubeadm.go:309] [bootstrap-token] Using token: 2k933v.i2eh3umskbh9b4cx

	I0305 04:34:34.822363 2259571 out.go:204]   - Configuring RBAC rules ...

	I0305 04:34:34.822486 2259571 kubeadm.go:309] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles

	I0305 04:34:34.830153 2259571 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes

	I0305 04:34:34.838867 2259571 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials

	I0305 04:34:34.847796 2259571 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token

	I0305 04:34:34.853333 2259571 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster

	I0305 04:34:34.857335 2259571 kubeadm.go:309] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace

	I0305 04:34:34.873525 2259571 kubeadm.go:309] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key

	I0305 04:34:35.123650 2259571 kubeadm.go:309] [addons] Applied essential addon: CoreDNS

	I0305 04:34:35.241578 2259571 kubeadm.go:309] [addons] Applied essential addon: kube-proxy

	I0305 04:34:35.241591 2259571 kubeadm.go:309] 

	I0305 04:34:35.241649 2259571 kubeadm.go:309] Your Kubernetes control-plane has initialized successfully!

	I0305 04:34:35.241654 2259571 kubeadm.go:309] 

	I0305 04:34:35.241728 2259571 kubeadm.go:309] To start using your cluster, you need to run the following as a regular user:

	I0305 04:34:35.241733 2259571 kubeadm.go:309] 

	I0305 04:34:35.241758 2259571 kubeadm.go:309]   mkdir -p $HOME/.kube

	I0305 04:34:35.241818 2259571 kubeadm.go:309]   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

	I0305 04:34:35.241866 2259571 kubeadm.go:309]   sudo chown $(id -u):$(id -g) $HOME/.kube/config

	I0305 04:34:35.241870 2259571 kubeadm.go:309] 

	I0305 04:34:35.241921 2259571 kubeadm.go:309] Alternatively, if you are the root user, you can run:

	I0305 04:34:35.241925 2259571 kubeadm.go:309] 

	I0305 04:34:35.241970 2259571 kubeadm.go:309]   export KUBECONFIG=/etc/kubernetes/admin.conf

	I0305 04:34:35.241974 2259571 kubeadm.go:309] 

	I0305 04:34:35.242023 2259571 kubeadm.go:309] You should now deploy a pod network to the cluster.

	I0305 04:34:35.242105 2259571 kubeadm.go:309] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

	I0305 04:34:35.242169 2259571 kubeadm.go:309]   https://kubernetes.io/docs/concepts/cluster-administration/addons/

	I0305 04:34:35.242173 2259571 kubeadm.go:309] 

	I0305 04:34:35.242253 2259571 kubeadm.go:309] You can now join any number of control-plane nodes by copying certificate authorities

	I0305 04:34:35.242326 2259571 kubeadm.go:309] and service account keys on each node and then running the following as root:

	I0305 04:34:35.242330 2259571 kubeadm.go:309] 

	I0305 04:34:35.242409 2259571 kubeadm.go:309]   kubeadm join control-plane.minikube.internal:8443 --token 2k933v.i2eh3umskbh9b4cx \

	I0305 04:34:35.242508 2259571 kubeadm.go:309] 	--discovery-token-ca-cert-hash sha256:cefcd7c0bfd2c452822fa36506aa2c27229f3d47a807a2142de836bb6edf18e7 \

	I0305 04:34:35.242527 2259571 kubeadm.go:309] 	--control-plane 

	I0305 04:34:35.242531 2259571 kubeadm.go:309] 

	I0305 04:34:35.242615 2259571 kubeadm.go:309] Then you can join any number of worker nodes by running the following on each as root:

	I0305 04:34:35.242619 2259571 kubeadm.go:309] 

	I0305 04:34:35.242697 2259571 kubeadm.go:309] kubeadm join control-plane.minikube.internal:8443 --token 2k933v.i2eh3umskbh9b4cx \

	I0305 04:34:35.242794 2259571 kubeadm.go:309] 	--discovery-token-ca-cert-hash sha256:cefcd7c0bfd2c452822fa36506aa2c27229f3d47a807a2142de836bb6edf18e7 

	I0305 04:34:35.246779 2259571 kubeadm.go:309] 	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1055-aws\n", err: exit status 1

	I0305 04:34:35.246887 2259571 kubeadm.go:309] 	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'

	I0305 04:34:35.246903 2259571 cni.go:84] Creating CNI manager for ""

	I0305 04:34:35.246911 2259571 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet

	I0305 04:34:35.249433 2259571 out.go:177] * Configuring CNI (Container Networking Interface) ...

	I0305 04:34:35.251791 2259571 ssh_runner.go:195] Run: stat /opt/cni/bin/portmap

	I0305 04:34:35.270765 2259571 cni.go:182] applying CNI manifest using /var/lib/minikube/binaries/v1.28.4/kubectl ...

	I0305 04:34:35.270776 2259571 ssh_runner.go:362] scp memory --> /var/tmp/minikube/cni.yaml (2438 bytes)

	I0305 04:34:35.333187 2259571 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml

	I0305 04:34:36.317033 2259571 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"

	I0305 04:34:36.317158 2259571 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig

	I0305 04:34:36.317234 2259571 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig label --overwrite nodes scheduled-stop-160289 minikube.k8s.io/updated_at=2024_03_05T04_34_36_0700 minikube.k8s.io/version=v1.32.0 minikube.k8s.io/commit=c10843c2362d401e82b228acf1f340a428845477 minikube.k8s.io/name=scheduled-stop-160289 minikube.k8s.io/primary=true

	I0305 04:34:36.488468 2259571 ops.go:34] apiserver oom_adj: -16

	I0305 04:34:36.488486 2259571 kubeadm.go:1106] duration metric: took 171.384314ms to wait for elevateKubeSystemPrivileges

	W0305 04:34:36.488502 2259571 kubeadm.go:286] apiserver tunnel failed: apiserver port not set

	I0305 04:34:36.488507 2259571 kubeadm.go:393] duration metric: took 19.478758893s to StartCluster

	I0305 04:34:36.488521 2259571 settings.go:142] acquiring lock: {Name:mkcb5c641f18ebe4ce7c5a24601fb23663fb648c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 04:34:36.488584 2259571 settings.go:150] Updating kubeconfig:  /home/jenkins/minikube-integration/17909-2126334/kubeconfig

	I0305 04:34:36.489265 2259571 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17909-2126334/kubeconfig: {Name:mkc7f94e2bde916bb2b18dc0ff5a1ab6ea679ee4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}

	I0305 04:34:36.489466 2259571 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}

	I0305 04:34:36.492149 2259571 out.go:177] * Verifying Kubernetes components...

	I0305 04:34:36.489568 2259571 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"

	I0305 04:34:36.489734 2259571 config.go:182] Loaded profile config "scheduled-stop-160289": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.28.4

	I0305 04:34:36.489743 2259571 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false yakd:false]

	I0305 04:34:36.494766 2259571 addons.go:69] Setting storage-provisioner=true in profile "scheduled-stop-160289"

	I0305 04:34:36.494811 2259571 addons.go:234] Setting addon storage-provisioner=true in "scheduled-stop-160289"

	I0305 04:34:36.494846 2259571 host.go:66] Checking if "scheduled-stop-160289" exists ...

	I0305 04:34:36.495470 2259571 cli_runner.go:164] Run: docker container inspect scheduled-stop-160289 --format={{.State.Status}}

	I0305 04:34:36.497820 2259571 ssh_runner.go:195] Run: sudo systemctl daemon-reload

	I0305 04:34:36.497950 2259571 addons.go:69] Setting default-storageclass=true in profile "scheduled-stop-160289"

	I0305 04:34:36.497983 2259571 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "scheduled-stop-160289"

	I0305 04:34:36.498261 2259571 cli_runner.go:164] Run: docker container inspect scheduled-stop-160289 --format={{.State.Status}}

	I0305 04:34:36.548313 2259571 out.go:177]   - Using image gcr.io/k8s-minikube/storage-provisioner:v5

	I0305 04:34:36.545900 2259571 addons.go:234] Setting addon default-storageclass=true in "scheduled-stop-160289"

	I0305 04:34:36.553563 2259571 host.go:66] Checking if "scheduled-stop-160289" exists ...

	I0305 04:34:36.554043 2259571 cli_runner.go:164] Run: docker container inspect scheduled-stop-160289 --format={{.State.Status}}

	I0305 04:34:36.554234 2259571 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml

	I0305 04:34:36.554242 2259571 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)

	I0305 04:34:36.554276 2259571 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" scheduled-stop-160289

	I0305 04:34:36.585212 2259571 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml

	I0305 04:34:36.585224 2259571 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)

	I0305 04:34:36.585286 2259571 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" scheduled-stop-160289

	I0305 04:34:36.602012 2259571 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35244 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/scheduled-stop-160289/id_rsa Username:docker}

	I0305 04:34:36.623487 2259571 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:35244 SSHKeyPath:/home/jenkins/minikube-integration/17909-2126334/.minikube/machines/scheduled-stop-160289/id_rsa Username:docker}

	I0305 04:34:36.745587 2259571 ssh_runner.go:195] Run: sudo systemctl start kubelet

	I0305 04:34:36.745740 2259571 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed -e '/^        forward . \/etc\/resolv.conf.*/i \        hosts {\n           192.168.76.1 host.minikube.internal\n           fallthrough\n        }' -e '/^        errors *$/i \        log' | sudo /var/lib/minikube/binaries/v1.28.4/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"

	I0305 04:34:36.776707 2259571 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml

	I0305 04:34:36.795547 2259571 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.28.4/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml

	I0305 04:34:37.323536 2259571 start.go:948] {"host.minikube.internal": 192.168.76.1} host record injected into CoreDNS's ConfigMap

	I0305 04:34:37.325613 2259571 api_server.go:52] waiting for apiserver process to appear ...

	I0305 04:34:37.325677 2259571 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*

	I0305 04:34:37.498116 2259571 api_server.go:72] duration metric: took 1.00862513s to wait for apiserver process to appear ...

	I0305 04:34:37.498127 2259571 api_server.go:88] waiting for apiserver healthz status ...

	I0305 04:34:37.498142 2259571 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...

	I0305 04:34:37.515600 2259571 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:

	ok

	I0305 04:34:37.519184 2259571 out.go:177] * Enabled addons: storage-provisioner, default-storageclass

	I0305 04:34:37.517411 2259571 api_server.go:141] control plane version: v1.28.4

	I0305 04:34:37.520919 2259571 addons.go:505] duration metric: took 1.031172693s for enable addons: enabled=[storage-provisioner default-storageclass]

	I0305 04:34:37.520939 2259571 api_server.go:131] duration metric: took 22.76247ms to wait for apiserver health ...

	I0305 04:34:37.520946 2259571 system_pods.go:43] waiting for kube-system pods to appear ...

	I0305 04:34:37.527930 2259571 system_pods.go:59] 5 kube-system pods found

	I0305 04:34:37.527950 2259571 system_pods.go:61] "etcd-scheduled-stop-160289" [8d160e43-aae4-477e-9660-b80541bc5895] Running / Ready:ContainersNotReady (containers with unready status: [etcd]) / ContainersReady:ContainersNotReady (containers with unready status: [etcd])

	I0305 04:34:37.527994 2259571 system_pods.go:61] "kube-apiserver-scheduled-stop-160289" [92a7ee75-9b75-486b-b153-5da1a3bd7b54] Running / Ready:ContainersNotReady (containers with unready status: [kube-apiserver]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-apiserver])

	I0305 04:34:37.528003 2259571 system_pods.go:61] "kube-controller-manager-scheduled-stop-160289" [127c01d3-9972-4626-92db-f2b5a996c540] Running / Ready:ContainersNotReady (containers with unready status: [kube-controller-manager]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-controller-manager])

	I0305 04:34:37.528009 2259571 system_pods.go:61] "kube-scheduler-scheduled-stop-160289" [82644b1a-390c-4b9f-bcb2-dc8ecfa8f1c3] Running / Ready:ContainersNotReady (containers with unready status: [kube-scheduler]) / ContainersReady:ContainersNotReady (containers with unready status: [kube-scheduler])

	I0305 04:34:37.528015 2259571 system_pods.go:61] "storage-provisioner" [dec33d97-2166-4f7b-a2af-05c28a6fc2f7] Pending: PodScheduled:Unschedulable (0/1 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..)

	I0305 04:34:37.528021 2259571 system_pods.go:74] duration metric: took 7.069566ms to wait for pod list to return data ...

	I0305 04:34:37.528031 2259571 kubeadm.go:576] duration metric: took 1.03854574s to wait for: map[apiserver:true system_pods:true]

	I0305 04:34:37.528044 2259571 node_conditions.go:102] verifying NodePressure condition ...

	I0305 04:34:37.531642 2259571 node_conditions.go:122] node storage ephemeral capacity is 203034800Ki

	I0305 04:34:37.531661 2259571 node_conditions.go:123] node cpu capacity is 2

	I0305 04:34:37.531672 2259571 node_conditions.go:105] duration metric: took 3.623443ms to run NodePressure ...

	I0305 04:34:37.531684 2259571 start.go:240] waiting for startup goroutines ...

	I0305 04:34:37.828108 2259571 kapi.go:248] "coredns" deployment in "kube-system" namespace and "scheduled-stop-160289" context rescaled to 1 replicas

	I0305 04:34:37.828141 2259571 start.go:245] waiting for cluster config update ...

	I0305 04:34:37.828151 2259571 start.go:254] writing updated cluster config ...

	I0305 04:34:37.828432 2259571 ssh_runner.go:195] Run: rm -f paused

	I0305 04:34:37.888957 2259571 start.go:600] kubectl: 1.29.2, cluster: 1.28.4 (minor skew: 1)

	I0305 04:34:37.891305 2259571 out.go:177] * Done! kubectl is now configured to use "scheduled-stop-160289" cluster and "default" namespace by default

	==> CRI-O <==

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.639048667Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:05c284c929889d88306fdb3dd14ee2d0132543740f9e247685243214fc3d2c54,RepoTags:[registry.k8s.io/kube-scheduler:v1.28.4],RepoDigests:[registry.k8s.io/kube-scheduler@sha256:335bba9e861b88fa8b7bb9250bcd69b7a33f83da4fee93f9fc0eedc6f34e28ba registry.k8s.io/kube-scheduler@sha256:ddb0fb05335238789e9a847f0c6731e1da918c42a389625cb2a7ec577ca20afe],Size_:59253556,Uid:&Int64Value{Value:0,},Username:,Spec:nil,},Info:map[string]string{},}" id=b28aaace-4f2e-4cc1-a1b1-67ed74cbc7ad name=/runtime.v1.ImageService/ImageStatus

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.641101655Z" level=info msg="Checking image status: registry.k8s.io/kube-scheduler:v1.28.4" id=6c7f97d8-0f1b-48f5-8038-94a4ff5e2630 name=/runtime.v1.ImageService/ImageStatus

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.641287008Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:05c284c929889d88306fdb3dd14ee2d0132543740f9e247685243214fc3d2c54,RepoTags:[registry.k8s.io/kube-scheduler:v1.28.4],RepoDigests:[registry.k8s.io/kube-scheduler@sha256:335bba9e861b88fa8b7bb9250bcd69b7a33f83da4fee93f9fc0eedc6f34e28ba registry.k8s.io/kube-scheduler@sha256:ddb0fb05335238789e9a847f0c6731e1da918c42a389625cb2a7ec577ca20afe],Size_:59253556,Uid:&Int64Value{Value:0,},Username:,Spec:nil,},Info:map[string]string{},}" id=6c7f97d8-0f1b-48f5-8038-94a4ff5e2630 name=/runtime.v1.ImageService/ImageStatus

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.641379233Z" level=info msg="Checking image status: registry.k8s.io/kube-controller-manager:v1.28.4" id=45b5057f-4dc9-4736-bfc1-beca67487b2b name=/runtime.v1.ImageService/ImageStatus

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.641506018Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:9961cbceaf234d59b7dcf8a197a024f3e3ce4b7fe2b67c2378efd3d209ca994b,RepoTags:[registry.k8s.io/kube-controller-manager:v1.28.4],RepoDigests:[registry.k8s.io/kube-controller-manager@sha256:65486c8c338f96dc022dd1a0abe8763e38f35095b84b208c78f44d9e99447d1c registry.k8s.io/kube-controller-manager@sha256:fe49ea386d014cbf10cd16f53900b91bb7e7c32c5cf4fdcea550c33bedfdc92e],Size_:117252916,Uid:&Int64Value{Value:0,},Username:,Spec:nil,},Info:map[string]string{},}" id=45b5057f-4dc9-4736-bfc1-beca67487b2b name=/runtime.v1.ImageService/ImageStatus

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.643464041Z" level=info msg="Creating container: kube-system/etcd-scheduled-stop-160289/etcd" id=ad22d1fe-17b4-4766-bf6d-b8166e081e87 name=/runtime.v1.RuntimeService/CreateContainer

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.643557899Z" level=warning msg="Allowed annotations are specified for workload []"

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.647054229Z" level=info msg="Creating container: kube-system/kube-scheduler-scheduled-stop-160289/kube-scheduler" id=54a2b6af-05ee-411e-9d5d-e7da9887b1c2 name=/runtime.v1.RuntimeService/CreateContainer

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.647397899Z" level=warning msg="Allowed annotations are specified for workload []"

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.647290855Z" level=info msg="Creating container: kube-system/kube-controller-manager-scheduled-stop-160289/kube-controller-manager" id=5e36ae69-663f-44cc-b3c6-bf6a4af8e42e name=/runtime.v1.RuntimeService/CreateContainer

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.649921622Z" level=warning msg="Allowed annotations are specified for workload []"

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.783661860Z" level=info msg="Created container 9892b0fc899b95f59e5f8d073007c1d1a870ccb2f0912f83e6561857405bce58: kube-system/kube-scheduler-scheduled-stop-160289/kube-scheduler" id=54a2b6af-05ee-411e-9d5d-e7da9887b1c2 name=/runtime.v1.RuntimeService/CreateContainer

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.784782429Z" level=info msg="Starting container: 9892b0fc899b95f59e5f8d073007c1d1a870ccb2f0912f83e6561857405bce58" id=1a067478-48c4-4c38-b578-cbd704ab43ab name=/runtime.v1.RuntimeService/StartContainer

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.793991662Z" level=info msg="Started container" PID=1406 containerID=9892b0fc899b95f59e5f8d073007c1d1a870ccb2f0912f83e6561857405bce58 description=kube-system/kube-scheduler-scheduled-stop-160289/kube-scheduler id=1a067478-48c4-4c38-b578-cbd704ab43ab name=/runtime.v1.RuntimeService/StartContainer sandboxID=d25563adf29f7d7014a9faf2ca55aa0025b6002dbc506bf50f5d38972b87bbb1

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.808617331Z" level=info msg="Created container ba7c079dee11d413dd6d39b0cd9788ebb25bf334802b0e03d99525e918a15c2b: kube-system/kube-controller-manager-scheduled-stop-160289/kube-controller-manager" id=5e36ae69-663f-44cc-b3c6-bf6a4af8e42e name=/runtime.v1.RuntimeService/CreateContainer

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.809384335Z" level=info msg="Created container 8faab4d13465f85cf8cb933affb35d3a10b803037e35c3615fb163a5a9166258: kube-system/kube-apiserver-scheduled-stop-160289/kube-apiserver" id=66f5069f-f0de-4c00-80da-0a30ea0f5aa2 name=/runtime.v1.RuntimeService/CreateContainer

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.810585970Z" level=info msg="Starting container: ba7c079dee11d413dd6d39b0cd9788ebb25bf334802b0e03d99525e918a15c2b" id=9a878d18-c8ae-4182-a45b-327259006f72 name=/runtime.v1.RuntimeService/StartContainer

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.810767106Z" level=info msg="Starting container: 8faab4d13465f85cf8cb933affb35d3a10b803037e35c3615fb163a5a9166258" id=e1991285-9036-4a66-b4a0-6b4f68975b78 name=/runtime.v1.RuntimeService/StartContainer

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.824713016Z" level=info msg="Started container" PID=1350 containerID=8faab4d13465f85cf8cb933affb35d3a10b803037e35c3615fb163a5a9166258 description=kube-system/kube-apiserver-scheduled-stop-160289/kube-apiserver id=e1991285-9036-4a66-b4a0-6b4f68975b78 name=/runtime.v1.RuntimeService/StartContainer sandboxID=d01bfa5f4b1d52eb7b61f4595df9157fd06303b1b02e172bfdca9c3c149954d9

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.827112718Z" level=info msg="Started container" PID=1396 containerID=ba7c079dee11d413dd6d39b0cd9788ebb25bf334802b0e03d99525e918a15c2b description=kube-system/kube-controller-manager-scheduled-stop-160289/kube-controller-manager id=9a878d18-c8ae-4182-a45b-327259006f72 name=/runtime.v1.RuntimeService/StartContainer sandboxID=3cb4632df5d629c2f87c9b30b95713ef7000d0ea880f3a9ee023d277e2f95408

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.829868136Z" level=info msg="Created container 4e20765eaefe8f92a8005ab6d74c233cd1571b759f3a33f0eac46a6439c6ef1a: kube-system/etcd-scheduled-stop-160289/etcd" id=ad22d1fe-17b4-4766-bf6d-b8166e081e87 name=/runtime.v1.RuntimeService/CreateContainer

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.839554446Z" level=info msg="Starting container: 4e20765eaefe8f92a8005ab6d74c233cd1571b759f3a33f0eac46a6439c6ef1a" id=e28f6a74-a581-42bf-9792-d4518b88b75c name=/runtime.v1.RuntimeService/StartContainer

	Mar 05 04:34:27 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:27.857928374Z" level=info msg="Started container" PID=1382 containerID=4e20765eaefe8f92a8005ab6d74c233cd1571b759f3a33f0eac46a6439c6ef1a description=kube-system/etcd-scheduled-stop-160289/etcd id=e28f6a74-a581-42bf-9792-d4518b88b75c name=/runtime.v1.RuntimeService/StartContainer sandboxID=94f264d8fb8e45185907f90dc5455a6dc672ccfed9ffe16a51d155b294e8f9d1

	Mar 05 04:34:35 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:35.188759827Z" level=info msg="Checking image status: registry.k8s.io/pause:3.9" id=ac26d885-f4be-493f-86be-6c8af91af1aa name=/runtime.v1.ImageService/ImageStatus

	Mar 05 04:34:35 scheduled-stop-160289 crio[912]: time="2024-03-05 04:34:35.188931329Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:829e9de338bd5fdd3f16f68f83a9fb288fbc8453e881e5d5cfd0f6f2ff72b43e,RepoTags:[registry.k8s.io/pause:3.9],RepoDigests:[registry.k8s.io/pause@sha256:3ec98b8452dc8ae265a6917dfb81587ac78849e520d5dbba6de524851d20eca6 registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097],Size_:520014,Uid:&Int64Value{Value:65535,},Username:,Spec:nil,},Info:map[string]string{},}" id=ac26d885-f4be-493f-86be-6c8af91af1aa name=/runtime.v1.ImageService/ImageStatus

	==> container status <==

	CONTAINER           IMAGE                                                              CREATED             STATE               NAME                      ATTEMPT             POD ID              POD

	9892b0fc899b9       05c284c929889d88306fdb3dd14ee2d0132543740f9e247685243214fc3d2c54   11 seconds ago      Running             kube-scheduler            0                   d25563adf29f7       kube-scheduler-scheduled-stop-160289

	ba7c079dee11d       9961cbceaf234d59b7dcf8a197a024f3e3ce4b7fe2b67c2378efd3d209ca994b   11 seconds ago      Running             kube-controller-manager   0                   3cb4632df5d62       kube-controller-manager-scheduled-stop-160289

	4e20765eaefe8       9cdd6470f48c8b127530b7ce6ea4b3524137984481e48bcde619735890840ace   11 seconds ago      Running             etcd                      0                   94f264d8fb8e4       etcd-scheduled-stop-160289

	8faab4d13465f       04b4c447bb9d4840af3bf7e836397379d65df87c86e55dcd27f31a8d11df2419   11 seconds ago      Running             kube-apiserver            0                   d01bfa5f4b1d5       kube-apiserver-scheduled-stop-160289

	==> describe nodes <==

	Name:               scheduled-stop-160289

	Roles:              control-plane

	Labels:             beta.kubernetes.io/arch=arm64

	                    beta.kubernetes.io/os=linux

	                    kubernetes.io/arch=arm64

	                    kubernetes.io/hostname=scheduled-stop-160289

	                    kubernetes.io/os=linux

	                    minikube.k8s.io/commit=c10843c2362d401e82b228acf1f340a428845477

	                    minikube.k8s.io/name=scheduled-stop-160289

	                    minikube.k8s.io/primary=true

	                    minikube.k8s.io/updated_at=2024_03_05T04_34_36_0700

	                    minikube.k8s.io/version=v1.32.0

	                    node-role.kubernetes.io/control-plane=

	                    node.kubernetes.io/exclude-from-external-load-balancers=

	Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/crio/crio.sock

	                    volumes.kubernetes.io/controller-managed-attach-detach: true

	CreationTimestamp:  Tue, 05 Mar 2024 04:34:32 +0000

	Taints:             node.kubernetes.io/not-ready:NoSchedule

	Unschedulable:      false

	Lease:

	  HolderIdentity:  scheduled-stop-160289

	  AcquireTime:     <unset>

	  RenewTime:       Tue, 05 Mar 2024 04:34:35 +0000

	Conditions:

	  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message

	  ----             ------  -----------------                 ------------------                ------                       -------

	  MemoryPressure   False   Tue, 05 Mar 2024 04:34:35 +0000   Tue, 05 Mar 2024 04:34:28 +0000   KubeletHasSufficientMemory   kubelet has sufficient memory available

	  DiskPressure     False   Tue, 05 Mar 2024 04:34:35 +0000   Tue, 05 Mar 2024 04:34:28 +0000   KubeletHasNoDiskPressure     kubelet has no disk pressure

	  PIDPressure      False   Tue, 05 Mar 2024 04:34:35 +0000   Tue, 05 Mar 2024 04:34:28 +0000   KubeletHasSufficientPID      kubelet has sufficient PID available

	  Ready            False   Tue, 05 Mar 2024 04:34:35 +0000   Tue, 05 Mar 2024 04:34:28 +0000   KubeletNotReady              [container runtime status check may not have completed yet, container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/cni/net.d/. Has your network provider started?]

	Addresses:

	  InternalIP:  192.168.76.2

	  Hostname:    scheduled-stop-160289

	Capacity:

	  cpu:                2

	  ephemeral-storage:  203034800Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             8022504Ki

	  pods:               110

	Allocatable:

	  cpu:                2

	  ephemeral-storage:  203034800Ki

	  hugepages-1Gi:      0

	  hugepages-2Mi:      0

	  hugepages-32Mi:     0

	  hugepages-64Ki:     0

	  memory:             8022504Ki

	  pods:               110

	System Info:

	  Machine ID:                 baa92a7794b94ac99316fadb1ca861e2

	  System UUID:                31590ced-857a-46c4-82e5-c635fab046ac

	  Boot ID:                    e693da36-93b6-4a73-8932-63823d30425e

	  Kernel Version:             5.15.0-1055-aws

	  OS Image:                   Ubuntu 22.04.3 LTS

	  Operating System:           linux

	  Architecture:               arm64

	  Container Runtime Version:  cri-o://1.24.6

	  Kubelet Version:            v1.28.4

	  Kube-Proxy Version:         v1.28.4

	Non-terminated Pods:          (4 in total)

	  Namespace                   Name                                             CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age

	  ---------                   ----                                             ------------  ----------  ---------------  -------------  ---

	  kube-system                 etcd-scheduled-stop-160289                       100m (5%!)(MISSING)     0 (0%!)(MISSING)      100Mi (1%!)(MISSING)       0 (0%!)(MISSING)         4s

	  kube-system                 kube-apiserver-scheduled-stop-160289             250m (12%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         4s

	  kube-system                 kube-controller-manager-scheduled-stop-160289    200m (10%!)(MISSING)    0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         4s

	  kube-system                 kube-scheduler-scheduled-stop-160289             100m (5%!)(MISSING)     0 (0%!)(MISSING)      0 (0%!)(MISSING)           0 (0%!)(MISSING)         6s

	Allocated resources:

	  (Total limits may be over 100 percent, i.e., overcommitted.)

	  Resource           Requests    Limits

	  --------           --------    ------

	  cpu                650m (32%!)(MISSING)  0 (0%!)(MISSING)

	  memory             100Mi (1%!)(MISSING)  0 (0%!)(MISSING)

	  ephemeral-storage  0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-1Gi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-2Mi      0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-32Mi     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	  hugepages-64Ki     0 (0%!)(MISSING)      0 (0%!)(MISSING)

	Events:

	  Type    Reason                   Age                From     Message

	  ----    ------                   ----               ----     -------

	  Normal  NodeHasSufficientMemory  12s (x8 over 12s)  kubelet  Node scheduled-stop-160289 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    12s (x8 over 12s)  kubelet  Node scheduled-stop-160289 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     12s (x8 over 12s)  kubelet  Node scheduled-stop-160289 status is now: NodeHasSufficientPID

	  Normal  Starting                 4s                 kubelet  Starting kubelet.

	  Normal  NodeHasSufficientMemory  4s                 kubelet  Node scheduled-stop-160289 status is now: NodeHasSufficientMemory

	  Normal  NodeHasNoDiskPressure    4s                 kubelet  Node scheduled-stop-160289 status is now: NodeHasNoDiskPressure

	  Normal  NodeHasSufficientPID     4s                 kubelet  Node scheduled-stop-160289 status is now: NodeHasSufficientPID

	==> dmesg <==

	[  +0.001190] FS-Cache: O-key=[8] '59455c0100000000'

	[  +0.000701] FS-Cache: N-cookie c=0000008a [p=00000081 fl=2 nc=0 na=1]

	[  +0.000934] FS-Cache: N-cookie d=0000000007158bf0{9p.inode} n=00000000c2ddbe15

	[  +0.001150] FS-Cache: N-key=[8] '59455c0100000000'

	[  +0.004363] FS-Cache: Duplicate cookie detected

	[  +0.000737] FS-Cache: O-cookie c=00000084 [p=00000081 fl=226 nc=0 na=1]

	[  +0.001004] FS-Cache: O-cookie d=0000000007158bf0{9p.inode} n=0000000023c83f24

	[  +0.001081] FS-Cache: O-key=[8] '59455c0100000000'

	[  +0.000717] FS-Cache: N-cookie c=0000008b [p=00000081 fl=2 nc=0 na=1]

	[  +0.001081] FS-Cache: N-cookie d=0000000007158bf0{9p.inode} n=000000007544fd9a

	[  +0.001067] FS-Cache: N-key=[8] '59455c0100000000'

	[  +2.604745] FS-Cache: Duplicate cookie detected

	[  +0.000862] FS-Cache: O-cookie c=00000082 [p=00000081 fl=226 nc=0 na=1]

	[  +0.001124] FS-Cache: O-cookie d=0000000007158bf0{9p.inode} n=00000000c8eeb094

	[  +0.001170] FS-Cache: O-key=[8] '58455c0100000000'

	[  +0.000810] FS-Cache: N-cookie c=0000008d [p=00000081 fl=2 nc=0 na=1]

	[  +0.001061] FS-Cache: N-cookie d=0000000007158bf0{9p.inode} n=00000000c2ddbe15

	[  +0.001288] FS-Cache: N-key=[8] '58455c0100000000'

	[  +0.375468] FS-Cache: Duplicate cookie detected

	[  +0.000709] FS-Cache: O-cookie c=00000087 [p=00000081 fl=226 nc=0 na=1]

	[  +0.000996] FS-Cache: O-cookie d=0000000007158bf0{9p.inode} n=00000000a055ac16

	[  +0.001203] FS-Cache: O-key=[8] '5e455c0100000000'

	[  +0.000793] FS-Cache: N-cookie c=0000008e [p=00000081 fl=2 nc=0 na=1]

	[  +0.000948] FS-Cache: N-cookie d=0000000007158bf0{9p.inode} n=00000000eaee5be9

	[  +0.001109] FS-Cache: N-key=[8] '5e455c0100000000'

	==> etcd [4e20765eaefe8f92a8005ab6d74c233cd1571b759f3a33f0eac46a6439c6ef1a] <==

	{"level":"info","ts":"2024-03-05T04:34:28.044211Z","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"/var/lib/minikube/etcd/member/wal","suffix":"wal","max":5,"interval":"30s"}

	{"level":"info","ts":"2024-03-05T04:34:28.044471Z","caller":"embed/etcd.go:597","msg":"serving peer traffic","address":"192.168.76.2:2380"}

	{"level":"info","ts":"2024-03-05T04:34:28.044494Z","caller":"embed/etcd.go:569","msg":"cmux::serve","address":"192.168.76.2:2380"}

	{"level":"info","ts":"2024-03-05T04:34:28.044681Z","caller":"embed/etcd.go:278","msg":"now serving peer/client/metrics","local-member-id":"ea7e25599daad906","initial-advertise-peer-urls":["https://192.168.76.2:2380"],"listen-peer-urls":["https://192.168.76.2:2380"],"advertise-client-urls":["https://192.168.76.2:2379"],"listen-client-urls":["https://127.0.0.1:2379","https://192.168.76.2:2379"],"listen-metrics-urls":["http://127.0.0.1:2381"]}

	{"level":"info","ts":"2024-03-05T04:34:28.044724Z","caller":"embed/etcd.go:855","msg":"serving metrics","address":"http://127.0.0.1:2381"}

	{"level":"info","ts":"2024-03-05T04:34:28.045396Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ea7e25599daad906 switched to configuration voters=(16896983918768216326)"}

	{"level":"info","ts":"2024-03-05T04:34:28.045483Z","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"6f20f2c4b2fb5f8a","local-member-id":"ea7e25599daad906","added-peer-id":"ea7e25599daad906","added-peer-peer-urls":["https://192.168.76.2:2380"]}

	{"level":"info","ts":"2024-03-05T04:34:28.872007Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ea7e25599daad906 is starting a new election at term 1"}

	{"level":"info","ts":"2024-03-05T04:34:28.872068Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ea7e25599daad906 became pre-candidate at term 1"}

	{"level":"info","ts":"2024-03-05T04:34:28.872095Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ea7e25599daad906 received MsgPreVoteResp from ea7e25599daad906 at term 1"}

	{"level":"info","ts":"2024-03-05T04:34:28.87211Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ea7e25599daad906 became candidate at term 2"}

	{"level":"info","ts":"2024-03-05T04:34:28.872116Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ea7e25599daad906 received MsgVoteResp from ea7e25599daad906 at term 2"}

	{"level":"info","ts":"2024-03-05T04:34:28.872126Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ea7e25599daad906 became leader at term 2"}

	{"level":"info","ts":"2024-03-05T04:34:28.872133Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: ea7e25599daad906 elected leader ea7e25599daad906 at term 2"}

	{"level":"info","ts":"2024-03-05T04:34:28.876142Z","caller":"etcdserver/server.go:2062","msg":"published local member to cluster through raft","local-member-id":"ea7e25599daad906","local-member-attributes":"{Name:scheduled-stop-160289 ClientURLs:[https://192.168.76.2:2379]}","request-path":"/0/members/ea7e25599daad906/attributes","cluster-id":"6f20f2c4b2fb5f8a","publish-timeout":"7s"}

	{"level":"info","ts":"2024-03-05T04:34:28.880158Z","caller":"etcdserver/server.go:2571","msg":"setting up initial cluster version using v2 API","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-05T04:34:28.880306Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2024-03-05T04:34:28.881028Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}

	{"level":"info","ts":"2024-03-05T04:34:28.881091Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}

	{"level":"info","ts":"2024-03-05T04:34:28.881155Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"6f20f2c4b2fb5f8a","local-member-id":"ea7e25599daad906","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-05T04:34:28.881251Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-05T04:34:28.8813Z","caller":"etcdserver/server.go:2595","msg":"cluster version is updated","cluster-version":"3.5"}

	{"level":"info","ts":"2024-03-05T04:34:28.881352Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"}

	{"level":"info","ts":"2024-03-05T04:34:28.882239Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"192.168.76.2:2379"}

	{"level":"info","ts":"2024-03-05T04:34:28.883107Z","caller":"embed/serve.go:250","msg":"serving client traffic securely","traffic":"grpc+http","address":"127.0.0.1:2379"}

	==> kernel <==

	 04:34:39 up 10:17,  0 users,  load average: 1.26, 1.66, 2.07

	Linux scheduled-stop-160289 5.15.0-1055-aws #60~20.04.1-Ubuntu SMP Thu Feb 22 15:54:21 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

	PRETTY_NAME="Ubuntu 22.04.3 LTS"

	==> kube-apiserver [8faab4d13465f85cf8cb933affb35d3a10b803037e35c3615fb163a5a9166258] <==

	I0305 04:34:32.215135       1 apf_controller.go:377] Running API Priority and Fairness config worker

	I0305 04:34:32.215167       1 apf_controller.go:380] Running API Priority and Fairness periodic rebalancing process

	I0305 04:34:32.215721       1 shared_informer.go:318] Caches are synced for crd-autoregister

	I0305 04:34:32.215850       1 aggregator.go:166] initial CRD sync complete...

	I0305 04:34:32.215888       1 autoregister_controller.go:141] Starting autoregister controller

	I0305 04:34:32.215940       1 cache.go:32] Waiting for caches to sync for autoregister controller

	I0305 04:34:32.215993       1 cache.go:39] Caches are synced for autoregister controller

	I0305 04:34:32.218192       1 shared_informer.go:318] Caches are synced for configmaps

	I0305 04:34:32.218307       1 shared_informer.go:318] Caches are synced for cluster_authentication_trust_controller

	I0305 04:34:32.222124       1 controller.go:624] quota admission added evaluator for: namespaces

	I0305 04:34:32.234257       1 shared_informer.go:318] Caches are synced for node_authorizer

	I0305 04:34:32.246379       1 controller.go:624] quota admission added evaluator for: leases.coordination.k8s.io

	I0305 04:34:32.919081       1 storage_scheduling.go:95] created PriorityClass system-node-critical with value 2000001000

	I0305 04:34:32.924663       1 storage_scheduling.go:95] created PriorityClass system-cluster-critical with value 2000000000

	I0305 04:34:32.924757       1 storage_scheduling.go:111] all system priority classes are created successfully or already exist.

	I0305 04:34:33.530309       1 controller.go:624] quota admission added evaluator for: roles.rbac.authorization.k8s.io

	I0305 04:34:33.585664       1 controller.go:624] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io

	I0305 04:34:33.638543       1 alloc.go:330] "allocated clusterIPs" service="default/kubernetes" clusterIPs={"IPv4":"10.96.0.1"}

	W0305 04:34:33.644766       1 lease.go:263] Resetting endpoints for master service "kubernetes" to [192.168.76.2]

	I0305 04:34:33.646172       1 controller.go:624] quota admission added evaluator for: endpoints

	I0305 04:34:33.651191       1 controller.go:624] quota admission added evaluator for: endpointslices.discovery.k8s.io

	I0305 04:34:34.134901       1 controller.go:624] quota admission added evaluator for: serviceaccounts

	I0305 04:34:35.106989       1 controller.go:624] quota admission added evaluator for: deployments.apps

	I0305 04:34:35.120997       1 alloc.go:330] "allocated clusterIPs" service="kube-system/kube-dns" clusterIPs={"IPv4":"10.96.0.10"}

	I0305 04:34:35.135665       1 controller.go:624] quota admission added evaluator for: daemonsets.apps

	==> kube-controller-manager [ba7c079dee11d413dd6d39b0cd9788ebb25bf334802b0e03d99525e918a15c2b] <==

	I0305 04:34:36.096387       1 resource_quota_monitor.go:305] "QuotaMonitor running"

	I0305 04:34:36.350011       1 controllermanager.go:642] "Started controller" controller="namespace-controller"

	I0305 04:34:36.350410       1 namespace_controller.go:197] "Starting namespace controller"

	I0305 04:34:36.350526       1 shared_informer.go:311] Waiting for caches to sync for namespace

	I0305 04:34:36.486045       1 controllermanager.go:642] "Started controller" controller="serviceaccount-controller"

	I0305 04:34:36.486120       1 serviceaccounts_controller.go:111] "Starting service account controller"

	I0305 04:34:36.486129       1 shared_informer.go:311] Waiting for caches to sync for service account

	I0305 04:34:36.630271       1 controllermanager.go:642] "Started controller" controller="statefulset-controller"

	I0305 04:34:36.630390       1 stateful_set.go:161] "Starting stateful set controller"

	I0305 04:34:36.630399       1 shared_informer.go:311] Waiting for caches to sync for stateful set

	I0305 04:34:36.782864       1 controllermanager.go:642] "Started controller" controller="persistentvolume-expander-controller"

	I0305 04:34:36.782928       1 expand_controller.go:328] "Starting expand controller"

	I0305 04:34:36.782936       1 shared_informer.go:311] Waiting for caches to sync for expand

	I0305 04:34:36.970680       1 controllermanager.go:642] "Started controller" controller="endpoints-controller"

	I0305 04:34:36.970790       1 endpoints_controller.go:174] "Starting endpoint controller"

	I0305 04:34:36.970798       1 shared_informer.go:311] Waiting for caches to sync for endpoint

	I0305 04:34:37.088721       1 controllermanager.go:642] "Started controller" controller="endpointslice-controller"

	I0305 04:34:37.089048       1 endpointslice_controller.go:264] "Starting endpoint slice controller"

	I0305 04:34:37.089274       1 shared_informer.go:311] Waiting for caches to sync for endpoint_slice

	I0305 04:34:37.230317       1 controllermanager.go:642] "Started controller" controller="replicaset-controller"

	I0305 04:34:37.230512       1 replica_set.go:214] "Starting controller" name="replicaset"

	I0305 04:34:37.230553       1 shared_informer.go:311] Waiting for caches to sync for ReplicaSet

	I0305 04:34:37.278707       1 controllermanager.go:642] "Started controller" controller="certificatesigningrequest-approving-controller"

	I0305 04:34:37.278848       1 certificate_controller.go:115] "Starting certificate controller" name="csrapproving"

	I0305 04:34:37.278948       1 shared_informer.go:311] Waiting for caches to sync for certificate-csrapproving

	==> kube-scheduler [9892b0fc899b95f59e5f8d073007c1d1a870ccb2f0912f83e6561857405bce58] <==

	W0305 04:34:32.181359       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	E0305 04:34:32.181395       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope

	W0305 04:34:32.181490       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	E0305 04:34:32.182043       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope

	W0305 04:34:32.181822       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	E0305 04:34:32.182233       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope

	W0305 04:34:32.181869       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	E0305 04:34:32.182327       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	W0305 04:34:32.181914       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	E0305 04:34:32.182983       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

	W0305 04:34:32.181967       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope

	E0305 04:34:32.183109       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope

	W0305 04:34:32.183259       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	E0305 04:34:32.183392       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope

	W0305 04:34:33.024655       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	E0305 04:34:33.024808       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope

	W0305 04:34:33.053181       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope

	E0305 04:34:33.053299       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope

	W0305 04:34:33.091034       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	E0305 04:34:33.091137       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope

	W0305 04:34:33.148785       1 reflector.go:535] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	E0305 04:34:33.148893       1 reflector.go:147] pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"

	W0305 04:34:33.232895       1 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	E0305 04:34:33.233017       1 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope

	I0305 04:34:34.867832       1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

	==> kubelet <==

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.557317    1523 topology_manager.go:215] "Topology Admit Handler" podUID="6cb8e0aa9448b404d4bfb40eec2c06dd" podNamespace="kube-system" podName="kube-scheduler-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.557358    1523 topology_manager.go:215] "Topology Admit Handler" podUID="fb31bd128fc1b8a4e1a8bdfa64612bc2" podNamespace="kube-system" podName="etcd-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.557399    1523 topology_manager.go:215] "Topology Admit Handler" podUID="37b722ad744cc99e3e8e7a449386ae7a" podNamespace="kube-system" podName="kube-apiserver-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: E0305 04:34:35.602478    1523 kubelet.go:1890] "Failed creating a mirror pod for" err="pods \"kube-scheduler-scheduled-stop-160289\" already exists" pod="kube-system/kube-scheduler-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607307    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"usr-local-share-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/87b44b945224d724aab36a0c9ea8ff90-usr-local-share-ca-certificates\") pod \"kube-controller-manager-scheduled-stop-160289\" (UID: \"87b44b945224d724aab36a0c9ea8ff90\") " pod="kube-system/kube-controller-manager-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607370    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etcd-certs\" (UniqueName: \"kubernetes.io/host-path/fb31bd128fc1b8a4e1a8bdfa64612bc2-etcd-certs\") pod \"etcd-scheduled-stop-160289\" (UID: \"fb31bd128fc1b8a4e1a8bdfa64612bc2\") " pod="kube-system/etcd-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607395    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etcd-data\" (UniqueName: \"kubernetes.io/host-path/fb31bd128fc1b8a4e1a8bdfa64612bc2-etcd-data\") pod \"etcd-scheduled-stop-160289\" (UID: \"fb31bd128fc1b8a4e1a8bdfa64612bc2\") " pod="kube-system/etcd-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607425    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"usr-share-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/37b722ad744cc99e3e8e7a449386ae7a-usr-share-ca-certificates\") pod \"kube-apiserver-scheduled-stop-160289\" (UID: \"37b722ad744cc99e3e8e7a449386ae7a\") " pod="kube-system/kube-apiserver-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607448    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"k8s-certs\" (UniqueName: \"kubernetes.io/host-path/37b722ad744cc99e3e8e7a449386ae7a-k8s-certs\") pod \"kube-apiserver-scheduled-stop-160289\" (UID: \"37b722ad744cc99e3e8e7a449386ae7a\") " pod="kube-system/kube-apiserver-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607472    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/6cb8e0aa9448b404d4bfb40eec2c06dd-kubeconfig\") pod \"kube-scheduler-scheduled-stop-160289\" (UID: \"6cb8e0aa9448b404d4bfb40eec2c06dd\") " pod="kube-system/kube-scheduler-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607495    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-certs\" (UniqueName: \"kubernetes.io/host-path/37b722ad744cc99e3e8e7a449386ae7a-ca-certs\") pod \"kube-apiserver-scheduled-stop-160289\" (UID: \"37b722ad744cc99e3e8e7a449386ae7a\") " pod="kube-system/kube-apiserver-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607519    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"k8s-certs\" (UniqueName: \"kubernetes.io/host-path/87b44b945224d724aab36a0c9ea8ff90-k8s-certs\") pod \"kube-controller-manager-scheduled-stop-160289\" (UID: \"87b44b945224d724aab36a0c9ea8ff90\") " pod="kube-system/kube-controller-manager-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607544    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/87b44b945224d724aab36a0c9ea8ff90-kubeconfig\") pod \"kube-controller-manager-scheduled-stop-160289\" (UID: \"87b44b945224d724aab36a0c9ea8ff90\") " pod="kube-system/kube-controller-manager-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607568    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"usr-share-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/87b44b945224d724aab36a0c9ea8ff90-usr-share-ca-certificates\") pod \"kube-controller-manager-scheduled-stop-160289\" (UID: \"87b44b945224d724aab36a0c9ea8ff90\") " pod="kube-system/kube-controller-manager-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607606    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"flexvolume-dir\" (UniqueName: \"kubernetes.io/host-path/87b44b945224d724aab36a0c9ea8ff90-flexvolume-dir\") pod \"kube-controller-manager-scheduled-stop-160289\" (UID: \"87b44b945224d724aab36a0c9ea8ff90\") " pod="kube-system/kube-controller-manager-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607630    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"usr-local-share-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/37b722ad744cc99e3e8e7a449386ae7a-usr-local-share-ca-certificates\") pod \"kube-apiserver-scheduled-stop-160289\" (UID: \"37b722ad744cc99e3e8e7a449386ae7a\") " pod="kube-system/kube-apiserver-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607656    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-certs\" (UniqueName: \"kubernetes.io/host-path/87b44b945224d724aab36a0c9ea8ff90-ca-certs\") pod \"kube-controller-manager-scheduled-stop-160289\" (UID: \"87b44b945224d724aab36a0c9ea8ff90\") " pod="kube-system/kube-controller-manager-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607686    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/87b44b945224d724aab36a0c9ea8ff90-etc-ca-certificates\") pod \"kube-controller-manager-scheduled-stop-160289\" (UID: \"87b44b945224d724aab36a0c9ea8ff90\") " pod="kube-system/kube-controller-manager-scheduled-stop-160289"

	Mar 05 04:34:35 scheduled-stop-160289 kubelet[1523]: I0305 04:34:35.607709    1523 reconciler_common.go:258] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/37b722ad744cc99e3e8e7a449386ae7a-etc-ca-certificates\") pod \"kube-apiserver-scheduled-stop-160289\" (UID: \"37b722ad744cc99e3e8e7a449386ae7a\") " pod="kube-system/kube-apiserver-scheduled-stop-160289"

	Mar 05 04:34:36 scheduled-stop-160289 kubelet[1523]: I0305 04:34:36.169499    1523 apiserver.go:52] "Watching apiserver"

	Mar 05 04:34:36 scheduled-stop-160289 kubelet[1523]: I0305 04:34:36.204979    1523 desired_state_of_world_populator.go:159] "Finished populating initial desired state of world"

	Mar 05 04:34:36 scheduled-stop-160289 kubelet[1523]: I0305 04:34:36.257002    1523 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="kube-system/kube-controller-manager-scheduled-stop-160289" podStartSLOduration=1.256941153 podCreationTimestamp="2024-03-05 04:34:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2024-03-05 04:34:36.241597875 +0000 UTC m=+1.180889069" watchObservedRunningTime="2024-03-05 04:34:36.256941153 +0000 UTC m=+1.196232355"

	Mar 05 04:34:36 scheduled-stop-160289 kubelet[1523]: I0305 04:34:36.270950    1523 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="kube-system/kube-scheduler-scheduled-stop-160289" podStartSLOduration=3.2709079770000002 podCreationTimestamp="2024-03-05 04:34:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2024-03-05 04:34:36.258219999 +0000 UTC m=+1.197511192" watchObservedRunningTime="2024-03-05 04:34:36.270907977 +0000 UTC m=+1.210199171"

	Mar 05 04:34:36 scheduled-stop-160289 kubelet[1523]: I0305 04:34:36.271039    1523 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="kube-system/etcd-scheduled-stop-160289" podStartSLOduration=1.271019739 podCreationTimestamp="2024-03-05 04:34:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2024-03-05 04:34:36.270488721 +0000 UTC m=+1.209779915" watchObservedRunningTime="2024-03-05 04:34:36.271019739 +0000 UTC m=+1.210310949"

	Mar 05 04:34:36 scheduled-stop-160289 kubelet[1523]: I0305 04:34:36.285209    1523 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="kube-system/kube-apiserver-scheduled-stop-160289" podStartSLOduration=1.285169791 podCreationTimestamp="2024-03-05 04:34:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2024-03-05 04:34:36.284682285 +0000 UTC m=+1.223973479" watchObservedRunningTime="2024-03-05 04:34:36.285169791 +0000 UTC m=+1.224460984"

-- /stdout --

helpers_test.go:254: (dbg) Run:  out/minikube-linux-arm64 status --format={{.APIServer}} -p scheduled-stop-160289 -n scheduled-stop-160289

helpers_test.go:261: (dbg) Run:  kubectl --context scheduled-stop-160289 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running

helpers_test.go:272: non-running pods: storage-provisioner

helpers_test.go:274: ======> post-mortem[TestScheduledStopUnix]: describe non-running pods <======

helpers_test.go:277: (dbg) Run:  kubectl --context scheduled-stop-160289 describe pod storage-provisioner

helpers_test.go:277: (dbg) Non-zero exit: kubectl --context scheduled-stop-160289 describe pod storage-provisioner: exit status 1 (88.159473ms)

** stderr ** 

	Error from server (NotFound): pods "storage-provisioner" not found

** /stderr **

helpers_test.go:279: kubectl --context scheduled-stop-160289 describe pod storage-provisioner: exit status 1

helpers_test.go:175: Cleaning up "scheduled-stop-160289" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-linux-arm64 delete -p scheduled-stop-160289

helpers_test.go:178: (dbg) Done: out/minikube-linux-arm64 delete -p scheduled-stop-160289: (1.972231854s)

--- FAIL: TestScheduledStopUnix (37.26s)

=== RUN   TestDownloadOnly/v1.20.0/json-events

aaa_download_only_test.go:81: (dbg) Run:  out/minikube-linux-arm64 start -o=json --download-only -p download-only-565557 --force --alsologtostderr --kubernetes-version=v1.20.0 --container-runtime=crio --driver=docker  --container-runtime=crio

aaa_download_only_test.go:81: (dbg) Done: out/minikube-linux-arm64 start -o=json --download-only -p download-only-565557 --force --alsologtostderr --kubernetes-version=v1.20.0 --container-runtime=crio --driver=docker  --container-runtime=crio: (7.3965449s)

--- PASS: TestDownloadOnly/v1.20.0/json-events (7.40s)

=== RUN   TestDownloadOnly/v1.20.0/preload-exists

--- PASS: TestDownloadOnly/v1.20.0/preload-exists (0.00s)

=== RUN   TestDownloadOnly/v1.20.0/LogsDuration

aaa_download_only_test.go:184: (dbg) Run:  out/minikube-linux-arm64 logs -p download-only-565557

aaa_download_only_test.go:184: (dbg) Non-zero exit: out/minikube-linux-arm64 logs -p download-only-565557: exit status 85 (83.472707ms)

-- stdout --

	==> Audit <==

	|---------|--------------------------------|----------------------|---------|---------|---------------------|----------|

	| Command |              Args              |       Profile        |  User   | Version |     Start Time      | End Time |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|----------|

	| start   | -o=json --download-only        | download-only-565557 | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC |          |

	|         | -p download-only-565557        |                      |         |         |                     |          |

	|         | --force --alsologtostderr      |                      |         |         |                     |          |

	|         | --kubernetes-version=v1.20.0   |                      |         |         |                     |          |

	|         | --container-runtime=crio       |                      |         |         |                     |          |

	|         | --driver=docker                |                      |         |         |                     |          |

	|         | --container-runtime=crio       |                      |         |         |                     |          |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|----------|

	==> Last Start <==

	Log file created at: 2024/03/05 03:57:13

	Running on machine: ip-172-31-31-251

	Binary: Built with gc go1.22.0 for linux/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0305 03:57:13.706622 2131707 out.go:291] Setting OutFile to fd 1 ...

	I0305 03:57:13.706830 2131707 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 03:57:13.706859 2131707 out.go:304] Setting ErrFile to fd 2...

	I0305 03:57:13.706881 2131707 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 03:57:13.707150 2131707 root.go:338] Updating PATH: /home/jenkins/minikube-integration/17909-2126334/.minikube/bin

	W0305 03:57:13.707319 2131707 root.go:314] Error reading config file at /home/jenkins/minikube-integration/17909-2126334/.minikube/config/config.json: open /home/jenkins/minikube-integration/17909-2126334/.minikube/config/config.json: no such file or directory

	I0305 03:57:13.707750 2131707 out.go:298] Setting JSON to true

	I0305 03:57:13.708693 2131707 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":34780,"bootTime":1709576254,"procs":161,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1055-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}

	I0305 03:57:13.708782 2131707 start.go:139] virtualization:  

	I0305 03:57:13.711528 2131707 out.go:97] [download-only-565557] minikube v1.32.0 on Ubuntu 20.04 (arm64)

	W0305 03:57:13.711663 2131707 preload.go:294] Failed to list preload files: open /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball: no such file or directory

	I0305 03:57:13.711705 2131707 notify.go:220] Checking for updates...

	I0305 03:57:13.713219 2131707 out.go:169] MINIKUBE_LOCATION=17909

	I0305 03:57:13.715327 2131707 out.go:169] MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0305 03:57:13.717155 2131707 out.go:169] KUBECONFIG=/home/jenkins/minikube-integration/17909-2126334/kubeconfig

	I0305 03:57:13.718862 2131707 out.go:169] MINIKUBE_HOME=/home/jenkins/minikube-integration/17909-2126334/.minikube

	I0305 03:57:13.720425 2131707 out.go:169] MINIKUBE_BIN=out/minikube-linux-arm64

	W0305 03:57:13.723470 2131707 out.go:267] minikube skips various validations when --force is supplied; this may lead to unexpected behavior

	I0305 03:57:13.723725 2131707 driver.go:392] Setting default libvirt URI to qemu:///system

	I0305 03:57:13.743284 2131707 docker.go:122] docker version: linux-25.0.3:Docker Engine - Community

	I0305 03:57:13.743387 2131707 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0305 03:57:13.817808 2131707 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:28 OomKillDisable:true NGoroutines:56 SystemTime:2024-03-05 03:57:13.808804846 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar

ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro

rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.6]] Warnings:<nil>}}

	I0305 03:57:13.817916 2131707 docker.go:295] overlay module found

	I0305 03:57:13.820201 2131707 out.go:97] Using the docker driver based on user configuration

	I0305 03:57:13.820232 2131707 start.go:297] selected driver: docker

	I0305 03:57:13.820240 2131707 start.go:901] validating driver "docker" against <nil>

	I0305 03:57:13.820347 2131707 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0305 03:57:13.879065 2131707 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:28 OomKillDisable:true NGoroutines:56 SystemTime:2024-03-05 03:57:13.870299972 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar

ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro

rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.6]] Warnings:<nil>}}

	I0305 03:57:13.879245 2131707 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0305 03:57:13.879541 2131707 start_flags.go:393] Using suggested 2200MB memory alloc based on sys=7834MB, container=7834MB

	I0305 03:57:13.879699 2131707 start_flags.go:929] Wait components to verify : map[apiserver:true system_pods:true]

	I0305 03:57:13.881667 2131707 out.go:169] Using Docker driver with root privileges

	I0305 03:57:13.883934 2131707 cni.go:84] Creating CNI manager for ""

	I0305 03:57:13.884030 2131707 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet

	I0305 03:57:13.884049 2131707 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni

	I0305 03:57:13.884141 2131707 start.go:340] cluster config:

	{Name:download-only-565557 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:download-only-565557 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Co

ntainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0305 03:57:13.885862 2131707 out.go:97] Starting "download-only-565557" primary control-plane node in "download-only-565557" cluster

	I0305 03:57:13.885882 2131707 cache.go:121] Beginning downloading kic base image for docker with crio

	I0305 03:57:13.887438 2131707 out.go:97] Pulling base image v0.0.42-1708944392-18244 ...

	I0305 03:57:13.887463 2131707 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio

	I0305 03:57:13.887502 2131707 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local docker daemon

	I0305 03:57:13.901984 2131707 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 to local cache

	I0305 03:57:13.902179 2131707 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local cache directory

	I0305 03:57:13.902277 2131707 image.go:118] Writing gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 to local cache

	I0305 03:57:13.949783 2131707 preload.go:119] Found remote preload: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.20.0/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-arm64.tar.lz4

	I0305 03:57:13.949810 2131707 cache.go:56] Caching tarball of preloaded images

	I0305 03:57:13.950360 2131707 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio

	I0305 03:57:13.952558 2131707 out.go:97] Downloading Kubernetes v1.20.0 preload ...

	I0305 03:57:13.952584 2131707 preload.go:237] getting checksum for preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-arm64.tar.lz4 ...

	I0305 03:57:14.069079 2131707 download.go:107] Downloading: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.20.0/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-arm64.tar.lz4?checksum=md5:59cd2ef07b53f039bfd1761b921f2a02 -> /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-arm64.tar.lz4

	* The control-plane node download-only-565557 host does not exist

	  To start a cluster, run: "minikube start -p download-only-565557"

-- /stdout --

aaa_download_only_test.go:185: minikube logs failed with error: exit status 85

--- PASS: TestDownloadOnly/v1.20.0/LogsDuration (0.08s)

=== RUN   TestDownloadOnly/v1.20.0/DeleteAll

aaa_download_only_test.go:197: (dbg) Run:  out/minikube-linux-arm64 delete --all

--- PASS: TestDownloadOnly/v1.20.0/DeleteAll (0.22s)

=== RUN   TestDownloadOnly/v1.20.0/DeleteAlwaysSucceeds

aaa_download_only_test.go:208: (dbg) Run:  out/minikube-linux-arm64 delete -p download-only-565557

--- PASS: TestDownloadOnly/v1.20.0/DeleteAlwaysSucceeds (0.14s)

=== RUN   TestDownloadOnly/v1.28.4/json-events

aaa_download_only_test.go:81: (dbg) Run:  out/minikube-linux-arm64 start -o=json --download-only -p download-only-407140 --force --alsologtostderr --kubernetes-version=v1.28.4 --container-runtime=crio --driver=docker  --container-runtime=crio

aaa_download_only_test.go:81: (dbg) Done: out/minikube-linux-arm64 start -o=json --download-only -p download-only-407140 --force --alsologtostderr --kubernetes-version=v1.28.4 --container-runtime=crio --driver=docker  --container-runtime=crio: (8.90588092s)

--- PASS: TestDownloadOnly/v1.28.4/json-events (8.91s)

=== RUN   TestDownloadOnly/v1.28.4/preload-exists

--- PASS: TestDownloadOnly/v1.28.4/preload-exists (0.00s)

=== RUN   TestDownloadOnly/v1.28.4/LogsDuration

aaa_download_only_test.go:184: (dbg) Run:  out/minikube-linux-arm64 logs -p download-only-407140

aaa_download_only_test.go:184: (dbg) Non-zero exit: out/minikube-linux-arm64 logs -p download-only-407140: exit status 85 (84.004626ms)

-- stdout --

	==> Audit <==

	|---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| Command |              Args              |       Profile        |  User   | Version |     Start Time      |      End Time       |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| start   | -o=json --download-only        | download-only-565557 | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC |                     |

	|         | -p download-only-565557        |                      |         |         |                     |                     |

	|         | --force --alsologtostderr      |                      |         |         |                     |                     |

	|         | --kubernetes-version=v1.20.0   |                      |         |         |                     |                     |

	|         | --container-runtime=crio       |                      |         |         |                     |                     |

	|         | --driver=docker                |                      |         |         |                     |                     |

	|         | --container-runtime=crio       |                      |         |         |                     |                     |

	| delete  | --all                          | minikube             | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 03:57 UTC |

	| delete  | -p download-only-565557        | download-only-565557 | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 03:57 UTC |

	| start   | -o=json --download-only        | download-only-407140 | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC |                     |

	|         | -p download-only-407140        |                      |         |         |                     |                     |

	|         | --force --alsologtostderr      |                      |         |         |                     |                     |

	|         | --kubernetes-version=v1.28.4   |                      |         |         |                     |                     |

	|         | --container-runtime=crio       |                      |         |         |                     |                     |

	|         | --driver=docker                |                      |         |         |                     |                     |

	|         | --container-runtime=crio       |                      |         |         |                     |                     |

	|---------|--------------------------------|----------------------|---------|---------|---------------------|---------------------|

	==> Last Start <==

	Log file created at: 2024/03/05 03:57:21

	Running on machine: ip-172-31-31-251

	Binary: Built with gc go1.22.0 for linux/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0305 03:57:21.549445 2131865 out.go:291] Setting OutFile to fd 1 ...

	I0305 03:57:21.549599 2131865 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 03:57:21.549610 2131865 out.go:304] Setting ErrFile to fd 2...

	I0305 03:57:21.549615 2131865 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 03:57:21.549842 2131865 root.go:338] Updating PATH: /home/jenkins/minikube-integration/17909-2126334/.minikube/bin

	I0305 03:57:21.550221 2131865 out.go:298] Setting JSON to true

	I0305 03:57:21.551090 2131865 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":34788,"bootTime":1709576254,"procs":158,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1055-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}

	I0305 03:57:21.551155 2131865 start.go:139] virtualization:  

	I0305 03:57:21.553326 2131865 out.go:97] [download-only-407140] minikube v1.32.0 on Ubuntu 20.04 (arm64)

	I0305 03:57:21.555142 2131865 out.go:169] MINIKUBE_LOCATION=17909

	I0305 03:57:21.553581 2131865 notify.go:220] Checking for updates...

	I0305 03:57:21.558769 2131865 out.go:169] MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0305 03:57:21.560548 2131865 out.go:169] KUBECONFIG=/home/jenkins/minikube-integration/17909-2126334/kubeconfig

	I0305 03:57:21.562409 2131865 out.go:169] MINIKUBE_HOME=/home/jenkins/minikube-integration/17909-2126334/.minikube

	I0305 03:57:21.564481 2131865 out.go:169] MINIKUBE_BIN=out/minikube-linux-arm64

	W0305 03:57:21.567699 2131865 out.go:267] minikube skips various validations when --force is supplied; this may lead to unexpected behavior

	I0305 03:57:21.568088 2131865 driver.go:392] Setting default libvirt URI to qemu:///system

	I0305 03:57:21.589523 2131865 docker.go:122] docker version: linux-25.0.3:Docker Engine - Community

	I0305 03:57:21.589623 2131865 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0305 03:57:21.667320 2131865 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:28 OomKillDisable:true NGoroutines:50 SystemTime:2024-03-05 03:57:21.657123818 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar

ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro

rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.6]] Warnings:<nil>}}

	I0305 03:57:21.667425 2131865 docker.go:295] overlay module found

	I0305 03:57:21.669324 2131865 out.go:97] Using the docker driver based on user configuration

	I0305 03:57:21.669350 2131865 start.go:297] selected driver: docker

	I0305 03:57:21.669356 2131865 start.go:901] validating driver "docker" against <nil>

	I0305 03:57:21.669464 2131865 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0305 03:57:21.725236 2131865 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:28 OomKillDisable:true NGoroutines:50 SystemTime:2024-03-05 03:57:21.716457793 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar

ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro

rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.6]] Warnings:<nil>}}

	I0305 03:57:21.725396 2131865 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0305 03:57:21.725697 2131865 start_flags.go:393] Using suggested 2200MB memory alloc based on sys=7834MB, container=7834MB

	I0305 03:57:21.725856 2131865 start_flags.go:929] Wait components to verify : map[apiserver:true system_pods:true]

	I0305 03:57:21.727753 2131865 out.go:169] Using Docker driver with root privileges

	I0305 03:57:21.729225 2131865 cni.go:84] Creating CNI manager for ""

	I0305 03:57:21.729246 2131865 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet

	I0305 03:57:21.729257 2131865 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni

	I0305 03:57:21.729334 2131865 start.go:340] cluster config:

	{Name:download-only-407140 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.28.4 ClusterName:download-only-407140 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Co

ntainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.28.4 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0305 03:57:21.731037 2131865 out.go:97] Starting "download-only-407140" primary control-plane node in "download-only-407140" cluster

	I0305 03:57:21.731060 2131865 cache.go:121] Beginning downloading kic base image for docker with crio

	I0305 03:57:21.732748 2131865 out.go:97] Pulling base image v0.0.42-1708944392-18244 ...

	I0305 03:57:21.732774 2131865 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime crio

	I0305 03:57:21.732812 2131865 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local docker daemon

	I0305 03:57:21.747431 2131865 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 to local cache

	I0305 03:57:21.747563 2131865 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local cache directory

	I0305 03:57:21.747582 2131865 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local cache directory, skipping pull

	I0305 03:57:21.747587 2131865 image.go:105] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 exists in cache, skipping pull

	I0305 03:57:21.747595 2131865 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 as a tarball

	I0305 03:57:21.786117 2131865 preload.go:119] Found remote preload: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.28.4/preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4

	I0305 03:57:21.786155 2131865 cache.go:56] Caching tarball of preloaded images

	I0305 03:57:21.786655 2131865 preload.go:132] Checking if preload exists for k8s version v1.28.4 and runtime crio

	I0305 03:57:21.788578 2131865 out.go:97] Downloading Kubernetes v1.28.4 preload ...

	I0305 03:57:21.788603 2131865 preload.go:237] getting checksum for preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4 ...

	I0305 03:57:21.892974 2131865 download.go:107] Downloading: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.28.4/preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4?checksum=md5:23e2271fd1a7b32f52ce36ae8363c081 -> /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.28.4-cri-o-overlay-arm64.tar.lz4

	* The control-plane node download-only-407140 host does not exist

	  To start a cluster, run: "minikube start -p download-only-407140"

-- /stdout --

aaa_download_only_test.go:185: minikube logs failed with error: exit status 85

--- PASS: TestDownloadOnly/v1.28.4/LogsDuration (0.08s)

=== RUN   TestDownloadOnly/v1.28.4/DeleteAll

aaa_download_only_test.go:197: (dbg) Run:  out/minikube-linux-arm64 delete --all

--- PASS: TestDownloadOnly/v1.28.4/DeleteAll (0.21s)

=== RUN   TestDownloadOnly/v1.28.4/DeleteAlwaysSucceeds

aaa_download_only_test.go:208: (dbg) Run:  out/minikube-linux-arm64 delete -p download-only-407140

--- PASS: TestDownloadOnly/v1.28.4/DeleteAlwaysSucceeds (0.14s)

=== RUN   TestDownloadOnly/v1.29.0-rc.2/json-events

aaa_download_only_test.go:81: (dbg) Run:  out/minikube-linux-arm64 start -o=json --download-only -p download-only-574412 --force --alsologtostderr --kubernetes-version=v1.29.0-rc.2 --container-runtime=crio --driver=docker  --container-runtime=crio

aaa_download_only_test.go:81: (dbg) Done: out/minikube-linux-arm64 start -o=json --download-only -p download-only-574412 --force --alsologtostderr --kubernetes-version=v1.29.0-rc.2 --container-runtime=crio --driver=docker  --container-runtime=crio: (6.745080824s)

--- PASS: TestDownloadOnly/v1.29.0-rc.2/json-events (6.75s)

=== RUN   TestDownloadOnly/v1.29.0-rc.2/preload-exists

--- PASS: TestDownloadOnly/v1.29.0-rc.2/preload-exists (0.00s)

=== RUN   TestDownloadOnly/v1.29.0-rc.2/LogsDuration

aaa_download_only_test.go:184: (dbg) Run:  out/minikube-linux-arm64 logs -p download-only-574412

aaa_download_only_test.go:184: (dbg) Non-zero exit: out/minikube-linux-arm64 logs -p download-only-574412: exit status 85 (171.858409ms)

-- stdout --

	==> Audit <==

	|---------|-----------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| Command |               Args                |       Profile        |  User   | Version |     Start Time      |      End Time       |

	|---------|-----------------------------------|----------------------|---------|---------|---------------------|---------------------|

	| start   | -o=json --download-only           | download-only-565557 | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC |                     |

	|         | -p download-only-565557           |                      |         |         |                     |                     |

	|         | --force --alsologtostderr         |                      |         |         |                     |                     |

	|         | --kubernetes-version=v1.20.0      |                      |         |         |                     |                     |

	|         | --container-runtime=crio          |                      |         |         |                     |                     |

	|         | --driver=docker                   |                      |         |         |                     |                     |

	|         | --container-runtime=crio          |                      |         |         |                     |                     |

	| delete  | --all                             | minikube             | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 03:57 UTC |

	| delete  | -p download-only-565557           | download-only-565557 | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 03:57 UTC |

	| start   | -o=json --download-only           | download-only-407140 | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC |                     |

	|         | -p download-only-407140           |                      |         |         |                     |                     |

	|         | --force --alsologtostderr         |                      |         |         |                     |                     |

	|         | --kubernetes-version=v1.28.4      |                      |         |         |                     |                     |

	|         | --container-runtime=crio          |                      |         |         |                     |                     |

	|         | --driver=docker                   |                      |         |         |                     |                     |

	|         | --container-runtime=crio          |                      |         |         |                     |                     |

	| delete  | --all                             | minikube             | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 03:57 UTC |

	| delete  | -p download-only-407140           | download-only-407140 | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC | 05 Mar 24 03:57 UTC |

	| start   | -o=json --download-only           | download-only-574412 | jenkins | v1.32.0 | 05 Mar 24 03:57 UTC |                     |

	|         | -p download-only-574412           |                      |         |         |                     |                     |

	|         | --force --alsologtostderr         |                      |         |         |                     |                     |

	|         | --kubernetes-version=v1.29.0-rc.2 |                      |         |         |                     |                     |

	|         | --container-runtime=crio          |                      |         |         |                     |                     |

	|         | --driver=docker                   |                      |         |         |                     |                     |

	|         | --container-runtime=crio          |                      |         |         |                     |                     |

	|---------|-----------------------------------|----------------------|---------|---------|---------------------|---------------------|

	==> Last Start <==

	Log file created at: 2024/03/05 03:57:30

	Running on machine: ip-172-31-31-251

	Binary: Built with gc go1.22.0 for linux/arm64

	Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg

	I0305 03:57:30.892697 2132025 out.go:291] Setting OutFile to fd 1 ...

	I0305 03:57:30.892862 2132025 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 03:57:30.892887 2132025 out.go:304] Setting ErrFile to fd 2...

	I0305 03:57:30.892894 2132025 out.go:338] TERM=,COLORTERM=, which probably does not support color

	I0305 03:57:30.893177 2132025 root.go:338] Updating PATH: /home/jenkins/minikube-integration/17909-2126334/.minikube/bin

	I0305 03:57:30.893608 2132025 out.go:298] Setting JSON to true

	I0305 03:57:30.894487 2132025 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":34797,"bootTime":1709576254,"procs":158,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1055-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}

	I0305 03:57:30.894556 2132025 start.go:139] virtualization:  

	I0305 03:57:30.897152 2132025 out.go:97] [download-only-574412] minikube v1.32.0 on Ubuntu 20.04 (arm64)

	I0305 03:57:30.899382 2132025 out.go:169] MINIKUBE_LOCATION=17909

	I0305 03:57:30.897375 2132025 notify.go:220] Checking for updates...

	I0305 03:57:30.903020 2132025 out.go:169] MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true

	I0305 03:57:30.905264 2132025 out.go:169] KUBECONFIG=/home/jenkins/minikube-integration/17909-2126334/kubeconfig

	I0305 03:57:30.907176 2132025 out.go:169] MINIKUBE_HOME=/home/jenkins/minikube-integration/17909-2126334/.minikube

	I0305 03:57:30.909167 2132025 out.go:169] MINIKUBE_BIN=out/minikube-linux-arm64

	W0305 03:57:30.912996 2132025 out.go:267] minikube skips various validations when --force is supplied; this may lead to unexpected behavior

	I0305 03:57:30.913279 2132025 driver.go:392] Setting default libvirt URI to qemu:///system

	I0305 03:57:30.933926 2132025 docker.go:122] docker version: linux-25.0.3:Docker Engine - Community

	I0305 03:57:30.934027 2132025 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0305 03:57:31.007907 2132025 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:28 OomKillDisable:true NGoroutines:49 SystemTime:2024-03-05 03:57:30.996070032 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar

ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro

rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.6]] Warnings:<nil>}}

	I0305 03:57:31.008119 2132025 docker.go:295] overlay module found

	I0305 03:57:31.010186 2132025 out.go:97] Using the docker driver based on user configuration

	I0305 03:57:31.010233 2132025 start.go:297] selected driver: docker

	I0305 03:57:31.010240 2132025 start.go:901] validating driver "docker" against <nil>

	I0305 03:57:31.010361 2132025 cli_runner.go:164] Run: docker system info --format "{{json .}}"

	I0305 03:57:31.067801 2132025 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:28 OomKillDisable:true NGoroutines:49 SystemTime:2024-03-05 03:57:31.058901199 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1055-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar

ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215044096 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:25.0.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro

rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.12.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.24.6]] Warnings:<nil>}}

	I0305 03:57:31.068009 2132025 start_flags.go:310] no existing cluster config was found, will generate one from the flags 

	I0305 03:57:31.068296 2132025 start_flags.go:393] Using suggested 2200MB memory alloc based on sys=7834MB, container=7834MB

	I0305 03:57:31.068459 2132025 start_flags.go:929] Wait components to verify : map[apiserver:true system_pods:true]

	I0305 03:57:31.070664 2132025 out.go:169] Using Docker driver with root privileges

	I0305 03:57:31.072485 2132025 cni.go:84] Creating CNI manager for ""

	I0305 03:57:31.072506 2132025 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet

	I0305 03:57:31.072517 2132025 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni

	I0305 03:57:31.072610 2132025 start.go:340] cluster config:

	{Name:download-only-574412 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.29.0-rc.2 ClusterName:download-only-574412 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.loc

al ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.29.0-rc.2 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}

	I0305 03:57:31.074436 2132025 out.go:97] Starting "download-only-574412" primary control-plane node in "download-only-574412" cluster

	I0305 03:57:31.074456 2132025 cache.go:121] Beginning downloading kic base image for docker with crio

	I0305 03:57:31.076671 2132025 out.go:97] Pulling base image v0.0.42-1708944392-18244 ...

	I0305 03:57:31.076717 2132025 preload.go:132] Checking if preload exists for k8s version v1.29.0-rc.2 and runtime crio

	I0305 03:57:31.076946 2132025 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local docker daemon

	I0305 03:57:31.091868 2132025 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 to local cache

	I0305 03:57:31.092044 2132025 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local cache directory

	I0305 03:57:31.092071 2132025 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 in local cache directory, skipping pull

	I0305 03:57:31.092080 2132025 image.go:105] gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 exists in cache, skipping pull

	I0305 03:57:31.092089 2132025 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.42-1708944392-18244@sha256:8610dac8144c3f59a6cf50871eb10395cea122e148262744231a04c396033b08 as a tarball

	I0305 03:57:31.139312 2132025 preload.go:119] Found remote preload: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.29.0-rc.2/preloaded-images-k8s-v18-v1.29.0-rc.2-cri-o-overlay-arm64.tar.lz4

	I0305 03:57:31.139356 2132025 cache.go:56] Caching tarball of preloaded images

	I0305 03:57:31.139897 2132025 preload.go:132] Checking if preload exists for k8s version v1.29.0-rc.2 and runtime crio

	I0305 03:57:31.141869 2132025 out.go:97] Downloading Kubernetes v1.29.0-rc.2 preload ...

	I0305 03:57:31.141897 2132025 preload.go:237] getting checksum for preloaded-images-k8s-v18-v1.29.0-rc.2-cri-o-overlay-arm64.tar.lz4 ...

	I0305 03:57:31.268832 2132025 download.go:107] Downloading: https://storage.googleapis.com/minikube-preloaded-volume-tarballs/v18/v1.29.0-rc.2/preloaded-images-k8s-v18-v1.29.0-rc.2-cri-o-overlay-arm64.tar.lz4?checksum=md5:9d8119c6fd5c58f71de57a6fdbe27bf3 -> /home/jenkins/minikube-integration/17909-2126334/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.29.0-rc.2-cri-o-overlay-arm64.tar.lz4

	* The control-plane node download-only-574412 host does not exist

	  To start a cluster, run: "minikube start -p download-only-574412"

-- /stdout --

aaa_download_only_test.go:185: minikube logs failed with error: exit status 85

--- PASS: TestDownloadOnly/v1.29.0-rc.2/LogsDuration (0.17s)

=== RUN   TestDownloadOnly/v1.29.0-rc.2/DeleteAll

aaa_download_only_test.go:197: (dbg) Run:  out/minikube-linux-arm64 delete --all

--- PASS: TestDownloadOnly/v1.29.0-rc.2/DeleteAll (0.38s)

=== RUN   TestDownloadOnly/v1.29.0-rc.2/DeleteAlwaysSucceeds

aaa_download_only_test.go:208: (dbg) Run:  out/minikube-linux-arm64 delete -p download-only-574412

--- PASS: TestDownloadOnly/v1.29.0-rc.2/DeleteAlwaysSucceeds (0.26s)

=== RUN   TestBinaryMirror

aaa_download_only_test.go:314: (dbg) Run:  out/minikube-linux-arm64 start --download-only -p binary-mirror-734085 --alsologtostderr --binary-mirror http://127.0.0.1:35021 --driver=docker  --container-runtime=crio

helpers_test.go:175: Cleaning up "binary-mirror-734085" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-linux-arm64 delete -p binary-mirror-734085

--- PASS: TestBinaryMirror (0.57s)

=== RUN   TestAddons/PreSetup/EnablingAddonOnNonExistingCluster

=== PAUSE TestAddons/PreSetup/EnablingAddonOnNonExistingCluster

=== CONT  TestAddons/PreSetup/EnablingAddonOnNonExistingCluster

addons_test.go:928: (dbg) Run:  out/minikube-linux-arm64 addons enable dashboard -p addons-649326

addons_test.go:928: (dbg) Non-zero exit: out/minikube-linux-arm64 addons enable dashboard -p addons-649326: exit status 85 (86.550185ms)

-- stdout --

	* Profile "addons-649326" not found. Run "minikube profile list" to view all profiles.

	  To start a cluster, run: "minikube start -p addons-649326"

-- /stdout --

--- PASS: TestAddons/PreSetup/EnablingAddonOnNonExistingCluster (0.09s)

=== RUN   TestAddons/PreSetup/DisablingAddonOnNonExistingCluster

=== PAUSE TestAddons/PreSetup/DisablingAddonOnNonExistingCluster

=== CONT  TestAddons/PreSetup/DisablingAddonOnNonExistingCluster

addons_test.go:939: (dbg) Run:  out/minikube-linux-arm64 addons disable dashboard -p addons-649326

addons_test.go:939: (dbg) Non-zero exit: out/minikube-linux-arm64 addons disable dashboard -p addons-649326: exit status 85 (74.185432ms)

-- stdout --

	* Profile "addons-649326" not found. Run "minikube profile list" to view all profiles.

	  To start a cluster, run: "minikube start -p addons-649326"

-- /stdout --

--- PASS: TestAddons/PreSetup/DisablingAddonOnNonExistingCluster (0.07s)

=== RUN   TestAddons/Setup

addons_test.go:109: (dbg) Run:  out/minikube-linux-arm64 start -p addons-649326 --wait=true --memory=4000 --alsologtostderr --addons=registry --addons=metrics-server --addons=volumesnapshots --addons=csi-hostpath-driver --addons=gcp-auth --addons=cloud-spanner --addons=inspektor-gadget --addons=storage-provisioner-rancher --addons=nvidia-device-plugin --addons=yakd --driver=docker  --container-runtime=crio --addons=ingress --addons=ingress-dns

addons_test.go:109: (dbg) Done: out/minikube-linux-arm64 start -p addons-649326 --wait=true --memory=4000 --alsologtostderr --addons=registry --addons=metrics-server --addons=volumesnapshots --addons=csi-hostpath-driver --addons=gcp-auth --addons=cloud-spanner --addons=inspektor-gadget --addons=storage-provisioner-rancher --addons=nvidia-device-plugin --addons=yakd --driver=docker  --container-runtime=crio --addons=ingress --addons=ingress-dns: (2m49.651091882s)

--- PASS: TestAddons/Setup (169.65s)

=== RUN   TestAddons/parallel/Registry

=== PAUSE TestAddons/parallel/Registry

=== CONT  TestAddons/parallel/Registry

addons_test.go:330: registry stabilized in 91.908722ms

addons_test.go:332: (dbg) TestAddons/parallel/Registry: waiting 6m0s for pods matching "actual-registry=true" in namespace "kube-system" ...

helpers_test.go:344: "registry-g7rd8" [939c48ab-c821-49f0-ba12-25e9e9903c0c] Running

addons_test.go:332: (dbg) TestAddons/parallel/Registry: actual-registry=true healthy within 5.007479172s

addons_test.go:335: (dbg) TestAddons/parallel/Registry: waiting 10m0s for pods matching "registry-proxy=true" in namespace "kube-system" ...

helpers_test.go:344: "registry-proxy-8xs2x" [5cc5f670-a6f3-41a6-bfd3-90a772af9ca1] Running

addons_test.go:335: (dbg) TestAddons/parallel/Registry: registry-proxy=true healthy within 6.005134647s

addons_test.go:340: (dbg) Run:  kubectl --context addons-649326 delete po -l run=registry-test --now

addons_test.go:345: (dbg) Run:  kubectl --context addons-649326 run --rm registry-test --restart=Never --image=gcr.io/k8s-minikube/busybox -it -- sh -c "wget --spider -S http://registry.kube-system.svc.cluster.local"

addons_test.go:345: (dbg) Done: kubectl --context addons-649326 run --rm registry-test --restart=Never --image=gcr.io/k8s-minikube/busybox -it -- sh -c "wget --spider -S http://registry.kube-system.svc.cluster.local": (4.220445036s)

addons_test.go:359: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 ip

2024/03/05 04:00:45 [DEBUG] GET http://192.168.49.2:5000

addons_test.go:388: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 addons disable registry --alsologtostderr -v=1

--- PASS: TestAddons/parallel/Registry (16.40s)

=== RUN   TestAddons/parallel/InspektorGadget

=== PAUSE TestAddons/parallel/InspektorGadget

=== CONT  TestAddons/parallel/InspektorGadget

addons_test.go:838: (dbg) TestAddons/parallel/InspektorGadget: waiting 8m0s for pods matching "k8s-app=gadget" in namespace "gadget" ...

helpers_test.go:344: "gadget-4sbpz" [37a74c90-760a-481a-9603-7139a64c7a2f] Running / Ready:ContainersNotReady (containers with unready status: [gadget]) / ContainersReady:ContainersNotReady (containers with unready status: [gadget])

addons_test.go:838: (dbg) TestAddons/parallel/InspektorGadget: k8s-app=gadget healthy within 6.005131384s

addons_test.go:841: (dbg) Run:  out/minikube-linux-arm64 addons disable inspektor-gadget -p addons-649326

addons_test.go:841: (dbg) Done: out/minikube-linux-arm64 addons disable inspektor-gadget -p addons-649326: (5.814951825s)

--- PASS: TestAddons/parallel/InspektorGadget (11.82s)

=== RUN   TestAddons/parallel/MetricsServer

=== PAUSE TestAddons/parallel/MetricsServer

=== CONT  TestAddons/parallel/MetricsServer

addons_test.go:407: metrics-server stabilized in 3.61536ms

addons_test.go:409: (dbg) TestAddons/parallel/MetricsServer: waiting 6m0s for pods matching "k8s-app=metrics-server" in namespace "kube-system" ...

helpers_test.go:344: "metrics-server-69cf46c98-7gfhq" [4ecac735-3006-4634-b22e-5bbbd02c53a3] Running

addons_test.go:409: (dbg) TestAddons/parallel/MetricsServer: k8s-app=metrics-server healthy within 5.006181586s

addons_test.go:415: (dbg) Run:  kubectl --context addons-649326 top pods -n kube-system

addons_test.go:432: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 addons disable metrics-server --alsologtostderr -v=1

--- PASS: TestAddons/parallel/MetricsServer (5.76s)

=== RUN   TestAddons/parallel/CSI

=== PAUSE TestAddons/parallel/CSI

=== CONT  TestAddons/parallel/CSI

addons_test.go:561: csi-hostpath-driver pods stabilized in 91.700747ms

addons_test.go:564: (dbg) Run:  kubectl --context addons-649326 create -f testdata/csi-hostpath-driver/pvc.yaml

addons_test.go:569: (dbg) TestAddons/parallel/CSI: waiting 6m0s for pvc "hpvc" in namespace "default" ...

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc -o jsonpath={.status.phase} -n default

addons_test.go:574: (dbg) Run:  kubectl --context addons-649326 create -f testdata/csi-hostpath-driver/pv-pod.yaml

addons_test.go:579: (dbg) TestAddons/parallel/CSI: waiting 6m0s for pods matching "app=task-pv-pod" in namespace "default" ...

helpers_test.go:344: "task-pv-pod" [ab978245-5244-492f-9ef9-54e31c8477eb] Pending

helpers_test.go:344: "task-pv-pod" [ab978245-5244-492f-9ef9-54e31c8477eb] Pending / Ready:ContainersNotReady (containers with unready status: [task-pv-container]) / ContainersReady:ContainersNotReady (containers with unready status: [task-pv-container])

helpers_test.go:344: "task-pv-pod" [ab978245-5244-492f-9ef9-54e31c8477eb] Running

addons_test.go:579: (dbg) TestAddons/parallel/CSI: app=task-pv-pod healthy within 12.011640812s

addons_test.go:584: (dbg) Run:  kubectl --context addons-649326 create -f testdata/csi-hostpath-driver/snapshot.yaml

addons_test.go:589: (dbg) TestAddons/parallel/CSI: waiting 6m0s for volume snapshot "new-snapshot-demo" in namespace "default" ...

helpers_test.go:419: (dbg) Run:  kubectl --context addons-649326 get volumesnapshot new-snapshot-demo -o jsonpath={.status.readyToUse} -n default

helpers_test.go:419: (dbg) Run:  kubectl --context addons-649326 get volumesnapshot new-snapshot-demo -o jsonpath={.status.readyToUse} -n default

addons_test.go:594: (dbg) Run:  kubectl --context addons-649326 delete pod task-pv-pod

addons_test.go:600: (dbg) Run:  kubectl --context addons-649326 delete pvc hpvc

addons_test.go:606: (dbg) Run:  kubectl --context addons-649326 create -f testdata/csi-hostpath-driver/pvc-restore.yaml

addons_test.go:611: (dbg) TestAddons/parallel/CSI: waiting 6m0s for pvc "hpvc-restore" in namespace "default" ...

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc-restore -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc-restore -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc-restore -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc-restore -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc-restore -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc-restore -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc hpvc-restore -o jsonpath={.status.phase} -n default

addons_test.go:616: (dbg) Run:  kubectl --context addons-649326 create -f testdata/csi-hostpath-driver/pv-pod-restore.yaml

addons_test.go:621: (dbg) TestAddons/parallel/CSI: waiting 6m0s for pods matching "app=task-pv-pod-restore" in namespace "default" ...

helpers_test.go:344: "task-pv-pod-restore" [71787a61-fb18-47df-ab87-209c6686c847] Pending

helpers_test.go:344: "task-pv-pod-restore" [71787a61-fb18-47df-ab87-209c6686c847] Pending / Ready:ContainersNotReady (containers with unready status: [task-pv-container]) / ContainersReady:ContainersNotReady (containers with unready status: [task-pv-container])

helpers_test.go:344: "task-pv-pod-restore" [71787a61-fb18-47df-ab87-209c6686c847] Running

addons_test.go:621: (dbg) TestAddons/parallel/CSI: app=task-pv-pod-restore healthy within 8.004472604s

addons_test.go:626: (dbg) Run:  kubectl --context addons-649326 delete pod task-pv-pod-restore

addons_test.go:630: (dbg) Run:  kubectl --context addons-649326 delete pvc hpvc-restore

addons_test.go:634: (dbg) Run:  kubectl --context addons-649326 delete volumesnapshot new-snapshot-demo

addons_test.go:638: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 addons disable csi-hostpath-driver --alsologtostderr -v=1

addons_test.go:638: (dbg) Done: out/minikube-linux-arm64 -p addons-649326 addons disable csi-hostpath-driver --alsologtostderr -v=1: (6.774777399s)

addons_test.go:642: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 addons disable volumesnapshots --alsologtostderr -v=1

--- PASS: TestAddons/parallel/CSI (62.09s)

=== RUN   TestAddons/parallel/Headlamp

=== PAUSE TestAddons/parallel/Headlamp

=== CONT  TestAddons/parallel/Headlamp

addons_test.go:824: (dbg) Run:  out/minikube-linux-arm64 addons enable headlamp -p addons-649326 --alsologtostderr -v=1

addons_test.go:824: (dbg) Done: out/minikube-linux-arm64 addons enable headlamp -p addons-649326 --alsologtostderr -v=1: (1.54990405s)

addons_test.go:829: (dbg) TestAddons/parallel/Headlamp: waiting 8m0s for pods matching "app.kubernetes.io/name=headlamp" in namespace "headlamp" ...

helpers_test.go:344: "headlamp-7ddfbb94ff-9nxrl" [2a8928b9-0303-4338-bce1-fcfb34789293] Pending

helpers_test.go:344: "headlamp-7ddfbb94ff-9nxrl" [2a8928b9-0303-4338-bce1-fcfb34789293] Pending / Ready:ContainersNotReady (containers with unready status: [headlamp]) / ContainersReady:ContainersNotReady (containers with unready status: [headlamp])

helpers_test.go:344: "headlamp-7ddfbb94ff-9nxrl" [2a8928b9-0303-4338-bce1-fcfb34789293] Running

addons_test.go:829: (dbg) TestAddons/parallel/Headlamp: app.kubernetes.io/name=headlamp healthy within 12.003511139s

--- PASS: TestAddons/parallel/Headlamp (13.55s)

=== RUN   TestAddons/parallel/CloudSpanner

=== PAUSE TestAddons/parallel/CloudSpanner

=== CONT  TestAddons/parallel/CloudSpanner

addons_test.go:857: (dbg) TestAddons/parallel/CloudSpanner: waiting 6m0s for pods matching "app=cloud-spanner-emulator" in namespace "default" ...

helpers_test.go:344: "cloud-spanner-emulator-6548d5df46-btl9w" [84f1d969-5dc1-4ade-9e0c-a1cabbdec9b6] Running

addons_test.go:857: (dbg) TestAddons/parallel/CloudSpanner: app=cloud-spanner-emulator healthy within 6.004210763s

addons_test.go:860: (dbg) Run:  out/minikube-linux-arm64 addons disable cloud-spanner -p addons-649326

--- PASS: TestAddons/parallel/CloudSpanner (6.79s)

=== RUN   TestAddons/parallel/LocalPath

=== PAUSE TestAddons/parallel/LocalPath

=== CONT  TestAddons/parallel/LocalPath

addons_test.go:873: (dbg) Run:  kubectl --context addons-649326 apply -f testdata/storage-provisioner-rancher/pvc.yaml

addons_test.go:879: (dbg) Run:  kubectl --context addons-649326 apply -f testdata/storage-provisioner-rancher/pod.yaml

addons_test.go:883: (dbg) TestAddons/parallel/LocalPath: waiting 5m0s for pvc "test-pvc" in namespace "default" ...

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc test-pvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc test-pvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc test-pvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc test-pvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc test-pvc -o jsonpath={.status.phase} -n default

helpers_test.go:394: (dbg) Run:  kubectl --context addons-649326 get pvc test-pvc -o jsonpath={.status.phase} -n default

addons_test.go:886: (dbg) TestAddons/parallel/LocalPath: waiting 3m0s for pods matching "run=test-local-path" in namespace "default" ...

helpers_test.go:344: "test-local-path" [24ad59c5-4e5f-4147-8216-1144470594b8] Pending / Ready:ContainersNotReady (containers with unready status: [busybox]) / ContainersReady:ContainersNotReady (containers with unready status: [busybox])

helpers_test.go:344: "test-local-path" [24ad59c5-4e5f-4147-8216-1144470594b8] Pending: Initialized:PodCompleted / Ready:PodCompleted / ContainersReady:PodCompleted

helpers_test.go:344: "test-local-path" [24ad59c5-4e5f-4147-8216-1144470594b8] Succeeded: Initialized:PodCompleted / Ready:PodCompleted / ContainersReady:PodCompleted

addons_test.go:886: (dbg) TestAddons/parallel/LocalPath: run=test-local-path healthy within 3.003672936s

addons_test.go:891: (dbg) Run:  kubectl --context addons-649326 get pvc test-pvc -o=json

addons_test.go:900: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 ssh "cat /opt/local-path-provisioner/pvc-07b1e0a4-0914-45b3-a2bf-4ae142a30731_default_test-pvc/file1"

addons_test.go:912: (dbg) Run:  kubectl --context addons-649326 delete pod test-local-path

addons_test.go:916: (dbg) Run:  kubectl --context addons-649326 delete pvc test-pvc

addons_test.go:920: (dbg) Run:  out/minikube-linux-arm64 -p addons-649326 addons disable storage-provisioner-rancher --alsologtostderr -v=1

--- PASS: TestAddons/parallel/LocalPath (9.81s)

=== RUN   TestAddons/parallel/NvidiaDevicePlugin

=== PAUSE TestAddons/parallel/NvidiaDevicePlugin

=== CONT  TestAddons/parallel/NvidiaDevicePlugin

addons_test.go:952: (dbg) TestAddons/parallel/NvidiaDevicePlugin: waiting 6m0s for pods matching "name=nvidia-device-plugin-ds" in namespace "kube-system" ...

helpers_test.go:344: "nvidia-device-plugin-daemonset-mjt8p" [9e26fee0-c9a4-43c8-b502-59bd1eb43478] Running

addons_test.go:952: (dbg) TestAddons/parallel/NvidiaDevicePlugin: name=nvidia-device-plugin-ds healthy within 6.019217712s

addons_test.go:955: (dbg) Run:  out/minikube-linux-arm64 addons disable nvidia-device-plugin -p addons-649326

--- PASS: TestAddons/parallel/NvidiaDevicePlugin (6.81s)

=== RUN   TestAddons/parallel/Yakd

=== PAUSE TestAddons/parallel/Yakd

=== CONT  TestAddons/parallel/Yakd

addons_test.go:963: (dbg) TestAddons/parallel/Yakd: waiting 2m0s for pods matching "app.kubernetes.io/name=yakd-dashboard" in namespace "yakd-dashboard" ...

helpers_test.go:344: "yakd-dashboard-9947fc6bf-xjgvd" [4ed4689c-d4ef-43f8-8170-8714530cec4e] Running

addons_test.go:963: (dbg) TestAddons/parallel/Yakd: app.kubernetes.io/name=yakd-dashboard healthy within 6.004624456s

--- PASS: TestAddons/parallel/Yakd (6.01s)

=== RUN   TestAddons/serial/GCPAuth/Namespaces

addons_test.go:650: (dbg) Run:  kubectl --context addons-649326 create ns new-namespace

addons_test.go:664: (dbg) Run:  kubectl --context addons-649326 get secret gcp-auth -n new-namespace

--- PASS: TestAddons/serial/GCPAuth/Namespaces (0.17s)

=== RUN   TestAddons/StoppedEnableDisable

addons_test.go:172: (dbg) Run:  out/minikube-linux-arm64 stop -p addons-649326

addons_test.go:172: (dbg) Done: out/minikube-linux-arm64 stop -p addons-649326: (11.909977978s)

addons_test.go:176: (dbg) Run:  out/minikube-linux-arm64 addons enable dashboard -p addons-649326

addons_test.go:180: (dbg) Run:  out/minikube-linux-arm64 addons disable dashboard -p addons-649326

addons_test.go:185: (dbg) Run:  out/minikube-linux-arm64 addons disable gvisor -p addons-649326

--- PASS: TestAddons/StoppedEnableDisable (12.20s)

=== RUN   TestCertOptions

=== PAUSE TestCertOptions

=== CONT  TestCertOptions

cert_options_test.go:49: (dbg) Run:  out/minikube-linux-arm64 start -p cert-options-557052 --memory=2048 --apiserver-ips=127.0.0.1 --apiserver-ips=192.168.15.15 --apiserver-names=localhost --apiserver-names=www.google.com --apiserver-port=8555 --driver=docker  --container-runtime=crio

E0305 04:43:08.248614 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/functional-908360/client.crt: no such file or directory

cert_options_test.go:49: (dbg) Done: out/minikube-linux-arm64 start -p cert-options-557052 --memory=2048 --apiserver-ips=127.0.0.1 --apiserver-ips=192.168.15.15 --apiserver-names=localhost --apiserver-names=www.google.com --apiserver-port=8555 --driver=docker  --container-runtime=crio: (36.965870065s)

cert_options_test.go:60: (dbg) Run:  out/minikube-linux-arm64 -p cert-options-557052 ssh "openssl x509 -text -noout -in /var/lib/minikube/certs/apiserver.crt"

cert_options_test.go:88: (dbg) Run:  kubectl --context cert-options-557052 config view

cert_options_test.go:100: (dbg) Run:  out/minikube-linux-arm64 ssh -p cert-options-557052 -- "sudo cat /etc/kubernetes/admin.conf"

helpers_test.go:175: Cleaning up "cert-options-557052" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-linux-arm64 delete -p cert-options-557052

helpers_test.go:178: (dbg) Done: out/minikube-linux-arm64 delete -p cert-options-557052: (2.219523266s)

--- PASS: TestCertOptions (40.10s)

=== RUN   TestCertExpiration

=== PAUSE TestCertExpiration

=== CONT  TestCertExpiration

cert_options_test.go:123: (dbg) Run:  out/minikube-linux-arm64 start -p cert-expiration-553546 --memory=2048 --cert-expiration=3m --driver=docker  --container-runtime=crio

cert_options_test.go:123: (dbg) Done: out/minikube-linux-arm64 start -p cert-expiration-553546 --memory=2048 --cert-expiration=3m --driver=docker  --container-runtime=crio: (39.9177246s)

cert_options_test.go:131: (dbg) Run:  out/minikube-linux-arm64 start -p cert-expiration-553546 --memory=2048 --cert-expiration=8760h --driver=docker  --container-runtime=crio

cert_options_test.go:131: (dbg) Done: out/minikube-linux-arm64 start -p cert-expiration-553546 --memory=2048 --cert-expiration=8760h --driver=docker  --container-runtime=crio: (29.097092238s)

helpers_test.go:175: Cleaning up "cert-expiration-553546" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-linux-arm64 delete -p cert-expiration-553546

helpers_test.go:178: (dbg) Done: out/minikube-linux-arm64 delete -p cert-expiration-553546: (2.562799889s)

--- PASS: TestCertExpiration (251.58s)

=== RUN   TestForceSystemdFlag

=== PAUSE TestForceSystemdFlag

=== CONT  TestForceSystemdFlag

docker_test.go:91: (dbg) Run:  out/minikube-linux-arm64 start -p force-systemd-flag-938642 --memory=2048 --force-systemd --alsologtostderr -v=5 --driver=docker  --container-runtime=crio

docker_test.go:91: (dbg) Done: out/minikube-linux-arm64 start -p force-systemd-flag-938642 --memory=2048 --force-systemd --alsologtostderr -v=5 --driver=docker  --container-runtime=crio: (41.187289124s)

docker_test.go:132: (dbg) Run:  out/minikube-linux-arm64 -p force-systemd-flag-938642 ssh "cat /etc/crio/crio.conf.d/02-crio.conf"

helpers_test.go:175: Cleaning up "force-systemd-flag-938642" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-linux-arm64 delete -p force-systemd-flag-938642

helpers_test.go:178: (dbg) Done: out/minikube-linux-arm64 delete -p force-systemd-flag-938642: (2.442982186s)

--- PASS: TestForceSystemdFlag (43.92s)

=== RUN   TestForceSystemdEnv

=== PAUSE TestForceSystemdEnv

=== CONT  TestForceSystemdEnv

docker_test.go:155: (dbg) Run:  out/minikube-linux-arm64 start -p force-systemd-env-740231 --memory=2048 --alsologtostderr -v=5 --driver=docker  --container-runtime=crio

docker_test.go:155: (dbg) Done: out/minikube-linux-arm64 start -p force-systemd-env-740231 --memory=2048 --alsologtostderr -v=5 --driver=docker  --container-runtime=crio: (35.490905806s)

helpers_test.go:175: Cleaning up "force-systemd-env-740231" profile ...

helpers_test.go:178: (dbg) Run:  out/minikube-linux-arm64 delete -p force-systemd-env-740231

helpers_test.go:178: (dbg) Done: out/minikube-linux-arm64 delete -p force-systemd-env-740231: (2.631654424s)

--- PASS: TestForceSystemdEnv (38.12s)

=== RUN   TestErrorSpam/setup

error_spam_test.go:81: (dbg) Run:  out/minikube-linux-arm64 start -p nospam-979440 -n=1 --memory=2250 --wait=false --log_dir=/tmp/nospam-979440 --driver=docker  --container-runtime=crio

E0305 04:05:29.936865 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

E0305 04:05:29.944741 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

E0305 04:05:29.954971 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

E0305 04:05:29.975229 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

E0305 04:05:30.015476 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

E0305 04:05:30.095783 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

E0305 04:05:30.256100 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

E0305 04:05:30.576590 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

error_spam_test.go:81: (dbg) Done: out/minikube-linux-arm64 start -p nospam-979440 -n=1 --memory=2250 --wait=false --log_dir=/tmp/nospam-979440 --driver=docker  --container-runtime=crio: (33.631530232s)

--- PASS: TestErrorSpam/setup (33.63s)

=== RUN   TestErrorSpam/start

error_spam_test.go:216: Cleaning up 1 logfile(s) ...

error_spam_test.go:159: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 start --dry-run

error_spam_test.go:159: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 start --dry-run

E0305 04:05:31.217476 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

error_spam_test.go:182: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 start --dry-run

--- PASS: TestErrorSpam/start (0.80s)

=== RUN   TestErrorSpam/status

error_spam_test.go:216: Cleaning up 0 logfile(s) ...

error_spam_test.go:159: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 status

error_spam_test.go:159: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 status

error_spam_test.go:182: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 status

E0305 04:05:32.498498 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

--- PASS: TestErrorSpam/status (1.06s)

=== RUN   TestErrorSpam/pause

error_spam_test.go:216: Cleaning up 0 logfile(s) ...

error_spam_test.go:159: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 pause

error_spam_test.go:159: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 pause

error_spam_test.go:182: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 pause

--- PASS: TestErrorSpam/pause (1.69s)

=== RUN   TestErrorSpam/unpause

error_spam_test.go:216: Cleaning up 0 logfile(s) ...

error_spam_test.go:159: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 unpause

error_spam_test.go:159: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 unpause

E0305 04:05:35.058770 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

error_spam_test.go:182: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 unpause

--- PASS: TestErrorSpam/unpause (1.79s)

=== RUN   TestErrorSpam/stop

error_spam_test.go:216: Cleaning up 0 logfile(s) ...

error_spam_test.go:159: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 stop

error_spam_test.go:159: (dbg) Done: out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 stop: (1.223374669s)

error_spam_test.go:159: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 stop

error_spam_test.go:182: (dbg) Run:  out/minikube-linux-arm64 -p nospam-979440 --log_dir /tmp/nospam-979440 stop

--- PASS: TestErrorSpam/stop (1.44s)

=== RUN   TestFunctional/serial/CopySyncFile

functional_test.go:1851: local sync path: /home/jenkins/minikube-integration/17909-2126334/.minikube/files/etc/test/nested/copy/2131701/hosts

--- PASS: TestFunctional/serial/CopySyncFile (0.00s)

=== RUN   TestFunctional/serial/StartWithProxy

functional_test.go:2230: (dbg) Run:  out/minikube-linux-arm64 start -p functional-908360 --memory=4000 --apiserver-port=8441 --wait=all --driver=docker  --container-runtime=crio

E0305 04:05:50.420295 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

E0305 04:06:10.901382 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

functional_test.go:2230: (dbg) Done: out/minikube-linux-arm64 start -p functional-908360 --memory=4000 --apiserver-port=8441 --wait=all --driver=docker  --container-runtime=crio: (51.64810197s)

--- PASS: TestFunctional/serial/StartWithProxy (51.65s)

=== RUN   TestFunctional/serial/AuditLog

--- PASS: TestFunctional/serial/AuditLog (0.00s)

=== RUN   TestFunctional/serial/SoftStart

functional_test.go:655: (dbg) Run:  out/minikube-linux-arm64 start -p functional-908360 --alsologtostderr -v=8

E0305 04:06:51.862546 2131701 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/17909-2126334/.minikube/profiles/addons-649326/client.crt: no such file or directory

functional_test.go:655: (dbg) Done: out/minikube-linux-arm64 start -p functional-908360 --alsologtostderr -v=8: (24.427018007s)

functional_test.go:659: soft start took 24.429358611s for "functional-908360" cluster.

--- PASS: TestFunctional/serial/SoftStart (24.43s)

=== RUN   TestFunctional/serial/KubeContext

functional_test.go:677: (dbg) Run:  kubectl config current-context

--- PASS: TestFunctional/serial/KubeContext (0.06s)