wrapper.sh] [INFO] Wrapping Test Command: `bash -c curl -sSL https://kind.sigs.k8s.io/dl/latest/linux-amd64.tgz | tar xvfz - -C "${PATH%%:*}/" && e2e-k8s.sh` wrapper.sh] [INFO] Running in: gcr.io/k8s-staging-test-infra/krte:v20250904-c89b045f57-master wrapper.sh] [INFO] See: https://github.com/kubernetes/test-infra/blob/master/images/krte/wrapper.sh ================================================================================ wrapper.sh] [SETUP] Performing pre-test setup ... wrapper.sh] [SETUP] Docker in Docker enabled, initializing ... net.ipv6.conf.all.disable_ipv6 = 0 net.ipv6.conf.all.forwarding = 1 Starting Docker: docker. wrapper.sh] [SETUP] Waiting for Docker to be ready, sleeping for 1 seconds ... wrapper.sh] [SETUP] Done setting up Docker in Docker. wrapper.sh] [SETUP] Setting SOURCE_DATE_EPOCH for build reproducibility ... wrapper.sh] [SETUP] exported SOURCE_DATE_EPOCH=1757558517 ================================================================================ wrapper.sh] [TEST] Running Test Command: `bash -c curl -sSL https://kind.sigs.k8s.io/dl/latest/linux-amd64.tgz | tar xvfz - -C "${PATH%%:*}/" && e2e-k8s.sh` ... kind e2e-k8s.sh + CLEANED_UP=false + trap signal_handler INT TERM + main + mktemp -d + TMP_DIR=/tmp/tmp.C1AsOONFz5 + export ARTIFACTS=/logs/artifacts + mkdir -p /logs/artifacts + KUBECONFIG=/root/.kube/kind-test-config + export KUBECONFIG + echo exported KUBECONFIG=/root/.kube/kind-test-config exported KUBECONFIG=/root/.kube/kind-test-config + kind version kind v0.31.0-alpha+9e6df7df674c77 go1.24.6 linux/amd64 + build + kind build node-image -v 1 Building using source: "/home/prow/go/src/k8s.io/kubernetes" Starting to build Kubernetes make: Entering directory '/home/prow/go/src/k8s.io/kubernetes' +++ [0911 03:48:13] Verifying Prerequisites.... +++ [0911 03:48:13] Building Docker image kube-build:build-e8dce5e45a-5-v1.34.0-go1.24.6-bullseye.0 +++ [0911 03:48:56] Creating data container kube-build-data-e8dce5e45a-5-v1.34.0-go1.24.6-bullseye.0 +++ [0911 03:48:57] Syncing sources to container +++ [0911 03:49:01] Running build command... +++ [0911 03:49:10] Building go targets for linux/amd64 k8s.io/kubernetes/cmd/kube-apiserver (static) k8s.io/kubernetes/cmd/kube-controller-manager (static) k8s.io/kubernetes/cmd/kube-proxy (static) k8s.io/kubernetes/cmd/kube-scheduler (static) k8s.io/kubernetes/cmd/kubeadm (static) k8s.io/kubernetes/cmd/kubectl (static) k8s.io/kubernetes/cmd/kubelet (non-static) +++ [0911 03:51:09] Syncing out of container +++ [0911 03:51:11] Building images: linux-amd64 +++ [0911 03:51:11] Starting docker build for image: kube-apiserver-amd64 with base registry.k8s.io/build-image/go-runner:v2.4.0-go1.24.6-bookworm.0 +++ [0911 03:51:11] Starting docker build for image: kube-controller-manager-amd64 with base registry.k8s.io/build-image/go-runner:v2.4.0-go1.24.6-bookworm.0 +++ [0911 03:51:11] Starting docker build for image: kube-scheduler-amd64 with base registry.k8s.io/build-image/go-runner:v2.4.0-go1.24.6-bookworm.0 +++ [0911 03:51:11] Starting docker build for image: kube-proxy-amd64 with base registry.k8s.io/build-image/distroless-iptables:v0.7.8 +++ [0911 03:51:11] Starting docker build for image: kubectl-amd64 with base registry.k8s.io/build-image/go-runner:v2.4.0-go1.24.6-bookworm.0 +++ [0911 03:51:15] Deleting docker image registry.k8s.io/kube-proxy-amd64:v1.35.0-alpha.0.618_1f23e8ce7dead4 +++ [0911 03:51:17] Deleting docker image registry.k8s.io/kubectl-amd64:v1.35.0-alpha.0.618_1f23e8ce7dead4 +++ [0911 03:51:17] Deleting docker image registry.k8s.io/kube-scheduler-amd64:v1.35.0-alpha.0.618_1f23e8ce7dead4 +++ [0911 03:51:17] Deleting docker image registry.k8s.io/kube-controller-manager-amd64:v1.35.0-alpha.0.618_1f23e8ce7dead4 +++ [0911 03:51:19] Deleting docker image registry.k8s.io/kube-apiserver-amd64:v1.35.0-alpha.0.618_1f23e8ce7dead4 +++ [0911 03:51:19] Docker builds done make: Leaving directory '/home/prow/go/src/k8s.io/kubernetes' Finished building Kubernetes Building node image ... DEBUG: docker/pull.go:28] Pulling image: docker.io/kindest/base:v20250710-v0.29.0-alpha-25-g9ac770cf for platform linux/amd64 ... Building in container: kind-build-1757562684-888212940 DEBUG: nodeimage/buildcontext.go:184] fixed: registry.k8s.io/kube-apiserver-amd64 -> registry.k8s.io/kube-apiserver DEBUG: nodeimage/buildcontext.go:184] fixed: registry.k8s.io/kube-controller-manager-amd64 -> registry.k8s.io/kube-controller-manager DEBUG: nodeimage/buildcontext.go:184] fixed: registry.k8s.io/kube-proxy-amd64 -> registry.k8s.io/kube-proxy DEBUG: nodeimage/buildcontext.go:184] fixed: registry.k8s.io/kube-scheduler-amd64 -> registry.k8s.io/kube-scheduler DEBUG: nodeimage/buildcontext.go:205] Detected built images: registry.k8s.io/kube-apiserver:v1.35.0-alpha.0.618_1f23e8ce7dead4, registry.k8s.io/kube-controller-manager:v1.35.0-alpha.0.618_1f23e8ce7dead4, registry.k8s.io/kube-proxy:v1.35.0-alpha.0.618_1f23e8ce7dead4, registry.k8s.io/kube-scheduler:v1.35.0-alpha.0.618_1f23e8ce7dead4 registry.k8s.io/kube apiserver amd64:v1. saved application/vnd.oci.image.manifest.v1+json sha256:f237b93788b700d1d3c42f63dfc860b625d6b24898c267f49f7e281726574ea7 Importing elapsed: 1.0 s total: 0.0 B (0.0 B/s) registry.k8s.io/kube controller manager saved application/vnd.oci.image.manifest.v1+json sha256:70522e35199f485a2192fff823778198e8a3e0654ee58d445c13f69718057a4a Importing elapsed: 1.0 s total: 0.0 B (0.0 B/s) registry.k8s.io/kube scheduler amd64:v1. saved application/vnd.oci.image.manifest.v1+json sha256:d81f5df558335a10a6596c5eae79e228b55457db017d70d7e000dd72de183e9b Importing elapsed: 1.0 s total: 0.0 B (0.0 B/s) registry.k8s.io/kube proxy amd64:v1.35.0 saved application/vnd.oci.image.manifest.v1+json sha256:61408077c9982000e7a9fbd14042ec28fd1218af81e510b5d2d6f7bbd8811897 Importing elapsed: 1.1 s total: 0.0 B (0.0 B/s) Image "kindest/node:latest" build completed. + GINKGO_SRC_DIR=vendor/github.com/onsi/ginkgo/v2/ginkgo + [ ! -d vendor/github.com/onsi/ginkgo/v2/ginkgo ] + make all WHAT=cmd/kubectl test/e2e/e2e.test vendor/github.com/onsi/ginkgo/v2/ginkgo +++ [0911 03:51:37] Building go targets for linux/amd64 github.com/onsi/ginkgo/v2/ginkgo (non-static) k8s.io/kubernetes/cmd/kubectl (static) k8s.io/kubernetes/test/e2e/e2e.test (test) + export PATH=/home/prow/go/src/k8s.io/kubernetes/_output/bin:/home/prow/go/bin:/home/prow/go/bin:/usr/local/go/bin:/google-cloud-sdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin + [ -n ] + res=0 + create_cluster + docker run --rm --entrypoint=cat kindest/node:latest /kind/version + KUBE_VERSION=v1.35.0-alpha.0.618+1f23e8ce7dead4 + KIND_CLUSTER_LOG_LEVEL=4 + CLUSTER_LOG_FORMAT= + scheduler_extra_args= "v": "4" + controllerManager_extra_args= "v": "4" + apiServer_extra_args= "v": "4" + [ -n ] + kubelet_extra_args= "v": "4" "container-log-max-files": "10" "container-log-max-size": "100Mi" + KUBELET_LOG_FORMAT= + [ -n ] + feature_gates={"AllAlpha":true,"AllBeta":true,"EventedPLEG":false} + runtime_config={"api/all":"true"} + : + cat + NUM_NODES=2 + KIND_CREATE_ATTEMPTED=true + kind create cluster --image=kindest/node:latest --retain --wait=1m -v=3 --config=/logs/artifacts/kind-config.yaml Creating cluster "kind" ... • Ensuring node image (kindest/node:latest) 🖼 ... DEBUG: docker/images.go:58] Image: kindest/node:latest present locally ✓ Ensuring node image (kindest/node:latest) 🖼 • Preparing nodes 📦 📦 📦 ... ✓ Preparing nodes 📦 📦 📦 • Writing configuration 📜 ... DEBUG: config/config.go:96] Using the following kubeadm config for node kind-worker2: apiServer: certSANs: - localhost - 127.0.0.1 extraArgs: feature-gates: AllAlpha=true,AllBeta=true,EventedPLEG=false runtime-config: api/all=true v: "4" apiVersion: kubeadm.k8s.io/v1beta3 clusterName: kind controlPlaneEndpoint: kind-control-plane:6443 controllerManager: extraArgs: enable-hostpath-provisioner: "true" feature-gates: AllAlpha=true,AllBeta=true,EventedPLEG=false v: "4" kind: ClusterConfiguration kubernetesVersion: v1.35.0-alpha.0.618+1f23e8ce7dead4 networking: podSubnet: 10.244.0.0/16 serviceSubnet: 10.96.0.0/16 scheduler: extraArgs: feature-gates: AllAlpha=true,AllBeta=true,EventedPLEG=false v: "4" --- apiVersion: kubeadm.k8s.io/v1beta3 bootstrapTokens: - token: abcdef.0123456789abcdef kind: InitConfiguration localAPIEndpoint: advertiseAddress: 172.18.0.3 bindPort: 6443 nodeRegistration: criSocket: unix:///run/containerd/containerd.sock kubeletExtraArgs: container-log-max-files: "10" container-log-max-size: 100Mi node-ip: 172.18.0.3 node-labels: "" provider-id: kind://docker/kind/kind-worker2 v: "4" skipPhases: - preflight --- apiVersion: kubeadm.k8s.io/v1beta3 discovery: bootstrapToken: apiServerEndpoint: kind-control-plane:6443 token: abcdef.0123456789abcdef unsafeSkipCAVerification: true kind: JoinConfiguration nodeRegistration: criSocket: unix:///run/containerd/containerd.sock kubeletExtraArgs: container-log-max-files: "10" container-log-max-size: 100Mi node-ip: 172.18.0.3 node-labels: "" provider-id: kind://docker/kind/kind-worker2 v: "4" skipPhases: - preflight --- apiVersion: kubelet.config.k8s.io/v1beta1 cgroupDriver: systemd cgroupRoot: /kubelet evictionHard: imagefs.available: 0% nodefs.available: 0% nodefs.inodesFree: 0% failSwapOn: false featureGates: AllAlpha: true AllBeta: true EventedPLEG: false imageGCHighThresholdPercent: 100 kind: KubeletConfiguration --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 conntrack: maxPerCore: 0 featureGates: AllAlpha: true AllBeta: true EventedPLEG: false iptables: minSyncPeriod: 1s kind: KubeProxyConfiguration mode: iptables DEBUG: config/config.go:96] Using the following kubeadm config for node kind-control-plane: apiServer: certSANs: - localhost - 127.0.0.1 extraArgs: feature-gates: AllAlpha=true,AllBeta=true,EventedPLEG=false runtime-config: api/all=true v: "4" apiVersion: kubeadm.k8s.io/v1beta3 clusterName: kind controlPlaneEndpoint: kind-control-plane:6443 controllerManager: extraArgs: enable-hostpath-provisioner: "true" feature-gates: AllAlpha=true,AllBeta=true,EventedPLEG=false v: "4" kind: ClusterConfiguration kubernetesVersion: v1.35.0-alpha.0.618+1f23e8ce7dead4 networking: podSubnet: 10.244.0.0/16 serviceSubnet: 10.96.0.0/16 scheduler: extraArgs: feature-gates: AllAlpha=true,AllBeta=true,EventedPLEG=false v: "4" --- apiVersion: kubeadm.k8s.io/v1beta3 bootstrapTokens: - token: abcdef.0123456789abcdef kind: InitConfiguration localAPIEndpoint: advertiseAddress: 172.18.0.4 bindPort: 6443 nodeRegistration: criSocket: unix:///run/containerd/containerd.sock kubeletExtraArgs: container-log-max-files: "10" container-log-max-size: 100Mi node-ip: 172.18.0.4 node-labels: "" provider-id: kind://docker/kind/kind-control-plane v: "4" skipPhases: - preflight --- apiVersion: kubeadm.k8s.io/v1beta3 controlPlane: localAPIEndpoint: advertiseAddress: 172.18.0.4 bindPort: 6443 discovery: bootstrapToken: apiServerEndpoint: kind-control-plane:6443 token: abcdef.0123456789abcdef unsafeSkipCAVerification: true kind: JoinConfiguration nodeRegistration: criSocket: unix:///run/containerd/containerd.sock kubeletExtraArgs: container-log-max-files: "10" container-log-max-size: 100Mi node-ip: 172.18.0.4 node-labels: "" provider-id: kind://docker/kind/kind-control-plane v: "4" skipPhases: - preflight --- apiVersion: kubelet.config.k8s.io/v1beta1 cgroupDriver: systemd cgroupRoot: /kubelet evictionHard: imagefs.available: 0% nodefs.available: 0% nodefs.inodesFree: 0% failSwapOn: false featureGates: AllAlpha: true AllBeta: true EventedPLEG: false imageGCHighThresholdPercent: 100 kind: KubeletConfiguration --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 conntrack: maxPerCore: 0 featureGates: AllAlpha: true AllBeta: true EventedPLEG: false iptables: minSyncPeriod: 1s kind: KubeProxyConfiguration mode: iptables DEBUG: config/config.go:96] Using the following kubeadm config for node kind-worker: apiServer: certSANs: - localhost - 127.0.0.1 extraArgs: feature-gates: AllAlpha=true,AllBeta=true,EventedPLEG=false runtime-config: api/all=true v: "4" apiVersion: kubeadm.k8s.io/v1beta3 clusterName: kind controlPlaneEndpoint: kind-control-plane:6443 controllerManager: extraArgs: enable-hostpath-provisioner: "true" feature-gates: AllAlpha=true,AllBeta=true,EventedPLEG=false v: "4" kind: ClusterConfiguration kubernetesVersion: v1.35.0-alpha.0.618+1f23e8ce7dead4 networking: podSubnet: 10.244.0.0/16 serviceSubnet: 10.96.0.0/16 scheduler: extraArgs: feature-gates: AllAlpha=true,AllBeta=true,EventedPLEG=false v: "4" --- apiVersion: kubeadm.k8s.io/v1beta3 bootstrapTokens: - token: abcdef.0123456789abcdef kind: InitConfiguration localAPIEndpoint: advertiseAddress: 172.18.0.2 bindPort: 6443 nodeRegistration: criSocket: unix:///run/containerd/containerd.sock kubeletExtraArgs: container-log-max-files: "10" container-log-max-size: 100Mi node-ip: 172.18.0.2 node-labels: "" provider-id: kind://docker/kind/kind-worker v: "4" skipPhases: - preflight --- apiVersion: kubeadm.k8s.io/v1beta3 discovery: bootstrapToken: apiServerEndpoint: kind-control-plane:6443 token: abcdef.0123456789abcdef unsafeSkipCAVerification: true kind: JoinConfiguration nodeRegistration: criSocket: unix:///run/containerd/containerd.sock kubeletExtraArgs: container-log-max-files: "10" container-log-max-size: 100Mi node-ip: 172.18.0.2 node-labels: "" provider-id: kind://docker/kind/kind-worker v: "4" skipPhases: - preflight --- apiVersion: kubelet.config.k8s.io/v1beta1 cgroupDriver: systemd cgroupRoot: /kubelet evictionHard: imagefs.available: 0% nodefs.available: 0% nodefs.inodesFree: 0% failSwapOn: false featureGates: AllAlpha: true AllBeta: true EventedPLEG: false imageGCHighThresholdPercent: 100 kind: KubeletConfiguration --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 conntrack: maxPerCore: 0 featureGates: AllAlpha: true AllBeta: true EventedPLEG: false iptables: minSyncPeriod: 1s kind: KubeProxyConfiguration mode: iptables ✓ Writing configuration 📜 • Starting control-plane 🕹️ ... DEBUG: kubeadminit/init.go:96] I0911 03:53:31.395149 178 initconfiguration.go:260] loading configuration from "/kind/kubeadm.conf" W0911 03:53:31.395423 178 common.go:100] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "ClusterConfiguration"). Please use 'kubeadm config migrate --old-config old-config-file --new-config new-config-file', which will write the new, similar spec using a newer API version. W0911 03:53:31.395650 178 common.go:100] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "InitConfiguration"). Please use 'kubeadm config migrate --old-config old-config-file --new-config new-config-file', which will write the new, similar spec using a newer API version. W0911 03:53:31.395792 178 common.go:100] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "JoinConfiguration"). Please use 'kubeadm config migrate --old-config old-config-file --new-config new-config-file', which will write the new, similar spec using a newer API version. W0911 03:53:31.395896 178 initconfiguration.go:361] [config] WARNING: Ignored configuration document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=JoinConfiguration I0911 03:53:31.396395 178 common.go:148] WARNING: tolerating control plane version v1.35.0-alpha.0.618+1f23e8ce7dead4 as a pre-release version [init] Using Kubernetes version: v1.35.0-alpha.0.618+1f23e8ce7dead4 [certs] Using certificateDir folder "/etc/kubernetes/pki" I0911 03:53:31.396966 178 certs.go:111] creating a new certificate authority for ca [certs] Generating "ca" certificate and key I0911 03:53:31.525604 178 certs.go:472] validating certificate period for ca certificate [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [kind-control-plane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 172.18.0.4 127.0.0.1] [certs] Generating "apiserver-kubelet-client" certificate and key I0911 03:53:31.806514 178 certs.go:111] creating a new certificate authority for front-proxy-ca [certs] Generating "front-proxy-ca" certificate and key I0911 03:53:31.965533 178 certs.go:472] validating certificate period for front-proxy-ca certificate [certs] Generating "front-proxy-client" certificate and key I0911 03:53:32.147349 178 certs.go:111] creating a new certificate authority for etcd-ca [certs] Generating "etcd/ca" certificate and key I0911 03:53:32.190960 178 certs.go:472] validating certificate period for etcd/ca certificate [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.18.0.4 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.18.0.4 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key I0911 03:53:32.534155 178 certs.go:77] creating new public/private key files for signing service account users [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" I0911 03:53:32.560063 178 kubeconfig.go:111] creating kubeconfig file for admin.conf [kubeconfig] Writing "admin.conf" kubeconfig file I0911 03:53:32.603856 178 kubeconfig.go:111] creating kubeconfig file for super-admin.conf [kubeconfig] Writing "super-admin.conf" kubeconfig file I0911 03:53:32.634273 178 kubeconfig.go:111] creating kubeconfig file for kubelet.conf [kubeconfig] Writing "kubelet.conf" kubeconfig file I0911 03:53:32.854679 178 kubeconfig.go:111] creating kubeconfig file for controller-manager.conf [kubeconfig] Writing "controller-manager.conf" kubeconfig file I0911 03:53:32.963364 178 kubeconfig.go:111] creating kubeconfig file for scheduler.conf [kubeconfig] Writing "scheduler.conf" kubeconfig file [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" I0911 03:53:33.255518 178 images.go:86] WARNING: could not find officially supported version of etcd for Kubernetes v1.35.0-alpha.0.618+1f23e8ce7dead4, falling back to the nearest etcd version (3.6.4-0) I0911 03:53:33.255554 178 images.go:86] WARNING: could not find officially supported version of etcd for Kubernetes v1.35.0-alpha.0.618+1f23e8ce7dead4, falling back to the nearest etcd version (3.6.4-0) I0911 03:53:33.257069 178 local.go:67] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml" I0911 03:53:33.257081 178 manifests.go:125] [control-plane] getting StaticPodSpecs [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" I0911 03:53:33.257197 178 certs.go:472] validating certificate period for CA certificate I0911 03:53:33.257234 178 manifests.go:151] [control-plane] adding volume "ca-certs" for component "kube-apiserver" I0911 03:53:33.257237 178 manifests.go:151] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver" I0911 03:53:33.257238 178 manifests.go:151] [control-plane] adding volume "k8s-certs" for component "kube-apiserver" I0911 03:53:33.257241 178 manifests.go:151] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver" I0911 03:53:33.257247 178 manifests.go:151] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver" I0911 03:53:33.257548 178 manifests.go:180] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml" [control-plane] Creating static Pod manifest for "kube-controller-manager" I0911 03:53:33.257552 178 manifests.go:125] [control-plane] getting StaticPodSpecs I0911 03:53:33.257627 178 manifests.go:151] [control-plane] adding volume "ca-certs" for component "kube-controller-manager" I0911 03:53:33.257630 178 manifests.go:151] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager" I0911 03:53:33.257632 178 manifests.go:151] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager" I0911 03:53:33.257634 178 manifests.go:151] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager" I0911 03:53:33.257635 178 manifests.go:151] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager" I0911 03:53:33.257637 178 manifests.go:151] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager" I0911 03:53:33.257639 178 manifests.go:151] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager" I0911 03:53:33.257911 178 manifests.go:180] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml" I0911 03:53:33.257915 178 manifests.go:125] [control-plane] getting StaticPodSpecs [control-plane] Creating static Pod manifest for "kube-scheduler" I0911 03:53:33.257984 178 manifests.go:151] [control-plane] adding volume "kubeconfig" for component "kube-scheduler" I0911 03:53:33.258169 178 manifests.go:180] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml" I0911 03:53:33.258174 178 kubelet.go:69] Stopping the kubelet [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml" [patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet I0911 03:53:33.328281 178 loader.go:405] Config loaded from file: /etc/kubernetes/admin.conf I0911 03:53:33.328513 178 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false I0911 03:53:33.328521 178 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0911 03:53:33.328523 178 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0911 03:53:33.328529 178 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true I0911 03:53:33.328531 178 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests" [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s [kubelet-check] The kubelet is healthy after 501.213603ms [control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s [control-plane-check] Checking kube-apiserver at https://172.18.0.4:6443/livez [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez I0911 03:53:33.832468 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0 I0911 03:53:34.333288 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0 I0911 03:53:36.057447 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="403 Forbidden" milliseconds=1224 I0911 03:53:36.060353 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="403 Forbidden" milliseconds=2 [control-plane-check] kube-scheduler is healthy after 2.238763777s I0911 03:53:36.333626 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="403 Forbidden" milliseconds=0 I0911 03:53:36.833244 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="403 Forbidden" milliseconds=0 I0911 03:53:37.332802 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="500 Internal Server Error" milliseconds=0 I0911 03:53:37.332939 178 request.go:1500] "Body was not decodable (unable to check for Status)" err="couldn't get version/kind; json parse error: json: cannot unmarshal array into Go value of type struct { APIVersion string \"json:\\\"apiVersion,omitempty\\\"\"; Kind string \"json:\\\"kind,omitempty\\\"\" }" I0911 03:53:37.833213 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="500 Internal Server Error" milliseconds=0 I0911 03:53:37.833346 178 request.go:1500] "Body was not decodable (unable to check for Status)" err="couldn't get version/kind; json parse error: json: cannot unmarshal array into Go value of type struct { APIVersion string \"json:\\\"apiVersion,omitempty\\\"\"; Kind string \"json:\\\"kind,omitempty\\\"\" }" [control-plane-check] kube-controller-manager is healthy after 4.002448412s I0911 03:53:38.333907 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="500 Internal Server Error" milliseconds=0 I0911 03:53:38.334049 178 request.go:1500] "Body was not decodable (unable to check for Status)" err="couldn't get version/kind; json parse error: json: cannot unmarshal array into Go value of type struct { APIVersion string \"json:\\\"apiVersion,omitempty\\\"\"; Kind string \"json:\\\"kind,omitempty\\\"\" }" I0911 03:53:38.833076 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="500 Internal Server Error" milliseconds=0 I0911 03:53:38.833207 178 request.go:1500] "Body was not decodable (unable to check for Status)" err="couldn't get version/kind; json parse error: json: cannot unmarshal array into Go value of type struct { APIVersion string \"json:\\\"apiVersion,omitempty\\\"\"; Kind string \"json:\\\"kind,omitempty\\\"\" }" I0911 03:53:39.333304 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="500 Internal Server Error" milliseconds=0 I0911 03:53:39.333436 178 request.go:1500] "Body was not decodable (unable to check for Status)" err="couldn't get version/kind; json parse error: json: cannot unmarshal array into Go value of type struct { APIVersion string \"json:\\\"apiVersion,omitempty\\\"\"; Kind string \"json:\\\"kind,omitempty\\\"\" }" I0911 03:53:39.833697 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="200 OK" milliseconds=0 [control-plane-check] kube-apiserver is healthy after 6.00164165s I0911 03:53:39.834039 178 loader.go:405] Config loaded from file: /etc/kubernetes/admin.conf I0911 03:53:39.834363 178 loader.go:405] Config loaded from file: /etc/kubernetes/super-admin.conf I0911 03:53:39.834569 178 kubeconfig.go:657] ensuring that the ClusterRoleBinding for the kubeadm:cluster-admins Group exists I0911 03:53:39.835062 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s" status="403 Forbidden" milliseconds=0 I0911 03:53:39.835111 178 kubeconfig.go:730] creating the ClusterRoleBinding for the kubeadm:cluster-admins Group by using super-admin.conf I0911 03:53:39.838839 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s" status="201 Created" milliseconds=3 [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace I0911 03:53:39.838882 178 uploadconfig.go:111] [upload-config] Uploading the kubeadm ClusterConfiguration to a ConfigMap I0911 03:53:39.840341 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:39.841551 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:39.842535 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s" status="201 Created" milliseconds=0 [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster I0911 03:53:39.842576 178 uploadconfig.go:125] [upload-config] Uploading the kubelet component config to a ConfigMap I0911 03:53:39.844406 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:39.845421 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s" status="201 Created" milliseconds=0 I0911 03:53:39.846363 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s" status="201 Created" milliseconds=0 [upload-certs] Skipping phase. Please see --upload-certs [mark-control-plane] Marking the node kind-control-plane as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] [mark-control-plane] Marking the node kind-control-plane as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule] I0911 03:53:39.847171 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/nodes/kind-control-plane?timeout=10s" status="200 OK" milliseconds=0 I0911 03:53:39.849357 178 round_trippers.go:632] "Response" verb="PATCH" url="https://kind-control-plane:6443/api/v1/nodes/kind-control-plane?timeout=10s" status="200 OK" milliseconds=1 I0911 03:53:39.849647 178 loader.go:405] Config loaded from file: /etc/kubernetes/admin.conf [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles I0911 03:53:39.850327 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/secrets/bootstrap-token-abcdef?timeout=10s" status="404 Not Found" milliseconds=0 I0911 03:53:39.851354 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/secrets?timeout=10s" status="201 Created" milliseconds=0 [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes I0911 03:53:39.852280 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s" status="201 Created" milliseconds=0 I0911 03:53:39.853196 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s" status="201 Created" milliseconds=0 [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials I0911 03:53:39.854039 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s" status="201 Created" milliseconds=0 [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token I0911 03:53:39.854869 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s" status="201 Created" milliseconds=0 [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster I0911 03:53:39.855706 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s" status="201 Created" milliseconds=0 [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace I0911 03:53:39.855749 178 clusterinfo.go:58] [bootstrap-token] copying the cluster from admin.conf to the bootstrap kubeconfig I0911 03:53:39.855851 178 clusterinfo.go:70] [bootstrap-token] creating/updating ConfigMap in kube-public namespace I0911 03:53:39.856871 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps?timeout=10s" status="201 Created" milliseconds=0 I0911 03:53:39.856899 178 clusterinfo.go:84] creating the RBAC rules for exposing the cluster-info ConfigMap in the kube-public namespace I0911 03:53:40.035136 178 request.go:683] "Waited before sending request" delay="178.182425ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/roles?timeout=10s" I0911 03:53:40.036249 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/roles?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:40.235504 178 request.go:683] "Waited before sending request" delay="199.162635ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/rolebindings?timeout=10s" I0911 03:53:40.236622 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-public/rolebindings?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:40.236684 178 kubeletfinalize.go:90] [kubelet-finalize] Assuming that kubelet client certificate rotation is enabled: found "/var/lib/kubelet/pki/kubelet-client-current.pem" [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key I0911 03:53:40.236934 178 loader.go:405] Config loaded from file: /etc/kubernetes/kubelet.conf I0911 03:53:40.237135 178 kubeletfinalize.go:144] [kubelet-finalize] Restarting the kubelet to enable client certificate rotation I0911 03:53:40.327783 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/apis/apps/v1/namespaces/kube-system/deployments?labelSelector=k8s-app%3Dkube-dns" status="200 OK" milliseconds=1 I0911 03:53:40.329098 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/coredns?timeout=10s" status="404 Not Found" milliseconds=0 I0911 03:53:40.330656 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:40.435084 178 request.go:683] "Waited before sending request" delay="104.201614ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s" I0911 03:53:40.436424 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterroles?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:40.635008 178 request.go:683] "Waited before sending request" delay="198.291719ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s" I0911 03:53:40.636135 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:40.637233 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s" status="201 Created" milliseconds=0 I0911 03:53:40.639619 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/apps/v1/namespaces/kube-system/deployments?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:40.642016 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/services?timeout=10s" status="201 Created" milliseconds=2 [addons] Applied essential addon: CoreDNS I0911 03:53:40.643677 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:40.645575 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/apps/v1/namespaces/kube-system/daemonsets?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:40.646587 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/serviceaccounts?timeout=10s" status="201 Created" milliseconds=0 I0911 03:53:40.834932 178 request.go:683] "Waited before sending request" delay="188.26579ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s" I0911 03:53:40.835907 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings?timeout=10s" status="201 Created" milliseconds=0 I0911 03:53:41.035301 178 request.go:683] "Waited before sending request" delay="199.29246ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s" I0911 03:53:41.036825 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s" status="201 Created" milliseconds=1 I0911 03:53:41.235140 178 request.go:683] "Waited before sending request" delay="198.17908ms" reason="client-side throttling, not priority and fairness" verb="POST" URL="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s" I0911 03:53:41.236438 178 round_trippers.go:632] "Response" verb="POST" url="https://kind-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s" status="201 Created" milliseconds=1 [addons] Applied essential addon: kube-proxy I0911 03:53:41.236774 178 loader.go:405] Config loaded from file: /etc/kubernetes/admin.conf I0911 03:53:41.237040 178 loader.go:405] Config loaded from file: /etc/kubernetes/admin.conf Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of control-plane nodes by copying certificate authorities and service account keys on each node and then running the following as root: kubeadm join kind-control-plane:6443 --token \ --discovery-token-ca-cert-hash sha256:8880bbe43e7abfda89bf83fc0a0caea1769a72991b82a8f04221e4d4af5db7de \ --control-plane Then you can join any number of worker nodes by running the following on each as root: kubeadm join kind-control-plane:6443 --token \ --discovery-token-ca-cert-hash sha256:8880bbe43e7abfda89bf83fc0a0caea1769a72991b82a8f04221e4d4af5db7de ✓ Starting control-plane 🕹️ • Installing CNI 🔌 ... ✓ Installing CNI 🔌 • Installing StorageClass 💾 ... ✓ Installing StorageClass 💾 • Joining worker nodes 🚜 ... DEBUG: kubeadmjoin/join.go:147] I0911 03:53:41.834976 178 join.go:421] [preflight] found NodeName empty; using OS hostname as NodeName I0911 03:53:41.835023 178 joinconfiguration.go:82] loading configuration from "/kind/kubeadm.conf" W0911 03:53:41.835327 178 joinconfiguration.go:112] [config] WARNING: Ignored configuration document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=ClusterConfiguration W0911 03:53:41.835353 178 joinconfiguration.go:112] [config] WARNING: Ignored configuration document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=InitConfiguration W0911 03:53:41.835359 178 common.go:100] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "JoinConfiguration"). Please use 'kubeadm config migrate --old-config old-config-file --new-config new-config-file', which will write the new, similar spec using a newer API version. W0911 03:53:41.835536 178 joinconfiguration.go:112] [config] WARNING: Ignored configuration document with GroupVersionKind kubelet.config.k8s.io/v1beta1, Kind=KubeletConfiguration W0911 03:53:41.835539 178 joinconfiguration.go:112] [config] WARNING: Ignored configuration document with GroupVersionKind kubeproxy.config.k8s.io/v1alpha1, Kind=KubeProxyConfiguration I0911 03:53:41.835635 178 controlplaneprepare.go:225] [download-certs] Skipping certs download I0911 03:53:41.835642 178 join.go:551] [preflight] Discovering cluster-info I0911 03:53:41.835653 178 token.go:71] [discovery] Created cluster-info discovery client, requesting info from "kind-control-plane:6443" I0911 03:53:41.835738 178 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0911 03:53:41.835746 178 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0911 03:53:41.835750 178 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true I0911 03:53:41.835752 178 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0911 03:53:41.835753 178 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false I0911 03:53:41.835963 178 token.go:229] [discovery] Waiting for the cluster-info ConfigMap to receive a JWS signature for token ID "abcdef" I0911 03:53:41.840504 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s" status="200 OK" milliseconds=4 I0911 03:53:41.840649 178 token.go:249] [discovery] Retrying due to error: could not find a JWS signature in the cluster-info ConfigMap for token ID "abcdef" I0911 03:53:46.840064 178 token.go:229] [discovery] Waiting for the cluster-info ConfigMap to receive a JWS signature for token ID "abcdef" I0911 03:53:46.842838 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s" status="200 OK" milliseconds=2 I0911 03:53:46.843292 178 token.go:249] [discovery] Retrying due to error: could not find a JWS signature in the cluster-info ConfigMap for token ID "abcdef" I0911 03:53:51.836087 178 token.go:229] [discovery] Waiting for the cluster-info ConfigMap to receive a JWS signature for token ID "abcdef" I0911 03:53:51.840512 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s" status="200 OK" milliseconds=4 I0911 03:53:51.840819 178 token.go:113] [discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "kind-control-plane:6443" I0911 03:53:51.840825 178 discovery.go:52] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process I0911 03:53:51.840833 178 join.go:565] [preflight] Fetching init configuration I0911 03:53:51.840836 178 join.go:654] [preflight] Retrieving KubeConfig objects [preflight] Reading configuration from the "kubeadm-config" ConfigMap in namespace "kube-system"... [preflight] Use 'kubeadm init phase upload-config kubeadm --config your-config-file' to re-upload it. I0911 03:53:51.844311 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s" status="200 OK" milliseconds=3 I0911 03:53:51.844815 178 kubeproxy.go:55] attempting to download the KubeProxyConfiguration from ConfigMap "kube-proxy" I0911 03:53:51.845556 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kube-proxy?timeout=10s" status="200 OK" milliseconds=0 I0911 03:53:51.846554 178 kubelet.go:73] attempting to download the KubeletConfiguration from ConfigMap "kubelet-config" I0911 03:53:51.847228 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config?timeout=10s" status="200 OK" milliseconds=0 I0911 03:53:51.847953 178 initconfiguration.go:114] skip CRI socket detection, fill with the default CRI socket unix:///var/run/containerd/containerd.sock I0911 03:53:51.848052 178 interface.go:432] Looking for default routes with IPv4 addresses I0911 03:53:51.848057 178 interface.go:437] Default route transits interface "eth0" I0911 03:53:51.848101 178 interface.go:209] Interface eth0 is up I0911 03:53:51.848125 178 interface.go:257] Interface "eth0" has 3 addresses :[172.18.0.3/16 fc00:f853:ccd:e793::3/64 fe80::cca1:aeff:fe1a:d6b0/64]. I0911 03:53:51.848136 178 interface.go:224] Checking addr 172.18.0.3/16. I0911 03:53:51.848141 178 interface.go:231] IP found 172.18.0.3 I0911 03:53:51.848149 178 interface.go:263] Found valid IPv4 address 172.18.0.3 for interface "eth0". I0911 03:53:51.848152 178 interface.go:443] Found active IP 172.18.0.3 I0911 03:53:51.848187 178 common.go:148] WARNING: tolerating control plane version v1.35.0-alpha.0.618+1f23e8ce7dead4 as a pre-release version I0911 03:53:51.848311 178 kubelet.go:166] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf I0911 03:53:51.848688 178 kubelet.go:181] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt I0911 03:53:51.848739 178 kubelet.go:197] [kubelet-start] Checking for an existing Node in the cluster with name "kind-worker2" and status "Ready" I0911 03:53:51.849497 178 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/nodes/kind-worker2?timeout=10s" status="404 Not Found" milliseconds=0 I0911 03:53:51.849527 178 kubelet.go:212] [kubelet-start] Stopping the kubelet [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml" [patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s [kubelet-check] The kubelet is healthy after 501.254734ms [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap I0911 03:53:52.433690 178 loader.go:405] Config loaded from file: /etc/kubernetes/kubelet.conf I0911 03:53:52.434049 178 cert_rotation.go:141] "Starting client certificate rotation controller" logger="tls-transport-cache" I0911 03:53:52.434155 178 loader.go:405] Config loaded from file: /etc/kubernetes/kubelet.conf This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the control-plane to see this node join the cluster. DEBUG: kubeadmjoin/join.go:147] I0911 03:53:41.835168 176 join.go:421] [preflight] found NodeName empty; using OS hostname as NodeName I0911 03:53:41.835210 176 joinconfiguration.go:82] loading configuration from "/kind/kubeadm.conf" W0911 03:53:41.835449 176 joinconfiguration.go:112] [config] WARNING: Ignored configuration document with GroupVersionKind kubelet.config.k8s.io/v1beta1, Kind=KubeletConfiguration W0911 03:53:41.835457 176 joinconfiguration.go:112] [config] WARNING: Ignored configuration document with GroupVersionKind kubeproxy.config.k8s.io/v1alpha1, Kind=KubeProxyConfiguration W0911 03:53:41.835459 176 joinconfiguration.go:112] [config] WARNING: Ignored configuration document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=ClusterConfiguration W0911 03:53:41.835461 176 joinconfiguration.go:112] [config] WARNING: Ignored configuration document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=InitConfiguration W0911 03:53:41.835464 176 common.go:100] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "JoinConfiguration"). Please use 'kubeadm config migrate --old-config old-config-file --new-config new-config-file', which will write the new, similar spec using a newer API version. I0911 03:53:41.835681 176 controlplaneprepare.go:225] [download-certs] Skipping certs download I0911 03:53:41.835686 176 join.go:551] [preflight] Discovering cluster-info I0911 03:53:41.835694 176 token.go:71] [discovery] Created cluster-info discovery client, requesting info from "kind-control-plane:6443" I0911 03:53:41.835752 176 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true I0911 03:53:41.835759 176 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0911 03:53:41.835764 176 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false I0911 03:53:41.835767 176 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0911 03:53:41.835776 176 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0911 03:53:41.835962 176 token.go:229] [discovery] Waiting for the cluster-info ConfigMap to receive a JWS signature for token ID "abcdef" I0911 03:53:41.840500 176 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s" status="200 OK" milliseconds=4 I0911 03:53:41.840638 176 token.go:249] [discovery] Retrying due to error: could not find a JWS signature in the cluster-info ConfigMap for token ID "abcdef" I0911 03:53:46.836732 176 token.go:229] [discovery] Waiting for the cluster-info ConfigMap to receive a JWS signature for token ID "abcdef" I0911 03:53:46.839650 176 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s" status="200 OK" milliseconds=2 I0911 03:53:46.839930 176 token.go:249] [discovery] Retrying due to error: could not find a JWS signature in the cluster-info ConfigMap for token ID "abcdef" I0911 03:53:51.836087 176 token.go:229] [discovery] Waiting for the cluster-info ConfigMap to receive a JWS signature for token ID "abcdef" I0911 03:53:51.840460 176 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s" status="200 OK" milliseconds=4 I0911 03:53:51.840819 176 token.go:113] [discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "kind-control-plane:6443" I0911 03:53:51.840825 176 discovery.go:52] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process I0911 03:53:51.840833 176 join.go:565] [preflight] Fetching init configuration I0911 03:53:51.840836 176 join.go:654] [preflight] Retrieving KubeConfig objects [preflight] Reading configuration from the "kubeadm-config" ConfigMap in namespace "kube-system"... [preflight] Use 'kubeadm init phase upload-config kubeadm --config your-config-file' to re-upload it. I0911 03:53:51.844468 176 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s" status="200 OK" milliseconds=3 I0911 03:53:51.844944 176 kubeproxy.go:55] attempting to download the KubeProxyConfiguration from ConfigMap "kube-proxy" I0911 03:53:51.845648 176 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kube-proxy?timeout=10s" status="200 OK" milliseconds=0 I0911 03:53:51.846272 176 kubelet.go:73] attempting to download the KubeletConfiguration from ConfigMap "kubelet-config" I0911 03:53:51.846991 176 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config?timeout=10s" status="200 OK" milliseconds=0 I0911 03:53:51.847883 176 initconfiguration.go:114] skip CRI socket detection, fill with the default CRI socket unix:///var/run/containerd/containerd.sock I0911 03:53:51.848022 176 interface.go:432] Looking for default routes with IPv4 addresses I0911 03:53:51.848036 176 interface.go:437] Default route transits interface "eth0" I0911 03:53:51.848100 176 interface.go:209] Interface eth0 is up I0911 03:53:51.848125 176 interface.go:257] Interface "eth0" has 3 addresses :[172.18.0.2/16 fc00:f853:ccd:e793::2/64 fe80::9884:c7ff:fed4:f7c0/64]. I0911 03:53:51.848143 176 interface.go:224] Checking addr 172.18.0.2/16. I0911 03:53:51.848149 176 interface.go:231] IP found 172.18.0.2 I0911 03:53:51.848156 176 interface.go:263] Found valid IPv4 address 172.18.0.2 for interface "eth0". I0911 03:53:51.848160 176 interface.go:443] Found active IP 172.18.0.2 I0911 03:53:51.848204 176 common.go:148] WARNING: tolerating control plane version v1.35.0-alpha.0.618+1f23e8ce7dead4 as a pre-release version I0911 03:53:51.848313 176 kubelet.go:166] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf I0911 03:53:51.848644 176 kubelet.go:181] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt I0911 03:53:51.848691 176 kubelet.go:197] [kubelet-start] Checking for an existing Node in the cluster with name "kind-worker" and status "Ready" I0911 03:53:51.849430 176 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/api/v1/nodes/kind-worker?timeout=10s" status="404 Not Found" milliseconds=0 I0911 03:53:51.849476 176 kubelet.go:212] [kubelet-start] Stopping the kubelet [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml" [patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s [kubelet-check] The kubelet is healthy after 500.776618ms [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap I0911 03:53:52.437710 176 loader.go:405] Config loaded from file: /etc/kubernetes/kubelet.conf I0911 03:53:52.438073 176 cert_rotation.go:141] "Starting client certificate rotation controller" logger="tls-transport-cache" This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the control-plane to see this node join the cluster. I0911 03:53:52.438257 176 loader.go:405] Config loaded from file: /etc/kubernetes/kubelet.conf ✓ Joining worker nodes 🚜 • Waiting ≤ 1m0s for control-plane = Ready ⏳ ... ✓ Waiting ≤ 1m0s for control-plane = Ready ⏳ • Ready after 6s 💚 Set kubectl context to "kind-kind" You can now use your cluster with: kubectl cluster-info --context kind-kind Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂 + kubectl version Client Version: v1.35.0-alpha.0.618+1f23e8ce7dead4 Kustomize Version: v5.7.1 Server Version: v1.35.0-alpha.0.618+1f23e8ce7dead4 + kubectl patch -n kube-system daemonset/kube-proxy --type=json -p=[{"op": "add", "path": "/spec/template/spec/containers/0/command/-", "value": "--v=4" }] daemonset.apps/kube-proxy patched + run_tests + [ ipv4 = ipv6 ] + SKIP=PodSecurityPolicy|LoadBalancer|load.balancer|In-tree.Volumes.\[Driver:.nfs\]|PersistentVolumes.NFS|Network.should.set.TCP.CLOSE_WAIT.timeout|Simple.pod.should.support.exec.through.an.HTTP.proxy|subPath.should.support.existing|should.provide.basic.identity + FOCUS= + LABEL_FILTER=Feature: isSubsetOf OffByDefault && !Deprecated && !Slow && !Disruptive && !Flaky + [ -z ] + [ -z Feature: isSubsetOf OffByDefault && !Deprecated && !Slow && !Disruptive && !Flaky ] + [ true = true ] + export GINKGO_PARALLEL=y + [ -z PodSecurityPolicy|LoadBalancer|load.balancer|In-tree.Volumes.\[Driver:.nfs\]|PersistentVolumes.NFS|Network.should.set.TCP.CLOSE_WAIT.timeout|Simple.pod.should.support.exec.through.an.HTTP.proxy|subPath.should.support.existing|should.provide.basic.identity ] + SKIP=\[Serial\]|PodSecurityPolicy|LoadBalancer|load.balancer|In-tree.Volumes.\[Driver:.nfs\]|PersistentVolumes.NFS|Network.should.set.TCP.CLOSE_WAIT.timeout|Simple.pod.should.support.exec.through.an.HTTP.proxy|subPath.should.support.existing|should.provide.basic.identity + export KUBERNETES_CONFORMANCE_TEST=y + export KUBE_CONTAINER_RUNTIME=remote + export KUBE_CONTAINER_RUNTIME_ENDPOINT=unix:///run/containerd/containerd.sock + export KUBE_CONTAINER_RUNTIME_NAME=containerd + GINKGO_PID=73496 + wait 73496 + ./hack/ginkgo-e2e.sh --provider=skeleton --num-nodes=2 --ginkgo.focus= --ginkgo.skip=\[Serial\]|PodSecurityPolicy|LoadBalancer|load.balancer|In-tree.Volumes.\[Driver:.nfs\]|PersistentVolumes.NFS|Network.should.set.TCP.CLOSE_WAIT.timeout|Simple.pod.should.support.exec.through.an.HTTP.proxy|subPath.should.support.existing|should.provide.basic.identity --ginkgo.label-filter=Feature: isSubsetOf OffByDefault && !Deprecated && !Slow && !Disruptive && !Flaky --report-dir=/logs/artifacts --disable-log-dump=true Conformance test: not doing test setup. + set +x + /home/prow/go/src/k8s.io/kubernetes/_output/bin/ginkgo --nodes=25 --poll-progress-after=60m --poll-progress-interval=5m --source-root=/home/prow/go/src/k8s.io/kubernetes/cluster/.. --timeout=24h --silence-skips --force-newlines --no-color /home/prow/go/src/k8s.io/kubernetes/_output/bin/e2e.test -- --kubeconfig=/root/.kube/kind-test-config --host=https://127.0.0.1:33387 --provider=skeleton --gce-project= --gce-zone= --gce-region= --gce-multizone=false --gke-cluster= --kube-master= --cluster-tag= --cloud-config-file= --repo-root=/home/prow/go/src/k8s.io/kubernetes/cluster/.. --node-instance-group= --prefix=e2e --network=e2e --node-tag= --master-tag= --docker-config-file= --dns-domain=cluster.local --prepull-images=false --report-complete-ginkgo --report-complete-junit --provider=skeleton --num-nodes=2 --ginkgo.focus= '--ginkgo.skip=\[Serial\]|PodSecurityPolicy|LoadBalancer|load.balancer|In-tree.Volumes.\[Driver:.nfs\]|PersistentVolumes.NFS|Network.should.set.TCP.CLOSE_WAIT.timeout|Simple.pod.should.support.exec.through.an.HTTP.proxy|subPath.should.support.existing|should.provide.basic.identity' '--ginkgo.label-filter=Feature: isSubsetOf OffByDefault && !Deprecated && !Slow && !Disruptive && !Flaky' --report-dir=/logs/artifacts --disable-log-dump=true Running Suite: Kubernetes e2e suite - /home/prow/go/src/k8s.io/kubernetes/_output/bin ===================================================================================== Random Seed: 1757562838 - will randomize all specs Will run 2317 of 7146 specs Running in parallel across 25 processes • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • Ran 1076 of 7146 Specs in 965.823 seconds SUCCESS! -- 1076 Passed | 0 Failed | 0 Pending | 6070 Skipped Ginkgo ran 1 suite in 16m11.091813647s Test Suite Passed + cleanup + [ false = true ] + [ true = true ] + kind export logs /logs/artifacts Exporting logs for cluster "kind" to: /logs/artifacts + kind delete cluster Deleting cluster "kind" ... Deleted nodes: ["kind-worker" "kind-control-plane" "kind-worker2"] + rm -f _output/bin/e2e.test + [ -n /tmp/tmp.C1AsOONFz5 ] + rm -rf /tmp/tmp.C1AsOONFz5 + CLEANED_UP=true + exit 0 wrapper.sh] [TEST] Test Command exit code: 0 wrapper.sh] [CLEANUP] Waiting 30 seconds for pods stopped with terminationGracePeriod:30 wrapper.sh] [CLEANUP] Cleaning up after Docker in Docker ... 9dfbbc8c5261 wrapper.sh] [CLEANUP] Waiting for docker to stop for 30 seconds Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die. wrapper.sh] [CLEANUP] Done cleaning up after Docker in Docker. ================================================================================ wrapper.sh] Exiting 0