W1025 08:51:37.855] ************************************************************************** bootstrap.py is deprecated! test-infra oncall does not support any job still using bootstrap.py. Please migrate your job to podutils! https://github.com/kubernetes/test-infra/blob/master/prow/pod-utilities.md ************************************************************************** I1025 08:51:37.855] Args: --job=ci-kubernetes-charts-gce --service-account=/etc/service-account/service-account.json --upload=gs://kubernetes-jenkins/logs --timeout=200 --repo=github.com/helm/charts --root=/go/src --service-account=/etc/service-account/service-account.json --scenario=kubernetes_e2e -- --charts --check-leaked-resources --extract=ci/latest --gcp-node-image=gci --gcp-zone=us-central1-f --provider=gce --test=false --timeout=180m I1025 08:51:37.856] Bootstrap ci-kubernetes-charts-gce... I1025 08:51:37.858] Builder: 42261741-169f-11eb-b256-6ee25ea2e440 I1025 08:51:37.858] Image: gcr.io/k8s-testimages/kubekins-e2e:v20201021-a62ce8a-master I1025 08:51:37.858] Gubernator results at https://gubernator.k8s.io/build/kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320286823146065920 I1025 08:51:37.858] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1025 08:51:38.541] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] I1025 08:51:38.664] process 34 exited with code 0 after 0.0m I1025 08:51:38.664] Call: gcloud config get-value account I1025 08:51:39.281] process 47 exited with code 0 after 0.0m I1025 08:51:39.282] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com I1025 08:51:39.282] Root: /go/src I1025 08:51:39.282] cd to /go/src I1025 08:51:39.282] Checkout: /go/src/github.com/helm/charts master to /go/src/github.com/helm/charts I1025 08:51:39.282] Call: git init github.com/helm/charts I1025 08:51:39.289] Initialized empty Git repository in /go/src/github.com/helm/charts/.git/ I1025 08:51:39.289] process 60 exited with code 0 after 0.0m I1025 08:51:39.289] Call: git config --local user.name 'K8S Bootstrap' I1025 08:51:39.293] process 61 exited with code 0 after 0.0m I1025 08:51:39.294] Call: git config --local user.email k8s_bootstrap@localhost I1025 08:51:39.298] process 62 exited with code 0 after 0.0m I1025 08:51:39.298] Call: git fetch --quiet --tags https://github.com/helm/charts master I1025 08:51:42.765] process 63 exited with code 0 after 0.1m I1025 08:51:42.765] Call: git checkout -B test FETCH_HEAD W1025 08:51:43.229] Switched to a new branch 'test' I1025 08:51:43.235] process 74 exited with code 0 after 0.0m I1025 08:51:43.235] Call: git show -s --format=format:%ct HEAD I1025 08:51:43.240] process 75 exited with code 0 after 0.0m I1025 08:51:43.241] Configure environment... I1025 08:51:43.241] Call: git show -s --format=format:%ct HEAD I1025 08:51:43.246] process 76 exited with code 0 after 0.0m I1025 08:51:43.246] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1025 08:51:43.941] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] I1025 08:51:44.164] process 77 exited with code 0 after 0.0m I1025 08:51:44.165] Call: gcloud config get-value account I1025 08:51:44.774] process 90 exited with code 0 after 0.0m I1025 08:51:44.774] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com I1025 08:51:44.774] Start 1320286823146065920 at unknown... I1025 08:51:44.776] Call: gsutil -q -h Content-Type:application/json cp /tmp/gsutil_wWusO5 gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320286823146065920/started.json I1025 08:51:46.328] process 103 exited with code 0 after 0.0m I1025 08:51:46.329] Call: /workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py --charts --check-leaked-resources --extract=ci/latest --gcp-node-image=gci --gcp-zone=us-central1-f --provider=gce --test=false --timeout=180m W1025 08:51:46.375] starts with local mode W1025 08:51:46.376] Environment: W1025 08:51:46.376] ARTIFACTS=/workspace/_artifacts W1025 08:51:46.376] AWS_SSH_PRIVATE_KEY_FILE=/root/.ssh/kube_aws_rsa W1025 08:51:46.376] AWS_SSH_PUBLIC_KEY_FILE=/root/.ssh/kube_aws_rsa.pub W1025 08:51:46.377] BAZEL_REMOTE_CACHE_ENABLED=false W1025 08:51:46.377] BAZEL_VERSION=2.2.0 W1025 08:51:46.377] BOOTSTRAP_MIGRATION=yes W1025 08:51:46.377] BOSKOS_METRICS_PORT=tcp://10.63.252.110:9090 W1025 08:51:46.377] BOSKOS_METRICS_PORT_9090_TCP=tcp://10.63.252.110:9090 W1025 08:51:46.377] BOSKOS_METRICS_PORT_9090_TCP_ADDR=10.63.252.110 W1025 08:51:46.377] BOSKOS_METRICS_PORT_9090_TCP_PORT=9090 W1025 08:51:46.378] BOSKOS_METRICS_PORT_9090_TCP_PROTO=tcp W1025 08:51:46.378] BOSKOS_METRICS_SERVICE_HOST=10.63.252.110 W1025 08:51:46.378] BOSKOS_METRICS_SERVICE_PORT=9090 W1025 08:51:46.378] BOSKOS_METRICS_SERVICE_PORT_METRICS=9090 W1025 08:51:46.378] BOSKOS_PORT=tcp://10.63.250.132:80 W1025 08:51:46.378] BOSKOS_PORT_80_TCP=tcp://10.63.250.132:80 W1025 08:51:46.378] BOSKOS_PORT_80_TCP_ADDR=10.63.250.132 W1025 08:51:46.378] BOSKOS_PORT_80_TCP_PORT=80 W1025 08:51:46.379] BOSKOS_PORT_80_TCP_PROTO=tcp W1025 08:51:46.379] BOSKOS_SERVICE_HOST=10.63.250.132 W1025 08:51:46.379] BOSKOS_SERVICE_PORT=80 W1025 08:51:46.379] BOSKOS_SERVICE_PORT_DEFAULT=80 W1025 08:51:46.379] BUILD_ID=1320286823146065920 W1025 08:51:46.379] BUILD_NUMBER=1320286823146065920 W1025 08:51:46.379] CI=true W1025 08:51:46.380] CLOUDSDK_COMPONENT_MANAGER_DISABLE_UPDATE_CHECK=true W1025 08:51:46.380] CLOUDSDK_CONFIG=/go/src/github.com/helm/charts/.config/gcloud W1025 08:51:46.380] CLOUDSDK_CORE_DISABLE_PROMPTS=1 W1025 08:51:46.380] CLOUDSDK_EXPERIMENTAL_FAST_COMPONENT_UPDATE=false W1025 08:51:46.380] DOCKER_IN_DOCKER_ENABLED=false W1025 08:51:46.380] DOCKER_IN_DOCKER_IPV6_ENABLED=false W1025 08:51:46.380] E2E_GOOGLE_APPLICATION_CREDENTIALS=/etc/service-account/service-account.json W1025 08:51:46.381] GCS_ARTIFACTS_DIR=gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320286823146065920/artifacts W1025 08:51:46.381] GOOGLE_APPLICATION_CREDENTIALS=/etc/service-account/service-account.json W1025 08:51:46.381] GOOGLE_APPLICATION_CREDENTIALS_DEPRECATED=Migrate to workload identity, contact sig-testing W1025 08:51:46.381] GOPATH=/go W1025 08:51:46.381] GOPROXY=https://proxy.golang.org W1025 08:51:46.381] GO_TARBALL=go1.15.2.linux-amd64.tar.gz W1025 08:51:46.382] HOME=/workspace W1025 08:51:46.382] HOSTNAME=42261741-169f-11eb-b256-6ee25ea2e440 W1025 08:51:46.382] IMAGE=gcr.io/k8s-testimages/kubekins-e2e:v20201021-a62ce8a-master W1025 08:51:46.382] INSTANCE_PREFIX=bootstrap-e2e W1025 08:51:46.382] JENKINS_GCE_SSH_PRIVATE_KEY_FILE=/workspace/.ssh/google_compute_engine W1025 08:51:46.382] JENKINS_GCE_SSH_PUBLIC_KEY_FILE=/workspace/.ssh/google_compute_engine.pub W1025 08:51:46.382] JOB_NAME=ci-kubernetes-charts-gce W1025 08:51:46.383] JOB_SPEC={"type":"periodic","job":"ci-kubernetes-charts-gce","buildid":"1320286823146065920","prowjobid":"42261741-169f-11eb-b256-6ee25ea2e440"} W1025 08:51:46.383] JOB_TYPE=periodic W1025 08:51:46.383] KUBERNETES_PORT=tcp://10.63.240.1:443 W1025 08:51:46.383] KUBERNETES_PORT_443_TCP=tcp://10.63.240.1:443 W1025 08:51:46.383] KUBERNETES_PORT_443_TCP_ADDR=10.63.240.1 W1025 08:51:46.383] KUBERNETES_PORT_443_TCP_PORT=443 W1025 08:51:46.383] KUBERNETES_PORT_443_TCP_PROTO=tcp W1025 08:51:46.384] KUBERNETES_SERVICE_HOST=10.63.240.1 W1025 08:51:46.384] KUBERNETES_SERVICE_PORT=443 W1025 08:51:46.384] KUBERNETES_SERVICE_PORT_HTTPS=443 W1025 08:51:46.384] KUBETEST_IN_DOCKER=true W1025 08:51:46.384] KUBETEST_MANUAL_DUMP=y W1025 08:51:46.384] KUBE_AWS_INSTANCE_PREFIX=bootstrap-e2e W1025 08:51:46.384] KUBE_GCE_INSTANCE_PREFIX=bootstrap-e2e W1025 08:51:46.385] LOG_DUMP_SCRIPT_PATH=/workspace/log-dump.sh W1025 08:51:46.385] NODE_NAME=42261741-169f-11eb-b256-6ee25ea2e440 W1025 08:51:46.385] PATH=/go/bin:/go/bin:/usr/local/go/bin:/google-cloud-sdk/bin:/workspace:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin W1025 08:51:46.385] PROW_JOB_ID=42261741-169f-11eb-b256-6ee25ea2e440 W1025 08:51:46.385] PWD=/workspace W1025 08:51:46.385] SHLVL=2 W1025 08:51:46.385] SOURCE_DATE_EPOCH=1603361254 W1025 08:51:46.385] TERM=xterm W1025 08:51:46.386] USER=prow W1025 08:51:46.386] WORKSPACE=/workspace W1025 08:51:46.386] _=./test-infra/jenkins/bootstrap.py W1025 08:51:46.386] Run: ('kubetest', '--dump=/workspace/_artifacts', '--gcp-service-account=/etc/service-account/service-account.json', '--up', '--down', '--provider=gce', '--cluster=bootstrap-e2e', '--gcp-network=bootstrap-e2e', '--charts', '--check-leaked-resources', '--extract=ci/latest', '--gcp-node-image=gci', '--gcp-zone=us-central1-f', '--timeout=180m') W1025 08:51:46.407] 2020/10/25 08:51:46 Warning: Couldn't find directory src/sigs.k8s.io/cloud-provider-azure under any of GOPATH /go, defaulting to /go/src/k8s.io/cloud-provider-azure W1025 08:51:46.410] 2020/10/25 08:51:46 extract_k8s.go:112: Matched extraction strategy: ^ci/(.+)$ W1025 08:51:46.410] 2020/10/25 08:51:46 main.go:329: Limiting testing to 3h0m0s W1025 08:51:46.410] 2020/10/25 08:51:46 process.go:153: Running: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1025 08:51:47.051] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] W1025 08:51:47.168] 2020/10/25 08:51:47 process.go:155: Step 'gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json' finished in 757.31693ms W1025 08:51:47.168] 2020/10/25 08:51:47 main.go:728: --gcp-project is missing, trying to fetch a project from boskos. W1025 08:51:47.169] (for local runs please set --gcp-project to your dev project) W1025 08:51:47.169] 2020/10/25 08:51:47 main.go:740: provider gce, will acquire project type gce-project from boskos W1025 08:51:47.177] 2020/10/25 08:51:47 process.go:153: Running: gcloud config set project k8s-boskos-gce-project-17 W1025 08:51:47.720] Updated property [core/project]. W1025 08:51:47.993] 2020/10/25 08:51:47 process.go:155: Step 'gcloud config set project k8s-boskos-gce-project-17' finished in 815.921868ms W1025 08:51:47.994] 2020/10/25 08:51:47 main.go:779: Checking existing of GCP ssh keys... W1025 08:51:47.994] 2020/10/25 08:51:47 main.go:789: Checking presence of public key in k8s-boskos-gce-project-17 W1025 08:51:47.994] 2020/10/25 08:51:47 process.go:153: Running: gcloud compute --project=k8s-boskos-gce-project-17 project-info describe W1025 08:51:49.536] 2020/10/25 08:51:49 process.go:155: Step 'gcloud compute --project=k8s-boskos-gce-project-17 project-info describe' finished in 1.542562163s W1025 08:51:49.623] 2020/10/25 08:51:49 extract_k8s.go:295: U=https://storage.googleapis.com/kubernetes-release-dev/ci R=v1.20.0-alpha.3.114+5935fcd704fe89 get-kube.sh W1025 08:51:49.623] 2020/10/25 08:51:49 process.go:153: Running: /workspace/get-kube.sh I1025 08:51:49.724] Downloading kubernetes release v1.20.0-alpha.3.114+5935fcd704fe89 I1025 08:51:49.724] from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.114+5935fcd704fe89/kubernetes.tar.gz I1025 08:51:49.724] to /go/src/github.com/helm/charts/kubernetes.tar.gz W1025 08:51:50.835] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.114+5935fcd704fe89/kubernetes.tar.gz... W1025 08:51:50.997] / [0 files][ 0.0 B/487.6 KiB] / [1 files][487.6 KiB/487.6 KiB] W1025 08:51:50.997] Operation completed over 1 objects/487.6 KiB. I1025 08:51:51.235] Unpacking kubernetes release v1.20.0-alpha.3.114+5935fcd704fe89 I1025 08:51:51.304] Kubernetes release: v1.20.0-alpha.3.114+5935fcd704fe89 I1025 08:51:51.304] Server: linux/amd64 (to override, set KUBERNETES_SERVER_ARCH) I1025 08:51:51.305] Client: linux/amd64 (autodetected) (to override, set KUBERNETES_CLIENT_OS and/or KUBERNETES_CLIENT_ARCH) I1025 08:51:51.305] I1025 08:51:51.305] Will download kubernetes-server-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.114+5935fcd704fe89 I1025 08:51:51.306] Will download and extract kubernetes-client-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.114+5935fcd704fe89 I1025 08:51:51.306] Will download and extract kubernetes-test tarball(s) from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.114+5935fcd704fe89 W1025 08:51:52.359] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.114+5935fcd704fe89/kubernetes-server-linux-amd64.tar.gz... W1025 08:51:52.361] / [0 files][ 0.0 B/502.2 MiB] ==> NOTE: You are downloading one or more large file(s), which would W1025 08:51:52.362] run significantly faster if you enabled sliced object downloads. This W1025 08:51:52.362] feature is enabled by default but requires that compiled crcmod be W1025 08:51:52.362] installed (see "gsutil help crcmod"). W1025 08:51:52.362] W1025 08:51:58.274] - - [0 files][114.2 MiB/502.2 MiB] \ | | [0 files][246.5 MiB/502.2 MiB] / / [0 files][378.7 MiB/502.2 MiB] - \ \ [1 files][502.2 MiB/502.2 MiB] | W1025 08:51:58.275] Operation completed over 1 objects/502.2 MiB. I1025 08:51:58.528] I1025 08:51:59.711] md5sum(kubernetes-server-linux-amd64.tar.gz)=b24185e1c269b81eaf8535934156c994 I1025 08:52:01.829] sha512sum(kubernetes-server-linux-amd64.tar.gz)=3dbaafa5dbddeaa308d3423091c8adf4cf022affd7893e8a5f6f1cd6c3bd5969d470511692cf93a79f60d374bd8d66cd1d265088c567f8ca8ca5709be161b5b2 I1025 08:52:01.830] W1025 08:52:02.970] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.114+5935fcd704fe89/kubernetes-client-linux-amd64.tar.gz... W1025 08:52:03.583] / [0 files][ 0.0 B/ 22.7 MiB] / [1 files][ 22.7 MiB/ 22.7 MiB] - W1025 08:52:03.583] Operation completed over 1 objects/22.7 MiB. I1025 08:52:03.799] I1025 08:52:03.859] md5sum(kubernetes-client-linux-amd64.tar.gz)=b3589af481f4436a1ebee67513b0d8de I1025 08:52:03.957] sha512sum(kubernetes-client-linux-amd64.tar.gz)=04aeb4a4f990505689861d66b1be085f64e6b9a6ccefd7d85f36cc9f8e3900a3fe3b6e64d524f928008727f5a65472a09613e7b635e75cc82a118a9213ca7d0b I1025 08:52:03.957] I1025 08:52:03.957] Extracting /go/src/github.com/helm/charts/kubernetes/client/kubernetes-client-linux-amd64.tar.gz into /go/src/github.com/helm/charts/kubernetes/platforms/linux/amd64 I1025 08:52:04.490] Add '/go/src/github.com/helm/charts/kubernetes/client/bin' to your PATH to use newly-installed binaries. W1025 08:52:05.856] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.114+5935fcd704fe89/kubernetes-test-portable.tar.gz... W1025 08:52:06.016] / [0 files][ 0.0 B/219.8 KiB] / [1 files][219.8 KiB/219.8 KiB] W1025 08:52:06.017] Operation completed over 1 objects/219.8 KiB. I1025 08:52:06.772] I1025 08:52:06.789] md5sum(kubernetes-test-portable.tar.gz)=a167870eac14644a1d7fda4e717936da I1025 08:52:06.812] sha512sum(kubernetes-test-portable.tar.gz)=3d2a03914c52b8c35cf313b2317ccd71d22b4af5e5c4f079cfb7d205d332dbd1d449a5a8f326881f04be459846af3c67f5f9b6f18e58d6a6bada6438f4b1f2a4 I1025 08:52:06.813] I1025 08:52:06.813] Extracting kubernetes-test-portable.tar.gz into /go/src/github.com/helm/charts/kubernetes W1025 08:52:10.048] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.114+5935fcd704fe89/kubernetes-test-linux-amd64.tar.gz... W1025 08:52:10.088] / [0 files][ 0.0 B/351.2 MiB] ==> NOTE: You are downloading one or more large file(s), which would W1025 08:52:10.089] run significantly faster if you enabled sliced object downloads. This W1025 08:52:10.089] feature is enabled by default but requires that compiled crcmod be W1025 08:52:10.089] installed (see "gsutil help crcmod"). W1025 08:52:10.089] W1025 08:52:15.813] - - [0 files][ 48.7 MiB/351.2 MiB] \ | | [0 files][148.2 MiB/351.2 MiB] / / [0 files][238.7 MiB/351.2 MiB] - \ \ [0 files][337.0 MiB/351.2 MiB] \ [1 files][351.2 MiB/351.2 MiB] | W1025 08:52:15.813] Operation completed over 1 objects/351.2 MiB. I1025 08:52:16.602] I1025 08:52:17.577] md5sum(kubernetes-test-linux-amd64.tar.gz)=a95b81bc19c80260af1f11d7dbcecfb4 I1025 08:52:20.105] sha512sum(kubernetes-test-linux-amd64.tar.gz)=52bfb16bf072c239489323d095c49c539d38b511273578e03a6a79197e8c4ed1c8e2bdaa3545c72580b10104a94a765c00412c6a8d206dadec0b89b8378a429f I1025 08:52:20.105] I1025 08:52:20.106] Extracting /go/src/github.com/helm/charts/kubernetes/test/kubernetes-test-linux-amd64.tar.gz into /go/src/github.com/helm/charts/kubernetes/platforms/linux/amd64 W1025 08:52:30.998] 2020/10/25 08:52:30 process.go:155: Step '/workspace/get-kube.sh' finished in 41.375076033s W1025 08:52:30.998] 2020/10/25 08:52:30 process.go:153: Running: ./hack/e2e-internal/e2e-down.sh W1025 08:52:31.074] Project: k8s-boskos-gce-project-17 W1025 08:52:31.074] Network Project: k8s-boskos-gce-project-17 W1025 08:52:31.075] Zone: us-central1-f I1025 08:52:31.175] Shutting down test cluster in background. I1025 08:52:33.729] Bringing down cluster using provider: gce W1025 08:52:33.830] ... calling verify-prereqs W1025 08:52:34.396] ... calling verify-kube-binaries W1025 08:52:34.558] ... calling kube-down W1025 08:52:34.558] Project: k8s-boskos-gce-project-17 W1025 08:52:34.558] Network Project: k8s-boskos-gce-project-17 W1025 08:52:34.559] Zone: us-central1-f I1025 08:52:36.986] Bringing down cluster W1025 08:52:37.086] INSTANCE_GROUPS= W1025 08:52:37.086] NODE_NAMES= I1025 08:53:03.945] Deleting firewall rules remaining in network bootstrap-e2e: W1025 08:53:07.038] W1025 08:53:07.038289 1869 loader.go:223] Config not found: /workspace/.kube/config W1025 08:53:07.194] W1025 08:53:07.194043 1920 loader.go:223] Config not found: /workspace/.kube/config W1025 08:53:07.194] W1025 08:53:07.194310 1920 loader.go:223] Config not found: /workspace/.kube/config I1025 08:53:07.295] Property "clusters.k8s-boskos-gce-project-17_bootstrap-e2e" unset. I1025 08:53:07.371] Property "users.k8s-boskos-gce-project-17_bootstrap-e2e" unset. W1025 08:53:07.471] W1025 08:53:07.370475 1971 loader.go:223] Config not found: /workspace/.kube/config W1025 08:53:07.472] W1025 08:53:07.370740 1971 loader.go:223] Config not found: /workspace/.kube/config W1025 08:53:07.546] W1025 08:53:07.546614 2022 loader.go:223] Config not found: /workspace/.kube/config W1025 08:53:07.547] W1025 08:53:07.546841 2022 loader.go:223] Config not found: /workspace/.kube/config I1025 08:53:07.647] Property "users.k8s-boskos-gce-project-17_bootstrap-e2e-basic-auth" unset. I1025 08:53:07.717] Property "contexts.k8s-boskos-gce-project-17_bootstrap-e2e" unset. I1025 08:53:07.722] Cleared config for k8s-boskos-gce-project-17_bootstrap-e2e from /workspace/.kube/config I1025 08:53:07.722] Done W1025 08:53:07.823] W1025 08:53:07.716643 2074 loader.go:223] Config not found: /workspace/.kube/config W1025 08:53:07.823] W1025 08:53:07.716910 2074 loader.go:223] Config not found: /workspace/.kube/config W1025 08:53:07.823] 2020/10/25 08:53:07 process.go:155: Step './hack/e2e-internal/e2e-down.sh' finished in 36.725375769s W1025 08:53:07.823] 2020/10/25 08:53:07 e2e.go:456: Listing resources... W1025 08:53:07.824] 2020/10/25 08:53:07 process.go:153: Running: ./cluster/gce/list-resources.sh W1025 08:53:08.675] Listed 0 items. W1025 08:53:09.751] Listed 0 items. W1025 08:53:11.464] Listed 0 items. W1025 08:53:12.547] Listed 0 items. W1025 08:53:13.636] Listed 0 items. W1025 08:53:15.891] W1025 08:53:15.892] To show all fields of the firewall, please show in JSON format: --format=json W1025 08:53:15.892] To show all fields in table format, please see the examples in --help. W1025 08:53:15.892] W1025 08:53:17.044] Listed 0 items. W1025 08:53:18.287] Listed 0 items. W1025 08:53:19.228] 2020/10/25 08:53:19 process.go:155: Step './cluster/gce/list-resources.sh' finished in 11.504295686s W1025 08:53:19.228] 2020/10/25 08:53:19 process.go:153: Running: ./hack/e2e-internal/e2e-up.sh W1025 08:53:19.288] Project: k8s-boskos-gce-project-17 W1025 08:53:19.288] Network Project: k8s-boskos-gce-project-17 W1025 08:53:19.289] Zone: us-central1-f W1025 08:53:19.342] ... Starting cluster in us-central1-f using provider gce W1025 08:53:19.342] ... calling verify-prereqs W1025 08:53:20.000] ... calling verify-kube-binaries W1025 08:53:20.157] ... calling verify-release-tars W1025 08:53:20.169] ... calling kube-up W1025 08:53:20.176] Project: k8s-boskos-gce-project-17 W1025 08:53:20.176] Network Project: k8s-boskos-gce-project-17 W1025 08:53:20.176] Zone: us-central1-f I1025 08:53:23.669] +++ Staging tars to Google Storage: gs://kubernetes-staging-a425961e09/bootstrap-e2e-devel I1025 08:53:27.197] +++ kubernetes-server-linux-amd64.tar.gz uploaded earlier, cloud and local file md5 match (md5 = b24185e1c269b81eaf8535934156c994) I1025 08:53:29.496] +++ kubernetes-manifests.tar.gz uploaded earlier, cloud and local file md5 match (md5 = 187f02ed9a45facb840f69dbe9bcd905) I1025 08:53:30.543] Creating new auto network: bootstrap-e2e W1025 08:54:27.881] Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/networks/bootstrap-e2e]. I1025 08:54:28.171] NAME SUBNET_MODE BGP_ROUTING_MODE IPV4_RANGE GATEWAY_IPV4 I1025 08:54:28.171] bootstrap-e2e AUTO REGIONAL W1025 08:54:28.272] W1025 08:54:28.272] Instances on this network will not be reachable until firewall rules W1025 08:54:28.272] are created. As an example, you can allow all internal traffic between W1025 08:54:28.272] instances as well as SSH, RDP, and ICMP by running: W1025 08:54:28.273] W1025 08:54:28.273] $ gcloud compute firewall-rules create --network bootstrap-e2e --allow tcp,udp,icmp --source-ranges W1025 08:54:28.273] $ gcloud compute firewall-rules create --network bootstrap-e2e --allow tcp:22,tcp:3389,icmp W1025 08:54:28.273] I1025 08:54:31.689] IP aliases are disabled. I1025 08:54:32.801] Found subnet for region us-central1 in network bootstrap-e2e: bootstrap-e2e I1025 08:54:32.805] Starting master and configuring firewalls I1025 08:54:32.806] Configuring firewall for apiserver konnectivity server W1025 08:54:35.421] Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/zones/us-central1-f/disks/bootstrap-e2e-master-pd]. I1025 08:54:35.602] NAME ZONE SIZE_GB TYPE STATUS I1025 08:54:35.602] bootstrap-e2e-master-pd us-central1-f 20 pd-ssd READY W1025 08:54:35.703] W1025 08:54:35.703] New disks are unformatted. You must format and mount a disk before it W1025 08:54:35.703] can be used. You can find instructions on how to do this at: W1025 08:54:35.703] W1025 08:54:35.704] https://cloud.google.com/compute/docs/disks/add-persistent-disk#formatting W1025 08:54:35.704] W1025 08:54:35.802] Creating firewall... W1025 08:54:35.803] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-default-internal-master]. I1025 08:54:35.995] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1025 08:54:35.996] bootstrap-e2e-default-internal-master bootstrap-e2e INGRESS 1000 tcp:1-2379,tcp:2382-65535,udp:1-65535,icmp False W1025 08:54:36.096] done. W1025 08:54:38.322] Creating firewall... W1025 08:54:38.322] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-default-internal-node]. W1025 08:54:38.412] Creating firewall... W1025 08:54:38.413] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-default-ssh]. I1025 08:54:38.543] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1025 08:54:38.543] bootstrap-e2e-default-internal-node bootstrap-e2e INGRESS 1000 tcp:1-65535,udp:1-65535,icmp False I1025 08:54:38.638] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1025 08:54:38.638] bootstrap-e2e-default-ssh bootstrap-e2e INGRESS 1000 tcp:22 False W1025 08:54:38.739] done. W1025 08:54:38.739] done. W1025 08:54:38.927] Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/regions/us-central1/addresses/bootstrap-e2e-master-ip]. W1025 08:54:41.260] Creating firewall... W1025 08:54:41.260] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-master-https]. I1025 08:54:41.466] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1025 08:54:41.467] bootstrap-e2e-master-https bootstrap-e2e INGRESS 1000 tcp:443 False I1025 08:54:41.487] Generating certs for alternate-names: IP:34.122.2.172,IP:10.0.0.1,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local,DNS:bootstrap-e2e-master W1025 08:54:41.587] done. W1025 08:54:43.997] Creating firewall... W1025 08:54:43.997] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-master-etcd]. W1025 08:54:44.136] 2020/10/25 08:54:44 [INFO] generating a new CA key and certificate from CSR W1025 08:54:44.136] 2020/10/25 08:54:44 [INFO] generate received request W1025 08:54:44.136] 2020/10/25 08:54:44 [INFO] received CSR W1025 08:54:44.137] 2020/10/25 08:54:44 [INFO] generating key: ecdsa-256 W1025 08:54:44.154] 2020/10/25 08:54:44 [INFO] encoded CSR W1025 08:54:44.155] 2020/10/25 08:54:44 [INFO] signed certificate with serial number 512184119030136278954118097810974595992207345966 W1025 08:54:44.165] 2020/10/25 08:54:44 [INFO] generate received request W1025 08:54:44.166] 2020/10/25 08:54:44 [INFO] received CSR W1025 08:54:44.166] 2020/10/25 08:54:44 [INFO] generating key: ecdsa-256 W1025 08:54:44.192] 2020/10/25 08:54:44 [INFO] encoded CSR W1025 08:54:44.192] 2020/10/25 08:54:44 [INFO] signed certificate with serial number 687248384852574927018447503878045490064781010214 W1025 08:54:44.194] done. W1025 08:54:44.225] 2020/10/25 08:54:44 [INFO] generate received request W1025 08:54:44.225] 2020/10/25 08:54:44 [INFO] received CSR W1025 08:54:44.226] 2020/10/25 08:54:44 [INFO] generating key: ecdsa-256 W1025 08:54:44.253] 2020/10/25 08:54:44 [INFO] encoded CSR W1025 08:54:44.253] 2020/10/25 08:54:44 [INFO] signed certificate with serial number 106068229119739476338600067895925897181264226241 W1025 08:54:44.270] 2020/10/25 08:54:44 [INFO] generate received request W1025 08:54:44.270] 2020/10/25 08:54:44 [INFO] received CSR W1025 08:54:44.270] 2020/10/25 08:54:44 [INFO] generating key: ecdsa-256 W1025 08:54:44.288] 2020/10/25 08:54:44 [INFO] encoded CSR W1025 08:54:44.289] 2020/10/25 08:54:44 [INFO] signed certificate with serial number 226831558536978903223811924809540310486295847242 W1025 08:54:44.289] 2020/10/25 08:54:44 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for W1025 08:54:44.290] websites. For more information see the Baseline Requirements for the Issuance and Management W1025 08:54:44.290] of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org); W1025 08:54:44.290] specifically, section 10.2.3 ("Information Requirements"). I1025 08:54:44.390] Generate peer certificates... I1025 08:54:44.391] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1025 08:54:44.391] bootstrap-e2e-master-etcd bootstrap-e2e INGRESS 1000 tcp:2380,tcp:2381 False I1025 08:54:44.391] Generate server certificates... I1025 08:54:44.391] Generate client certificates... W1025 08:54:49.675] Creating firewall... W1025 08:54:49.675] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-minion-all]. I1025 08:54:49.867] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1025 08:54:49.867] bootstrap-e2e-minion-all bootstrap-e2e INGRESS 1000 tcp,udp,icmp,esp,ah,sctp False W1025 08:54:49.968] done. W1025 08:54:57.554] WARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance. W1025 08:54:57.554] Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/zones/us-central1-f/instances/bootstrap-e2e-master]. W1025 08:54:57.554] WARNING: Some requests generated warnings: W1025 08:54:57.555] - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details. W1025 08:54:57.555] - The resource 'projects/cos-cloud/global/images/cos-85-13310-1041-9' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-85-13310-1041-14'. W1025 08:54:57.555] W1025 08:54:57.555] NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS W1025 08:54:57.555] bootstrap-e2e-master us-central1-f n1-standard-1 10.128.0.2 34.122.2.172 RUNNING I1025 08:54:57.656] Creating nodes. W1025 08:55:00.032] /go/src/github.com/helm/charts/kubernetes/cluster/../cluster/../cluster/gce/util.sh: line 1537: WINDOWS_CONTAINER_RUNTIME_ENDPOINT: unbound variable I1025 08:55:00.133] Using subnet bootstrap-e2e W1025 08:55:01.133] Attempt 1 to create bootstrap-e2e-minion-template W1025 08:55:01.797] WARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance. W1025 08:55:03.837] Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/instanceTemplates/bootstrap-e2e-minion-template]. W1025 08:55:04.135] NAME MACHINE_TYPE PREEMPTIBLE CREATION_TIMESTAMP W1025 08:55:04.135] bootstrap-e2e-minion-template n1-standard-2 2020-10-25T01:55:02.504-07:00 I1025 08:55:04.301] Using subnet bootstrap-e2e W1025 08:55:05.283] Attempt 1 to create bootstrap-e2e-windows-node-template W1025 08:55:05.947] WARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance. W1025 08:55:08.192] Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/instanceTemplates/bootstrap-e2e-windows-node-template]. W1025 08:55:08.425] NAME MACHINE_TYPE PREEMPTIBLE CREATION_TIMESTAMP W1025 08:55:08.426] bootstrap-e2e-windows-node-template n1-standard-2 2020-10-25T01:55:06.612-07:00 W1025 08:55:19.994] Created [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/zones/us-central1-f/instanceGroupManagers/bootstrap-e2e-minion-group]. I1025 08:55:20.411] NAME LOCATION SCOPE BASE_INSTANCE_NAME SIZE TARGET_SIZE INSTANCE_TEMPLATE AUTOSCALED I1025 08:55:20.411] bootstrap-e2e-minion-group us-central1-f zone bootstrap-e2e-minion-group 0 3 bootstrap-e2e-minion-template no I1025 08:55:43.093] Waiting for group to become stable, current operations: creating: 3 I1025 08:55:43.093] Waiting for group to become stable, current operations: creating: 1 I1025 08:55:43.093] Group is stable W1025 08:55:48.680] INSTANCE_GROUPS=bootstrap-e2e-minion-group W1025 08:55:48.681] NODE_NAMES=bootstrap-e2e-minion-group-316b bootstrap-e2e-minion-group-tn5f bootstrap-e2e-minion-group-xtbb W1025 08:55:48.681] Trying to find master named 'bootstrap-e2e-master' W1025 08:55:48.681] Looking for address 'bootstrap-e2e-master-ip' I1025 08:55:49.733] Waiting up to 300 seconds for cluster initialization. I1025 08:55:49.733] I1025 08:55:49.733] This will continually check to see if the API for kubernetes is reachable. I1025 08:55:49.733] This may time out if there was some uncaught error during start up. I1025 08:55:49.733] W1025 08:55:49.834] Using master: bootstrap-e2e-master (external IP: 34.122.2.172; internal IP: (not set)) I1025 08:56:54.342] .................Kubernetes cluster created. I1025 08:56:54.490] Cluster "k8s-boskos-gce-project-17_bootstrap-e2e" set. I1025 08:56:54.644] User "k8s-boskos-gce-project-17_bootstrap-e2e" set. I1025 08:56:54.790] Context "k8s-boskos-gce-project-17_bootstrap-e2e" created. I1025 08:56:54.938] Switched to context "k8s-boskos-gce-project-17_bootstrap-e2e". I1025 08:56:55.092] User "k8s-boskos-gce-project-17_bootstrap-e2e-basic-auth" set. I1025 08:56:55.096] Wrote config for k8s-boskos-gce-project-17_bootstrap-e2e to /workspace/.kube/config W1025 08:57:04.549] Updated [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/zones/us-central1-f/instances/bootstrap-e2e-master]. I1025 08:57:04.993] I1025 08:57:04.993] Kubernetes cluster is running. The master is running at: I1025 08:57:04.993] I1025 08:57:04.993]  https://34.122.2.172 I1025 08:57:04.993] I1025 08:57:04.994] The user name and password to use is located in /workspace/.kube/config. I1025 08:57:04.994] I1025 08:57:05.056] Validating gce cluster, MULTIZONE= W1025 08:57:05.157] ... calling validate-cluster W1025 08:57:05.157] Project: k8s-boskos-gce-project-17 W1025 08:57:05.157] Network Project: k8s-boskos-gce-project-17 W1025 08:57:05.157] Zone: us-central1-f W1025 08:57:07.139] No resources found I1025 08:57:07.239] Waiting for 4 ready nodes. 0 ready nodes, 0 registered. Retrying. I1025 08:57:22.493] Waiting for 4 ready nodes. 0 ready nodes, 1 registered. Retrying. I1025 08:57:37.722] Waiting for 4 ready nodes. 2 ready nodes, 4 registered. Retrying. I1025 08:57:52.909] Found 4 node(s). I1025 08:57:53.083] NAME STATUS ROLES AGE VERSION I1025 08:57:53.083] bootstrap-e2e-master Ready,SchedulingDisabled 31s v1.20.0-alpha.3.114+5935fcd704fe89 I1025 08:57:53.083] bootstrap-e2e-minion-group-316b Ready 18s v1.20.0-alpha.3.114+5935fcd704fe89 I1025 08:57:53.084] bootstrap-e2e-minion-group-tn5f Ready 18s v1.20.0-alpha.3.114+5935fcd704fe89 I1025 08:57:53.084] bootstrap-e2e-minion-group-xtbb Ready 18s v1.20.0-alpha.3.114+5935fcd704fe89 W1025 08:57:53.273] Warning: v1 ComponentStatus is deprecated in v1.19+ I1025 08:57:53.374] Validate output: W1025 08:57:53.485] Warning: v1 ComponentStatus is deprecated in v1.19+ W1025 08:57:53.492] Done, listing cluster services: W1025 08:57:53.492] I1025 08:57:53.593] NAME STATUS MESSAGE ERROR I1025 08:57:53.593] controller-manager Healthy ok I1025 08:57:53.593] etcd-1 Healthy {"health":"true"} I1025 08:57:53.593] scheduler Healthy ok I1025 08:57:53.594] etcd-0 Healthy {"health":"true"} I1025 08:57:53.594] Cluster validation succeeded I1025 08:57:53.675] Kubernetes control plane is running at https://34.122.2.172 I1025 08:57:53.675] GLBCDefaultBackend is running at https://34.122.2.172/api/v1/namespaces/kube-system/services/default-http-backend:http/proxy I1025 08:57:53.675] CoreDNS is running at https://34.122.2.172/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy I1025 08:57:53.676] Metrics-server is running at https://34.122.2.172/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy I1025 08:57:53.676] I1025 08:57:53.676] To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. I1025 08:57:53.680] I1025 08:57:59.847] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1025 08:57:59.847] bootstrap-e2e-minion-http-alt bootstrap-e2e INGRESS 1000 tcp:80,tcp:8080 False I1025 08:58:00.849] allowed: I1025 08:58:00.849] - IPProtocol: tcp I1025 08:58:00.849] ports: I1025 08:58:00.849] - '80' I1025 08:58:00.849] - IPProtocol: tcp I1025 08:58:00.849] ports: I1025 08:58:00.850] - '8080' I1025 08:58:00.850] creationTimestamp: '2020-10-25T01:57:54.802-07:00' I1025 08:58:00.850] description: '' I1025 08:58:00.850] direction: INGRESS I1025 08:58:00.850] disabled: false I1025 08:58:00.850] id: '4451503200951369981' I1025 08:58:00.851] kind: compute#firewall I1025 08:58:00.851] logConfig: I1025 08:58:00.851] enable: false I1025 08:58:00.851] name: bootstrap-e2e-minion-http-alt I1025 08:58:00.851] network: https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/networks/bootstrap-e2e I1025 08:58:00.851] priority: 1000 I1025 08:58:00.852] selfLink: https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-minion-http-alt I1025 08:58:00.852] sourceRanges: I1025 08:58:00.852] - 0.0.0.0/0 I1025 08:58:00.852] targetTags: I1025 08:58:00.852] - bootstrap-e2e-minion I1025 08:58:06.915] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1025 08:58:06.915] bootstrap-e2e-minion-nodeports bootstrap-e2e INGRESS 1000 tcp:30000-32767,udp:30000-32767 False I1025 08:58:08.039] allowed: I1025 08:58:08.039] - IPProtocol: tcp I1025 08:58:08.039] ports: I1025 08:58:08.039] - 30000-32767 I1025 08:58:08.039] - IPProtocol: udp I1025 08:58:08.039] ports: I1025 08:58:08.040] - 30000-32767 I1025 08:58:08.040] creationTimestamp: '2020-10-25T01:58:02.098-07:00' I1025 08:58:08.040] description: '' I1025 08:58:08.040] direction: INGRESS I1025 08:58:08.040] disabled: false I1025 08:58:08.040] id: '4102137560228724981' I1025 08:58:08.040] kind: compute#firewall I1025 08:58:08.040] logConfig: I1025 08:58:08.040] enable: false I1025 08:58:08.040] name: bootstrap-e2e-minion-nodeports I1025 08:58:08.041] network: https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/networks/bootstrap-e2e I1025 08:58:08.041] priority: 1000 I1025 08:58:08.041] selfLink: https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-minion-nodeports I1025 08:58:08.041] sourceRanges: I1025 08:58:08.041] - 0.0.0.0/0 I1025 08:58:08.041] targetTags: I1025 08:58:08.041] - bootstrap-e2e-minion W1025 08:58:08.226] 2020/10/25 08:58:08 process.go:155: Step './hack/e2e-internal/e2e-up.sh' finished in 4m48.997865145s W1025 08:58:08.226] 2020/10/25 08:58:08 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false version W1025 08:58:08.447] 2020/10/25 08:58:08 process.go:155: Step './cluster/kubectl.sh --match-server-version=false version' finished in 220.957067ms W1025 08:58:08.447] 2020/10/25 08:58:08 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false get nodes -oyaml W1025 08:58:08.855] 2020/10/25 08:58:08 process.go:155: Step './cluster/kubectl.sh --match-server-version=false get nodes -oyaml' finished in 407.831558ms W1025 08:58:08.855] 2020/10/25 08:58:08 e2e.go:456: Listing resources... W1025 08:58:08.855] 2020/10/25 08:58:08 process.go:153: Running: ./cluster/gce/list-resources.sh W1025 08:58:17.250] W1025 08:58:17.250] To show all fields of the firewall, please show in JSON format: --format=json W1025 08:58:17.250] To show all fields in table format, please see the examples in --help. W1025 08:58:17.250] W1025 08:58:18.416] Listed 0 items. W1025 08:58:19.513] Listed 0 items. W1025 08:58:20.383] 2020/10/25 08:58:20 process.go:155: Step './cluster/gce/list-resources.sh' finished in 11.528189018s W1025 08:58:20.384] 2020/10/25 08:58:20 util.go:56: Warning: Couldn't find directory src/k8s.io/charts under any of GOPATH /go, defaulting to /go/src/k8s.io/charts W1025 08:58:20.384] 2020/10/25 08:58:20 process.go:153: Running: /go/src/k8s.io/charts/test/helm-test-e2e.sh W1025 08:58:20.384] 2020/10/25 08:58:20 process.go:155: Step '/go/src/k8s.io/charts/test/helm-test-e2e.sh' finished in 313.137µs W1025 08:58:20.384] 2020/10/25 08:58:20 e2e.go:544: Dumping logs locally to: /workspace/_artifacts W1025 08:58:20.385] 2020/10/25 08:58:20 process.go:153: Running: ./cluster/log-dump/log-dump.sh /workspace/_artifacts W1025 08:58:20.460] Trying to find master named 'bootstrap-e2e-master' W1025 08:58:20.460] Looking for address 'bootstrap-e2e-master-ip' I1025 08:58:20.561] Checking for custom logdump instances, if any I1025 08:58:20.561] Sourcing kube-util.sh I1025 08:58:20.561] Detecting project I1025 08:58:20.561] Project: k8s-boskos-gce-project-17 I1025 08:58:20.562] Network Project: k8s-boskos-gce-project-17 I1025 08:58:20.562] Zone: us-central1-f I1025 08:58:20.562] Dumping logs from master locally to '/workspace/_artifacts' W1025 08:58:21.476] Using master: bootstrap-e2e-master (external IP: 34.122.2.172; internal IP: (not set)) I1025 08:58:47.319] Changing logfiles to be world-readable for download I1025 08:58:50.163] Copying 'kube-apiserver.log kube-apiserver-audit.log kube-scheduler.log kube-controller-manager.log etcd.log etcd-events.log glbc.log cluster-autoscaler.log kube-addon-manager.log konnectivity-server.log fluentd.log kubelet.cov startupscript.log' from bootstrap-e2e-master W1025 08:58:51.078] W1025 08:58:51.078] Specify --start=56845 in the next get-serial-port-output invocation to get only the new output starting from here. W1025 08:58:52.852] scp: /var/log/cluster-autoscaler.log*: No such file or directory W1025 08:58:52.854] scp: /var/log/konnectivity-server.log*: No such file or directory W1025 08:58:52.854] scp: /var/log/fluentd.log*: No such file or directory W1025 08:58:52.854] scp: /var/log/kubelet.cov*: No such file or directory W1025 08:58:52.854] scp: /var/log/startupscript.log*: No such file or directory W1025 08:58:52.859] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. I1025 08:58:53.041] Dumping logs from nodes locally to '/workspace/_artifacts' I1025 08:58:53.041] Detecting nodes in the cluster I1025 08:59:25.103] Changing logfiles to be world-readable for download I1025 08:59:25.462] Changing logfiles to be world-readable for download I1025 08:59:25.800] Changing logfiles to be world-readable for download I1025 08:59:28.074] Copying 'kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov startupscript.log' from bootstrap-e2e-minion-group-xtbb I1025 08:59:28.342] Copying 'kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov startupscript.log' from bootstrap-e2e-minion-group-316b I1025 08:59:28.782] Copying 'kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov startupscript.log' from bootstrap-e2e-minion-group-tn5f W1025 08:59:29.046] W1025 08:59:29.046] Specify --start=67620 in the next get-serial-port-output invocation to get only the new output starting from here. W1025 08:59:29.585] W1025 08:59:29.585] Specify --start=67568 in the next get-serial-port-output invocation to get only the new output starting from here. W1025 08:59:29.940] W1025 08:59:29.940] Specify --start=67418 in the next get-serial-port-output invocation to get only the new output starting from here. W1025 08:59:30.422] scp: /var/log/fluentd.log*: No such file or directory W1025 08:59:30.422] scp: /var/log/node-problem-detector.log*: No such file or directory W1025 08:59:30.422] scp: /var/log/kubelet.cov*: No such file or directory W1025 08:59:30.422] scp: /var/log/startupscript.log*: No such file or directory W1025 08:59:30.425] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1025 08:59:30.942] scp: /var/log/fluentd.log*: No such file or directory W1025 08:59:30.942] scp: /var/log/node-problem-detector.log*: No such file or directory W1025 08:59:30.942] scp: /var/log/kubelet.cov*: No such file or directory W1025 08:59:30.942] scp: /var/log/startupscript.log*: No such file or directory W1025 08:59:30.947] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1025 08:59:31.400] scp: /var/log/fluentd.log*: No such file or directory W1025 08:59:31.401] scp: /var/log/node-problem-detector.log*: No such file or directory W1025 08:59:31.401] scp: /var/log/kubelet.cov*: No such file or directory W1025 08:59:31.401] scp: /var/log/startupscript.log*: No such file or directory W1025 08:59:31.405] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1025 08:59:35.332] INSTANCE_GROUPS=bootstrap-e2e-minion-group W1025 08:59:35.332] NODE_NAMES=bootstrap-e2e-minion-group-316b bootstrap-e2e-minion-group-tn5f bootstrap-e2e-minion-group-xtbb I1025 08:59:36.509] Failures for bootstrap-e2e-minion-group (if any): W1025 08:59:38.823] 2020/10/25 08:59:38 process.go:155: Step './cluster/log-dump/log-dump.sh /workspace/_artifacts' finished in 1m18.439274797s W1025 08:59:38.823] 2020/10/25 08:59:38 e2e.go:456: Listing resources... W1025 08:59:38.824] 2020/10/25 08:59:38 process.go:153: Running: ./cluster/gce/list-resources.sh W1025 08:59:47.024] W1025 08:59:47.025] To show all fields of the firewall, please show in JSON format: --format=json W1025 08:59:47.025] To show all fields in table format, please see the examples in --help. W1025 08:59:47.025] W1025 08:59:48.172] Listed 0 items. W1025 08:59:49.196] Listed 0 items. W1025 08:59:50.057] 2020/10/25 08:59:50 process.go:155: Step './cluster/gce/list-resources.sh' finished in 11.233528725s W1025 08:59:50.057] 2020/10/25 08:59:50 process.go:153: Running: ./hack/e2e-internal/e2e-down.sh W1025 08:59:50.115] Project: k8s-boskos-gce-project-17 W1025 08:59:50.115] Network Project: k8s-boskos-gce-project-17 W1025 08:59:50.115] Zone: us-central1-f I1025 08:59:50.215] Shutting down test cluster in background. W1025 08:59:55.570] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-minion-http-alt]. W1025 08:59:57.663] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-minion-nodeports]. I1025 08:59:57.954] Bringing down cluster using provider: gce W1025 08:59:58.055] ... calling verify-prereqs W1025 08:59:58.611] ... calling verify-kube-binaries W1025 08:59:58.761] ... calling kube-down W1025 08:59:58.761] Project: k8s-boskos-gce-project-17 W1025 08:59:58.761] Network Project: k8s-boskos-gce-project-17 W1025 08:59:58.761] Zone: us-central1-f W1025 09:00:02.688] INSTANCE_GROUPS=bootstrap-e2e-minion-group W1025 09:00:02.689] NODE_NAMES=bootstrap-e2e-minion-group-316b bootstrap-e2e-minion-group-tn5f bootstrap-e2e-minion-group-xtbb I1025 09:00:02.789] Bringing down cluster W1025 09:02:12.794] Deleting Managed Instance Group... W1025 09:02:12.795] ...Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/zones/us-central1-f/instanceGroupManagers/bootstrap-e2e-minion-group]. W1025 09:02:12.798] done. W1025 09:02:16.284] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/instanceTemplates/bootstrap-e2e-minion-template]. W1025 09:02:17.717] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/instanceTemplates/bootstrap-e2e-windows-node-template]. I1025 09:02:22.971] Successfully executed 'curl -s --cacert /etc/srv/kubernetes/pki/etcd-apiserver-ca.crt --cert /etc/srv/kubernetes/pki/etcd-apiserver-client.crt --key /etc/srv/kubernetes/pki/etcd-apiserver-client.key https://127.0.0.1:2379/v2/members/$(curl -s --cacert /etc/srv/kubernetes/pki/etcd-apiserver-ca.crt --cert /etc/srv/kubernetes/pki/etcd-apiserver-client.crt --key /etc/srv/kubernetes/pki/etcd-apiserver-client.key https://127.0.0.1:2379/v2/members -XGET | sed 's/{\"id/\n/g' | grep bootstrap-e2e-master\" | cut -f 3 -d \") -XDELETE -L 2>/dev/null' on bootstrap-e2e-master I1025 09:02:22.971] Removing etcd replica, name: bootstrap-e2e-master, port: 2379, result: 0 I1025 09:02:24.490] Successfully executed 'curl -s http://127.0.0.1:4002/v2/members/$(curl -s http://127.0.0.1:4002/v2/members -XGET | sed 's/{\"id/\n/g' | grep bootstrap-e2e-master\" | cut -f 3 -d \") -XDELETE -L 2>/dev/null' on bootstrap-e2e-master I1025 09:02:24.490] Removing etcd replica, name: bootstrap-e2e-master, port: 4002, result: 0 W1025 09:02:29.435] Updated [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/zones/us-central1-f/instances/bootstrap-e2e-master]. W1025 09:04:21.249] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/zones/us-central1-f/instances/bootstrap-e2e-master]. W1025 09:04:30.916] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-master-https]. W1025 09:04:33.254] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-minion-all]. W1025 09:04:33.445] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-master-etcd]. W1025 09:04:40.776] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/regions/us-central1/addresses/bootstrap-e2e-master-ip]. I1025 09:04:46.570] Deleting routes bootstrap-e2e-b6e5de63-7b36-4d3f-9e78-6b8bb106a753 W1025 09:04:53.167] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/routes/bootstrap-e2e-b6e5de63-7b36-4d3f-9e78-6b8bb106a753]. W1025 09:05:02.201] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-default-internal-master]. W1025 09:05:04.841] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-default-ssh]. W1025 09:05:05.012] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/firewalls/bootstrap-e2e-default-internal-node]. I1025 09:05:06.371] Deleting firewall rules remaining in network bootstrap-e2e: W1025 09:05:54.852] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-boskos-gce-project-17/global/networks/bootstrap-e2e]. I1025 09:05:55.368] Property "current-context" unset. I1025 09:05:55.523] Property "clusters.k8s-boskos-gce-project-17_bootstrap-e2e" unset. I1025 09:05:55.670] Property "users.k8s-boskos-gce-project-17_bootstrap-e2e" unset. I1025 09:05:55.817] Property "users.k8s-boskos-gce-project-17_bootstrap-e2e-basic-auth" unset. I1025 09:05:55.966] Property "contexts.k8s-boskos-gce-project-17_bootstrap-e2e" unset. I1025 09:05:55.970] Cleared config for k8s-boskos-gce-project-17_bootstrap-e2e from /workspace/.kube/config I1025 09:05:55.970] Done W1025 09:05:56.071] 2020/10/25 09:05:55 process.go:155: Step './hack/e2e-internal/e2e-down.sh' finished in 6m5.915428345s W1025 09:05:56.071] 2020/10/25 09:05:55 e2e.go:278: Sleeping for 30 seconds... W1025 09:06:25.973] 2020/10/25 09:06:25 e2e.go:456: Listing resources... W1025 09:06:25.973] 2020/10/25 09:06:25 process.go:153: Running: ./cluster/gce/list-resources.sh W1025 09:06:26.962] Listed 0 items. W1025 09:06:28.039] Listed 0 items. W1025 09:06:29.925] Listed 0 items. W1025 09:06:31.045] Listed 0 items. W1025 09:06:32.107] Listed 0 items. W1025 09:06:34.309] W1025 09:06:34.309] To show all fields of the firewall, please show in JSON format: --format=json W1025 09:06:34.309] To show all fields in table format, please see the examples in --help. W1025 09:06:34.309] W1025 09:06:35.417] Listed 0 items. W1025 09:06:36.531] Listed 0 items. W1025 09:06:37.420] 2020/10/25 09:06:37 process.go:155: Step './cluster/gce/list-resources.sh' finished in 11.447632738s W1025 09:06:37.421] 2020/10/25 09:06:37 process.go:153: Running: diff -sw -U0 -F^\[.*\]$ /workspace/_artifacts/gcp-resources-before.txt /workspace/_artifacts/gcp-resources-after.txt W1025 09:06:37.426] 2020/10/25 09:06:37 process.go:155: Step 'diff -sw -U0 -F^\[.*\]$ /workspace/_artifacts/gcp-resources-before.txt /workspace/_artifacts/gcp-resources-after.txt' finished in 4.781603ms W1025 09:06:37.426] 2020/10/25 09:06:37 process.go:96: Saved XML output to /workspace/_artifacts/junit_runner.xml. W1025 09:06:37.435] 2020/10/25 09:06:37 main.go:316: Something went wrong: encountered 1 errors: [error starting /go/src/k8s.io/charts/test/helm-test-e2e.sh: fork/exec /go/src/k8s.io/charts/test/helm-test-e2e.sh: no such file or directory] W1025 09:06:37.440] Traceback (most recent call last): W1025 09:06:37.441] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 720, in W1025 09:06:37.441] main(parse_args()) W1025 09:06:37.441] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 570, in main W1025 09:06:37.441] mode.start(runner_args) W1025 09:06:37.441] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 228, in start W1025 09:06:37.442] check_env(env, self.command, *args) W1025 09:06:37.442] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 111, in check_env W1025 09:06:37.442] subprocess.check_call(cmd, env=env) W1025 09:06:37.442] File "/usr/lib/python2.7/subprocess.py", line 190, in check_call W1025 09:06:37.442] raise CalledProcessError(retcode, cmd) W1025 09:06:37.443] subprocess.CalledProcessError: Command '('kubetest', '--dump=/workspace/_artifacts', '--gcp-service-account=/etc/service-account/service-account.json', '--up', '--down', '--provider=gce', '--cluster=bootstrap-e2e', '--gcp-network=bootstrap-e2e', '--charts', '--check-leaked-resources', '--extract=ci/latest', '--gcp-node-image=gci', '--gcp-zone=us-central1-f', '--timeout=180m')' returned non-zero exit status 1 E1025 09:06:37.451] Command failed I1025 09:06:37.451] process 282 exited with code 1 after 14.9m E1025 09:06:37.451] FAIL: ci-kubernetes-charts-gce I1025 09:06:37.452] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1025 09:06:38.122] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] I1025 09:06:38.231] process 10804 exited with code 0 after 0.0m I1025 09:06:38.231] Call: gcloud config get-value account I1025 09:06:38.858] process 10817 exited with code 0 after 0.0m I1025 09:06:38.859] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com I1025 09:06:38.859] Upload result and artifacts... I1025 09:06:38.859] Gubernator results at https://gubernator.k8s.io/build/kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320286823146065920 I1025 09:06:38.859] Call: gsutil ls gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320286823146065920/artifacts W1025 09:06:39.820] CommandException: One or more URLs matched no objects. E1025 09:06:40.005] Command failed I1025 09:06:40.006] process 10830 exited with code 1 after 0.0m W1025 09:06:40.006] Remote dir gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320286823146065920/artifacts not exist yet I1025 09:06:40.006] Call: gsutil -m -q -o GSUtil:use_magicfile=True cp -r -c -z log,txt,xml /workspace/_artifacts gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320286823146065920/artifacts I1025 09:06:42.522] process 10973 exited with code 0 after 0.0m I1025 09:06:42.523] Call: git rev-parse HEAD I1025 09:06:42.527] process 11626 exited with code 0 after 0.0m I1025 09:06:42.527] Call: git rev-parse HEAD I1025 09:06:42.531] process 11627 exited with code 0 after 0.0m I1025 09:06:42.531] Call: gsutil stat gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/jobResultsCache.json I1025 09:06:43.636] process 11628 exited with code 0 after 0.0m I1025 09:06:43.637] Call: gsutil -q cat 'gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/jobResultsCache.json#1603613087950731' I1025 09:06:44.798] process 11771 exited with code 0 after 0.0m I1025 09:06:44.802] Call: gsutil -q -h Content-Type:application/json -h x-goog-if-generation-match:1603613087950731 cp /tmp/gsutil_54VBnm gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/jobResultsCache.json I1025 09:06:46.186] process 11916 exited with code 0 after 0.0m I1025 09:06:46.187] Call: gsutil -q -h Content-Type:application/json cp /tmp/gsutil_MXar9Q gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320286823146065920/finished.json I1025 09:06:47.566] process 12095 exited with code 0 after 0.0m I1025 09:06:47.567] Call: gsutil -q -h Content-Type:text/plain -h 'Cache-Control:private, max-age=0, no-transform' cp /tmp/gsutil_8KGes7 gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/latest-build.txt I1025 09:06:49.017] process 12274 exited with code 0 after 0.0m