W1026 08:14:37.518] ************************************************************************** bootstrap.py is deprecated! test-infra oncall does not support any job still using bootstrap.py. Please migrate your job to podutils! https://github.com/kubernetes/test-infra/blob/master/prow/pod-utilities.md ************************************************************************** I1026 08:14:37.519] Args: --job=ci-kubernetes-charts-gce --service-account=/etc/service-account/service-account.json --upload=gs://kubernetes-jenkins/logs --timeout=200 --repo=github.com/helm/charts --root=/go/src --service-account=/etc/service-account/service-account.json --scenario=kubernetes_e2e -- --charts --check-leaked-resources --extract=ci/latest --gcp-node-image=gci --gcp-zone=us-central1-f --provider=gce --test=false --timeout=180m I1026 08:14:37.519] Bootstrap ci-kubernetes-charts-gce... I1026 08:14:37.521] Builder: 40f2d0e8-1763-11eb-b256-6ee25ea2e440 I1026 08:14:37.521] Image: gcr.io/k8s-testimages/kubekins-e2e:v20201021-a62ce8a-master I1026 08:14:37.522] Gubernator results at https://gubernator.k8s.io/build/kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320639896909516800 I1026 08:14:37.522] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1026 08:14:38.277] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] I1026 08:14:38.401] process 34 exited with code 0 after 0.0m I1026 08:14:38.401] Call: gcloud config get-value account I1026 08:14:39.059] process 47 exited with code 0 after 0.0m I1026 08:14:39.059] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com I1026 08:14:39.060] Root: /go/src I1026 08:14:39.060] cd to /go/src I1026 08:14:39.060] Checkout: /go/src/github.com/helm/charts master to /go/src/github.com/helm/charts I1026 08:14:39.060] Call: git init github.com/helm/charts I1026 08:14:39.067] Initialized empty Git repository in /go/src/github.com/helm/charts/.git/ I1026 08:14:39.068] process 60 exited with code 0 after 0.0m I1026 08:14:39.068] Call: git config --local user.name 'K8S Bootstrap' I1026 08:14:39.073] process 61 exited with code 0 after 0.0m I1026 08:14:39.073] Call: git config --local user.email k8s_bootstrap@localhost I1026 08:14:39.078] process 62 exited with code 0 after 0.0m I1026 08:14:39.078] Call: git fetch --quiet --tags https://github.com/helm/charts master I1026 08:14:43.092] process 63 exited with code 0 after 0.1m I1026 08:14:43.093] Call: git checkout -B test FETCH_HEAD W1026 08:14:43.611] Switched to a new branch 'test' I1026 08:14:43.616] process 74 exited with code 0 after 0.0m I1026 08:14:43.616] Call: git show -s --format=format:%ct HEAD I1026 08:14:43.621] process 75 exited with code 0 after 0.0m I1026 08:14:43.622] Configure environment... I1026 08:14:43.623] Call: git show -s --format=format:%ct HEAD I1026 08:14:43.627] process 76 exited with code 0 after 0.0m I1026 08:14:43.627] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1026 08:14:44.342] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] I1026 08:14:44.568] process 77 exited with code 0 after 0.0m I1026 08:14:44.568] Call: gcloud config get-value account I1026 08:14:45.203] process 90 exited with code 0 after 0.0m I1026 08:14:45.204] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com I1026 08:14:45.204] Start 1320639896909516800 at unknown... I1026 08:14:45.206] Call: gsutil -q -h Content-Type:application/json cp /tmp/gsutil_bB6V3p gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320639896909516800/started.json I1026 08:14:46.815] process 103 exited with code 0 after 0.0m I1026 08:14:46.816] Call: /workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py --charts --check-leaked-resources --extract=ci/latest --gcp-node-image=gci --gcp-zone=us-central1-f --provider=gce --test=false --timeout=180m W1026 08:14:46.865] starts with local mode W1026 08:14:46.866] Environment: W1026 08:14:46.866] ARTIFACTS=/workspace/_artifacts W1026 08:14:46.866] AWS_SSH_PRIVATE_KEY_FILE=/root/.ssh/kube_aws_rsa W1026 08:14:46.866] AWS_SSH_PUBLIC_KEY_FILE=/root/.ssh/kube_aws_rsa.pub W1026 08:14:46.866] BAZEL_REMOTE_CACHE_ENABLED=false W1026 08:14:46.866] BAZEL_VERSION=2.2.0 W1026 08:14:46.867] BOOTSTRAP_MIGRATION=yes W1026 08:14:46.867] BOSKOS_METRICS_PORT=tcp://10.63.252.110:9090 W1026 08:14:46.867] BOSKOS_METRICS_PORT_9090_TCP=tcp://10.63.252.110:9090 W1026 08:14:46.867] BOSKOS_METRICS_PORT_9090_TCP_ADDR=10.63.252.110 W1026 08:14:46.867] BOSKOS_METRICS_PORT_9090_TCP_PORT=9090 W1026 08:14:46.867] BOSKOS_METRICS_PORT_9090_TCP_PROTO=tcp W1026 08:14:46.867] BOSKOS_METRICS_SERVICE_HOST=10.63.252.110 W1026 08:14:46.868] BOSKOS_METRICS_SERVICE_PORT=9090 W1026 08:14:46.868] BOSKOS_METRICS_SERVICE_PORT_METRICS=9090 W1026 08:14:46.868] BOSKOS_PORT=tcp://10.63.250.132:80 W1026 08:14:46.868] BOSKOS_PORT_80_TCP=tcp://10.63.250.132:80 W1026 08:14:46.868] BOSKOS_PORT_80_TCP_ADDR=10.63.250.132 W1026 08:14:46.868] BOSKOS_PORT_80_TCP_PORT=80 W1026 08:14:46.868] BOSKOS_PORT_80_TCP_PROTO=tcp W1026 08:14:46.869] BOSKOS_SERVICE_HOST=10.63.250.132 W1026 08:14:46.869] BOSKOS_SERVICE_PORT=80 W1026 08:14:46.869] BOSKOS_SERVICE_PORT_DEFAULT=80 W1026 08:14:46.869] BUILD_ID=1320639896909516800 W1026 08:14:46.869] BUILD_NUMBER=1320639896909516800 W1026 08:14:46.869] CI=true W1026 08:14:46.869] CLOUDSDK_COMPONENT_MANAGER_DISABLE_UPDATE_CHECK=true W1026 08:14:46.869] CLOUDSDK_CONFIG=/go/src/github.com/helm/charts/.config/gcloud W1026 08:14:46.870] CLOUDSDK_CORE_DISABLE_PROMPTS=1 W1026 08:14:46.870] CLOUDSDK_EXPERIMENTAL_FAST_COMPONENT_UPDATE=false W1026 08:14:46.870] DOCKER_IN_DOCKER_ENABLED=false W1026 08:14:46.870] DOCKER_IN_DOCKER_IPV6_ENABLED=false W1026 08:14:46.870] E2E_GOOGLE_APPLICATION_CREDENTIALS=/etc/service-account/service-account.json W1026 08:14:46.870] GCS_ARTIFACTS_DIR=gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320639896909516800/artifacts W1026 08:14:46.870] GOOGLE_APPLICATION_CREDENTIALS=/etc/service-account/service-account.json W1026 08:14:46.871] GOOGLE_APPLICATION_CREDENTIALS_DEPRECATED=Migrate to workload identity, contact sig-testing W1026 08:14:46.871] GOPATH=/go W1026 08:14:46.871] GOPROXY=https://proxy.golang.org W1026 08:14:46.871] GO_TARBALL=go1.15.2.linux-amd64.tar.gz W1026 08:14:46.871] HOME=/workspace W1026 08:14:46.871] HOSTNAME=40f2d0e8-1763-11eb-b256-6ee25ea2e440 W1026 08:14:46.871] IMAGE=gcr.io/k8s-testimages/kubekins-e2e:v20201021-a62ce8a-master W1026 08:14:46.872] INSTANCE_PREFIX=bootstrap-e2e W1026 08:14:46.872] JENKINS_GCE_SSH_PRIVATE_KEY_FILE=/workspace/.ssh/google_compute_engine W1026 08:14:46.872] JENKINS_GCE_SSH_PUBLIC_KEY_FILE=/workspace/.ssh/google_compute_engine.pub W1026 08:14:46.872] JOB_NAME=ci-kubernetes-charts-gce W1026 08:14:46.872] JOB_SPEC={"type":"periodic","job":"ci-kubernetes-charts-gce","buildid":"1320639896909516800","prowjobid":"40f2d0e8-1763-11eb-b256-6ee25ea2e440"} W1026 08:14:46.872] JOB_TYPE=periodic W1026 08:14:46.873] KUBERNETES_PORT=tcp://10.63.240.1:443 W1026 08:14:46.873] KUBERNETES_PORT_443_TCP=tcp://10.63.240.1:443 W1026 08:14:46.873] KUBERNETES_PORT_443_TCP_ADDR=10.63.240.1 W1026 08:14:46.873] KUBERNETES_PORT_443_TCP_PORT=443 W1026 08:14:46.873] KUBERNETES_PORT_443_TCP_PROTO=tcp W1026 08:14:46.873] KUBERNETES_SERVICE_HOST=10.63.240.1 W1026 08:14:46.873] KUBERNETES_SERVICE_PORT=443 W1026 08:14:46.874] KUBERNETES_SERVICE_PORT_HTTPS=443 W1026 08:14:46.874] KUBETEST_IN_DOCKER=true W1026 08:14:46.874] KUBETEST_MANUAL_DUMP=y W1026 08:14:46.874] KUBE_AWS_INSTANCE_PREFIX=bootstrap-e2e W1026 08:14:46.874] KUBE_GCE_INSTANCE_PREFIX=bootstrap-e2e W1026 08:14:46.874] LOG_DUMP_SCRIPT_PATH=/workspace/log-dump.sh W1026 08:14:46.874] NODE_NAME=40f2d0e8-1763-11eb-b256-6ee25ea2e440 W1026 08:14:46.875] PATH=/go/bin:/go/bin:/usr/local/go/bin:/google-cloud-sdk/bin:/workspace:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin W1026 08:14:46.875] PROW_JOB_ID=40f2d0e8-1763-11eb-b256-6ee25ea2e440 W1026 08:14:46.875] PWD=/workspace W1026 08:14:46.875] SHLVL=2 W1026 08:14:46.875] SOURCE_DATE_EPOCH=1603659301 W1026 08:14:46.875] TERM=xterm W1026 08:14:46.875] USER=prow W1026 08:14:46.875] WORKSPACE=/workspace W1026 08:14:46.876] _=./test-infra/jenkins/bootstrap.py W1026 08:14:46.876] Run: ('kubetest', '--dump=/workspace/_artifacts', '--gcp-service-account=/etc/service-account/service-account.json', '--up', '--down', '--provider=gce', '--cluster=bootstrap-e2e', '--gcp-network=bootstrap-e2e', '--charts', '--check-leaked-resources', '--extract=ci/latest', '--gcp-node-image=gci', '--gcp-zone=us-central1-f', '--timeout=180m') W1026 08:14:46.900] 2020/10/26 08:14:46 Warning: Couldn't find directory src/sigs.k8s.io/cloud-provider-azure under any of GOPATH /go, defaulting to /go/src/k8s.io/cloud-provider-azure W1026 08:14:46.902] 2020/10/26 08:14:46 extract_k8s.go:112: Matched extraction strategy: ^ci/(.+)$ W1026 08:14:46.902] 2020/10/26 08:14:46 main.go:329: Limiting testing to 3h0m0s W1026 08:14:46.902] 2020/10/26 08:14:46 process.go:153: Running: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1026 08:14:47.548] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] W1026 08:14:47.654] 2020/10/26 08:14:47 process.go:155: Step 'gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json' finished in 751.534649ms W1026 08:14:47.654] 2020/10/26 08:14:47 main.go:728: --gcp-project is missing, trying to fetch a project from boskos. W1026 08:14:47.655] (for local runs please set --gcp-project to your dev project) W1026 08:14:47.655] 2020/10/26 08:14:47 main.go:740: provider gce, will acquire project type gce-project from boskos W1026 08:14:47.667] 2020/10/26 08:14:47 process.go:153: Running: gcloud config set project k8s-jkns-gce-sd-log W1026 08:14:48.193] Updated property [core/project]. W1026 08:14:48.481] 2020/10/26 08:14:48 process.go:155: Step 'gcloud config set project k8s-jkns-gce-sd-log' finished in 813.949704ms W1026 08:14:48.481] 2020/10/26 08:14:48 main.go:779: Checking existing of GCP ssh keys... W1026 08:14:48.482] 2020/10/26 08:14:48 main.go:789: Checking presence of public key in k8s-jkns-gce-sd-log W1026 08:14:48.482] 2020/10/26 08:14:48 process.go:153: Running: gcloud compute --project=k8s-jkns-gce-sd-log project-info describe W1026 08:14:50.097] 2020/10/26 08:14:50 process.go:155: Step 'gcloud compute --project=k8s-jkns-gce-sd-log project-info describe' finished in 1.616020117s W1026 08:14:50.205] 2020/10/26 08:14:50 extract_k8s.go:295: U=https://storage.googleapis.com/kubernetes-release-dev/ci R=v1.20.0-alpha.3.116+16e18a590f32b9 get-kube.sh W1026 08:14:50.205] 2020/10/26 08:14:50 process.go:153: Running: /workspace/get-kube.sh I1026 08:14:50.306] Downloading kubernetes release v1.20.0-alpha.3.116+16e18a590f32b9 I1026 08:14:50.306] from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes.tar.gz I1026 08:14:50.306] to /go/src/github.com/helm/charts/kubernetes.tar.gz W1026 08:14:51.320] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes.tar.gz... W1026 08:14:51.426] / [0 files][ 0.0 B/487.6 KiB] / [1 files][487.6 KiB/487.6 KiB] W1026 08:14:51.426] Operation completed over 1 objects/487.6 KiB. I1026 08:14:51.639] Unpacking kubernetes release v1.20.0-alpha.3.116+16e18a590f32b9 I1026 08:14:51.718] Kubernetes release: v1.20.0-alpha.3.116+16e18a590f32b9 I1026 08:14:51.718] Server: linux/amd64 (to override, set KUBERNETES_SERVER_ARCH) I1026 08:14:51.719] Client: linux/amd64 (autodetected) (to override, set KUBERNETES_CLIENT_OS and/or KUBERNETES_CLIENT_ARCH) I1026 08:14:51.719] I1026 08:14:51.719] Will download kubernetes-server-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9 I1026 08:14:51.719] Will download and extract kubernetes-client-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9 I1026 08:14:51.720] Will download and extract kubernetes-test tarball(s) from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9 W1026 08:14:52.769] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes-server-linux-amd64.tar.gz... W1026 08:14:52.771] / [0 files][ 0.0 B/502.2 MiB] ==> NOTE: You are downloading one or more large file(s), which would W1026 08:14:52.771] run significantly faster if you enabled sliced object downloads. This W1026 08:14:52.771] feature is enabled by default but requires that compiled crcmod be W1026 08:14:52.771] installed (see "gsutil help crcmod"). W1026 08:14:52.772] W1026 08:14:58.464] - - [0 files][121.7 MiB/502.2 MiB] \ | | [0 files][260.4 MiB/502.2 MiB] / / [0 files][395.0 MiB/502.2 MiB] - \ \ [1 files][502.2 MiB/502.2 MiB] | W1026 08:14:58.465] Operation completed over 1 objects/502.2 MiB. I1026 08:14:58.676] I1026 08:14:59.864] md5sum(kubernetes-server-linux-amd64.tar.gz)=e7d002a3e2857262b353740982d62b0c I1026 08:15:01.862] sha512sum(kubernetes-server-linux-amd64.tar.gz)=912eb1aa30aaf2af864bf7f124781093f856889d80dc11146e2a4e1edbb777a4ee2233be2c6847f9369591c279ae3f08858f35d49f94a7ce900bcf0b38780a19 I1026 08:15:01.863] W1026 08:15:02.919] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes-client-linux-amd64.tar.gz... W1026 08:15:03.409] / [0 files][ 0.0 B/ 22.7 MiB] / [1 files][ 22.7 MiB/ 22.7 MiB] W1026 08:15:03.410] Operation completed over 1 objects/22.7 MiB. I1026 08:15:03.617] I1026 08:15:03.677] md5sum(kubernetes-client-linux-amd64.tar.gz)=33717c5a370ea5f05cd05560e3cb8e0c I1026 08:15:03.775] sha512sum(kubernetes-client-linux-amd64.tar.gz)=4d4c5c8d0a454d1f3814652f542755d27dc1f0bedc16c84cdf6d53858035abbff664a15e4130dd84f8e0006c2a4d6279fc3fa8f352ba775ecf40a1dacc1e5cac I1026 08:15:03.775] I1026 08:15:03.776] Extracting /go/src/github.com/helm/charts/kubernetes/client/kubernetes-client-linux-amd64.tar.gz into /go/src/github.com/helm/charts/kubernetes/platforms/linux/amd64 I1026 08:15:04.273] Add '/go/src/github.com/helm/charts/kubernetes/client/bin' to your PATH to use newly-installed binaries. W1026 08:15:05.307] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes-test-portable.tar.gz... W1026 08:15:05.412] / [0 files][ 0.0 B/219.8 KiB] / [1 files][219.8 KiB/219.8 KiB] W1026 08:15:05.412] Operation completed over 1 objects/219.8 KiB. I1026 08:15:05.613] I1026 08:15:05.618] md5sum(kubernetes-test-portable.tar.gz)=d550def479686663ba5c84b42613f3c3 I1026 08:15:05.624] sha512sum(kubernetes-test-portable.tar.gz)=4aecc7823b95e9c3b2538024347f1d58975327b48201603a109cfc624f04365373dda638dd9d8109cecedd359b1e53121a534aa095e0fead8a28295b49e58094 I1026 08:15:05.624] I1026 08:15:05.624] Extracting kubernetes-test-portable.tar.gz into /go/src/github.com/helm/charts/kubernetes W1026 08:15:06.701] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes-test-linux-amd64.tar.gz... W1026 08:15:06.703] / [0 files][ 0.0 B/351.2 MiB] ==> NOTE: You are downloading one or more large file(s), which would W1026 08:15:06.703] run significantly faster if you enabled sliced object downloads. This W1026 08:15:06.703] feature is enabled by default but requires that compiled crcmod be W1026 08:15:06.703] installed (see "gsutil help crcmod"). W1026 08:15:06.704] W1026 08:15:10.825] - - [0 files][108.8 MiB/351.2 MiB] \ | | [0 files][256.8 MiB/351.2 MiB] / / [1 files][351.2 MiB/351.2 MiB] - W1026 08:15:10.826] Operation completed over 1 objects/351.2 MiB. I1026 08:15:11.040] I1026 08:15:11.870] md5sum(kubernetes-test-linux-amd64.tar.gz)=8e6a818d5d85bb3c231742ee6a3ac0be I1026 08:15:13.275] sha512sum(kubernetes-test-linux-amd64.tar.gz)=1dfd88ed99f6369e8f56bea0384cb1f51bb4bee0c3dae06134ac24d7b366125d36132b3fb92d972ced62f3d55fe8b5d94357af7b7a9bb456b03d07f161e8acf9 I1026 08:15:13.275] I1026 08:15:13.275] Extracting /go/src/github.com/helm/charts/kubernetes/test/kubernetes-test-linux-amd64.tar.gz into /go/src/github.com/helm/charts/kubernetes/platforms/linux/amd64 W1026 08:15:21.637] 2020/10/26 08:15:21 process.go:155: Step '/workspace/get-kube.sh' finished in 31.431952822s W1026 08:15:21.637] 2020/10/26 08:15:21 process.go:153: Running: ./hack/e2e-internal/e2e-down.sh W1026 08:15:21.700] Project: k8s-jkns-gce-sd-log W1026 08:15:21.700] Network Project: k8s-jkns-gce-sd-log W1026 08:15:21.700] Zone: us-central1-f I1026 08:15:21.800] Shutting down test cluster in background. I1026 08:15:23.949] Bringing down cluster using provider: gce W1026 08:15:24.050] ... calling verify-prereqs W1026 08:15:24.591] ... calling verify-kube-binaries W1026 08:15:24.747] ... calling kube-down W1026 08:15:24.748] Project: k8s-jkns-gce-sd-log W1026 08:15:24.748] Network Project: k8s-jkns-gce-sd-log W1026 08:15:24.748] Zone: us-central1-f W1026 08:15:27.005] INSTANCE_GROUPS= W1026 08:15:27.005] NODE_NAMES= I1026 08:15:27.105] Bringing down cluster I1026 08:15:52.127] Deleting firewall rules remaining in network bootstrap-e2e: W1026 08:15:55.243] W1026 08:15:55.243414 1871 loader.go:223] Config not found: /workspace/.kube/config W1026 08:15:55.395] W1026 08:15:55.395403 1923 loader.go:223] Config not found: /workspace/.kube/config W1026 08:15:55.396] W1026 08:15:55.395681 1923 loader.go:223] Config not found: /workspace/.kube/config I1026 08:15:55.496] Property "clusters.k8s-jkns-gce-sd-log_bootstrap-e2e" unset. I1026 08:15:55.545] Property "users.k8s-jkns-gce-sd-log_bootstrap-e2e" unset. W1026 08:15:55.645] W1026 08:15:55.544922 1975 loader.go:223] Config not found: /workspace/.kube/config W1026 08:15:55.646] W1026 08:15:55.545154 1975 loader.go:223] Config not found: /workspace/.kube/config W1026 08:15:55.696] W1026 08:15:55.696314 2026 loader.go:223] Config not found: /workspace/.kube/config W1026 08:15:55.697] W1026 08:15:55.696543 2026 loader.go:223] Config not found: /workspace/.kube/config I1026 08:15:55.797] Property "users.k8s-jkns-gce-sd-log_bootstrap-e2e-basic-auth" unset. I1026 08:15:55.849] Property "contexts.k8s-jkns-gce-sd-log_bootstrap-e2e" unset. I1026 08:15:55.852] Cleared config for k8s-jkns-gce-sd-log_bootstrap-e2e from /workspace/.kube/config I1026 08:15:55.853] Done W1026 08:15:55.953] W1026 08:15:55.848298 2077 loader.go:223] Config not found: /workspace/.kube/config W1026 08:15:55.954] W1026 08:15:55.848834 2077 loader.go:223] Config not found: /workspace/.kube/config W1026 08:15:55.954] 2020/10/26 08:15:55 process.go:155: Step './hack/e2e-internal/e2e-down.sh' finished in 34.21713116s W1026 08:15:55.954] 2020/10/26 08:15:55 e2e.go:456: Listing resources... W1026 08:15:55.954] 2020/10/26 08:15:55 process.go:153: Running: ./cluster/gce/list-resources.sh W1026 08:15:56.768] Listed 0 items. W1026 08:15:57.920] Listed 0 items. W1026 08:15:59.559] Listed 0 items. W1026 08:16:00.652] Listed 0 items. W1026 08:16:01.714] Listed 0 items. W1026 08:16:03.908] W1026 08:16:03.909] To show all fields of the firewall, please show in JSON format: --format=json W1026 08:16:03.909] To show all fields in table format, please see the examples in --help. W1026 08:16:03.909] W1026 08:16:05.026] Listed 0 items. W1026 08:16:06.162] Listed 0 items. W1026 08:16:07.070] 2020/10/26 08:16:07 process.go:155: Step './cluster/gce/list-resources.sh' finished in 11.216238596s W1026 08:16:07.071] 2020/10/26 08:16:07 process.go:153: Running: ./hack/e2e-internal/e2e-up.sh W1026 08:16:07.129] Project: k8s-jkns-gce-sd-log W1026 08:16:07.129] Network Project: k8s-jkns-gce-sd-log W1026 08:16:07.130] Zone: us-central1-f W1026 08:16:07.182] ... Starting cluster in us-central1-f using provider gce W1026 08:16:07.182] ... calling verify-prereqs W1026 08:16:07.831] ... calling verify-kube-binaries W1026 08:16:07.992] ... calling verify-release-tars W1026 08:16:08.004] ... calling kube-up W1026 08:16:08.011] Project: k8s-jkns-gce-sd-log W1026 08:16:08.011] Network Project: k8s-jkns-gce-sd-log W1026 08:16:08.012] Zone: us-central1-f I1026 08:16:11.587] +++ Staging tars to Google Storage: gs://kubernetes-staging-e3f16598c6/bootstrap-e2e-devel I1026 08:16:15.209] +++ kubernetes-server-linux-amd64.tar.gz uploaded earlier, cloud and local file md5 match (md5 = e7d002a3e2857262b353740982d62b0c) I1026 08:16:17.418] +++ kubernetes-manifests.tar.gz uploaded earlier, cloud and local file md5 match (md5 = 205a19d9e256a8f082fa6c3cb535b857) I1026 08:16:18.497] Creating new auto network: bootstrap-e2e W1026 08:17:14.620] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/networks/bootstrap-e2e]. I1026 08:17:14.890] NAME SUBNET_MODE BGP_ROUTING_MODE IPV4_RANGE GATEWAY_IPV4 I1026 08:17:14.890] bootstrap-e2e AUTO REGIONAL W1026 08:17:14.990] W1026 08:17:14.991] Instances on this network will not be reachable until firewall rules W1026 08:17:14.991] are created. As an example, you can allow all internal traffic between W1026 08:17:14.991] instances as well as SSH, RDP, and ICMP by running: W1026 08:17:14.991] W1026 08:17:14.991] $ gcloud compute firewall-rules create --network bootstrap-e2e --allow tcp,udp,icmp --source-ranges W1026 08:17:14.992] $ gcloud compute firewall-rules create --network bootstrap-e2e --allow tcp:22,tcp:3389,icmp W1026 08:17:14.992] I1026 08:17:18.214] IP aliases are disabled. I1026 08:17:19.293] Found subnet for region us-central1 in network bootstrap-e2e: bootstrap-e2e I1026 08:17:19.297] Starting master and configuring firewalls I1026 08:17:19.297] Configuring firewall for apiserver konnectivity server W1026 08:17:22.398] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/zones/us-central1-f/disks/bootstrap-e2e-master-pd]. W1026 08:17:22.409] Creating firewall... W1026 08:17:22.409] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-default-internal-master]. I1026 08:17:22.550] NAME ZONE SIZE_GB TYPE STATUS I1026 08:17:22.551] bootstrap-e2e-master-pd us-central1-f 20 pd-ssd READY W1026 08:17:22.651] W1026 08:17:22.652] New disks are unformatted. You must format and mount a disk before it W1026 08:17:22.652] can be used. You can find instructions on how to do this at: W1026 08:17:22.652] W1026 08:17:22.652] https://cloud.google.com/compute/docs/disks/add-persistent-disk#formatting W1026 08:17:22.652] I1026 08:17:22.882] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 08:17:22.883] bootstrap-e2e-default-internal-master bootstrap-e2e INGRESS 1000 tcp:1-2379,tcp:2382-65535,udp:1-65535,icmp False W1026 08:17:22.983] done. W1026 08:17:24.512] Creating firewall... W1026 08:17:24.513] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-default-ssh]. W1026 08:17:24.635] Creating firewall... W1026 08:17:24.636] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-default-internal-node]. I1026 08:17:24.748] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 08:17:24.748] bootstrap-e2e-default-ssh bootstrap-e2e INGRESS 1000 tcp:22 False W1026 08:17:24.849] done. W1026 08:17:24.858] done. I1026 08:17:24.959] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 08:17:24.959] bootstrap-e2e-default-internal-node bootstrap-e2e INGRESS 1000 tcp:1-65535,udp:1-65535,icmp False W1026 08:17:27.205] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/regions/us-central1/addresses/bootstrap-e2e-master-ip]. W1026 08:17:27.277] Creating firewall... W1026 08:17:27.278] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-master-https]. I1026 08:17:27.485] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 08:17:27.486] bootstrap-e2e-master-https bootstrap-e2e INGRESS 1000 tcp:443 False W1026 08:17:27.586] done. W1026 08:17:29.826] Creating firewall... W1026 08:17:29.826] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-master-etcd]. I1026 08:17:29.927] Generating certs for alternate-names: IP:35.232.13.96,IP:10.0.0.1,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local,DNS:bootstrap-e2e-master I1026 08:17:30.042] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 08:17:30.043] bootstrap-e2e-master-etcd bootstrap-e2e INGRESS 1000 tcp:2380,tcp:2381 False W1026 08:17:30.143] done. W1026 08:17:32.179] 2020/10/26 08:17:32 [INFO] generating a new CA key and certificate from CSR W1026 08:17:32.180] 2020/10/26 08:17:32 [INFO] generate received request W1026 08:17:32.180] 2020/10/26 08:17:32 [INFO] received CSR W1026 08:17:32.180] 2020/10/26 08:17:32 [INFO] generating key: ecdsa-256 W1026 08:17:32.198] 2020/10/26 08:17:32 [INFO] encoded CSR W1026 08:17:32.199] 2020/10/26 08:17:32 [INFO] signed certificate with serial number 499012853115324608530156676156600103780623575438 W1026 08:17:32.210] 2020/10/26 08:17:32 [INFO] generate received request W1026 08:17:32.210] 2020/10/26 08:17:32 [INFO] received CSR W1026 08:17:32.210] 2020/10/26 08:17:32 [INFO] generating key: ecdsa-256 W1026 08:17:32.228] 2020/10/26 08:17:32 [INFO] encoded CSR W1026 08:17:32.229] 2020/10/26 08:17:32 [INFO] signed certificate with serial number 95815428411183965304525796370397835434659314270 W1026 08:17:32.261] 2020/10/26 08:17:32 [INFO] generate received request W1026 08:17:32.261] 2020/10/26 08:17:32 [INFO] received CSR W1026 08:17:32.261] 2020/10/26 08:17:32 [INFO] generating key: ecdsa-256 W1026 08:17:32.278] 2020/10/26 08:17:32 [INFO] encoded CSR W1026 08:17:32.279] 2020/10/26 08:17:32 [INFO] signed certificate with serial number 454001684206585543295934947375357263107585304805 W1026 08:17:32.293] 2020/10/26 08:17:32 [INFO] generate received request W1026 08:17:32.294] 2020/10/26 08:17:32 [INFO] received CSR W1026 08:17:32.294] 2020/10/26 08:17:32 [INFO] generating key: ecdsa-256 W1026 08:17:32.311] 2020/10/26 08:17:32 [INFO] encoded CSR W1026 08:17:32.312] 2020/10/26 08:17:32 [INFO] signed certificate with serial number 264886012911122462121641016665019938164648544755 W1026 08:17:32.312] 2020/10/26 08:17:32 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for W1026 08:17:32.312] websites. For more information see the Baseline Requirements for the Issuance and Management W1026 08:17:32.313] of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org); W1026 08:17:32.313] specifically, section 10.2.3 ("Information Requirements"). I1026 08:17:32.413] Generate peer certificates... I1026 08:17:32.413] Generate server certificates... I1026 08:17:32.414] Generate client certificates... W1026 08:17:37.215] Creating firewall... W1026 08:17:37.215] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-minion-all]. I1026 08:17:37.459] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 08:17:37.460] bootstrap-e2e-minion-all bootstrap-e2e INGRESS 1000 tcp,udp,icmp,esp,ah,sctp False W1026 08:17:37.560] done. W1026 08:18:09.385] WARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance. W1026 08:18:09.385] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/zones/us-central1-f/instances/bootstrap-e2e-master]. W1026 08:18:09.385] WARNING: Some requests generated warnings: W1026 08:18:09.385] - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details. W1026 08:18:09.386] - The resource 'projects/cos-cloud/global/images/cos-85-13310-1041-9' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-85-13310-1041-14'. W1026 08:18:09.386] W1026 08:18:09.386] NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS W1026 08:18:09.386] bootstrap-e2e-master us-central1-f n1-standard-1 10.128.0.2 35.232.13.96 RUNNING I1026 08:18:09.486] Creating nodes. W1026 08:18:11.971] /go/src/github.com/helm/charts/kubernetes/cluster/../cluster/../cluster/gce/util.sh: line 1537: WINDOWS_CONTAINER_RUNTIME_ENDPOINT: unbound variable I1026 08:18:12.074] Using subnet bootstrap-e2e W1026 08:18:13.091] Attempt 1 to create bootstrap-e2e-minion-template W1026 08:18:13.753] WARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance. W1026 08:18:15.615] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/instanceTemplates/bootstrap-e2e-minion-template]. W1026 08:18:15.906] NAME MACHINE_TYPE PREEMPTIBLE CREATION_TIMESTAMP W1026 08:18:15.906] bootstrap-e2e-minion-template n1-standard-2 2020-10-26T01:18:14.467-07:00 I1026 08:18:16.098] Using subnet bootstrap-e2e W1026 08:18:17.146] Attempt 1 to create bootstrap-e2e-windows-node-template W1026 08:18:17.862] WARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance. W1026 08:18:20.367] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/instanceTemplates/bootstrap-e2e-windows-node-template]. W1026 08:18:20.725] NAME MACHINE_TYPE PREEMPTIBLE CREATION_TIMESTAMP W1026 08:18:20.726] bootstrap-e2e-windows-node-template n1-standard-2 2020-10-26T01:18:18.491-07:00 W1026 08:18:34.183] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/zones/us-central1-f/instanceGroupManagers/bootstrap-e2e-minion-group]. I1026 08:18:34.729] NAME LOCATION SCOPE BASE_INSTANCE_NAME SIZE TARGET_SIZE INSTANCE_TEMPLATE AUTOSCALED I1026 08:18:34.730] bootstrap-e2e-minion-group us-central1-f zone bootstrap-e2e-minion-group 0 3 bootstrap-e2e-minion-template no I1026 08:19:41.316] Waiting for group to become stable, current operations: creating: 3 I1026 08:19:41.317] Waiting for group to become stable, current operations: creating: 3 I1026 08:19:41.317] Waiting for group to become stable, current operations: creating: 3 I1026 08:19:41.317] Waiting for group to become stable, current operations: creating: 1 I1026 08:19:41.317] Waiting for group to become stable, current operations: creating: 1 I1026 08:19:41.317] Waiting for group to become stable, current operations: creating: 1 I1026 08:19:41.317] Group is stable W1026 08:19:45.797] INSTANCE_GROUPS=bootstrap-e2e-minion-group W1026 08:19:45.797] NODE_NAMES=bootstrap-e2e-minion-group-1vk3 bootstrap-e2e-minion-group-rjmp bootstrap-e2e-minion-group-vfqt W1026 08:19:45.798] Trying to find master named 'bootstrap-e2e-master' W1026 08:19:45.798] Looking for address 'bootstrap-e2e-master-ip' W1026 08:19:46.955] Using master: bootstrap-e2e-master (external IP: 35.232.13.96; internal IP: (not set)) I1026 08:19:47.056] Waiting up to 300 seconds for cluster initialization. I1026 08:19:47.056] I1026 08:19:47.056] This will continually check to see if the API for kubernetes is reachable. I1026 08:19:47.056] This may time out if there was some uncaught error during start up. I1026 08:19:47.056] I1026 08:20:01.191] ..Kubernetes cluster created. I1026 08:20:01.363] Cluster "k8s-jkns-gce-sd-log_bootstrap-e2e" set. I1026 08:20:01.523] User "k8s-jkns-gce-sd-log_bootstrap-e2e" set. I1026 08:20:01.680] Context "k8s-jkns-gce-sd-log_bootstrap-e2e" created. I1026 08:20:01.845] Switched to context "k8s-jkns-gce-sd-log_bootstrap-e2e". I1026 08:20:02.005] User "k8s-jkns-gce-sd-log_bootstrap-e2e-basic-auth" set. I1026 08:20:02.009] Wrote config for k8s-jkns-gce-sd-log_bootstrap-e2e to /workspace/.kube/config W1026 08:20:07.731] Updated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/zones/us-central1-f/instances/bootstrap-e2e-master]. I1026 08:20:08.164] I1026 08:20:08.164] Kubernetes cluster is running. The master is running at: I1026 08:20:08.164] I1026 08:20:08.164]  https://35.232.13.96 I1026 08:20:08.164] I1026 08:20:08.165] The user name and password to use is located in /workspace/.kube/config. I1026 08:20:08.165] I1026 08:20:08.234] Validating gce cluster, MULTIZONE= W1026 08:20:08.334] ... calling validate-cluster W1026 08:20:08.335] Project: k8s-jkns-gce-sd-log W1026 08:20:08.335] Network Project: k8s-jkns-gce-sd-log W1026 08:20:08.335] Zone: us-central1-f W1026 08:20:10.458] No resources found I1026 08:20:10.559] Waiting for 4 ready nodes. 0 ready nodes, 0 registered. Retrying. W1026 08:20:25.650] No resources found I1026 08:20:25.751] Waiting for 4 ready nodes. 0 ready nodes, 0 registered. Retrying. I1026 08:20:40.850] Waiting for 4 ready nodes. 0 ready nodes, 3 registered. Retrying. I1026 08:20:56.048] Found 4 node(s). I1026 08:20:56.225] NAME STATUS ROLES AGE VERSION I1026 08:20:56.225] bootstrap-e2e-master Ready,SchedulingDisabled 28s v1.20.0-alpha.3.116+16e18a590f32b9 I1026 08:20:56.225] bootstrap-e2e-minion-group-1vk3 Ready 24s v1.20.0-alpha.3.116+16e18a590f32b9 I1026 08:20:56.226] bootstrap-e2e-minion-group-rjmp Ready 11s v1.20.0-alpha.3.116+16e18a590f32b9 I1026 08:20:56.226] bootstrap-e2e-minion-group-vfqt Ready 26s v1.20.0-alpha.3.116+16e18a590f32b9 W1026 08:20:56.413] Warning: v1 ComponentStatus is deprecated in v1.19+ I1026 08:20:56.513] Validate output: I1026 08:20:56.604] NAME STATUS MESSAGE ERROR I1026 08:20:56.604] controller-manager Healthy ok I1026 08:20:56.605] scheduler Healthy ok I1026 08:20:56.605] etcd-1 Healthy {"health":"true"} I1026 08:20:56.605] etcd-0 Healthy {"health":"true"} I1026 08:20:56.609] Cluster validation succeeded W1026 08:20:56.709] Warning: v1 ComponentStatus is deprecated in v1.19+ W1026 08:20:56.710] Done, listing cluster services: W1026 08:20:56.710] I1026 08:20:56.810] Kubernetes control plane is running at https://35.232.13.96 I1026 08:20:56.811] GLBCDefaultBackend is running at https://35.232.13.96/api/v1/namespaces/kube-system/services/default-http-backend:http/proxy I1026 08:20:56.811] CoreDNS is running at https://35.232.13.96/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy I1026 08:20:56.811] Metrics-server is running at https://35.232.13.96/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy I1026 08:20:56.811] I1026 08:20:56.812] To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. I1026 08:20:56.812] I1026 08:21:02.968] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 08:21:02.968] bootstrap-e2e-minion-http-alt bootstrap-e2e INGRESS 1000 tcp:80,tcp:8080 False I1026 08:21:04.001] allowed: I1026 08:21:04.001] - IPProtocol: tcp I1026 08:21:04.001] ports: I1026 08:21:04.002] - '80' I1026 08:21:04.002] - IPProtocol: tcp I1026 08:21:04.002] ports: I1026 08:21:04.002] - '8080' I1026 08:21:04.002] creationTimestamp: '2020-10-26T01:20:57.894-07:00' I1026 08:21:04.002] description: '' I1026 08:21:04.002] direction: INGRESS I1026 08:21:04.002] disabled: false I1026 08:21:04.003] id: '8338981093876905990' I1026 08:21:04.003] kind: compute#firewall I1026 08:21:04.003] logConfig: I1026 08:21:04.003] enable: false I1026 08:21:04.003] name: bootstrap-e2e-minion-http-alt I1026 08:21:04.004] network: https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/networks/bootstrap-e2e I1026 08:21:04.004] priority: 1000 I1026 08:21:04.004] selfLink: https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-minion-http-alt I1026 08:21:04.004] sourceRanges: I1026 08:21:04.004] - 0.0.0.0/0 I1026 08:21:04.004] targetTags: I1026 08:21:04.004] - bootstrap-e2e-minion I1026 08:21:10.227] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 08:21:10.228] bootstrap-e2e-minion-nodeports bootstrap-e2e INGRESS 1000 tcp:30000-32767,udp:30000-32767 False I1026 08:21:11.264] allowed: I1026 08:21:11.264] - IPProtocol: tcp I1026 08:21:11.264] ports: I1026 08:21:11.264] - 30000-32767 I1026 08:21:11.264] - IPProtocol: udp I1026 08:21:11.265] ports: I1026 08:21:11.265] - 30000-32767 I1026 08:21:11.265] creationTimestamp: '2020-10-26T01:21:05.264-07:00' I1026 08:21:11.265] description: '' I1026 08:21:11.265] direction: INGRESS I1026 08:21:11.265] disabled: false I1026 08:21:11.265] id: '571493581844068382' I1026 08:21:11.265] kind: compute#firewall I1026 08:21:11.266] logConfig: I1026 08:21:11.266] enable: false I1026 08:21:11.266] name: bootstrap-e2e-minion-nodeports I1026 08:21:11.266] network: https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/networks/bootstrap-e2e I1026 08:21:11.266] priority: 1000 I1026 08:21:11.266] selfLink: https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-minion-nodeports I1026 08:21:11.267] sourceRanges: I1026 08:21:11.267] - 0.0.0.0/0 I1026 08:21:11.267] targetTags: I1026 08:21:11.267] - bootstrap-e2e-minion W1026 08:21:11.443] 2020/10/26 08:21:11 process.go:155: Step './hack/e2e-internal/e2e-up.sh' finished in 5m4.372112117s W1026 08:21:11.443] 2020/10/26 08:21:11 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false version W1026 08:21:11.614] 2020/10/26 08:21:11 process.go:155: Step './cluster/kubectl.sh --match-server-version=false version' finished in 171.497985ms W1026 08:21:11.615] 2020/10/26 08:21:11 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false get nodes -oyaml W1026 08:21:11.814] 2020/10/26 08:21:11 process.go:155: Step './cluster/kubectl.sh --match-server-version=false get nodes -oyaml' finished in 199.097592ms W1026 08:21:11.814] 2020/10/26 08:21:11 e2e.go:456: Listing resources... W1026 08:21:11.814] 2020/10/26 08:21:11 process.go:153: Running: ./cluster/gce/list-resources.sh W1026 08:21:20.518] W1026 08:21:20.519] To show all fields of the firewall, please show in JSON format: --format=json W1026 08:21:20.519] To show all fields in table format, please see the examples in --help. W1026 08:21:20.519] W1026 08:21:21.644] Listed 0 items. W1026 08:21:22.733] Listed 0 items. W1026 08:21:23.611] 2020/10/26 08:21:23 process.go:155: Step './cluster/gce/list-resources.sh' finished in 11.797061563s W1026 08:21:23.611] 2020/10/26 08:21:23 util.go:56: Warning: Couldn't find directory src/k8s.io/charts under any of GOPATH /go, defaulting to /go/src/k8s.io/charts W1026 08:21:23.612] 2020/10/26 08:21:23 process.go:153: Running: /go/src/k8s.io/charts/test/helm-test-e2e.sh W1026 08:21:23.612] 2020/10/26 08:21:23 process.go:155: Step '/go/src/k8s.io/charts/test/helm-test-e2e.sh' finished in 407.064µs W1026 08:21:23.612] 2020/10/26 08:21:23 e2e.go:544: Dumping logs locally to: /workspace/_artifacts W1026 08:21:23.612] 2020/10/26 08:21:23 process.go:153: Running: ./cluster/log-dump/log-dump.sh /workspace/_artifacts W1026 08:21:23.686] Trying to find master named 'bootstrap-e2e-master' W1026 08:21:23.686] Looking for address 'bootstrap-e2e-master-ip' I1026 08:21:23.787] Checking for custom logdump instances, if any I1026 08:21:23.787] Sourcing kube-util.sh I1026 08:21:23.787] Detecting project I1026 08:21:23.787] Project: k8s-jkns-gce-sd-log I1026 08:21:23.787] Network Project: k8s-jkns-gce-sd-log I1026 08:21:23.787] Zone: us-central1-f I1026 08:21:23.788] Dumping logs from master locally to '/workspace/_artifacts' W1026 08:21:24.757] Using master: bootstrap-e2e-master (external IP: 35.232.13.96; internal IP: (not set)) I1026 08:21:51.599] Changing logfiles to be world-readable for download I1026 08:21:54.655] Copying 'kube-apiserver.log kube-apiserver-audit.log kube-scheduler.log kube-controller-manager.log etcd.log etcd-events.log glbc.log cluster-autoscaler.log kube-addon-manager.log konnectivity-server.log fluentd.log kubelet.cov startupscript.log' from bootstrap-e2e-master W1026 08:21:55.650] W1026 08:21:55.650] Specify --start=56930 in the next get-serial-port-output invocation to get only the new output starting from here. W1026 08:21:57.311] scp: /var/log/cluster-autoscaler.log*: No such file or directory W1026 08:21:57.313] scp: /var/log/konnectivity-server.log*: No such file or directory W1026 08:21:57.313] scp: /var/log/fluentd.log*: No such file or directory W1026 08:21:57.313] scp: /var/log/kubelet.cov*: No such file or directory W1026 08:21:57.313] scp: /var/log/startupscript.log*: No such file or directory W1026 08:21:57.321] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. I1026 08:21:57.524] Dumping logs from nodes locally to '/workspace/_artifacts' I1026 08:21:57.525] Detecting nodes in the cluster I1026 08:22:31.096] Changing logfiles to be world-readable for download I1026 08:22:31.197] Changing logfiles to be world-readable for download I1026 08:22:32.037] Changing logfiles to be world-readable for download I1026 08:22:34.172] Copying 'kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov startupscript.log' from bootstrap-e2e-minion-group-rjmp I1026 08:22:34.173] Copying 'kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov startupscript.log' from bootstrap-e2e-minion-group-1vk3 I1026 08:22:35.077] Copying 'kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov startupscript.log' from bootstrap-e2e-minion-group-vfqt W1026 08:22:35.177] W1026 08:22:35.178] Specify --start=68157 in the next get-serial-port-output invocation to get only the new output starting from here. W1026 08:22:35.178] W1026 08:22:35.178] Specify --start=67930 in the next get-serial-port-output invocation to get only the new output starting from here. W1026 08:22:36.012] W1026 08:22:36.013] Specify --start=67699 in the next get-serial-port-output invocation to get only the new output starting from here. W1026 08:22:36.558] scp: /var/log/fluentd.log*: No such file or directory W1026 08:22:36.558] scp: /var/log/node-problem-detector.log*: No such file or directory W1026 08:22:36.558] scp: /var/log/kubelet.cov*: No such file or directory W1026 08:22:36.559] scp: /var/log/startupscript.log*: No such file or directory W1026 08:22:36.563] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1026 08:22:36.586] scp: /var/log/fluentd.log*: No such file or directory W1026 08:22:36.586] scp: /var/log/node-problem-detector.log*: No such file or directory W1026 08:22:36.586] scp: /var/log/kubelet.cov*: No such file or directory W1026 08:22:36.587] scp: /var/log/startupscript.log*: No such file or directory W1026 08:22:36.591] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1026 08:22:37.421] scp: /var/log/fluentd.log*: No such file or directory W1026 08:22:37.422] scp: /var/log/node-problem-detector.log*: No such file or directory W1026 08:22:37.422] scp: /var/log/kubelet.cov*: No such file or directory W1026 08:22:37.422] scp: /var/log/startupscript.log*: No such file or directory W1026 08:22:37.426] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1026 08:22:41.504] INSTANCE_GROUPS=bootstrap-e2e-minion-group W1026 08:22:41.504] NODE_NAMES=bootstrap-e2e-minion-group-1vk3 bootstrap-e2e-minion-group-rjmp bootstrap-e2e-minion-group-vfqt I1026 08:22:42.743] Failures for bootstrap-e2e-minion-group (if any): W1026 08:22:44.955] 2020/10/26 08:22:44 process.go:155: Step './cluster/log-dump/log-dump.sh /workspace/_artifacts' finished in 1m21.34319885s W1026 08:22:44.955] 2020/10/26 08:22:44 e2e.go:456: Listing resources... W1026 08:22:44.955] 2020/10/26 08:22:44 process.go:153: Running: ./cluster/gce/list-resources.sh W1026 08:22:53.486] W1026 08:22:53.487] To show all fields of the firewall, please show in JSON format: --format=json W1026 08:22:53.487] To show all fields in table format, please see the examples in --help. W1026 08:22:53.487] W1026 08:22:54.617] Listed 0 items. W1026 08:22:55.725] Listed 0 items. W1026 08:22:56.624] 2020/10/26 08:22:56 process.go:155: Step './cluster/gce/list-resources.sh' finished in 11.66860967s W1026 08:22:56.624] 2020/10/26 08:22:56 process.go:153: Running: ./hack/e2e-internal/e2e-down.sh W1026 08:22:56.682] Project: k8s-jkns-gce-sd-log W1026 08:22:56.683] Network Project: k8s-jkns-gce-sd-log W1026 08:22:56.683] Zone: us-central1-f I1026 08:22:56.783] Shutting down test cluster in background. W1026 08:23:04.359] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-minion-http-alt]. W1026 08:23:04.915] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-minion-nodeports]. I1026 08:23:05.201] Bringing down cluster using provider: gce W1026 08:23:05.302] ... calling verify-prereqs W1026 08:23:05.817] ... calling verify-kube-binaries W1026 08:23:05.967] ... calling kube-down W1026 08:23:05.968] Project: k8s-jkns-gce-sd-log W1026 08:23:05.968] Network Project: k8s-jkns-gce-sd-log W1026 08:23:05.968] Zone: us-central1-f W1026 08:23:09.944] INSTANCE_GROUPS=bootstrap-e2e-minion-group W1026 08:23:09.945] NODE_NAMES=bootstrap-e2e-minion-group-1vk3 bootstrap-e2e-minion-group-rjmp bootstrap-e2e-minion-group-vfqt I1026 08:23:10.045] Bringing down cluster W1026 08:25:23.740] Deleting Managed Instance Group... W1026 08:25:23.741] ...Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/zones/us-central1-f/instanceGroupManagers/bootstrap-e2e-minion-group]. W1026 08:25:23.744] done. W1026 08:25:27.900] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/instanceTemplates/bootstrap-e2e-minion-template]. W1026 08:25:28.939] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/instanceTemplates/bootstrap-e2e-windows-node-template]. I1026 08:25:34.564] Successfully executed 'curl -s --cacert /etc/srv/kubernetes/pki/etcd-apiserver-ca.crt --cert /etc/srv/kubernetes/pki/etcd-apiserver-client.crt --key /etc/srv/kubernetes/pki/etcd-apiserver-client.key https://127.0.0.1:2379/v2/members/$(curl -s --cacert /etc/srv/kubernetes/pki/etcd-apiserver-ca.crt --cert /etc/srv/kubernetes/pki/etcd-apiserver-client.crt --key /etc/srv/kubernetes/pki/etcd-apiserver-client.key https://127.0.0.1:2379/v2/members -XGET | sed 's/{\"id/\n/g' | grep bootstrap-e2e-master\" | cut -f 3 -d \") -XDELETE -L 2>/dev/null' on bootstrap-e2e-master I1026 08:25:34.564] Removing etcd replica, name: bootstrap-e2e-master, port: 2379, result: 0 I1026 08:25:36.095] Successfully executed 'curl -s http://127.0.0.1:4002/v2/members/$(curl -s http://127.0.0.1:4002/v2/members -XGET | sed 's/{\"id/\n/g' | grep bootstrap-e2e-master\" | cut -f 3 -d \") -XDELETE -L 2>/dev/null' on bootstrap-e2e-master I1026 08:25:36.096] Removing etcd replica, name: bootstrap-e2e-master, port: 4002, result: 0 W1026 08:25:41.592] Updated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/zones/us-central1-f/instances/bootstrap-e2e-master]. W1026 08:27:31.698] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/zones/us-central1-f/instances/bootstrap-e2e-master]. W1026 08:27:42.196] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-master-https]. W1026 08:27:45.032] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-master-etcd]. W1026 08:27:45.100] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-minion-all]. W1026 08:27:51.421] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/regions/us-central1/addresses/bootstrap-e2e-master-ip]. I1026 08:27:55.318] Deleting routes bootstrap-e2e-b23e8ad8-9ac0-4b59-a590-8ba0412508cb W1026 08:28:02.285] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/routes/bootstrap-e2e-b23e8ad8-9ac0-4b59-a590-8ba0412508cb]. W1026 08:28:10.920] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-default-internal-master]. W1026 08:28:13.599] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-default-ssh]. W1026 08:28:13.809] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/firewalls/bootstrap-e2e-default-internal-node]. I1026 08:28:15.101] Deleting firewall rules remaining in network bootstrap-e2e: W1026 08:29:01.202] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-sd-log/global/networks/bootstrap-e2e]. I1026 08:29:01.719] Property "current-context" unset. I1026 08:29:01.870] Property "clusters.k8s-jkns-gce-sd-log_bootstrap-e2e" unset. I1026 08:29:02.020] Property "users.k8s-jkns-gce-sd-log_bootstrap-e2e" unset. I1026 08:29:02.167] Property "users.k8s-jkns-gce-sd-log_bootstrap-e2e-basic-auth" unset. I1026 08:29:02.315] Property "contexts.k8s-jkns-gce-sd-log_bootstrap-e2e" unset. I1026 08:29:02.318] Cleared config for k8s-jkns-gce-sd-log_bootstrap-e2e from /workspace/.kube/config I1026 08:29:02.319] Done W1026 08:29:02.419] 2020/10/26 08:29:02 process.go:155: Step './hack/e2e-internal/e2e-down.sh' finished in 6m5.696760406s W1026 08:29:02.419] 2020/10/26 08:29:02 e2e.go:278: Sleeping for 30 seconds... W1026 08:29:32.321] 2020/10/26 08:29:32 e2e.go:456: Listing resources... W1026 08:29:32.321] 2020/10/26 08:29:32 process.go:153: Running: ./cluster/gce/list-resources.sh W1026 08:29:33.249] Listed 0 items. W1026 08:29:34.286] Listed 0 items. W1026 08:29:36.029] Listed 0 items. W1026 08:29:37.056] Listed 0 items. W1026 08:29:38.094] Listed 0 items. W1026 08:29:40.246] W1026 08:29:40.247] To show all fields of the firewall, please show in JSON format: --format=json W1026 08:29:40.247] To show all fields in table format, please see the examples in --help. W1026 08:29:40.247] W1026 08:29:41.330] Listed 0 items. W1026 08:29:42.396] Listed 0 items. W1026 08:29:43.270] 2020/10/26 08:29:43 process.go:155: Step './cluster/gce/list-resources.sh' finished in 10.949289839s W1026 08:29:43.271] 2020/10/26 08:29:43 process.go:153: Running: diff -sw -U0 -F^\[.*\]$ /workspace/_artifacts/gcp-resources-before.txt /workspace/_artifacts/gcp-resources-after.txt W1026 08:29:43.272] 2020/10/26 08:29:43 process.go:155: Step 'diff -sw -U0 -F^\[.*\]$ /workspace/_artifacts/gcp-resources-before.txt /workspace/_artifacts/gcp-resources-after.txt' finished in 1.659368ms W1026 08:29:43.273] 2020/10/26 08:29:43 process.go:96: Saved XML output to /workspace/_artifacts/junit_runner.xml. W1026 08:29:43.281] 2020/10/26 08:29:43 main.go:316: Something went wrong: encountered 1 errors: [error starting /go/src/k8s.io/charts/test/helm-test-e2e.sh: fork/exec /go/src/k8s.io/charts/test/helm-test-e2e.sh: no such file or directory] W1026 08:29:43.286] Traceback (most recent call last): W1026 08:29:43.286] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 720, in W1026 08:29:43.286] main(parse_args()) W1026 08:29:43.286] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 570, in main W1026 08:29:43.286] mode.start(runner_args) W1026 08:29:43.287] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 228, in start W1026 08:29:43.287] check_env(env, self.command, *args) W1026 08:29:43.287] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 111, in check_env W1026 08:29:43.287] subprocess.check_call(cmd, env=env) W1026 08:29:43.287] File "/usr/lib/python2.7/subprocess.py", line 190, in check_call W1026 08:29:43.287] raise CalledProcessError(retcode, cmd) W1026 08:29:43.288] subprocess.CalledProcessError: Command '('kubetest', '--dump=/workspace/_artifacts', '--gcp-service-account=/etc/service-account/service-account.json', '--up', '--down', '--provider=gce', '--cluster=bootstrap-e2e', '--gcp-network=bootstrap-e2e', '--charts', '--check-leaked-resources', '--extract=ci/latest', '--gcp-node-image=gci', '--gcp-zone=us-central1-f', '--timeout=180m')' returned non-zero exit status 1 E1026 08:29:43.297] Command failed I1026 08:29:43.298] process 282 exited with code 1 after 14.9m E1026 08:29:43.298] FAIL: ci-kubernetes-charts-gce I1026 08:29:43.298] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1026 08:29:43.925] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] I1026 08:29:44.033] process 10763 exited with code 0 after 0.0m I1026 08:29:44.033] Call: gcloud config get-value account I1026 08:29:44.609] process 10776 exited with code 0 after 0.0m I1026 08:29:44.609] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com I1026 08:29:44.609] Upload result and artifacts... I1026 08:29:44.609] Gubernator results at https://gubernator.k8s.io/build/kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320639896909516800 I1026 08:29:44.610] Call: gsutil ls gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320639896909516800/artifacts W1026 08:29:45.545] CommandException: One or more URLs matched no objects. E1026 08:29:45.730] Command failed I1026 08:29:45.730] process 10789 exited with code 1 after 0.0m W1026 08:29:45.730] Remote dir gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320639896909516800/artifacts not exist yet I1026 08:29:45.731] Call: gsutil -m -q -o GSUtil:use_magicfile=True cp -r -c -z log,txt,xml /workspace/_artifacts gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320639896909516800/artifacts I1026 08:29:48.206] process 10932 exited with code 0 after 0.0m I1026 08:29:48.207] Call: git rev-parse HEAD I1026 08:29:48.211] process 11585 exited with code 0 after 0.0m I1026 08:29:48.212] Call: git rev-parse HEAD I1026 08:29:48.216] process 11586 exited with code 0 after 0.0m I1026 08:29:48.217] Call: gsutil stat gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/jobResultsCache.json I1026 08:29:49.376] process 11587 exited with code 0 after 0.0m I1026 08:29:49.377] Call: gsutil -q cat 'gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/jobResultsCache.json#1603697331075350' I1026 08:29:50.560] process 11730 exited with code 0 after 0.0m I1026 08:29:50.564] Call: gsutil -q -h Content-Type:application/json -h x-goog-if-generation-match:1603697331075350 cp /tmp/gsutil_uPca0v gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/jobResultsCache.json I1026 08:29:51.965] process 11875 exited with code 0 after 0.0m I1026 08:29:51.966] Call: gsutil -q -h Content-Type:application/json cp /tmp/gsutil_3z3LJI gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320639896909516800/finished.json I1026 08:29:53.317] process 12054 exited with code 0 after 0.0m I1026 08:29:53.318] Call: gsutil -q -h Content-Type:text/plain -h 'Cache-Control:private, max-age=0, no-transform' cp /tmp/gsutil_NjEQf8 gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/latest-build.txt I1026 08:29:54.713] process 12233 exited with code 0 after 0.0m