W1026 04:10:35.672] ************************************************************************** bootstrap.py is deprecated! test-infra oncall does not support any job still using bootstrap.py. Please migrate your job to podutils! https://github.com/kubernetes/test-infra/blob/master/prow/pod-utilities.md ************************************************************************** I1026 04:10:35.672] Args: --job=ci-kubernetes-charts-gce --service-account=/etc/service-account/service-account.json --upload=gs://kubernetes-jenkins/logs --timeout=200 --repo=github.com/helm/charts --root=/go/src --service-account=/etc/service-account/service-account.json --scenario=kubernetes_e2e -- --charts --check-leaked-resources --extract=ci/latest --gcp-node-image=gci --gcp-zone=us-central1-f --provider=gce --test=false --timeout=180m I1026 04:10:35.672] Bootstrap ci-kubernetes-charts-gce... I1026 04:10:35.674] Builder: 2aad6a15-1741-11eb-b256-6ee25ea2e440 I1026 04:10:35.674] Image: gcr.io/k8s-testimages/kubekins-e2e:v20201021-a62ce8a-master I1026 04:10:35.675] Gubernator results at https://gubernator.k8s.io/build/kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320578491233603584 I1026 04:10:35.675] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1026 04:10:36.261] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] I1026 04:10:36.366] process 34 exited with code 0 after 0.0m I1026 04:10:36.366] Call: gcloud config get-value account I1026 04:10:36.914] process 47 exited with code 0 after 0.0m I1026 04:10:36.914] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com I1026 04:10:36.914] Root: /go/src I1026 04:10:36.914] cd to /go/src I1026 04:10:36.914] Checkout: /go/src/github.com/helm/charts master to /go/src/github.com/helm/charts I1026 04:10:36.915] Call: git init github.com/helm/charts I1026 04:10:36.921] Initialized empty Git repository in /go/src/github.com/helm/charts/.git/ I1026 04:10:36.921] process 60 exited with code 0 after 0.0m I1026 04:10:36.921] Call: git config --local user.name 'K8S Bootstrap' I1026 04:10:36.925] process 61 exited with code 0 after 0.0m I1026 04:10:36.925] Call: git config --local user.email k8s_bootstrap@localhost I1026 04:10:36.929] process 62 exited with code 0 after 0.0m I1026 04:10:36.929] Call: git fetch --quiet --tags https://github.com/helm/charts master I1026 04:10:40.645] process 63 exited with code 0 after 0.1m I1026 04:10:40.645] Call: git checkout -B test FETCH_HEAD W1026 04:10:41.078] Switched to a new branch 'test' I1026 04:10:41.083] process 74 exited with code 0 after 0.0m I1026 04:10:41.083] Call: git show -s --format=format:%ct HEAD I1026 04:10:41.088] process 75 exited with code 0 after 0.0m I1026 04:10:41.088] Configure environment... I1026 04:10:41.089] Call: git show -s --format=format:%ct HEAD I1026 04:10:41.093] process 76 exited with code 0 after 0.0m I1026 04:10:41.093] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1026 04:10:41.721] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] I1026 04:10:41.937] process 77 exited with code 0 after 0.0m I1026 04:10:41.937] Call: gcloud config get-value account I1026 04:10:42.496] process 90 exited with code 0 after 0.0m I1026 04:10:42.497] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com I1026 04:10:42.497] Start 1320578491233603584 at unknown... I1026 04:10:42.499] Call: gsutil -q -h Content-Type:application/json cp /tmp/gsutil_ykeNeV gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320578491233603584/started.json I1026 04:10:44.059] process 103 exited with code 0 after 0.0m I1026 04:10:44.060] Call: /workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py --charts --check-leaked-resources --extract=ci/latest --gcp-node-image=gci --gcp-zone=us-central1-f --provider=gce --test=false --timeout=180m W1026 04:10:44.103] starts with local mode W1026 04:10:44.103] Environment: W1026 04:10:44.103] ARTIFACTS=/workspace/_artifacts W1026 04:10:44.104] AWS_SSH_PRIVATE_KEY_FILE=/root/.ssh/kube_aws_rsa W1026 04:10:44.104] AWS_SSH_PUBLIC_KEY_FILE=/root/.ssh/kube_aws_rsa.pub W1026 04:10:44.104] BAZEL_REMOTE_CACHE_ENABLED=false W1026 04:10:44.104] BAZEL_VERSION=2.2.0 W1026 04:10:44.104] BOOTSTRAP_MIGRATION=yes W1026 04:10:44.104] BOSKOS_METRICS_PORT=tcp://10.63.252.110:9090 W1026 04:10:44.104] BOSKOS_METRICS_PORT_9090_TCP=tcp://10.63.252.110:9090 W1026 04:10:44.105] BOSKOS_METRICS_PORT_9090_TCP_ADDR=10.63.252.110 W1026 04:10:44.105] BOSKOS_METRICS_PORT_9090_TCP_PORT=9090 W1026 04:10:44.105] BOSKOS_METRICS_PORT_9090_TCP_PROTO=tcp W1026 04:10:44.105] BOSKOS_METRICS_SERVICE_HOST=10.63.252.110 W1026 04:10:44.105] BOSKOS_METRICS_SERVICE_PORT=9090 W1026 04:10:44.105] BOSKOS_METRICS_SERVICE_PORT_METRICS=9090 W1026 04:10:44.105] BOSKOS_PORT=tcp://10.63.250.132:80 W1026 04:10:44.106] BOSKOS_PORT_80_TCP=tcp://10.63.250.132:80 W1026 04:10:44.106] BOSKOS_PORT_80_TCP_ADDR=10.63.250.132 W1026 04:10:44.106] BOSKOS_PORT_80_TCP_PORT=80 W1026 04:10:44.106] BOSKOS_PORT_80_TCP_PROTO=tcp W1026 04:10:44.106] BOSKOS_SERVICE_HOST=10.63.250.132 W1026 04:10:44.106] BOSKOS_SERVICE_PORT=80 W1026 04:10:44.106] BOSKOS_SERVICE_PORT_DEFAULT=80 W1026 04:10:44.107] BUILD_ID=1320578491233603584 W1026 04:10:44.107] BUILD_NUMBER=1320578491233603584 W1026 04:10:44.107] CI=true W1026 04:10:44.107] CLOUDSDK_COMPONENT_MANAGER_DISABLE_UPDATE_CHECK=true W1026 04:10:44.107] CLOUDSDK_CONFIG=/go/src/github.com/helm/charts/.config/gcloud W1026 04:10:44.107] CLOUDSDK_CORE_DISABLE_PROMPTS=1 W1026 04:10:44.108] CLOUDSDK_EXPERIMENTAL_FAST_COMPONENT_UPDATE=false W1026 04:10:44.108] DOCKER_IN_DOCKER_ENABLED=false W1026 04:10:44.108] DOCKER_IN_DOCKER_IPV6_ENABLED=false W1026 04:10:44.108] E2E_GOOGLE_APPLICATION_CREDENTIALS=/etc/service-account/service-account.json W1026 04:10:44.108] GCS_ARTIFACTS_DIR=gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320578491233603584/artifacts W1026 04:10:44.108] GOOGLE_APPLICATION_CREDENTIALS=/etc/service-account/service-account.json W1026 04:10:44.109] GOOGLE_APPLICATION_CREDENTIALS_DEPRECATED=Migrate to workload identity, contact sig-testing W1026 04:10:44.109] GOPATH=/go W1026 04:10:44.109] GOPROXY=https://proxy.golang.org W1026 04:10:44.109] GO_TARBALL=go1.15.2.linux-amd64.tar.gz W1026 04:10:44.109] HOME=/workspace W1026 04:10:44.109] HOSTNAME=2aad6a15-1741-11eb-b256-6ee25ea2e440 W1026 04:10:44.110] IMAGE=gcr.io/k8s-testimages/kubekins-e2e:v20201021-a62ce8a-master W1026 04:10:44.110] INSTANCE_PREFIX=bootstrap-e2e W1026 04:10:44.110] JENKINS_GCE_SSH_PRIVATE_KEY_FILE=/workspace/.ssh/google_compute_engine W1026 04:10:44.110] JENKINS_GCE_SSH_PUBLIC_KEY_FILE=/workspace/.ssh/google_compute_engine.pub W1026 04:10:44.110] JOB_NAME=ci-kubernetes-charts-gce W1026 04:10:44.110] JOB_SPEC={"type":"periodic","job":"ci-kubernetes-charts-gce","buildid":"1320578491233603584","prowjobid":"2aad6a15-1741-11eb-b256-6ee25ea2e440"} W1026 04:10:44.111] JOB_TYPE=periodic W1026 04:10:44.111] KUBERNETES_PORT=tcp://10.63.240.1:443 W1026 04:10:44.111] KUBERNETES_PORT_443_TCP=tcp://10.63.240.1:443 W1026 04:10:44.111] KUBERNETES_PORT_443_TCP_ADDR=10.63.240.1 W1026 04:10:44.111] KUBERNETES_PORT_443_TCP_PORT=443 W1026 04:10:44.111] KUBERNETES_PORT_443_TCP_PROTO=tcp W1026 04:10:44.111] KUBERNETES_SERVICE_HOST=10.63.240.1 W1026 04:10:44.111] KUBERNETES_SERVICE_PORT=443 W1026 04:10:44.112] KUBERNETES_SERVICE_PORT_HTTPS=443 W1026 04:10:44.112] KUBETEST_IN_DOCKER=true W1026 04:10:44.112] KUBETEST_MANUAL_DUMP=y W1026 04:10:44.112] KUBE_AWS_INSTANCE_PREFIX=bootstrap-e2e W1026 04:10:44.112] KUBE_GCE_INSTANCE_PREFIX=bootstrap-e2e W1026 04:10:44.112] LOG_DUMP_SCRIPT_PATH=/workspace/log-dump.sh W1026 04:10:44.112] NODE_NAME=2aad6a15-1741-11eb-b256-6ee25ea2e440 W1026 04:10:44.113] PATH=/go/bin:/go/bin:/usr/local/go/bin:/google-cloud-sdk/bin:/workspace:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin W1026 04:10:44.113] PROW_JOB_ID=2aad6a15-1741-11eb-b256-6ee25ea2e440 W1026 04:10:44.113] PWD=/workspace W1026 04:10:44.113] SHLVL=2 W1026 04:10:44.113] SOURCE_DATE_EPOCH=1603659301 W1026 04:10:44.113] TERM=xterm W1026 04:10:44.113] USER=prow W1026 04:10:44.113] WORKSPACE=/workspace W1026 04:10:44.114] _=./test-infra/jenkins/bootstrap.py W1026 04:10:44.114] Run: ('kubetest', '--dump=/workspace/_artifacts', '--gcp-service-account=/etc/service-account/service-account.json', '--up', '--down', '--provider=gce', '--cluster=bootstrap-e2e', '--gcp-network=bootstrap-e2e', '--charts', '--check-leaked-resources', '--extract=ci/latest', '--gcp-node-image=gci', '--gcp-zone=us-central1-f', '--timeout=180m') W1026 04:10:44.140] 2020/10/26 04:10:44 Warning: Couldn't find directory src/sigs.k8s.io/cloud-provider-azure under any of GOPATH /go, defaulting to /go/src/k8s.io/cloud-provider-azure W1026 04:10:44.141] 2020/10/26 04:10:44 extract_k8s.go:112: Matched extraction strategy: ^ci/(.+)$ W1026 04:10:44.141] 2020/10/26 04:10:44 main.go:329: Limiting testing to 3h0m0s W1026 04:10:44.141] 2020/10/26 04:10:44 process.go:153: Running: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1026 04:10:44.753] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] W1026 04:10:44.856] 2020/10/26 04:10:44 process.go:155: Step 'gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json' finished in 714.449757ms W1026 04:10:44.856] 2020/10/26 04:10:44 main.go:728: --gcp-project is missing, trying to fetch a project from boskos. W1026 04:10:44.856] (for local runs please set --gcp-project to your dev project) W1026 04:10:44.856] 2020/10/26 04:10:44 main.go:740: provider gce, will acquire project type gce-project from boskos W1026 04:10:44.865] 2020/10/26 04:10:44 process.go:153: Running: gcloud config set project k8s-jkns-gci-gce-serial-1-6 W1026 04:10:45.312] Updated property [core/project]. W1026 04:10:45.544] 2020/10/26 04:10:45 process.go:155: Step 'gcloud config set project k8s-jkns-gci-gce-serial-1-6' finished in 678.256839ms W1026 04:10:45.544] 2020/10/26 04:10:45 main.go:779: Checking existing of GCP ssh keys... W1026 04:10:45.544] 2020/10/26 04:10:45 main.go:789: Checking presence of public key in k8s-jkns-gci-gce-serial-1-6 W1026 04:10:45.545] 2020/10/26 04:10:45 process.go:153: Running: gcloud compute --project=k8s-jkns-gci-gce-serial-1-6 project-info describe W1026 04:10:46.700] 2020/10/26 04:10:46 process.go:155: Step 'gcloud compute --project=k8s-jkns-gci-gce-serial-1-6 project-info describe' finished in 1.156332312s W1026 04:10:46.794] 2020/10/26 04:10:46 extract_k8s.go:295: U=https://storage.googleapis.com/kubernetes-release-dev/ci R=v1.20.0-alpha.3.116+16e18a590f32b9 get-kube.sh W1026 04:10:46.795] 2020/10/26 04:10:46 process.go:153: Running: /workspace/get-kube.sh I1026 04:10:46.895] Downloading kubernetes release v1.20.0-alpha.3.116+16e18a590f32b9 I1026 04:10:46.895] from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes.tar.gz I1026 04:10:46.896] to /go/src/github.com/helm/charts/kubernetes.tar.gz W1026 04:10:47.866] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes.tar.gz... W1026 04:10:47.979] / [0 files][ 0.0 B/487.6 KiB] / [1 files][487.6 KiB/487.6 KiB] W1026 04:10:47.980] Operation completed over 1 objects/487.6 KiB. I1026 04:10:48.188] Unpacking kubernetes release v1.20.0-alpha.3.116+16e18a590f32b9 I1026 04:10:48.252] Kubernetes release: v1.20.0-alpha.3.116+16e18a590f32b9 I1026 04:10:48.252] Server: linux/amd64 (to override, set KUBERNETES_SERVER_ARCH) I1026 04:10:48.252] Client: linux/amd64 (autodetected) (to override, set KUBERNETES_CLIENT_OS and/or KUBERNETES_CLIENT_ARCH) I1026 04:10:48.253] I1026 04:10:48.253] Will download kubernetes-server-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9 I1026 04:10:48.253] Will download and extract kubernetes-client-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9 I1026 04:10:48.254] Will download and extract kubernetes-test tarball(s) from https://storage.googleapis.com/kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9 W1026 04:10:49.279] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes-server-linux-amd64.tar.gz... W1026 04:10:49.281] / [0 files][ 0.0 B/502.2 MiB] ==> NOTE: You are downloading one or more large file(s), which would W1026 04:10:49.282] run significantly faster if you enabled sliced object downloads. This W1026 04:10:49.282] feature is enabled by default but requires that compiled crcmod be W1026 04:10:49.282] installed (see "gsutil help crcmod"). W1026 04:10:49.282] W1026 04:10:55.251] - - [0 files][ 97.2 MiB/502.2 MiB] \ | | [0 files][243.6 MiB/502.2 MiB] / / [0 files][386.5 MiB/502.2 MiB] - \ \ [1 files][502.2 MiB/502.2 MiB] | W1026 04:10:55.252] Operation completed over 1 objects/502.2 MiB. I1026 04:10:55.441] I1026 04:10:56.583] md5sum(kubernetes-server-linux-amd64.tar.gz)=e7d002a3e2857262b353740982d62b0c I1026 04:10:58.459] sha512sum(kubernetes-server-linux-amd64.tar.gz)=912eb1aa30aaf2af864bf7f124781093f856889d80dc11146e2a4e1edbb777a4ee2233be2c6847f9369591c279ae3f08858f35d49f94a7ce900bcf0b38780a19 I1026 04:10:58.459] W1026 04:10:59.450] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes-client-linux-amd64.tar.gz... W1026 04:10:59.784] / [0 files][ 0.0 B/ 22.7 MiB] / [1 files][ 22.7 MiB/ 22.7 MiB] W1026 04:10:59.784] Operation completed over 1 objects/22.7 MiB. I1026 04:10:59.985] I1026 04:11:00.039] md5sum(kubernetes-client-linux-amd64.tar.gz)=33717c5a370ea5f05cd05560e3cb8e0c I1026 04:11:00.127] sha512sum(kubernetes-client-linux-amd64.tar.gz)=4d4c5c8d0a454d1f3814652f542755d27dc1f0bedc16c84cdf6d53858035abbff664a15e4130dd84f8e0006c2a4d6279fc3fa8f352ba775ecf40a1dacc1e5cac I1026 04:11:00.128] I1026 04:11:00.128] Extracting /go/src/github.com/helm/charts/kubernetes/client/kubernetes-client-linux-amd64.tar.gz into /go/src/github.com/helm/charts/kubernetes/platforms/linux/amd64 I1026 04:11:00.589] Add '/go/src/github.com/helm/charts/kubernetes/client/bin' to your PATH to use newly-installed binaries. W1026 04:11:01.547] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes-test-portable.tar.gz... W1026 04:11:01.645] / [0 files][ 0.0 B/219.8 KiB] / [1 files][219.8 KiB/219.8 KiB] W1026 04:11:01.646] Operation completed over 1 objects/219.8 KiB. I1026 04:11:01.829] I1026 04:11:01.833] md5sum(kubernetes-test-portable.tar.gz)=d550def479686663ba5c84b42613f3c3 I1026 04:11:01.838] sha512sum(kubernetes-test-portable.tar.gz)=4aecc7823b95e9c3b2538024347f1d58975327b48201603a109cfc624f04365373dda638dd9d8109cecedd359b1e53121a534aa095e0fead8a28295b49e58094 I1026 04:11:01.838] I1026 04:11:01.838] Extracting kubernetes-test-portable.tar.gz into /go/src/github.com/helm/charts/kubernetes W1026 04:11:02.826] Copying gs://kubernetes-release-dev/ci/v1.20.0-alpha.3.116+16e18a590f32b9/kubernetes-test-linux-amd64.tar.gz... W1026 04:11:02.827] / [0 files][ 0.0 B/351.2 MiB] ==> NOTE: You are downloading one or more large file(s), which would W1026 04:11:02.828] run significantly faster if you enabled sliced object downloads. This W1026 04:11:02.828] feature is enabled by default but requires that compiled crcmod be W1026 04:11:02.828] installed (see "gsutil help crcmod"). W1026 04:11:02.828] W1026 04:11:06.811] - - [0 files][117.0 MiB/351.2 MiB] \ | | [0 files][258.1 MiB/351.2 MiB] / / [1 files][351.2 MiB/351.2 MiB] - W1026 04:11:06.811] Operation completed over 1 objects/351.2 MiB. I1026 04:11:06.999] I1026 04:11:07.759] md5sum(kubernetes-test-linux-amd64.tar.gz)=8e6a818d5d85bb3c231742ee6a3ac0be I1026 04:11:09.061] sha512sum(kubernetes-test-linux-amd64.tar.gz)=1dfd88ed99f6369e8f56bea0384cb1f51bb4bee0c3dae06134ac24d7b366125d36132b3fb92d972ced62f3d55fe8b5d94357af7b7a9bb456b03d07f161e8acf9 I1026 04:11:09.061] I1026 04:11:09.061] Extracting /go/src/github.com/helm/charts/kubernetes/test/kubernetes-test-linux-amd64.tar.gz into /go/src/github.com/helm/charts/kubernetes/platforms/linux/amd64 W1026 04:11:16.638] 2020/10/26 04:11:16 process.go:155: Step '/workspace/get-kube.sh' finished in 29.843408895s W1026 04:11:16.638] 2020/10/26 04:11:16 process.go:153: Running: ./hack/e2e-internal/e2e-down.sh W1026 04:11:16.696] Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:11:16.697] Network Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:11:16.697] Zone: us-central1-f I1026 04:11:16.797] Shutting down test cluster in background. I1026 04:11:18.692] Bringing down cluster using provider: gce W1026 04:11:18.793] ... calling verify-prereqs W1026 04:11:19.261] ... calling verify-kube-binaries W1026 04:11:19.396] ... calling kube-down W1026 04:11:19.396] Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:11:19.396] Network Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:11:19.396] Zone: us-central1-f W1026 04:11:21.446] INSTANCE_GROUPS= W1026 04:11:21.446] NODE_NAMES= I1026 04:11:21.546] Bringing down cluster I1026 04:11:43.939] Deleting firewall rules remaining in network bootstrap-e2e: W1026 04:11:46.716] W1026 04:11:46.716266 1869 loader.go:223] Config not found: /workspace/.kube/config W1026 04:11:46.857] W1026 04:11:46.856823 1920 loader.go:223] Config not found: /workspace/.kube/config W1026 04:11:46.857] W1026 04:11:46.857118 1920 loader.go:223] Config not found: /workspace/.kube/config I1026 04:11:46.957] Property "clusters.k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e" unset. I1026 04:11:46.996] Property "users.k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e" unset. W1026 04:11:47.096] W1026 04:11:46.995632 1971 loader.go:223] Config not found: /workspace/.kube/config W1026 04:11:47.097] W1026 04:11:46.995835 1971 loader.go:223] Config not found: /workspace/.kube/config W1026 04:11:47.136] W1026 04:11:47.135987 2023 loader.go:223] Config not found: /workspace/.kube/config W1026 04:11:47.136] W1026 04:11:47.136274 2023 loader.go:223] Config not found: /workspace/.kube/config I1026 04:11:47.237] Property "users.k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e-basic-auth" unset. I1026 04:11:47.275] Property "contexts.k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e" unset. I1026 04:11:47.279] Cleared config for k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e from /workspace/.kube/config I1026 04:11:47.280] Done W1026 04:11:47.380] W1026 04:11:47.275487 2075 loader.go:223] Config not found: /workspace/.kube/config W1026 04:11:47.380] W1026 04:11:47.275682 2075 loader.go:223] Config not found: /workspace/.kube/config W1026 04:11:47.380] 2020/10/26 04:11:47 process.go:155: Step './hack/e2e-internal/e2e-down.sh' finished in 30.643268697s W1026 04:11:47.381] 2020/10/26 04:11:47 e2e.go:456: Listing resources... W1026 04:11:47.381] 2020/10/26 04:11:47 process.go:153: Running: ./cluster/gce/list-resources.sh W1026 04:11:48.116] Listed 0 items. W1026 04:11:49.163] Listed 0 items. W1026 04:11:50.844] Listed 0 items. W1026 04:11:51.864] Listed 0 items. W1026 04:11:52.885] Listed 0 items. W1026 04:11:54.898] W1026 04:11:54.899] To show all fields of the firewall, please show in JSON format: --format=json W1026 04:11:54.899] To show all fields in table format, please see the examples in --help. W1026 04:11:54.899] W1026 04:11:55.972] Listed 0 items. W1026 04:11:57.042] Listed 0 items. W1026 04:11:57.943] 2020/10/26 04:11:57 process.go:155: Step './cluster/gce/list-resources.sh' finished in 10.661564261s W1026 04:11:57.943] 2020/10/26 04:11:57 process.go:153: Running: ./hack/e2e-internal/e2e-up.sh W1026 04:11:58.002] Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:11:58.002] Network Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:11:58.002] Zone: us-central1-f W1026 04:11:58.056] ... Starting cluster in us-central1-f using provider gce W1026 04:11:58.057] ... calling verify-prereqs W1026 04:11:58.678] ... calling verify-kube-binaries W1026 04:11:58.826] ... calling verify-release-tars W1026 04:11:58.837] ... calling kube-up W1026 04:11:58.843] Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:11:58.844] Network Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:11:58.844] Zone: us-central1-f I1026 04:12:02.263] +++ Staging tars to Google Storage: gs://kubernetes-staging-66dc9d4aa2/bootstrap-e2e-devel I1026 04:12:14.397] +++ kubernetes-server-linux-amd64.tar.gz uploaded (sha512 = 912eb1aa30aaf2af864bf7f124781093f856889d80dc11146e2a4e1edbb777a4ee2233be2c6847f9369591c279ae3f08858f35d49f94a7ce900bcf0b38780a19) I1026 04:12:19.404] +++ kubernetes-manifests.tar.gz uploaded (sha512 = 60b41962f38d3b05f59ce5d6e20ce3b0ac12028aec10808d52c848eae35b1c793679051424db83e491e3a91894f800e38d5d2cb9eef03dc5ed383eeefa30539a) I1026 04:12:20.358] Creating new auto network: bootstrap-e2e W1026 04:12:58.184] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/networks/bootstrap-e2e]. I1026 04:12:58.435] NAME SUBNET_MODE BGP_ROUTING_MODE IPV4_RANGE GATEWAY_IPV4 I1026 04:12:58.435] bootstrap-e2e AUTO REGIONAL W1026 04:12:58.536] W1026 04:12:58.536] Instances on this network will not be reachable until firewall rules W1026 04:12:58.536] are created. As an example, you can allow all internal traffic between W1026 04:12:58.536] instances as well as SSH, RDP, and ICMP by running: W1026 04:12:58.536] W1026 04:12:58.536] $ gcloud compute firewall-rules create --network bootstrap-e2e --allow tcp,udp,icmp --source-ranges W1026 04:12:58.537] $ gcloud compute firewall-rules create --network bootstrap-e2e --allow tcp:22,tcp:3389,icmp W1026 04:12:58.537] I1026 04:13:01.790] IP aliases are disabled. I1026 04:13:02.919] Found subnet for region us-central1 in network bootstrap-e2e: bootstrap-e2e I1026 04:13:02.923] Starting master and configuring firewalls I1026 04:13:02.924] Configuring firewall for apiserver konnectivity server W1026 04:13:05.051] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/zones/us-central1-f/disks/bootstrap-e2e-master-pd]. I1026 04:13:05.255] NAME ZONE SIZE_GB TYPE STATUS I1026 04:13:05.255] bootstrap-e2e-master-pd us-central1-f 20 pd-ssd READY W1026 04:13:05.356] W1026 04:13:05.356] New disks are unformatted. You must format and mount a disk before it W1026 04:13:05.356] can be used. You can find instructions on how to do this at: W1026 04:13:05.356] W1026 04:13:05.357] https://cloud.google.com/compute/docs/disks/add-persistent-disk#formatting W1026 04:13:05.357] W1026 04:13:05.357] Creating firewall... W1026 04:13:05.357] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-default-internal-master]. I1026 04:13:05.476] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 04:13:05.477] bootstrap-e2e-default-internal-master bootstrap-e2e INGRESS 1000 tcp:1-2379,tcp:2382-65535,udp:1-65535,icmp False W1026 04:13:05.577] done. W1026 04:13:07.785] Creating firewall... W1026 04:13:07.786] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-default-internal-node]. W1026 04:13:07.883] Creating firewall... W1026 04:13:07.884] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-default-ssh]. W1026 04:13:07.972] done. I1026 04:13:08.072] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 04:13:08.073] bootstrap-e2e-default-internal-node bootstrap-e2e INGRESS 1000 tcp:1-65535,udp:1-65535,icmp False I1026 04:13:08.082] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 04:13:08.083] bootstrap-e2e-default-ssh bootstrap-e2e INGRESS 1000 tcp:22 False W1026 04:13:08.183] done. W1026 04:13:08.707] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/regions/us-central1/addresses/bootstrap-e2e-master-ip]. W1026 04:13:10.503] Creating firewall... W1026 04:13:10.503] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-master-https]. I1026 04:13:10.689] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 04:13:10.690] bootstrap-e2e-master-https bootstrap-e2e INGRESS 1000 tcp:443 False W1026 04:13:10.790] done. I1026 04:13:11.568] Generating certs for alternate-names: IP:35.224.198.170,IP:10.0.0.1,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local,DNS:bootstrap-e2e-master W1026 04:13:12.867] Creating firewall... W1026 04:13:12.867] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-master-etcd]. I1026 04:13:13.083] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 04:13:13.084] bootstrap-e2e-master-etcd bootstrap-e2e INGRESS 1000 tcp:2380,tcp:2381 False W1026 04:13:13.184] done. W1026 04:13:14.164] 2020/10/26 04:13:14 [INFO] generating a new CA key and certificate from CSR W1026 04:13:14.164] 2020/10/26 04:13:14 [INFO] generate received request W1026 04:13:14.164] 2020/10/26 04:13:14 [INFO] received CSR W1026 04:13:14.164] 2020/10/26 04:13:14 [INFO] generating key: ecdsa-256 W1026 04:13:14.180] 2020/10/26 04:13:14 [INFO] encoded CSR W1026 04:13:14.182] 2020/10/26 04:13:14 [INFO] signed certificate with serial number 346167302932989079929678304431250187141319169042 W1026 04:13:14.192] 2020/10/26 04:13:14 [INFO] generate received request W1026 04:13:14.192] 2020/10/26 04:13:14 [INFO] received CSR W1026 04:13:14.192] 2020/10/26 04:13:14 [INFO] generating key: ecdsa-256 W1026 04:13:14.215] 2020/10/26 04:13:14 [INFO] encoded CSR W1026 04:13:14.216] 2020/10/26 04:13:14 [INFO] signed certificate with serial number 308926104032080588368264677843382779634124535030 W1026 04:13:14.249] 2020/10/26 04:13:14 [INFO] generate received request W1026 04:13:14.249] 2020/10/26 04:13:14 [INFO] received CSR W1026 04:13:14.249] 2020/10/26 04:13:14 [INFO] generating key: ecdsa-256 W1026 04:13:14.267] 2020/10/26 04:13:14 [INFO] encoded CSR W1026 04:13:14.269] 2020/10/26 04:13:14 [INFO] signed certificate with serial number 624443617629917626279113083820670922589361307408 W1026 04:13:14.284] 2020/10/26 04:13:14 [INFO] generate received request W1026 04:13:14.284] 2020/10/26 04:13:14 [INFO] received CSR W1026 04:13:14.285] 2020/10/26 04:13:14 [INFO] generating key: ecdsa-256 W1026 04:13:14.302] 2020/10/26 04:13:14 [INFO] encoded CSR W1026 04:13:14.303] 2020/10/26 04:13:14 [INFO] signed certificate with serial number 692973354419578781729455317753680514462977128168 W1026 04:13:14.303] 2020/10/26 04:13:14 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for W1026 04:13:14.304] websites. For more information see the Baseline Requirements for the Issuance and Management W1026 04:13:14.304] of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org); W1026 04:13:14.304] specifically, section 10.2.3 ("Information Requirements"). I1026 04:13:14.404] Generate peer certificates... I1026 04:13:14.405] Generate server certificates... I1026 04:13:14.405] Generate client certificates... W1026 04:13:19.991] Creating firewall... W1026 04:13:19.991] ..Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-minion-all]. I1026 04:13:20.179] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 04:13:20.179] bootstrap-e2e-minion-all bootstrap-e2e INGRESS 1000 tcp,udp,icmp,esp,ah,sctp False W1026 04:13:20.280] done. W1026 04:13:24.123] WARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance. W1026 04:13:24.124] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/zones/us-central1-f/instances/bootstrap-e2e-master]. W1026 04:13:24.124] WARNING: Some requests generated warnings: W1026 04:13:24.124] - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details. W1026 04:13:24.125] - The resource 'projects/cos-cloud/global/images/cos-85-13310-1041-9' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-85-13310-1041-14'. W1026 04:13:24.125] W1026 04:13:24.125] NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS W1026 04:13:24.125] bootstrap-e2e-master us-central1-f n1-standard-1 10.128.0.2 35.224.198.170 RUNNING I1026 04:13:24.226] Creating nodes. W1026 04:13:26.465] /go/src/github.com/helm/charts/kubernetes/cluster/../cluster/../cluster/gce/util.sh: line 1537: WINDOWS_CONTAINER_RUNTIME_ENDPOINT: unbound variable I1026 04:13:26.566] Using subnet bootstrap-e2e W1026 04:13:27.508] Attempt 1 to create bootstrap-e2e-minion-template W1026 04:13:28.166] WARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance. W1026 04:13:30.252] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/instanceTemplates/bootstrap-e2e-minion-template]. W1026 04:13:30.443] NAME MACHINE_TYPE PREEMPTIBLE CREATION_TIMESTAMP W1026 04:13:30.443] bootstrap-e2e-minion-template n1-standard-2 2020-10-25T21:13:28.787-07:00 I1026 04:13:30.619] Using subnet bootstrap-e2e W1026 04:13:31.573] Attempt 1 to create bootstrap-e2e-windows-node-template W1026 04:13:32.237] WARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance. W1026 04:13:34.560] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/instanceTemplates/bootstrap-e2e-windows-node-template]. W1026 04:13:34.796] NAME MACHINE_TYPE PREEMPTIBLE CREATION_TIMESTAMP W1026 04:13:34.796] bootstrap-e2e-windows-node-template n1-standard-2 2020-10-25T21:13:33.060-07:00 W1026 04:13:43.795] Created [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/zones/us-central1-f/instanceGroupManagers/bootstrap-e2e-minion-group]. I1026 04:13:44.188] NAME LOCATION SCOPE BASE_INSTANCE_NAME SIZE TARGET_SIZE INSTANCE_TEMPLATE AUTOSCALED I1026 04:13:44.188] bootstrap-e2e-minion-group us-central1-f zone bootstrap-e2e-minion-group 0 3 bootstrap-e2e-minion-template no I1026 04:13:56.260] Waiting for group to become stable, current operations: creating: 3 I1026 04:13:56.261] Group is stable W1026 04:14:00.227] INSTANCE_GROUPS=bootstrap-e2e-minion-group W1026 04:14:00.227] NODE_NAMES=bootstrap-e2e-minion-group-9b6t bootstrap-e2e-minion-group-c0f2 bootstrap-e2e-minion-group-hxdf W1026 04:14:00.228] Trying to find master named 'bootstrap-e2e-master' W1026 04:14:00.228] Looking for address 'bootstrap-e2e-master-ip' W1026 04:14:01.220] Using master: bootstrap-e2e-master (external IP: 35.224.198.170; internal IP: (not set)) I1026 04:14:01.321] Waiting up to 300 seconds for cluster initialization. I1026 04:14:01.321] I1026 04:14:01.321] This will continually check to see if the API for kubernetes is reachable. I1026 04:14:01.321] This may time out if there was some uncaught error during start up. I1026 04:14:01.321] I1026 04:15:16.700] .................Kubernetes cluster created. I1026 04:15:16.845] Cluster "k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e" set. I1026 04:15:16.983] User "k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e" set. I1026 04:15:17.131] Context "k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e" created. I1026 04:15:17.279] Switched to context "k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e". I1026 04:15:17.424] User "k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e-basic-auth" set. I1026 04:15:17.428] Wrote config for k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e to /workspace/.kube/config W1026 04:15:22.991] Updated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/zones/us-central1-f/instances/bootstrap-e2e-master]. I1026 04:15:23.378] I1026 04:15:23.379] Kubernetes cluster is running. The master is running at: I1026 04:15:23.379] I1026 04:15:23.379]  https://35.224.198.170 I1026 04:15:23.379] I1026 04:15:23.380] The user name and password to use is located in /workspace/.kube/config. I1026 04:15:23.380] I1026 04:15:23.434] Validating gce cluster, MULTIZONE= W1026 04:15:23.534] ... calling validate-cluster W1026 04:15:23.534] Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:15:23.535] Network Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:15:23.535] Zone: us-central1-f W1026 04:15:25.533] No resources found I1026 04:15:25.633] Waiting for 4 ready nodes. 0 ready nodes, 0 registered. Retrying. W1026 04:15:40.804] No resources found I1026 04:15:40.905] Waiting for 4 ready nodes. 0 ready nodes, 0 registered. Retrying. I1026 04:15:55.990] Waiting for 4 ready nodes. 0 ready nodes, 4 registered. Retrying. I1026 04:16:11.161] Found 4 node(s). I1026 04:16:11.320] NAME STATUS ROLES AGE VERSION I1026 04:16:11.320] bootstrap-e2e-master Ready,SchedulingDisabled 27s v1.20.0-alpha.3.116+16e18a590f32b9 I1026 04:16:11.321] bootstrap-e2e-minion-group-9b6t Ready 23s v1.20.0-alpha.3.116+16e18a590f32b9 I1026 04:16:11.321] bootstrap-e2e-minion-group-c0f2 Ready 24s v1.20.0-alpha.3.116+16e18a590f32b9 I1026 04:16:11.321] bootstrap-e2e-minion-group-hxdf Ready 23s v1.20.0-alpha.3.116+16e18a590f32b9 W1026 04:16:11.492] Warning: v1 ComponentStatus is deprecated in v1.19+ I1026 04:16:11.593] Validate output: I1026 04:16:11.666] NAME STATUS MESSAGE ERROR I1026 04:16:11.666] scheduler Healthy ok I1026 04:16:11.667] controller-manager Healthy ok I1026 04:16:11.667] etcd-1 Healthy {"health":"true"} I1026 04:16:11.667] etcd-0 Healthy {"health":"true"} I1026 04:16:11.671] Cluster validation succeeded W1026 04:16:11.771] Warning: v1 ComponentStatus is deprecated in v1.19+ W1026 04:16:11.771] Done, listing cluster services: W1026 04:16:11.772] I1026 04:16:11.872] Kubernetes control plane is running at https://35.224.198.170 I1026 04:16:11.872] GLBCDefaultBackend is running at https://35.224.198.170/api/v1/namespaces/kube-system/services/default-http-backend:http/proxy I1026 04:16:11.873] CoreDNS is running at https://35.224.198.170/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy I1026 04:16:11.873] Metrics-server is running at https://35.224.198.170/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy I1026 04:16:11.873] I1026 04:16:11.873] To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. I1026 04:16:11.873] I1026 04:16:18.046] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 04:16:18.047] bootstrap-e2e-minion-http-alt bootstrap-e2e INGRESS 1000 tcp:80,tcp:8080 False I1026 04:16:18.948] allowed: I1026 04:16:18.949] - IPProtocol: tcp I1026 04:16:18.949] ports: I1026 04:16:18.949] - '80' I1026 04:16:18.949] - IPProtocol: tcp I1026 04:16:18.949] ports: I1026 04:16:18.949] - '8080' I1026 04:16:18.949] creationTimestamp: '2020-10-25T21:16:12.848-07:00' I1026 04:16:18.949] description: '' I1026 04:16:18.949] direction: INGRESS I1026 04:16:18.950] disabled: false I1026 04:16:18.950] id: '1093821180529079139' I1026 04:16:18.950] kind: compute#firewall I1026 04:16:18.950] logConfig: I1026 04:16:18.950] enable: false I1026 04:16:18.950] name: bootstrap-e2e-minion-http-alt I1026 04:16:18.951] network: https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/networks/bootstrap-e2e I1026 04:16:18.951] priority: 1000 I1026 04:16:18.951] selfLink: https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-minion-http-alt I1026 04:16:18.951] sourceRanges: I1026 04:16:18.951] - 0.0.0.0/0 I1026 04:16:18.951] targetTags: I1026 04:16:18.952] - bootstrap-e2e-minion I1026 04:16:25.085] NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED I1026 04:16:25.086] bootstrap-e2e-minion-nodeports bootstrap-e2e INGRESS 1000 tcp:30000-32767,udp:30000-32767 False I1026 04:16:26.043] allowed: I1026 04:16:26.043] - IPProtocol: tcp I1026 04:16:26.043] ports: I1026 04:16:26.043] - 30000-32767 I1026 04:16:26.043] - IPProtocol: udp I1026 04:16:26.043] ports: I1026 04:16:26.043] - 30000-32767 I1026 04:16:26.044] creationTimestamp: '2020-10-25T21:16:20.080-07:00' I1026 04:16:26.044] description: '' I1026 04:16:26.044] direction: INGRESS I1026 04:16:26.044] disabled: false I1026 04:16:26.044] id: '3827258300370252667' I1026 04:16:26.044] kind: compute#firewall I1026 04:16:26.044] logConfig: I1026 04:16:26.045] enable: false I1026 04:16:26.045] name: bootstrap-e2e-minion-nodeports I1026 04:16:26.045] network: https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/networks/bootstrap-e2e I1026 04:16:26.045] priority: 1000 I1026 04:16:26.045] selfLink: https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-minion-nodeports I1026 04:16:26.045] sourceRanges: I1026 04:16:26.045] - 0.0.0.0/0 I1026 04:16:26.045] targetTags: I1026 04:16:26.045] - bootstrap-e2e-minion W1026 04:16:26.189] 2020/10/26 04:16:26 process.go:155: Step './hack/e2e-internal/e2e-up.sh' finished in 4m28.245960788s W1026 04:16:26.189] 2020/10/26 04:16:26 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false version W1026 04:16:26.359] 2020/10/26 04:16:26 process.go:155: Step './cluster/kubectl.sh --match-server-version=false version' finished in 169.490975ms W1026 04:16:26.359] 2020/10/26 04:16:26 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false get nodes -oyaml W1026 04:16:26.546] 2020/10/26 04:16:26 process.go:155: Step './cluster/kubectl.sh --match-server-version=false get nodes -oyaml' finished in 187.001378ms W1026 04:16:26.546] 2020/10/26 04:16:26 e2e.go:456: Listing resources... W1026 04:16:26.546] 2020/10/26 04:16:26 process.go:153: Running: ./cluster/gce/list-resources.sh W1026 04:16:34.117] W1026 04:16:34.118] To show all fields of the firewall, please show in JSON format: --format=json W1026 04:16:34.118] To show all fields in table format, please see the examples in --help. W1026 04:16:34.118] W1026 04:16:35.136] Listed 0 items. W1026 04:16:36.132] Listed 0 items. W1026 04:16:36.950] 2020/10/26 04:16:36 process.go:155: Step './cluster/gce/list-resources.sh' finished in 10.404095507s W1026 04:16:36.950] 2020/10/26 04:16:36 util.go:56: Warning: Couldn't find directory src/k8s.io/charts under any of GOPATH /go, defaulting to /go/src/k8s.io/charts W1026 04:16:36.951] 2020/10/26 04:16:36 process.go:153: Running: /go/src/k8s.io/charts/test/helm-test-e2e.sh W1026 04:16:36.951] 2020/10/26 04:16:36 process.go:155: Step '/go/src/k8s.io/charts/test/helm-test-e2e.sh' finished in 391.664µs W1026 04:16:36.951] 2020/10/26 04:16:36 e2e.go:544: Dumping logs locally to: /workspace/_artifacts W1026 04:16:36.951] 2020/10/26 04:16:36 process.go:153: Running: ./cluster/log-dump/log-dump.sh /workspace/_artifacts W1026 04:16:37.020] Trying to find master named 'bootstrap-e2e-master' W1026 04:16:37.021] Looking for address 'bootstrap-e2e-master-ip' I1026 04:16:37.121] Checking for custom logdump instances, if any I1026 04:16:37.121] Sourcing kube-util.sh I1026 04:16:37.122] Detecting project I1026 04:16:37.122] Project: k8s-jkns-gci-gce-serial-1-6 I1026 04:16:37.122] Network Project: k8s-jkns-gci-gce-serial-1-6 I1026 04:16:37.122] Zone: us-central1-f I1026 04:16:37.122] Dumping logs from master locally to '/workspace/_artifacts' W1026 04:16:37.958] Using master: bootstrap-e2e-master (external IP: 35.224.198.170; internal IP: (not set)) I1026 04:17:02.009] Changing logfiles to be world-readable for download I1026 04:17:04.722] Copying 'kube-apiserver.log kube-apiserver-audit.log kube-scheduler.log kube-controller-manager.log etcd.log etcd-events.log glbc.log cluster-autoscaler.log kube-addon-manager.log konnectivity-server.log fluentd.log kubelet.cov startupscript.log' from bootstrap-e2e-master W1026 04:17:05.860] W1026 04:17:05.860] Specify --start=57000 in the next get-serial-port-output invocation to get only the new output starting from here. W1026 04:17:07.515] scp: /var/log/cluster-autoscaler.log*: No such file or directory W1026 04:17:07.519] scp: /var/log/konnectivity-server.log*: No such file or directory W1026 04:17:07.519] scp: /var/log/fluentd.log*: No such file or directory W1026 04:17:07.519] scp: /var/log/kubelet.cov*: No such file or directory W1026 04:17:07.519] scp: /var/log/startupscript.log*: No such file or directory W1026 04:17:07.525] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. I1026 04:17:07.700] Dumping logs from nodes locally to '/workspace/_artifacts' I1026 04:17:07.701] Detecting nodes in the cluster I1026 04:17:37.692] Changing logfiles to be world-readable for download I1026 04:17:37.726] Changing logfiles to be world-readable for download I1026 04:17:37.794] Changing logfiles to be world-readable for download I1026 04:17:40.347] Copying 'kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov startupscript.log' from bootstrap-e2e-minion-group-c0f2 I1026 04:17:40.422] Copying 'kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov startupscript.log' from bootstrap-e2e-minion-group-hxdf I1026 04:17:40.484] Copying 'kube-proxy.log fluentd.log node-problem-detector.log kubelet.cov startupscript.log' from bootstrap-e2e-minion-group-9b6t W1026 04:17:41.264] W1026 04:17:41.264] Specify --start=68348 in the next get-serial-port-output invocation to get only the new output starting from here. W1026 04:17:41.302] W1026 04:17:41.302] Specify --start=67484 in the next get-serial-port-output invocation to get only the new output starting from here. W1026 04:17:41.317] W1026 04:17:41.317] Specify --start=67721 in the next get-serial-port-output invocation to get only the new output starting from here. W1026 04:17:42.575] scp: /var/log/fluentd.log*: No such file or directory W1026 04:17:42.576] scp: /var/log/node-problem-detector.log*: No such file or directory W1026 04:17:42.576] scp: /var/log/kubelet.cov*: No such file or directory W1026 04:17:42.576] scp: /var/log/startupscript.log*: No such file or directory W1026 04:17:42.580] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1026 04:17:42.591] scp: /var/log/fluentd.log*: No such file or directory W1026 04:17:42.592] scp: /var/log/node-problem-detector.log*: No such file or directory W1026 04:17:42.592] scp: /var/log/kubelet.cov*: No such file or directory W1026 04:17:42.592] scp: /var/log/startupscript.log*: No such file or directory W1026 04:17:42.596] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1026 04:17:42.600] scp: /var/log/fluentd.log*: No such file or directory W1026 04:17:42.600] scp: /var/log/node-problem-detector.log*: No such file or directory W1026 04:17:42.601] scp: /var/log/kubelet.cov*: No such file or directory W1026 04:17:42.601] scp: /var/log/startupscript.log*: No such file or directory W1026 04:17:42.604] ERROR: (gcloud.compute.scp) [/usr/bin/scp] exited with return code [1]. W1026 04:17:46.277] INSTANCE_GROUPS=bootstrap-e2e-minion-group W1026 04:17:46.278] NODE_NAMES=bootstrap-e2e-minion-group-9b6t bootstrap-e2e-minion-group-c0f2 bootstrap-e2e-minion-group-hxdf I1026 04:17:47.337] Failures for bootstrap-e2e-minion-group (if any): W1026 04:17:49.490] 2020/10/26 04:17:49 process.go:155: Step './cluster/log-dump/log-dump.sh /workspace/_artifacts' finished in 1m12.539163746s W1026 04:17:49.490] 2020/10/26 04:17:49 e2e.go:456: Listing resources... W1026 04:17:49.490] 2020/10/26 04:17:49 process.go:153: Running: ./cluster/gce/list-resources.sh W1026 04:17:57.178] W1026 04:17:57.178] To show all fields of the firewall, please show in JSON format: --format=json W1026 04:17:57.178] To show all fields in table format, please see the examples in --help. W1026 04:17:57.179] W1026 04:17:58.177] Listed 0 items. W1026 04:17:59.165] Listed 0 items. W1026 04:18:00.022] 2020/10/26 04:18:00 process.go:155: Step './cluster/gce/list-resources.sh' finished in 10.532324797s W1026 04:18:00.022] 2020/10/26 04:18:00 process.go:153: Running: ./hack/e2e-internal/e2e-down.sh W1026 04:18:00.077] Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:18:00.077] Network Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:18:00.077] Zone: us-central1-f I1026 04:18:00.177] Shutting down test cluster in background. W1026 04:18:05.589] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-minion-http-alt]. W1026 04:18:07.511] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-minion-nodeports]. I1026 04:18:07.763] Bringing down cluster using provider: gce W1026 04:18:07.863] ... calling verify-prereqs W1026 04:18:08.339] ... calling verify-kube-binaries W1026 04:18:08.479] ... calling kube-down W1026 04:18:08.479] Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:18:08.479] Network Project: k8s-jkns-gci-gce-serial-1-6 W1026 04:18:08.480] Zone: us-central1-f W1026 04:18:12.010] INSTANCE_GROUPS=bootstrap-e2e-minion-group W1026 04:18:12.010] NODE_NAMES=bootstrap-e2e-minion-group-9b6t bootstrap-e2e-minion-group-c0f2 bootstrap-e2e-minion-group-hxdf I1026 04:18:12.111] Bringing down cluster W1026 04:20:16.249] Deleting Managed Instance Group... W1026 04:20:16.249] ...Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/zones/us-central1-f/instanceGroupManagers/bootstrap-e2e-minion-group]. W1026 04:20:16.253] done. W1026 04:20:20.338] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/instanceTemplates/bootstrap-e2e-minion-template]. W1026 04:20:21.620] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/instanceTemplates/bootstrap-e2e-windows-node-template]. I1026 04:20:27.137] Successfully executed 'curl -s --cacert /etc/srv/kubernetes/pki/etcd-apiserver-ca.crt --cert /etc/srv/kubernetes/pki/etcd-apiserver-client.crt --key /etc/srv/kubernetes/pki/etcd-apiserver-client.key https://127.0.0.1:2379/v2/members/$(curl -s --cacert /etc/srv/kubernetes/pki/etcd-apiserver-ca.crt --cert /etc/srv/kubernetes/pki/etcd-apiserver-client.crt --key /etc/srv/kubernetes/pki/etcd-apiserver-client.key https://127.0.0.1:2379/v2/members -XGET | sed 's/{\"id/\n/g' | grep bootstrap-e2e-master\" | cut -f 3 -d \") -XDELETE -L 2>/dev/null' on bootstrap-e2e-master I1026 04:20:27.137] Removing etcd replica, name: bootstrap-e2e-master, port: 2379, result: 0 I1026 04:20:28.502] Successfully executed 'curl -s http://127.0.0.1:4002/v2/members/$(curl -s http://127.0.0.1:4002/v2/members -XGET | sed 's/{\"id/\n/g' | grep bootstrap-e2e-master\" | cut -f 3 -d \") -XDELETE -L 2>/dev/null' on bootstrap-e2e-master I1026 04:20:28.502] Removing etcd replica, name: bootstrap-e2e-master, port: 4002, result: 0 W1026 04:20:33.346] Updated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/zones/us-central1-f/instances/bootstrap-e2e-master]. W1026 04:22:25.960] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/zones/us-central1-f/instances/bootstrap-e2e-master]. W1026 04:22:36.124] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-master-https]. W1026 04:22:39.001] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-master-etcd]. W1026 04:22:39.192] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-minion-all]. W1026 04:22:43.345] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/regions/us-central1/addresses/bootstrap-e2e-master-ip]. I1026 04:22:47.301] Deleting routes bootstrap-e2e-f79e7b8e-1f57-4360-aaa0-b37779b9658e W1026 04:22:53.676] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/routes/bootstrap-e2e-f79e7b8e-1f57-4360-aaa0-b37779b9658e]. W1026 04:23:00.132] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-default-internal-master]. W1026 04:23:02.467] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-default-internal-node]. W1026 04:23:02.500] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/firewalls/bootstrap-e2e-default-ssh]. I1026 04:23:03.658] Deleting firewall rules remaining in network bootstrap-e2e: W1026 04:23:50.448] Deleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-serial-1-6/global/networks/bootstrap-e2e]. I1026 04:23:50.967] Property "current-context" unset. I1026 04:23:51.108] Property "clusters.k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e" unset. I1026 04:23:51.251] Property "users.k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e" unset. I1026 04:23:51.394] Property "users.k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e-basic-auth" unset. I1026 04:23:51.533] Property "contexts.k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e" unset. I1026 04:23:51.537] Cleared config for k8s-jkns-gci-gce-serial-1-6_bootstrap-e2e from /workspace/.kube/config I1026 04:23:51.538] Done W1026 04:23:51.638] 2020/10/26 04:23:51 process.go:155: Step './hack/e2e-internal/e2e-down.sh' finished in 5m51.516990626s W1026 04:23:51.638] 2020/10/26 04:23:51 e2e.go:278: Sleeping for 30 seconds... W1026 04:24:21.539] 2020/10/26 04:24:21 e2e.go:456: Listing resources... W1026 04:24:21.540] 2020/10/26 04:24:21 process.go:153: Running: ./cluster/gce/list-resources.sh W1026 04:24:22.419] Listed 0 items. W1026 04:24:23.471] Listed 0 items. W1026 04:24:25.238] Listed 0 items. W1026 04:24:26.313] Listed 0 items. W1026 04:24:27.342] Listed 0 items. W1026 04:24:29.600] W1026 04:24:29.600] To show all fields of the firewall, please show in JSON format: --format=json W1026 04:24:29.600] To show all fields in table format, please see the examples in --help. W1026 04:24:29.601] W1026 04:24:30.733] Listed 0 items. W1026 04:24:31.808] Listed 0 items. W1026 04:24:32.699] 2020/10/26 04:24:32 process.go:155: Step './cluster/gce/list-resources.sh' finished in 11.159196806s W1026 04:24:32.699] 2020/10/26 04:24:32 process.go:153: Running: diff -sw -U0 -F^\[.*\]$ /workspace/_artifacts/gcp-resources-before.txt /workspace/_artifacts/gcp-resources-after.txt W1026 04:24:32.704] 2020/10/26 04:24:32 process.go:155: Step 'diff -sw -U0 -F^\[.*\]$ /workspace/_artifacts/gcp-resources-before.txt /workspace/_artifacts/gcp-resources-after.txt' finished in 4.947827ms W1026 04:24:32.705] 2020/10/26 04:24:32 process.go:96: Saved XML output to /workspace/_artifacts/junit_runner.xml. W1026 04:24:32.713] 2020/10/26 04:24:32 main.go:316: Something went wrong: encountered 1 errors: [error starting /go/src/k8s.io/charts/test/helm-test-e2e.sh: fork/exec /go/src/k8s.io/charts/test/helm-test-e2e.sh: no such file or directory] W1026 04:24:32.720] Traceback (most recent call last): W1026 04:24:32.720] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 720, in W1026 04:24:32.720] main(parse_args()) W1026 04:24:32.721] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 570, in main W1026 04:24:32.721] mode.start(runner_args) W1026 04:24:32.721] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 228, in start W1026 04:24:32.721] check_env(env, self.command, *args) W1026 04:24:32.721] File "/workspace/./test-infra/jenkins/../scenarios/kubernetes_e2e.py", line 111, in check_env W1026 04:24:32.721] subprocess.check_call(cmd, env=env) W1026 04:24:32.722] File "/usr/lib/python2.7/subprocess.py", line 190, in check_call W1026 04:24:32.722] raise CalledProcessError(retcode, cmd) W1026 04:24:32.722] subprocess.CalledProcessError: Command '('kubetest', '--dump=/workspace/_artifacts', '--gcp-service-account=/etc/service-account/service-account.json', '--up', '--down', '--provider=gce', '--cluster=bootstrap-e2e', '--gcp-network=bootstrap-e2e', '--charts', '--check-leaked-resources', '--extract=ci/latest', '--gcp-node-image=gci', '--gcp-zone=us-central1-f', '--timeout=180m')' returned non-zero exit status 1 E1026 04:24:32.730] Command failed I1026 04:24:32.730] process 282 exited with code 1 after 13.8m E1026 04:24:32.730] FAIL: ci-kubernetes-charts-gce I1026 04:24:32.731] Call: gcloud auth activate-service-account --key-file=/etc/service-account/service-account.json W1026 04:24:33.376] Activated service account credentials for: [pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com] I1026 04:24:33.500] process 12857 exited with code 0 after 0.0m I1026 04:24:33.501] Call: gcloud config get-value account I1026 04:24:34.075] process 12870 exited with code 0 after 0.0m I1026 04:24:34.075] Will upload results to gs://kubernetes-jenkins/logs using pr-kubekins@kubernetes-jenkins-pull.iam.gserviceaccount.com I1026 04:24:34.075] Upload result and artifacts... I1026 04:24:34.075] Gubernator results at https://gubernator.k8s.io/build/kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320578491233603584 I1026 04:24:34.076] Call: gsutil ls gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320578491233603584/artifacts W1026 04:24:35.026] CommandException: One or more URLs matched no objects. E1026 04:24:35.208] Command failed I1026 04:24:35.208] process 12883 exited with code 1 after 0.0m W1026 04:24:35.209] Remote dir gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320578491233603584/artifacts not exist yet I1026 04:24:35.209] Call: gsutil -m -q -o GSUtil:use_magicfile=True cp -r -c -z log,txt,xml /workspace/_artifacts gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320578491233603584/artifacts I1026 04:24:37.668] process 13026 exited with code 0 after 0.0m I1026 04:24:37.669] Call: git rev-parse HEAD I1026 04:24:37.673] process 13679 exited with code 0 after 0.0m I1026 04:24:37.673] Call: git rev-parse HEAD I1026 04:24:37.676] process 13680 exited with code 0 after 0.0m I1026 04:24:37.677] Call: gsutil stat gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/jobResultsCache.json I1026 04:24:38.763] process 13681 exited with code 0 after 0.0m I1026 04:24:38.764] Call: gsutil -q cat 'gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/jobResultsCache.json#1603682640726672' I1026 04:24:39.992] process 13824 exited with code 0 after 0.0m I1026 04:24:39.996] Call: gsutil -q -h Content-Type:application/json -h x-goog-if-generation-match:1603682640726672 cp /tmp/gsutil_BCiZ7x gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/jobResultsCache.json I1026 04:24:41.349] process 13969 exited with code 0 after 0.0m I1026 04:24:41.351] Call: gsutil -q -h Content-Type:application/json cp /tmp/gsutil_SQaIv6 gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/1320578491233603584/finished.json I1026 04:24:42.698] process 14148 exited with code 0 after 0.0m I1026 04:24:42.699] Call: gsutil -q -h Content-Type:text/plain -h 'Cache-Control:private, max-age=0, no-transform' cp /tmp/gsutil__12p_j gs://kubernetes-jenkins/logs/ci-kubernetes-charts-gce/latest-build.txt I1026 04:24:44.746] process 14327 exited with code 0 after 0.0m