INFO[2024-08-09T12:02:04Z] ci-operator version v20240808-a34f155d6      
INFO[2024-08-09T12:02:04Z] Loading configuration from https://config.ci.openshift.org for openshift/cluster-node-tuning-operator@master 
INFO[2024-08-09T12:02:04Z] Resolved source https://github.com/openshift/cluster-node-tuning-operator to master@6d625b5a, merging: #1129 b24c4f28 @ffromani 
WARN[2024-08-09T12:02:04Z] skipped directory "..2024_08_09_12_01_51.3638902426" when creating secret from directory "/secrets/ci-pull-credentials" 
WARN[2024-08-09T12:02:04Z] skipped directory "..2024_08_09_12_01_51.2754130912" when creating secret from directory "/usr/local/e2e-aws-operator-cluster-profile" 
INFO[2024-08-09T12:02:04Z] Loading information from https://config.ci.openshift.org for integrated stream ocp/4.18 
INFO[2024-08-09T12:02:04Z] Loading information from https://config.ci.openshift.org for integrated stream ocp/4.18 
INFO[2024-08-09T12:02:04Z] Building release initial from a snapshot of ocp/4.18 
INFO[2024-08-09T12:02:04Z] Building release latest from a snapshot of ocp/4.18 
INFO[2024-08-09T12:02:05Z] Using namespace https://console-openshift-console.apps.build03.ky4t.p1.openshiftapps.com/k8s/cluster/projects/ci-op-n3l6f4pk 
INFO[2024-08-09T12:02:05Z] Running [input:root], [input:ocp_4.17_base-rhel9], [input:ocp_builder_rhel-9-golang-1.22-openshift-4.17], [input:origin-centos-8], [input:ocp-4.12-upi-installer], [input:ocp-4.14-upi-installer], [input:ocp-4.16-upi-installer], [release-inputs:initial], [release-inputs:latest], src, cluster-node-tuning-operator, [output:stable:cluster-node-tuning-operator], [images], [release:latest], e2e-aws-operator 
INFO[2024-08-09T12:02:06Z] Tagging ocp/builder:rhel-9-golang-1.22-openshift-4.17 into pipeline:ocp_builder_rhel-9-golang-1.22-openshift-4.17. 
INFO[2024-08-09T12:02:06Z] Tagging ocp/4.14:upi-installer into pipeline:ocp-4.14-upi-installer. 
INFO[2024-08-09T12:02:06Z] Tagging ocp/4.16:upi-installer into pipeline:ocp-4.16-upi-installer. 
INFO[2024-08-09T12:02:06Z] Tagging ocp/4.18:base-rhel9 into pipeline:ocp_4.17_base-rhel9. 
INFO[2024-08-09T12:02:06Z] Tagging origin/centos:8 into pipeline:origin-centos-8. 
INFO[2024-08-09T12:02:06Z] Tagging ocp/4.12:upi-installer into pipeline:ocp-4.12-upi-installer. 
INFO[2024-08-09T12:02:06Z] Tagging openshift/release:rhel-9-release-golang-1.22-openshift-4.17 into pipeline:root. 
INFO[2024-08-09T12:02:06Z] Waiting to import tags on imagestream (after taking snapshot) ci-op-n3l6f4pk/stable-initial ... 
INFO[2024-08-09T12:02:06Z] Waiting to import tags on imagestream (after taking snapshot) ci-op-n3l6f4pk/stable ... 
INFO[2024-08-09T12:02:14Z] Building src                                 
INFO[2024-08-09T12:02:14Z] Found existing build "src-amd64"             
INFO[2024-08-09T12:05:07Z] Imported tags on imagestream (after taking snapshot) ci-op-n3l6f4pk/stable-initial 
INFO[2024-08-09T12:05:07Z] Imported tags on imagestream (after taking snapshot) ci-op-n3l6f4pk/stable 
INFO[2024-08-09T12:06:41Z] Build src-amd64 succeeded after 4m27s        
INFO[2024-08-09T12:06:42Z] Image ci-op-n3l6f4pk/pipeline:src created     for-build=src
INFO[2024-08-09T12:06:42Z] Building cluster-node-tuning-operator        
INFO[2024-08-09T12:06:42Z] Found existing build "cluster-node-tuning-operator-amd64" 
INFO[2024-08-09T12:11:01Z] Build cluster-node-tuning-operator-amd64 succeeded after 4m20s 
INFO[2024-08-09T12:11:02Z] Image ci-op-n3l6f4pk/pipeline:cluster-node-tuning-operator created  for-build=cluster-node-tuning-operator
INFO[2024-08-09T12:11:02Z] Tagging cluster-node-tuning-operator into stable 
INFO[2024-08-09T12:11:02Z] Creating release image registry.build03.ci.openshift.org/ci-op-n3l6f4pk/release:latest. 
INFO[2024-08-09T12:12:13Z] Snapshot integration stream into release 4.18.0-0.ci.test-2024-08-09-121102-ci-op-n3l6f4pk-latest to tag release:latest  
INFO[2024-08-09T12:12:13Z] Acquiring leases for test e2e-aws-operator: [aws-quota-slice] 
INFO[2024-08-09T12:12:13Z] Acquired 1 lease(s) for aws-quota-slice: [us-east-2--aws-quota-slice-02] 
INFO[2024-08-09T12:12:13Z] Acquiring IP Pool leases for test e2e-aws-operator: aws-ip-pools-us-east-2 
INFO[2024-08-09T12:12:13Z] no leases of type: aws-ip-pools-us-east-2 available 
INFO[2024-08-09T12:12:13Z] Running multi-stage test e2e-aws-operator    
INFO[2024-08-09T12:12:14Z] Running multi-stage phase pre                
INFO[2024-08-09T12:12:14Z] Running step e2e-aws-operator-ipi-conf.      
INFO[2024-08-09T12:12:22Z] Step e2e-aws-operator-ipi-conf succeeded after 8s. 
INFO[2024-08-09T12:12:22Z] Running step e2e-aws-operator-ipi-conf-telemetry. 
INFO[2024-08-09T12:12:31Z] Step e2e-aws-operator-ipi-conf-telemetry succeeded after 8s. 
INFO[2024-08-09T12:12:31Z] Running step e2e-aws-operator-ipi-conf-aws.  
INFO[2024-08-09T12:12:43Z] Step e2e-aws-operator-ipi-conf-aws succeeded after 12s. 
INFO[2024-08-09T12:12:43Z] Running step e2e-aws-operator-ipi-conf-aws-byo-ipv4-pool-public. 
INFO[2024-08-09T12:12:52Z] Step e2e-aws-operator-ipi-conf-aws-byo-ipv4-pool-public succeeded after 8s. 
INFO[2024-08-09T12:12:52Z] Running step e2e-aws-operator-ipi-install-monitoringpvc. 
INFO[2024-08-09T12:13:01Z] Step e2e-aws-operator-ipi-install-monitoringpvc succeeded after 8s. 
INFO[2024-08-09T12:13:01Z] Running step e2e-aws-operator-ipi-install-rbac. 
INFO[2024-08-09T12:13:09Z] Step e2e-aws-operator-ipi-install-rbac succeeded after 8s. 
INFO[2024-08-09T12:13:09Z] Running step e2e-aws-operator-openshift-cluster-bot-rbac. 
INFO[2024-08-09T12:13:17Z] Step e2e-aws-operator-openshift-cluster-bot-rbac succeeded after 8s. 
INFO[2024-08-09T12:13:17Z] Running step e2e-aws-operator-ipi-install-hosted-loki. 
INFO[2024-08-09T12:13:25Z] Step e2e-aws-operator-ipi-install-hosted-loki succeeded after 7s. 
INFO[2024-08-09T12:13:25Z] Running step e2e-aws-operator-ipi-install-install. 
INFO[2024-08-09T12:43:35Z] Step e2e-aws-operator-ipi-install-install succeeded after 30m9s. 
INFO[2024-08-09T12:43:35Z] Running step e2e-aws-operator-ipi-install-times-collection. 
INFO[2024-08-09T12:43:43Z] Step e2e-aws-operator-ipi-install-times-collection succeeded after 8s. 
INFO[2024-08-09T12:43:43Z] Running step e2e-aws-operator-nodes-readiness. 
INFO[2024-08-09T12:44:42Z] Step e2e-aws-operator-nodes-readiness succeeded after 58s. 
INFO[2024-08-09T12:44:42Z] Running step e2e-aws-operator-multiarch-validate-nodes. 
INFO[2024-08-09T12:44:50Z] Step e2e-aws-operator-multiarch-validate-nodes succeeded after 8s. 
INFO[2024-08-09T12:44:50Z] Step phase pre succeeded after 32m36s.       
INFO[2024-08-09T12:44:50Z] Running multi-stage phase test               
INFO[2024-08-09T12:44:50Z] Running step e2e-aws-operator-test.          
INFO[2024-08-09T12:52:43Z] Logs for container test in pod e2e-aws-operator-test: 
INFO[2024-08-09T12:52:43Z] GOOS=linux GO111MODULE=on GOFLAGS=-mod=vendor go build -o _output/go-bindata ./vendor/github.com/kevinburke/go-bindata/go-bindata
_output/go-bindata -mode 420 -modtime 1 -pkg manifests -o pkg/manifests/bindata.go assets/...
gofmt -s -w pkg/manifests/bindata.go
for d in core basic reboots reboots/sno deferred; do \
  KUBERNETES_CONFIG="/tmp/kubeconfig-2615628509" GOOS=linux GO111MODULE=on GOFLAGS=-mod=vendor go test -v -timeout 40m ./test/e2e/$d -ginkgo.v -ginkgo.no-color -ginkgo.fail-fast -ginkgo.label-filter=!flaky || exit; \
done
=== RUN   TestNodeTuningOperator
Running Suite: Node Tuning Operator e2e tests: core - /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/core
=============================================================================================================================
Random Seed: 1723207531

Will run 1 of 1 specs
------------------------------
[core][cluster_version] Node Tuning Operator host, container OS and cluster version host, container OS and cluster version retrievable
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/core/cluster_version.go:20
  STEP: getting a list of worker nodes @ 08/09/24 12:45:31.742
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:45:31.826
  STEP: getting the host OS version on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:45:31.845
  NAME="Red Hat Enterprise Linux CoreOS"
  ID="rhcos"
  ID_LIKE="rhel fedora"
  VERSION="416.94.202405210347-0"
  VERSION_ID="4.16"
  VARIANT="CoreOS"
  VARIANT_ID=coreos
  PLATFORM_ID="platform:el9"
  PRETTY_NAME="Red Hat Enterprise Linux CoreOS 416.94.202405210347-0"
  ANSI_COLOR="0;31"
  CPE_NAME="cpe:/o:redhat:enterprise_linux:9::baseos::coreos"
  HOME_URL="https://www.redhat.com/"
  DOCUMENTATION_URL="https://docs.okd.io/latest/welcome/index.html"
  BUG_REPORT_URL="https://access.redhat.com/labs/rhir/"
  REDHAT_BUGZILLA_PRODUCT="OpenShift Container Platform"
  REDHAT_BUGZILLA_PRODUCT_VERSION="4.16"
  REDHAT_SUPPORT_PRODUCT="OpenShift Container Platform"
  REDHAT_SUPPORT_PRODUCT_VERSION="4.16"
  OPENSHIFT_VERSION="4.16"
  RHEL_VERSION=9.4
  OSTREE_VERSION="416.94.202405210347-0"

  STEP: getting the TuneD container OS version @ 08/09/24 12:45:32.311
  NAME="Red Hat Enterprise Linux"
  VERSION="9.2 (Plow)"
  ID="rhel"
  ID_LIKE="fedora"
  VERSION_ID="9.2"
  PLATFORM_ID="platform:el9"
  PRETTY_NAME="Red Hat Enterprise Linux 9.2 (Plow)"
  ANSI_COLOR="0;31"
  LOGO="fedora-logo-icon"
  CPE_NAME="cpe:/o:redhat:enterprise_linux:9::baseos"
  HOME_URL="https://www.redhat.com/"
  DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9"
  BUG_REPORT_URL="https://issues.redhat.com/"

  REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 9"
  REDHAT_BUGZILLA_PRODUCT_VERSION=9.2
  REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
  REDHAT_SUPPORT_PRODUCT_VERSION="9.2"

  STEP: getting the cluster version @ 08/09/24 12:45:32.634
  run command 'oc [get clusterversion version -o jsonpath='{.status.desired.version}']':
    out='4.18.0-0.ci.test-2024-08-09-121102-ci-op-n3l6f4pk-latest'
    err=
    ret=<nil>
• [1.022 seconds]
------------------------------

Ran 1 of 1 Specs in 1.022 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestNodeTuningOperator (1.02s)
PASS
ok  	github.com/openshift/cluster-node-tuning-operator/test/e2e/core	1.036s
=== RUN   TestNodeTuningOperator
Running Suite: Node Tuning Operator e2e tests: basic - /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic
===============================================================================================================================
Random Seed: 1723207535

Will run 15 of 15 specs
------------------------------
[basic][available] Node Tuning Operator availability ClusterOperator/node-tuning available and not degraded
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/available.go:29
  STEP: waiting for ClusterOperator/node-tuning available @ 08/09/24 12:45:35.189
  STEP: waiting for ClusterOperator/node-tuning not degraded @ 08/09/24 12:45:35.246
• [0.073 seconds]
------------------------------
[basic][available] Node Tuning Operator availability Tuned/default exists
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/available.go:39
  STEP: waiting for Tuned/default existence @ 08/09/24 12:45:35.262
• [0.024 seconds]
------------------------------
[basic][netdev_set_channels] Node Tuning Operator adjust netdev queue count adjust netdev queue count adjust netdev queue count for physical network devices via ethtool
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/netdev_set_channels.go:41
  STEP: getting a list of worker nodes @ 08/09/24 12:45:35.285
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:45:35.33
  found Pod tuned-6pxwr running on node ip-10-0-114-137.us-east-2.compute.internal
  STEP: getting a list of physical network devices: [find /sys/class/net -type l -not -lname *virtual* -printf %f ] @ 08/09/24 12:45:35.35
  STEP: using physical network device ens5 for testing @ 08/09/24 12:45:36.043
  STEP: ensuring the number of multi-purpose channels is set to 1 for ens5 on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:45:36.747
  STEP: labelling node ip-10-0-114-137.us-east-2.compute.internal with label tuned.openshift.io/netdev-set-queue-count @ 08/09/24 12:45:37.434
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/netdev-set-queue-count=]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: creating profile ../testing_manifests/netdev_set_channels.yaml @ 08/09/24 12:45:37.598
  run command 'oc [create -n openshift-cluster-node-tuning-operator -f ../testing_manifests/netdev_set_channels.yaml]':
    out=tuned.tuned.openshift.io/openshift-netdev-set-queue-count created
    err=
    ret=<nil>
  STEP: ensuring the number of multi-purpose channels is set to 2 for ens5 on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:45:37.736
  STEP: deleting profile ../testing_manifests/netdev_set_channels.yaml @ 08/09/24 12:45:39.091
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/netdev_set_channels.yaml]':
    out=tuned.tuned.openshift.io "openshift-netdev-set-queue-count" deleted
    err=
    ret=<nil>
  STEP: removing label tuned.openshift.io/netdev-set-queue-count from node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:45:39.243
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/netdev-set-queue-count-]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: ensuring the number of multi-purpose channels was rolled back to 1 for ens5 on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:45:39.423
  STEP: cluster changes rollback @ 08/09/24 12:45:40.091
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/netdev-set-queue-count-]':
    out=label "tuned.openshift.io/netdev-set-queue-count" not found.
  node/ip-10-0-114-137.us-east-2.compute.internal not labeled
    err=
    ret=<nil>
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/netdev_set_channels.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "../testing_manifests/netdev_set_channels.yaml": tuneds.tuned.openshift.io "openshift-netdev-set-queue-count" not found
    ret=exit status 1
• [5.100 seconds]
------------------------------
[basic][default_node_sysctl] Node Tuning Operator default profile set net.ipv4.neigh.default.gc_thresh1 set
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/default_node_sysctl.go:21
  STEP: getting a list of worker nodes @ 08/09/24 12:45:40.386
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:45:40.428
  STEP: waiting for TuneD profile openshift-node on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:45:40.444
  STEP: ensuring the default worker node profile was set @ 08/09/24 12:45:40.467
• [0.423 seconds]
------------------------------
[basic][metrics] Node Tuning Operator certificate rotation TLS certificate rotation delete certificate Secret and check that the server restarts with latest certificate
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/metrics_cert_rotation.go:20
  STEP: deleting node-tuning-operator-tls Secret to trigger certificate rotation @ 08/09/24 12:45:40.808
  run command 'oc [delete -n openshift-cluster-node-tuning-operator secret/node-tuning-operator-tls]':
    out=secret "node-tuning-operator-tls" deleted
    err=
    ret=<nil>
  Error getting secret/node-tuning-operator-tls. May not exist yet. Err: secrets "node-tuning-operator-tls" not found
  STEP: deleting node-tuning-operator-tls Secret a second time @ 08/09/24 12:45:42.998
  run command 'oc [delete -n openshift-cluster-node-tuning-operator secret/node-tuning-operator-tls]':
    out=secret "node-tuning-operator-tls" deleted
    err=
    ret=<nil>
  Error getting secret/node-tuning-operator-tls. May not exist yet. Err: secrets "node-tuning-operator-tls" not found
  STEP: getting a list of worker nodes @ 08/09/24 12:45:45.176
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:45:45.205
  STEP: getting cluster-node-tuning-operator Pod @ 08/09/24 12:45:45.22
  STEP: checking if server TLS certificate matches TLS certificate in Secret @ 08/09/24 12:45:45.236
• [49.792 seconds]
------------------------------
[basic][custom_node_labels] Node Tuning Operator custom profile, node labels custom profile: node labels vm.nr_hugepages set
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/custom_node_labels.go:39
  STEP: getting a list of worker nodes @ 08/09/24 12:46:30.601
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:30.642
  STEP: waiting for TuneD profile openshift-node on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:30.661
  STEP: getting the current value of vm.nr_hugepages in Pod tuned-6pxwr @ 08/09/24 12:46:30.676
  STEP: labelling node ip-10-0-114-137.us-east-2.compute.internal with label tuned.openshift.io/hugepages @ 08/09/24 12:46:30.999
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/hugepages=]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: creating the custom hugepages profile ../../../examples/hugepages.yaml @ 08/09/24 12:46:31.17
  run command 'oc [create -n openshift-cluster-node-tuning-operator -f ../../../examples/hugepages.yaml]':
    out=tuned.tuned.openshift.io/openshift-hugepages created
    err=
    ret=<nil>
  STEP: ensuring the custom worker node profile was set @ 08/09/24 12:46:31.3
  STEP: deleting the custom hugepages profile ../../../examples/hugepages.yaml @ 08/09/24 12:46:31.618
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../../../examples/hugepages.yaml]':
    out=tuned.tuned.openshift.io "openshift-hugepages" deleted
    err=
    ret=<nil>
  STEP: ensuring the original vm.nr_hugepages value (0) is set in Pod tuned-6pxwr @ 08/09/24 12:46:31.756
  STEP: removing label tuned.openshift.io/hugepages from node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:32.077
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/hugepages-]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: cluster changes rollback @ 08/09/24 12:46:32.24
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/hugepages-]':
    out=label "tuned.openshift.io/hugepages" not found.
  node/ip-10-0-114-137.us-east-2.compute.internal not labeled
    err=
    ret=<nil>
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../../../examples/hugepages.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "../../../examples/hugepages.yaml": tuneds.tuned.openshift.io "openshift-hugepages" not found
    ret=exit status 1
• [1.912 seconds]
------------------------------
[basic][tuned_builtin_expand] Node Tuning Operator custom profile, TuneD built-in expansion custom profile: built-in expansion vm.nr_hugepages set
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/tuned_builtin_expand.go:41
  STEP: getting a list of worker nodes @ 08/09/24 12:46:32.513
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:32.555
  STEP: waiting for TuneD profile openshift-node on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:32.572
  STEP: getting the current value of vm.nr_hugepages in Pod tuned-6pxwr @ 08/09/24 12:46:32.587
  STEP: labelling node ip-10-0-114-137.us-east-2.compute.internal with label tuned.openshift.io/tuned-built-in @ 08/09/24 12:46:32.934
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/tuned-built-in=]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: creating the custom profile ../../../examples/hugepages.yaml @ 08/09/24 12:46:33.106
  run command 'oc [create -n openshift-cluster-node-tuning-operator -f ../../../examples/hugepages.yaml]':
    out=tuned.tuned.openshift.io/openshift-hugepages created
    err=
    ret=<nil>
  STEP: creating the custom profile ../testing_manifests/tuned_builtin_expand.yaml @ 08/09/24 12:46:33.235
  run command 'oc [create -n openshift-cluster-node-tuning-operator -f ../testing_manifests/tuned_builtin_expand.yaml]':
    out=tuned.tuned.openshift.io/openshift-tuned-builtin created
    err=
    ret=<nil>
  STEP: ensuring the custom worker node profile was set @ 08/09/24 12:46:33.35
  STEP: deleting the custom profile ../testing_manifests/tuned_builtin_expand.yaml @ 08/09/24 12:46:38.669
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/tuned_builtin_expand.yaml]':
    out=tuned.tuned.openshift.io "openshift-tuned-builtin" deleted
    err=
    ret=<nil>
  STEP: deleting the custom profile ../../../examples/hugepages.yaml @ 08/09/24 12:46:38.814
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../../../examples/hugepages.yaml]':
    out=tuned.tuned.openshift.io "openshift-hugepages" deleted
    err=
    ret=<nil>
  STEP: ensuring the original vm.nr_hugepages value (0) is set in Pod tuned-6pxwr @ 08/09/24 12:46:38.955
  STEP: removing label tuned.openshift.io/tuned-built-in from node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:39.282
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/tuned-built-in-]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: cluster changes rollback @ 08/09/24 12:46:39.455
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/tuned-built-in-]':
    out=label "tuned.openshift.io/tuned-built-in" not found.
  node/ip-10-0-114-137.us-east-2.compute.internal not labeled
    err=
    ret=<nil>
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/tuned_builtin_expand.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "../testing_manifests/tuned_builtin_expand.yaml": tuneds.tuned.openshift.io "openshift-tuned-builtin" not found
    ret=exit status 1
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../../../examples/hugepages.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "../../../examples/hugepages.yaml": tuneds.tuned.openshift.io "openshift-hugepages" not found
    ret=exit status 1
• [7.341 seconds]
------------------------------
[basic][default_irq_smp_affinity] Node Tuning Operator set irq default smp affinity irq default smp affinity default_irq_smp_affinity: /proc/irq/default_smp_affinity set
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/default_irq_smp_affinity.go:45
  STEP: getting a list of worker nodes @ 08/09/24 12:46:39.854
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:39.896
  STEP: getting the number of CPUs on ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:39.912
  STEP: waiting for TuneD profile openshift-node on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:40.238
  STEP: getting the original value of /proc/irq/default_smp_affinity @ 08/09/24 12:46:40.253
  tuned-6pxwr has the last nibble of /proc/irq/default_smp_affinity: f
  STEP: labelling node ip-10-0-114-137.us-east-2.compute.internal with label tuned.openshift.io/default-irq-smp-affinity @ 08/09/24 12:46:40.583
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/default-irq-smp-affinity=]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: creating the custom affinity profile ../testing_manifests/default_irq_smp_affinity0.yaml @ 08/09/24 12:46:40.758
  run command 'oc [create -n openshift-cluster-node-tuning-operator -f ../testing_manifests/default_irq_smp_affinity0.yaml]':
    out=tuned.tuned.openshift.io/openshift-default-irq-smp-affinity created
    err=
    ret=<nil>
  STEP: ensuring the correct value of d was set in the last nibble of /proc/irq/default_smp_affinity @ 08/09/24 12:46:40.887
  STEP: applying the custom affinity profile ../testing_manifests/default_irq_smp_affinity1.yaml @ 08/09/24 12:46:41.219
  run command 'oc [apply -n openshift-cluster-node-tuning-operator -f ../testing_manifests/default_irq_smp_affinity1.yaml]':
    out=tuned.tuned.openshift.io/openshift-default-irq-smp-affinity configured
    err=Warning: resource tuneds/openshift-default-irq-smp-affinity is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically.
    ret=<nil>
  STEP: ensuring the correct value of 2 was set in the last nibble of /proc/irq/default_smp_affinity @ 08/09/24 12:46:43.029
  STEP: deleting the custom affinity profile ../testing_manifests/default_irq_smp_affinity0.yaml @ 08/09/24 12:46:43.358
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/default_irq_smp_affinity0.yaml]':
    out=tuned.tuned.openshift.io "openshift-default-irq-smp-affinity" deleted
    err=
    ret=<nil>
  STEP: ensuring the original value of f was set in the last nibble of /proc/irq/default_smp_affinity @ 08/09/24 12:46:43.512
  STEP: removing label tuned.openshift.io/default-irq-smp-affinity from node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:43.806
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/default-irq-smp-affinity-]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: cluster changes rollback @ 08/09/24 12:46:43.967
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/default-irq-smp-affinity-]':
    out=label "tuned.openshift.io/default-irq-smp-affinity" not found.
  node/ip-10-0-114-137.us-east-2.compute.internal not labeled
    err=
    ret=<nil>
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/default_irq_smp_affinity0.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "../testing_manifests/default_irq_smp_affinity0.yaml": tuneds.tuned.openshift.io "openshift-default-irq-smp-affinity" not found
    ret=exit status 1
• [4.504 seconds]
------------------------------
[basic][profile_status] Profile status conditions applied and not degraded Profile status conditions applied and not degraded
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/profile_status.go:29
• [0.017 seconds]
------------------------------
[basic][tuned_errors_and_recovery] Cause TuneD daemon errors and recover TuneD daemon errors and recovery Cause TuneD daemon errors on invalid profile load and recover after the profile deletion
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/tuned_errors_and_recovery.go:47
  STEP: getting a list of worker nodes @ 08/09/24 12:46:44.375
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:44.407
  STEP: labelling node ip-10-0-114-137.us-east-2.compute.internal with label tuned.openshift.io/cause-tuned-failure @ 08/09/24 12:46:44.424
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/cause-tuned-failure=]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: waiting for ClusterOperator/node-tuning condition Available reason AsExpected @ 08/09/24 12:46:44.653
  STEP: creating the custom profile ../testing_manifests/cause_tuned_failure.yaml @ 08/09/24 12:46:44.669
  run command 'oc [create -n openshift-cluster-node-tuning-operator -f ../testing_manifests/cause_tuned_failure.yaml]':
    out=tuned.tuned.openshift.io/openshift-cause-tuned-failure created
    err=
    ret=<nil>
  STEP: waiting for ClusterOperator/node-tuning condition Available reason ProfileDegraded @ 08/09/24 12:46:44.846
  STEP: creating the custom profile ../testing_manifests/dummy.yaml @ 08/09/24 12:46:49.865
  run command 'oc [create -n openshift-cluster-node-tuning-operator -f ../testing_manifests/dummy.yaml]':
    out=tuned.tuned.openshift.io/openshift-dummy created
    err=
    ret=<nil>
  STEP: waiting for ClusterOperator/node-tuning condition Available reason AsExpected @ 08/09/24 12:46:49.993
  STEP: deleting the custom profile ../testing_manifests/cause_tuned_failure.yaml @ 08/09/24 12:46:55.013
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/cause_tuned_failure.yaml]':
    out=tuned.tuned.openshift.io "openshift-cause-tuned-failure" deleted
    err=
    ret=<nil>
  STEP: deleting the custom profile ../testing_manifests/dummy.yaml @ 08/09/24 12:46:55.164
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/dummy.yaml]':
    out=tuned.tuned.openshift.io "openshift-dummy" deleted
    err=
    ret=<nil>
  STEP: waiting for ClusterOperator/node-tuning condition Available reason AsExpected @ 08/09/24 12:46:55.307
  STEP: removing label tuned.openshift.io/cause-tuned-failure from node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:55.323
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/cause-tuned-failure-]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: cluster changes rollback @ 08/09/24 12:46:55.483
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/cause-tuned-failure-]':
    out=label "tuned.openshift.io/cause-tuned-failure" not found.
  node/ip-10-0-114-137.us-east-2.compute.internal not labeled
    err=
    ret=<nil>
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/cause_tuned_failure.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "../testing_manifests/cause_tuned_failure.yaml": tuneds.tuned.openshift.io "openshift-cause-tuned-failure" not found
    ret=exit status 1
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/dummy.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "../testing_manifests/dummy.yaml": tuneds.tuned.openshift.io "openshift-dummy" not found
    ret=exit status 1
  run command 'oc [exec -n openshift-cluster-node-tuning-operator tuned-6pxwr -- rm -rf /etc/tuned/openshift-dummy]':
    out=
    err=
    ret=<nil>
• [11.823 seconds]
------------------------------
[basic][rollback] Node Tuning Operator settings rollback TuneD settings rollback net.ipv4.tcp_tw_reuse set
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/rollback.go:54
  STEP: getting a list of worker nodes @ 08/09/24 12:46:56.198
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:56.241
  STEP: waiting for TuneD profile openshift-node on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:46:56.256
  STEP: ensuring the default net.ipv4.tcp_tw_reuse value (2) is set in Pod tuned-6pxwr @ 08/09/24 12:46:56.271
  STEP: labelling Pod tuned-6pxwr with label tuned.openshift.io/ingress @ 08/09/24 12:46:56.604
  run command 'oc [label pod --overwrite -n openshift-cluster-node-tuning-operator tuned-6pxwr tuned.openshift.io/ingress=]':
    out=pod/tuned-6pxwr labeled
    err=
    ret=<nil>
  STEP: creating custom profile /go/src/github.com/openshift/cluster-node-tuning-operator/examples/ingress.yaml @ 08/09/24 12:46:56.755
  run command 'oc [create -n openshift-cluster-node-tuning-operator -f /go/src/github.com/openshift/cluster-node-tuning-operator/examples/ingress.yaml]':
    out=tuned.tuned.openshift.io/openshift-ingress created
    err=
    ret=<nil>
  STEP: ensuring the custom worker node profile was set @ 08/09/24 12:46:56.882
  STEP: deleting Pod tuned-6pxwr @ 08/09/24 12:47:02.234
  run command 'oc [delete -n openshift-cluster-node-tuning-operator pod tuned-6pxwr --wait]':
    out=pod "tuned-6pxwr" deleted
    err=
    ret=<nil>
  STEP: waiting for a new TuneD Pod to be ready on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:02.98
  STEP: ensuring the custom net.ipv4.tcp_tw_reuse value (1) is still set in Pod tuned-ptqtx @ 08/09/24 12:47:02.996
  STEP: deleting custom profile /go/src/github.com/openshift/cluster-node-tuning-operator/examples/ingress.yaml @ 08/09/24 12:47:03.319
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f /go/src/github.com/openshift/cluster-node-tuning-operator/examples/ingress.yaml]':
    out=tuned.tuned.openshift.io "openshift-ingress" deleted
    err=
    ret=<nil>
  STEP: waiting for TuneD profile openshift-node on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:03.464
  STEP: ensuring the default net.ipv4.tcp_tw_reuse value (2) is set in Pod tuned-ptqtx @ 08/09/24 12:47:03.824
  STEP: cluster changes rollback @ 08/09/24 12:47:04.132
  run command 'oc [label pod --overwrite -n openshift-cluster-node-tuning-operator tuned-ptqtx tuned.openshift.io/ingress-]':
    out=label "tuned.openshift.io/ingress" not found.
  pod/tuned-ptqtx not labeled
    err=
    ret=<nil>
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f /go/src/github.com/openshift/cluster-node-tuning-operator/examples/ingress.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "/go/src/github.com/openshift/cluster-node-tuning-operator/examples/ingress.yaml": tuneds.tuned.openshift.io "openshift-ingress" not found
    ret=exit status 1
• [8.214 seconds]
------------------------------
[basic][rollback] Node Tuning Operator settings rollback TuneD settings rollback without pod restart kernel.shmmni set
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/rollback.go:149
  STEP: getting a list of worker nodes @ 08/09/24 12:47:04.413
  STEP: waiting for TuneD profile openshift-node on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:04.455
  STEP: checking the pristine state on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:04.47
  STEP: verification command: [/usr/sbin/sysctl -n kernel.shmmni] @ 08/09/24 12:47:04.47
  STEP: verification expected output: "8192" @ 08/09/24 12:47:04.47
  STEP: verification current output: "4096" @ 08/09/24 12:47:04.794
  STEP: creating custom profile /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/testing_manifests/deferred/tuned-basic-00.yaml @ 08/09/24 12:47:04.794
  run command 'oc [create -n openshift-cluster-node-tuning-operator -f /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/testing_manifests/deferred/tuned-basic-00.yaml]':
    out=tuned.tuned.openshift.io/ocp-prof-deferred-basic-00 created
    err=
    ret=<nil>
  STEP: waiting for TuneD profile test-shmmni on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:04.927
  STEP: ensuring the custom worker node profile was set @ 08/09/24 12:47:09.943
  STEP: deleting custom profile /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/testing_manifests/deferred/tuned-basic-00.yaml @ 08/09/24 12:47:10.246
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/testing_manifests/deferred/tuned-basic-00.yaml]':
    out=tuned.tuned.openshift.io "ocp-prof-deferred-basic-00" deleted
    err=
    ret=<nil>
  STEP: waiting for TuneD profile openshift-node on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:10.393
  STEP: ensuring the pristine state is restored on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:15.409
  STEP: cluster changes rollback @ 08/09/24 12:47:15.731
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/testing_manifests/deferred/tuned-basic-00.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/testing_manifests/deferred/tuned-basic-00.yaml": tuneds.tuned.openshift.io "ocp-prof-deferred-basic-00" not found
    ret=exit status 1
• [11.436 seconds]
------------------------------
[basic][custom_pod_labels] Node Tuning Operator custom profile, Pod labels custom profile: Pod labels net.ipv4.tcp_tw_reuse set
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/custom_pod_labels.go:39
  STEP: getting a list of worker nodes @ 08/09/24 12:47:15.848
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:15.89
  STEP: waiting for TuneD profile openshift-node on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:15.907
  STEP: getting the current value of net.ipv4.tcp_tw_reuse in Pod tuned-ptqtx @ 08/09/24 12:47:15.921
  STEP: labelling Pod tuned-ptqtx with label tuned.openshift.io/ingress @ 08/09/24 12:47:16.244
  run command 'oc [label pod --overwrite -n openshift-cluster-node-tuning-operator tuned-ptqtx tuned.openshift.io/ingress=]':
    out=pod/tuned-ptqtx labeled
    err=
    ret=<nil>
  STEP: creating the custom ingress profile ../../../examples/ingress.yaml @ 08/09/24 12:47:16.413
  run command 'oc [create -n openshift-cluster-node-tuning-operator -f ../../../examples/ingress.yaml]':
    out=tuned.tuned.openshift.io/openshift-ingress created
    err=
    ret=<nil>
  STEP: ensuring the custom worker node profile was set @ 08/09/24 12:47:16.531
  STEP: removing label tuned.openshift.io/ingress from Pod tuned-ptqtx @ 08/09/24 12:47:21.869
  run command 'oc [label pod --overwrite -n openshift-cluster-node-tuning-operator tuned-ptqtx tuned.openshift.io/ingress-]':
    out=pod/tuned-ptqtx labeled
    err=
    ret=<nil>
  STEP: ensuring the original net.ipv4.tcp_tw_reuse value (2) is set in Pod tuned-ptqtx @ 08/09/24 12:47:22.021
  STEP: deleting the custom ingress profile ../../../examples/ingress.yaml @ 08/09/24 12:47:37.354
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../../../examples/ingress.yaml]':
    out=tuned.tuned.openshift.io "openshift-ingress" deleted
    err=
    ret=<nil>
  STEP: cluster changes rollback @ 08/09/24 12:47:37.493
  run command 'oc [label pod --overwrite -n openshift-cluster-node-tuning-operator tuned-ptqtx tuned.openshift.io/ingress-]':
    out=label "tuned.openshift.io/ingress" not found.
  pod/tuned-ptqtx not labeled
    err=
    ret=<nil>
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../../../examples/ingress.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "../../../examples/ingress.yaml": tuneds.tuned.openshift.io "openshift-ingress" not found
    ret=exit status 1
• [21.925 seconds]
------------------------------
[basic][modules] Node Tuning Operator load kernel module module loading modules: md4 loaded
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/modules.go:39
  STEP: getting a list of worker nodes @ 08/09/24 12:47:37.773
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:37.81
  STEP: labelling node ip-10-0-114-137.us-east-2.compute.internal with label tuned.openshift.io/module-load @ 08/09/24 12:47:37.824
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/module-load=]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: trying to remove the md4 module if loaded @ 08/09/24 12:47:37.984
  STEP: ensuring the md4 module is not loaded @ 08/09/24 12:47:38.287
  STEP: creating profile ../testing_manifests/tuned_modules_load.yaml @ 08/09/24 12:47:38.616
  run command 'oc [create -n openshift-cluster-node-tuning-operator -f ../testing_manifests/tuned_modules_load.yaml]':
    out=tuned.tuned.openshift.io/openshift-module-load created
    err=
    ret=<nil>
  STEP: ensuring the md4 module is loaded @ 08/09/24 12:47:38.738
  STEP: deleting profile ../testing_manifests/tuned_modules_load.yaml @ 08/09/24 12:47:39.063
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/tuned_modules_load.yaml]':
    out=tuned.tuned.openshift.io "openshift-module-load" deleted
    err=
    ret=<nil>
  STEP: removing label tuned.openshift.io/module-load from node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:39.2
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/module-load-]':
    out=node/ip-10-0-114-137.us-east-2.compute.internal labeled
    err=
    ret=<nil>
  STEP: cluster changes rollback @ 08/09/24 12:47:39.358
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/module-load-]':
    out=label "tuned.openshift.io/module-load" not found.
  node/ip-10-0-114-137.us-east-2.compute.internal not labeled
    err=
    ret=<nil>
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/tuned_modules_load.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "../testing_manifests/tuned_modules_load.yaml": tuneds.tuned.openshift.io "openshift-module-load" not found
    ret=exit status 1
• [1.870 seconds]
------------------------------
[basic][sysctl_d_override] Node Tuning Operator /etc/sysctl.d/*.conf override sysctl.d override net.ipv4.neigh.default.gc_thresh1 set
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/sysctl_d_override.go:48
  STEP: getting a list of worker nodes @ 08/09/24 12:47:39.643
  STEP: getting a TuneD Pod running on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:39.68
  STEP: waiting for TuneD profile openshift-node on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:39.694
  STEP: getting the current value of net.ipv4.neigh.default.gc_thresh1 in Pod tuned-ptqtx @ 08/09/24 12:47:39.706
  STEP: writing /host/etc/sysctl.d/zzz.conf override file on the host with net.ipv4.neigh.default.gc_thresh1=256 @ 08/09/24 12:47:40.022
  run command 'oc [exec -n openshift-cluster-node-tuning-operator tuned-ptqtx -- sh -c echo net.ipv4.neigh.default.gc_thresh1=256 > /host/etc/sysctl.d/zzz.conf; sync /host/etc/sysctl.d/zzz.conf]':
    out=
    err=
    ret=<nil>
  run command 'oc [rsh -n openshift-cluster-node-tuning-operator tuned-ptqtx cat /host/etc/sysctl.d/zzz.conf]':
    out=net.ipv4.neigh.default.gc_thresh1=256
    err=
    ret=<nil>
  STEP: deleting Pod tuned-ptqtx @ 08/09/24 12:47:40.663
  run command 'oc [delete -n openshift-cluster-node-tuning-operator pod tuned-ptqtx --wait]':
    out=pod "tuned-ptqtx" deleted
    err=
    ret=<nil>
  STEP: waiting for a new TuneD Pod to be ready on node ip-10-0-114-137.us-east-2.compute.internal @ 08/09/24 12:47:41.79
  STEP: ensuring new net.ipv4.neigh.default.gc_thresh1 value (256) is set in Pod tuned-pmxvn @ 08/09/24 12:47:41.805
  [FAILED] in [It] - /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/sysctl_d_override.go:98 @ 08/09/24 12:52:42.141
  STEP: cluster changes rollback @ 08/09/24 12:52:42.141
  run command 'oc [label node --overwrite ip-10-0-114-137.us-east-2.compute.internal tuned.openshift.io/sysctl-override-]':
    out=label "tuned.openshift.io/sysctl-override" not found.
  node/ip-10-0-114-137.us-east-2.compute.internal not labeled
    err=
    ret=<nil>
  run command 'oc [exec -n openshift-cluster-node-tuning-operator tuned-pmxvn -- rm /host/etc/sysctl.d/zzz.conf]':
    out=
    err=
    ret=<nil>
  run command 'oc [delete -n openshift-cluster-node-tuning-operator -f ../testing_manifests/sysctl_override.yaml]':
    out=
    err=Error from server (NotFound): error when deleting "../testing_manifests/sysctl_override.yaml": tuneds.tuned.openshift.io "openshift-profile" not found
    ret=exit status 1
• [FAILED] [303.133 seconds]
[basic][sysctl_d_override] Node Tuning Operator /etc/sysctl.d/*.conf override sysctl.d override [It] net.ipv4.neigh.default.gc_thresh1 set
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/sysctl_d_override.go:48

  [FAILED] Unexpected error:
      <*errors.errorString | 0xc00005e6e0>: 
      sysctl net.ipv4.neigh.default.gc_thresh1=8192 in Pod tuned-pmxvn, expected 256: command [sysctl -n net.ipv4.neigh.default.gc_thresh1] outputs 8192 (leading/trailing whitespace trimmed) in Pod tuned-pmxvn, expected 256 (waited 5m0.335624446s): <nil>
      {
          s: "sysctl net.ipv4.neigh.default.gc_thresh1=8192 in Pod tuned-pmxvn, expected 256: command [sysctl -n net.ipv4.neigh.default.gc_thresh1] outputs 8192 (leading/trailing whitespace trimmed) in Pod tuned-pmxvn, expected 256 (waited 5m0.335624446s): <nil>",
      }
  occurred
  In [It] at: /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/sysctl_d_override.go:98 @ 08/09/24 12:52:42.141
------------------------------

Summarizing 1 Failure:
  [FAIL] [basic][sysctl_d_override] Node Tuning Operator /etc/sysctl.d/*.conf override sysctl.d override [It] net.ipv4.neigh.default.gc_thresh1 set
  /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/sysctl_d_override.go:98

Ran 15 of 15 Specs in 427.588 seconds
FAIL! -- 14 Passed | 1 Failed | 0 Pending | 0 Skipped
--- FAIL: TestNodeTuningOperator (427.59s)
FAIL
FAIL	github.com/openshift/cluster-node-tuning-operator/test/e2e/basic	427.606s
FAIL
make: *** [Makefile:103: test-e2e] Error 1
{"component":"entrypoint","error":"wrapped process failed: exit status 2","file":"sigs.k8s.io/prow/pkg/entrypoint/run.go:84","func":"sigs.k8s.io/prow/pkg/entrypoint.Options.internalRun","level":"error","msg":"Error executing test process","severity":"error","time":"2024-08-09T12:52:42Z"}
error: failed to execute wrapped command: exit status 2 
INFO[2024-08-09T12:52:43Z] Step e2e-aws-operator-test failed after 7m52s. 
INFO[2024-08-09T12:52:43Z] Step phase test failed after 7m52s.          
INFO[2024-08-09T12:52:43Z] Running multi-stage phase post               
INFO[2024-08-09T12:52:43Z] Running step e2e-aws-operator-gather-aws-console. 
INFO[2024-08-09T12:52:59Z] Step e2e-aws-operator-gather-aws-console succeeded after 16s. 
INFO[2024-08-09T12:52:59Z] Running step e2e-aws-operator-gather-must-gather. 
INFO[2024-08-09T12:54:30Z] Step e2e-aws-operator-gather-must-gather succeeded after 1m30s. 
INFO[2024-08-09T12:54:30Z] Running step e2e-aws-operator-gather-extra.  
INFO[2024-08-09T12:56:02Z] Step e2e-aws-operator-gather-extra succeeded after 1m32s. 
INFO[2024-08-09T12:56:02Z] Running step e2e-aws-operator-gather-audit-logs. 
INFO[2024-08-09T12:56:22Z] Step e2e-aws-operator-gather-audit-logs succeeded after 19s. 
INFO[2024-08-09T12:56:22Z] Running step e2e-aws-operator-ipi-deprovision-deprovision. 
INFO[2024-08-09T13:02:05Z] Step e2e-aws-operator-ipi-deprovision-deprovision succeeded after 5m42s. 
INFO[2024-08-09T13:02:05Z] Step phase post succeeded after 9m22s.       
INFO[2024-08-09T13:02:05Z] Releasing ip pool leases for test e2e-aws-operator 
INFO[2024-08-09T13:02:05Z] Releasing leases for test e2e-aws-operator   
INFO[2024-08-09T13:02:06Z] Ran for 1h0m1s                               
ERRO[2024-08-09T13:02:06Z] Some steps failed:                           
ERRO[2024-08-09T13:02:06Z] 
  * could not run steps: step e2e-aws-operator failed: "e2e-aws-operator" test steps failed: "e2e-aws-operator" pod "e2e-aws-operator-test" failed: could not watch pod: the pod ci-op-n3l6f4pk/e2e-aws-operator-test failed after 7m53s (failed containers: test): ContainerFailed one or more containers exited

Container test exited with code 2, reason Error
---
nifests/sysctl_override.yaml": tuneds.tuned.openshift.io "openshift-profile" not found
    ret=exit status 1
• [FAILED] [303.133 seconds]
[basic][sysctl_d_override] Node Tuning Operator /etc/sysctl.d/*.conf override sysctl.d override [It] net.ipv4.neigh.default.gc_thresh1 set
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/sysctl_d_override.go:48

  [FAILED] Unexpected error:
      <*errors.errorString | 0xc00005e6e0>: 
      sysctl net.ipv4.neigh.default.gc_thresh1=8192 in Pod tuned-pmxvn, expected 256: command [sysctl -n net.ipv4.neigh.default.gc_thresh1] outputs 8192 (leading/trailing whitespace trimmed) in Pod tuned-pmxvn, expected 256 (waited 5m0.335624446s): <nil>
      {
          s: "sysctl net.ipv4.neigh.default.gc_thresh1=8192 in Pod tuned-pmxvn, expected 256: command [sysctl -n net.ipv4.neigh.default.gc_thresh1] outputs 8192 (leading/trailing whitespace trimmed) in Pod tuned-pmxvn, expected 256 (waited 5m0.335624446s): <nil>",
      }
  occurred
  In [It] at: /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/sysctl_d_override.go:98 @ 08/09/24 12:52:42.141
------------------------------

Summarizing 1 Failure:
  [FAIL] [basic][sysctl_d_override] Node Tuning Operator /etc/sysctl.d/*.conf override sysctl.d override [It] net.ipv4.neigh.default.gc_thresh1 set
  /go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/basic/sysctl_d_override.go:98

Ran 15 of 15 Specs in 427.588 seconds
FAIL! -- 14 Passed | 1 Failed | 0 Pending | 0 Skipped
--- FAIL: TestNodeTuningOperator (427.59s)
FAIL
FAIL	github.com/openshift/cluster-node-tuning-operator/test/e2e/basic	427.606s
FAIL
make: *** [Makefile:103: test-e2e] Error 1
{"component":"entrypoint","error":"wrapped process failed: exit status 2","file":"sigs.k8s.io/prow/pkg/entrypoint/run.go:84","func":"sigs.k8s.io/prow/pkg/entrypoint.Options.internalRun","level":"error","msg":"Error executing test process","severity":"error","time":"2024-08-09T12:52:42Z"}
error: failed to execute wrapped command: exit status 2
---
Link to step on registry info site: https://steps.ci.openshift.org/reference/test
Link to job on registry info site: https://steps.ci.openshift.org/job?org=openshift&repo=cluster-node-tuning-operator&branch=master&test=e2e-aws-operator 
INFO[2024-08-09T13:02:06Z] Reporting job state 'failed' with reason 'executing_graph:step_failed:utilizing_lease:executing_test:utilizing_ip_pool:executing_test:executing_multi_stage_test'