=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-809747 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-809747 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0: exit status 102 (6m14.24579557s)
-- stdout --
* [old-k8s-version-809747] minikube v1.34.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=19679
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19679-440039/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19679-440039/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.31.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.1
* Using the docker driver based on existing profile
* Starting "old-k8s-version-809747" primary control-plane node in "old-k8s-version-809747" cluster
* Pulling base image v0.0.45-1726589491-19662 ...
* Restarting existing docker container for "old-k8s-version-809747" ...
* Preparing Kubernetes v1.20.0 on containerd 1.7.22 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image fake.domain/registry.k8s.io/echoserver:1.4
- Using image registry.k8s.io/echoserver:1.4
- Using image docker.io/kubernetesui/dashboard:v2.7.0
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-809747 addons enable metrics-server
* Enabled addons: metrics-server, storage-provisioner, dashboard, default-storageclass
-- /stdout --
** stderr **
I0920 19:00:24.876450 653977 out.go:345] Setting OutFile to fd 1 ...
I0920 19:00:24.876882 653977 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0920 19:00:24.876893 653977 out.go:358] Setting ErrFile to fd 2...
I0920 19:00:24.876899 653977 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0920 19:00:24.877160 653977 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19679-440039/.minikube/bin
I0920 19:00:24.877559 653977 out.go:352] Setting JSON to false
I0920 19:00:24.878520 653977 start.go:129] hostinfo: {"hostname":"ip-172-31-30-239","uptime":9776,"bootTime":1726849049,"procs":200,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1070-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"92f46a7d-c249-4c12-924a-77f64874c910"}
I0920 19:00:24.878594 653977 start.go:139] virtualization:
I0920 19:00:24.882039 653977 out.go:177] * [old-k8s-version-809747] minikube v1.34.0 on Ubuntu 20.04 (arm64)
I0920 19:00:24.885159 653977 out.go:177] - MINIKUBE_LOCATION=19679
I0920 19:00:24.885243 653977 notify.go:220] Checking for updates...
I0920 19:00:24.891460 653977 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0920 19:00:24.893853 653977 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19679-440039/kubeconfig
I0920 19:00:24.896127 653977 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19679-440039/.minikube
I0920 19:00:24.898864 653977 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0920 19:00:24.901732 653977 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0920 19:00:24.904937 653977 config.go:182] Loaded profile config "old-k8s-version-809747": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0920 19:00:24.907676 653977 out.go:177] * Kubernetes 1.31.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.1
I0920 19:00:24.910092 653977 driver.go:394] Setting default libvirt URI to qemu:///system
I0920 19:00:24.960859 653977 docker.go:123] docker version: linux-27.3.1:Docker Engine - Community
I0920 19:00:24.961000 653977 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0920 19:00:25.063260 653977 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:55 OomKillDisable:true NGoroutines:68 SystemTime:2024-09-20 19:00:25.050134038 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0920 19:00:25.063377 653977 docker.go:318] overlay module found
I0920 19:00:25.066293 653977 out.go:177] * Using the docker driver based on existing profile
I0920 19:00:25.068491 653977 start.go:297] selected driver: docker
I0920 19:00:25.068521 653977 start.go:901] validating driver "docker" against &{Name:old-k8s-version-809747 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-809747 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:fa
lse MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0920 19:00:25.068634 653977 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0920 19:00:25.069283 653977 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0920 19:00:25.162128 653977 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:55 OomKillDisable:true NGoroutines:68 SystemTime:2024-09-20 19:00:25.150020358 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0920 19:00:25.162671 653977 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0920 19:00:25.162706 653977 cni.go:84] Creating CNI manager for ""
I0920 19:00:25.162750 653977 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0920 19:00:25.162795 653977 start.go:340] cluster config:
{Name:old-k8s-version-809747 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-809747 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:do
cker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0920 19:00:25.165133 653977 out.go:177] * Starting "old-k8s-version-809747" primary control-plane node in "old-k8s-version-809747" cluster
I0920 19:00:25.167310 653977 cache.go:121] Beginning downloading kic base image for docker with containerd
I0920 19:00:25.169325 653977 out.go:177] * Pulling base image v0.0.45-1726589491-19662 ...
I0920 19:00:25.171366 653977 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0920 19:00:25.171494 653977 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19679-440039/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4
I0920 19:00:25.171508 653977 cache.go:56] Caching tarball of preloaded images
I0920 19:00:25.171666 653977 preload.go:172] Found /home/jenkins/minikube-integration/19679-440039/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0920 19:00:25.171676 653977 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on containerd
I0920 19:00:25.171841 653977 profile.go:143] Saving config to /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/old-k8s-version-809747/config.json ...
I0920 19:00:25.172205 653977 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 in local docker daemon
W0920 19:00:25.216073 653977 image.go:95] image gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 is of wrong architecture
I0920 19:00:25.216093 653977 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 to local cache
I0920 19:00:25.216180 653977 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 in local cache directory
I0920 19:00:25.216204 653977 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 in local cache directory, skipping pull
I0920 19:00:25.216209 653977 image.go:135] gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 exists in cache, skipping pull
I0920 19:00:25.216217 653977 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 as a tarball
I0920 19:00:25.216222 653977 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 from local cache
I0920 19:00:25.343976 653977 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 from cached tarball
I0920 19:00:25.344022 653977 cache.go:194] Successfully downloaded all kic artifacts
I0920 19:00:25.344052 653977 start.go:360] acquireMachinesLock for old-k8s-version-809747: {Name:mk4e6a11f9e9fe3a1d94ce483f3b7f94de9083d6 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0920 19:00:25.344134 653977 start.go:364] duration metric: took 62.153µs to acquireMachinesLock for "old-k8s-version-809747"
I0920 19:00:25.344155 653977 start.go:96] Skipping create...Using existing machine configuration
I0920 19:00:25.344161 653977 fix.go:54] fixHost starting:
I0920 19:00:25.344464 653977 cli_runner.go:164] Run: docker container inspect old-k8s-version-809747 --format={{.State.Status}}
I0920 19:00:25.378782 653977 fix.go:112] recreateIfNeeded on old-k8s-version-809747: state=Stopped err=<nil>
W0920 19:00:25.378810 653977 fix.go:138] unexpected machine state, will restart: <nil>
I0920 19:00:25.381392 653977 out.go:177] * Restarting existing docker container for "old-k8s-version-809747" ...
I0920 19:00:25.383416 653977 cli_runner.go:164] Run: docker start old-k8s-version-809747
I0920 19:00:25.866897 653977 cli_runner.go:164] Run: docker container inspect old-k8s-version-809747 --format={{.State.Status}}
I0920 19:00:25.891785 653977 kic.go:430] container "old-k8s-version-809747" state is running.
I0920 19:00:25.892195 653977 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-809747
I0920 19:00:25.915015 653977 profile.go:143] Saving config to /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/old-k8s-version-809747/config.json ...
I0920 19:00:25.915418 653977 machine.go:93] provisionDockerMachine start ...
I0920 19:00:25.915594 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:25.940635 653977 main.go:141] libmachine: Using SSH client type: native
I0920 19:00:25.940915 653977 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33063 <nil> <nil>}
I0920 19:00:25.940932 653977 main.go:141] libmachine: About to run SSH command:
hostname
I0920 19:00:25.941478 653977 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:52218->127.0.0.1:33063: read: connection reset by peer
I0920 19:00:29.102531 653977 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-809747
I0920 19:00:29.102561 653977 ubuntu.go:169] provisioning hostname "old-k8s-version-809747"
I0920 19:00:29.102653 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:29.137178 653977 main.go:141] libmachine: Using SSH client type: native
I0920 19:00:29.137429 653977 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33063 <nil> <nil>}
I0920 19:00:29.137442 653977 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-809747 && echo "old-k8s-version-809747" | sudo tee /etc/hostname
I0920 19:00:29.324628 653977 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-809747
I0920 19:00:29.324745 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:29.356196 653977 main.go:141] libmachine: Using SSH client type: native
I0920 19:00:29.356456 653977 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33063 <nil> <nil>}
I0920 19:00:29.356482 653977 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-809747' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-809747/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-809747' | sudo tee -a /etc/hosts;
fi
fi
I0920 19:00:29.510884 653977 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0920 19:00:29.510913 653977 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19679-440039/.minikube CaCertPath:/home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19679-440039/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19679-440039/.minikube}
I0920 19:00:29.510939 653977 ubuntu.go:177] setting up certificates
I0920 19:00:29.510949 653977 provision.go:84] configureAuth start
I0920 19:00:29.511021 653977 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-809747
I0920 19:00:29.548222 653977 provision.go:143] copyHostCerts
I0920 19:00:29.548291 653977 exec_runner.go:144] found /home/jenkins/minikube-integration/19679-440039/.minikube/key.pem, removing ...
I0920 19:00:29.548300 653977 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19679-440039/.minikube/key.pem
I0920 19:00:29.548378 653977 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19679-440039/.minikube/key.pem (1675 bytes)
I0920 19:00:29.548535 653977 exec_runner.go:144] found /home/jenkins/minikube-integration/19679-440039/.minikube/ca.pem, removing ...
I0920 19:00:29.548542 653977 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19679-440039/.minikube/ca.pem
I0920 19:00:29.548573 653977 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19679-440039/.minikube/ca.pem (1082 bytes)
I0920 19:00:29.548638 653977 exec_runner.go:144] found /home/jenkins/minikube-integration/19679-440039/.minikube/cert.pem, removing ...
I0920 19:00:29.548643 653977 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19679-440039/.minikube/cert.pem
I0920 19:00:29.548672 653977 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19679-440039/.minikube/cert.pem (1123 bytes)
I0920 19:00:29.548725 653977 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19679-440039/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-809747 san=[127.0.0.1 192.168.76.2 localhost minikube old-k8s-version-809747]
I0920 19:00:29.950083 653977 provision.go:177] copyRemoteCerts
I0920 19:00:29.950157 653977 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0920 19:00:29.950215 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:29.967190 653977 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/old-k8s-version-809747/id_rsa Username:docker}
I0920 19:00:30.078064 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0920 19:00:30.126063 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0920 19:00:30.179958 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0920 19:00:30.222932 653977 provision.go:87] duration metric: took 711.963992ms to configureAuth
I0920 19:00:30.222966 653977 ubuntu.go:193] setting minikube options for container-runtime
I0920 19:00:30.223203 653977 config.go:182] Loaded profile config "old-k8s-version-809747": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0920 19:00:30.223219 653977 machine.go:96] duration metric: took 4.307784257s to provisionDockerMachine
I0920 19:00:30.223229 653977 start.go:293] postStartSetup for "old-k8s-version-809747" (driver="docker")
I0920 19:00:30.223245 653977 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0920 19:00:30.223316 653977 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0920 19:00:30.223364 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:30.256887 653977 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/old-k8s-version-809747/id_rsa Username:docker}
I0920 19:00:30.375920 653977 ssh_runner.go:195] Run: cat /etc/os-release
I0920 19:00:30.379715 653977 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0920 19:00:30.379757 653977 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0920 19:00:30.379768 653977 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0920 19:00:30.379784 653977 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I0920 19:00:30.379798 653977 filesync.go:126] Scanning /home/jenkins/minikube-integration/19679-440039/.minikube/addons for local assets ...
I0920 19:00:30.379859 653977 filesync.go:126] Scanning /home/jenkins/minikube-integration/19679-440039/.minikube/files for local assets ...
I0920 19:00:30.379954 653977 filesync.go:149] local asset: /home/jenkins/minikube-integration/19679-440039/.minikube/files/etc/ssl/certs/4467832.pem -> 4467832.pem in /etc/ssl/certs
I0920 19:00:30.380065 653977 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0920 19:00:30.394626 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/files/etc/ssl/certs/4467832.pem --> /etc/ssl/certs/4467832.pem (1708 bytes)
I0920 19:00:30.430948 653977 start.go:296] duration metric: took 207.699465ms for postStartSetup
I0920 19:00:30.431050 653977 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0920 19:00:30.431091 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:30.459854 653977 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/old-k8s-version-809747/id_rsa Username:docker}
I0920 19:00:30.567781 653977 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0920 19:00:30.572858 653977 fix.go:56] duration metric: took 5.228688785s for fixHost
I0920 19:00:30.572887 653977 start.go:83] releasing machines lock for "old-k8s-version-809747", held for 5.228743011s
I0920 19:00:30.572966 653977 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-809747
I0920 19:00:30.604073 653977 ssh_runner.go:195] Run: cat /version.json
I0920 19:00:30.604127 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:30.604137 653977 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0920 19:00:30.604199 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:30.642519 653977 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/old-k8s-version-809747/id_rsa Username:docker}
I0920 19:00:30.643213 653977 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/old-k8s-version-809747/id_rsa Username:docker}
I0920 19:00:30.754050 653977 ssh_runner.go:195] Run: systemctl --version
I0920 19:00:30.925755 653977 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0920 19:00:30.931937 653977 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0920 19:00:30.973150 653977 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0920 19:00:30.973231 653977 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0920 19:00:30.988739 653977 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0920 19:00:30.988769 653977 start.go:495] detecting cgroup driver to use...
I0920 19:00:30.988817 653977 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0920 19:00:30.988887 653977 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0920 19:00:31.010109 653977 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0920 19:00:31.033753 653977 docker.go:217] disabling cri-docker service (if available) ...
I0920 19:00:31.033872 653977 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0920 19:00:31.049044 653977 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0920 19:00:31.063358 653977 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0920 19:00:31.228694 653977 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0920 19:00:31.391307 653977 docker.go:233] disabling docker service ...
I0920 19:00:31.391434 653977 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0920 19:00:31.413123 653977 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0920 19:00:31.426807 653977 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0920 19:00:31.584301 653977 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0920 19:00:31.734988 653977 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0920 19:00:31.756684 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0920 19:00:31.784859 653977 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0920 19:00:31.798603 653977 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0920 19:00:31.817982 653977 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0920 19:00:31.818104 653977 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0920 19:00:31.833086 653977 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0920 19:00:31.848398 653977 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0920 19:00:31.859757 653977 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0920 19:00:31.874050 653977 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0920 19:00:31.887927 653977 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0920 19:00:31.904131 653977 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0920 19:00:31.916170 653977 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0920 19:00:31.931978 653977 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0920 19:00:32.076726 653977 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0920 19:00:32.361540 653977 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0920 19:00:32.361687 653977 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0920 19:00:32.369397 653977 start.go:563] Will wait 60s for crictl version
I0920 19:00:32.369514 653977 ssh_runner.go:195] Run: which crictl
I0920 19:00:32.382829 653977 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0920 19:00:32.486071 653977 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.22
RuntimeApiVersion: v1
I0920 19:00:32.486211 653977 ssh_runner.go:195] Run: containerd --version
I0920 19:00:32.531190 653977 ssh_runner.go:195] Run: containerd --version
I0920 19:00:32.585342 653977 out.go:177] * Preparing Kubernetes v1.20.0 on containerd 1.7.22 ...
I0920 19:00:32.587700 653977 cli_runner.go:164] Run: docker network inspect old-k8s-version-809747 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0920 19:00:32.618112 653977 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0920 19:00:32.626780 653977 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0920 19:00:32.641346 653977 kubeadm.go:883] updating cluster {Name:old-k8s-version-809747 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-809747 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/
home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0920 19:00:32.641474 653977 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0920 19:00:32.641541 653977 ssh_runner.go:195] Run: sudo crictl images --output json
I0920 19:00:32.712063 653977 containerd.go:627] all images are preloaded for containerd runtime.
I0920 19:00:32.712091 653977 containerd.go:534] Images already preloaded, skipping extraction
I0920 19:00:32.712157 653977 ssh_runner.go:195] Run: sudo crictl images --output json
I0920 19:00:32.792461 653977 containerd.go:627] all images are preloaded for containerd runtime.
I0920 19:00:32.792491 653977 cache_images.go:84] Images are preloaded, skipping loading
I0920 19:00:32.792500 653977 kubeadm.go:934] updating node { 192.168.76.2 8443 v1.20.0 containerd true true} ...
I0920 19:00:32.792654 653977 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=old-k8s-version-809747 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-809747 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0920 19:00:32.792729 653977 ssh_runner.go:195] Run: sudo crictl info
I0920 19:00:32.862290 653977 cni.go:84] Creating CNI manager for ""
I0920 19:00:32.862337 653977 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0920 19:00:32.862355 653977 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0920 19:00:32.862375 653977 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-809747 NodeName:old-k8s-version-809747 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0920 19:00:32.862520 653977 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "old-k8s-version-809747"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0920 19:00:32.862597 653977 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0920 19:00:32.874825 653977 binaries.go:44] Found k8s binaries, skipping transfer
I0920 19:00:32.874896 653977 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0920 19:00:32.894657 653977 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (442 bytes)
I0920 19:00:32.927900 653977 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0920 19:00:32.959991 653977 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2125 bytes)
I0920 19:00:32.995934 653977 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0920 19:00:33.002869 653977 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0920 19:00:33.020320 653977 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0920 19:00:33.182353 653977 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0920 19:00:33.200166 653977 certs.go:68] Setting up /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/old-k8s-version-809747 for IP: 192.168.76.2
I0920 19:00:33.200190 653977 certs.go:194] generating shared ca certs ...
I0920 19:00:33.200206 653977 certs.go:226] acquiring lock for ca certs: {Name:mk3d7fcf9ade00248d7372a8cec4403eeffc64da Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:00:33.200354 653977 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19679-440039/.minikube/ca.key
I0920 19:00:33.200404 653977 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19679-440039/.minikube/proxy-client-ca.key
I0920 19:00:33.200414 653977 certs.go:256] generating profile certs ...
I0920 19:00:33.200497 653977 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/old-k8s-version-809747/client.key
I0920 19:00:33.200574 653977 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/old-k8s-version-809747/apiserver.key.641197a2
I0920 19:00:33.200623 653977 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/old-k8s-version-809747/proxy-client.key
I0920 19:00:33.200735 653977 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/446783.pem (1338 bytes)
W0920 19:00:33.200769 653977 certs.go:480] ignoring /home/jenkins/minikube-integration/19679-440039/.minikube/certs/446783_empty.pem, impossibly tiny 0 bytes
I0920 19:00:33.200780 653977 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca-key.pem (1679 bytes)
I0920 19:00:33.200807 653977 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca.pem (1082 bytes)
I0920 19:00:33.200833 653977 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/cert.pem (1123 bytes)
I0920 19:00:33.200859 653977 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/key.pem (1675 bytes)
I0920 19:00:33.200898 653977 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/files/etc/ssl/certs/4467832.pem (1708 bytes)
I0920 19:00:33.201591 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0920 19:00:33.292630 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0920 19:00:33.371796 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0920 19:00:33.428964 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0920 19:00:33.460463 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/old-k8s-version-809747/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0920 19:00:33.487940 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/old-k8s-version-809747/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0920 19:00:33.515402 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/old-k8s-version-809747/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0920 19:00:33.542432 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/old-k8s-version-809747/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0920 19:00:33.569270 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/certs/446783.pem --> /usr/share/ca-certificates/446783.pem (1338 bytes)
I0920 19:00:33.595258 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/files/etc/ssl/certs/4467832.pem --> /usr/share/ca-certificates/4467832.pem (1708 bytes)
I0920 19:00:33.629562 653977 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0920 19:00:33.663984 653977 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0920 19:00:33.684244 653977 ssh_runner.go:195] Run: openssl version
I0920 19:00:33.690656 653977 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/446783.pem && ln -fs /usr/share/ca-certificates/446783.pem /etc/ssl/certs/446783.pem"
I0920 19:00:33.701816 653977 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/446783.pem
I0920 19:00:33.705834 653977 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 20 18:20 /usr/share/ca-certificates/446783.pem
I0920 19:00:33.705979 653977 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/446783.pem
I0920 19:00:33.713430 653977 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/446783.pem /etc/ssl/certs/51391683.0"
I0920 19:00:33.724387 653977 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4467832.pem && ln -fs /usr/share/ca-certificates/4467832.pem /etc/ssl/certs/4467832.pem"
I0920 19:00:33.734937 653977 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/4467832.pem
I0920 19:00:33.739058 653977 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 20 18:20 /usr/share/ca-certificates/4467832.pem
I0920 19:00:33.739203 653977 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/4467832.pem
I0920 19:00:33.746764 653977 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/4467832.pem /etc/ssl/certs/3ec20f2e.0"
I0920 19:00:33.756872 653977 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0920 19:00:33.767514 653977 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0920 19:00:33.771735 653977 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 20 18:10 /usr/share/ca-certificates/minikubeCA.pem
I0920 19:00:33.771875 653977 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0920 19:00:33.779671 653977 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0920 19:00:33.790276 653977 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0920 19:00:33.794565 653977 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0920 19:00:33.806774 653977 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0920 19:00:33.831549 653977 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0920 19:00:33.842801 653977 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0920 19:00:33.851435 653977 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0920 19:00:33.859535 653977 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0920 19:00:33.867065 653977 kubeadm.go:392] StartCluster: {Name:old-k8s-version-809747 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-809747 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/hom
e/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0920 19:00:33.867228 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0920 19:00:33.867324 653977 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0920 19:00:33.925047 653977 cri.go:89] found id: "4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada"
I0920 19:00:33.925077 653977 cri.go:89] found id: "6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c"
I0920 19:00:33.925082 653977 cri.go:89] found id: "19f7587837edd2296001abf813b8f85bc66215e3208e13e9ab0f2b81524e8a9f"
I0920 19:00:33.925090 653977 cri.go:89] found id: "899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d"
I0920 19:00:33.925093 653977 cri.go:89] found id: "ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5"
I0920 19:00:33.925097 653977 cri.go:89] found id: "dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32"
I0920 19:00:33.925100 653977 cri.go:89] found id: "7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043"
I0920 19:00:33.925103 653977 cri.go:89] found id: "fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4"
I0920 19:00:33.925106 653977 cri.go:89] found id: ""
I0920 19:00:33.925161 653977 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0920 19:00:33.938762 653977 cri.go:116] JSON = null
W0920 19:00:33.938811 653977 kubeadm.go:399] unpause failed: list paused: list returned 0 containers, but ps returned 8
I0920 19:00:33.938878 653977 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0920 19:00:33.949230 653977 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0920 19:00:33.949247 653977 kubeadm.go:593] restartPrimaryControlPlane start ...
I0920 19:00:33.949300 653977 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0920 19:00:33.958983 653977 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0920 19:00:33.959423 653977 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-809747" does not appear in /home/jenkins/minikube-integration/19679-440039/kubeconfig
I0920 19:00:33.959521 653977 kubeconfig.go:62] /home/jenkins/minikube-integration/19679-440039/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-809747" cluster setting kubeconfig missing "old-k8s-version-809747" context setting]
I0920 19:00:33.959780 653977 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19679-440039/kubeconfig: {Name:mkc0c275236e567d398d3ba786de8188e8f878bb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:00:33.960997 653977 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0920 19:00:33.976853 653977 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.76.2
I0920 19:00:33.976938 653977 kubeadm.go:597] duration metric: took 27.684298ms to restartPrimaryControlPlane
I0920 19:00:33.976962 653977 kubeadm.go:394] duration metric: took 109.906388ms to StartCluster
I0920 19:00:33.977011 653977 settings.go:142] acquiring lock: {Name:mk1135c1a1ce95063626d6fac03fabf56993cb73 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:00:33.977116 653977 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19679-440039/kubeconfig
I0920 19:00:33.977850 653977 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19679-440039/kubeconfig: {Name:mkc0c275236e567d398d3ba786de8188e8f878bb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:00:33.978165 653977 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0920 19:00:33.978590 653977 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0920 19:00:33.978668 653977 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-809747"
I0920 19:00:33.978684 653977 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-809747"
W0920 19:00:33.978690 653977 addons.go:243] addon storage-provisioner should already be in state true
I0920 19:00:33.978715 653977 host.go:66] Checking if "old-k8s-version-809747" exists ...
I0920 19:00:33.979165 653977 cli_runner.go:164] Run: docker container inspect old-k8s-version-809747 --format={{.State.Status}}
I0920 19:00:33.979989 653977 config.go:182] Loaded profile config "old-k8s-version-809747": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0920 19:00:33.980175 653977 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-809747"
I0920 19:00:33.980226 653977 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-809747"
W0920 19:00:33.980248 653977 addons.go:243] addon metrics-server should already be in state true
I0920 19:00:33.980299 653977 host.go:66] Checking if "old-k8s-version-809747" exists ...
I0920 19:00:33.980886 653977 cli_runner.go:164] Run: docker container inspect old-k8s-version-809747 --format={{.State.Status}}
I0920 19:00:33.981152 653977 addons.go:69] Setting dashboard=true in profile "old-k8s-version-809747"
I0920 19:00:33.981200 653977 addons.go:234] Setting addon dashboard=true in "old-k8s-version-809747"
W0920 19:00:33.981240 653977 addons.go:243] addon dashboard should already be in state true
I0920 19:00:33.981290 653977 host.go:66] Checking if "old-k8s-version-809747" exists ...
I0920 19:00:33.981870 653977 cli_runner.go:164] Run: docker container inspect old-k8s-version-809747 --format={{.State.Status}}
I0920 19:00:33.982345 653977 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-809747"
I0920 19:00:33.982389 653977 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-809747"
I0920 19:00:33.982755 653977 cli_runner.go:164] Run: docker container inspect old-k8s-version-809747 --format={{.State.Status}}
I0920 19:00:33.990505 653977 out.go:177] * Verifying Kubernetes components...
I0920 19:00:33.996404 653977 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0920 19:00:34.043509 653977 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0920 19:00:34.045750 653977 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0920 19:00:34.045774 653977 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0920 19:00:34.045844 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:34.066684 653977 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0920 19:00:34.069094 653977 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0920 19:00:34.069122 653977 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0920 19:00:34.069209 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:34.082618 653977 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-809747"
W0920 19:00:34.082642 653977 addons.go:243] addon default-storageclass should already be in state true
I0920 19:00:34.082669 653977 host.go:66] Checking if "old-k8s-version-809747" exists ...
I0920 19:00:34.083099 653977 cli_runner.go:164] Run: docker container inspect old-k8s-version-809747 --format={{.State.Status}}
I0920 19:00:34.106400 653977 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0920 19:00:34.112085 653977 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0920 19:00:34.118347 653977 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0920 19:00:34.118385 653977 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0920 19:00:34.118464 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:34.150458 653977 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/old-k8s-version-809747/id_rsa Username:docker}
I0920 19:00:34.150997 653977 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/old-k8s-version-809747/id_rsa Username:docker}
I0920 19:00:34.158446 653977 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0920 19:00:34.158478 653977 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0920 19:00:34.158547 653977 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-809747
I0920 19:00:34.206537 653977 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/old-k8s-version-809747/id_rsa Username:docker}
I0920 19:00:34.222589 653977 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0920 19:00:34.230708 653977 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/old-k8s-version-809747/id_rsa Username:docker}
I0920 19:00:34.255645 653977 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-809747" to be "Ready" ...
I0920 19:00:34.357792 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0920 19:00:34.386772 653977 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0920 19:00:34.386806 653977 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0920 19:00:34.467819 653977 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0920 19:00:34.467873 653977 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0920 19:00:34.485604 653977 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0920 19:00:34.485653 653977 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0920 19:00:34.509370 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0920 19:00:34.569023 653977 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0920 19:00:34.569072 653977 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0920 19:00:34.596263 653977 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0920 19:00:34.596305 653977 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
W0920 19:00:34.678654 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:34.678712 653977 retry.go:31] will retry after 126.39832ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:34.683921 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0920 19:00:34.721285 653977 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0920 19:00:34.721318 653977 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0920 19:00:34.806296 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0920 19:00:34.844861 653977 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0920 19:00:34.844891 653977 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
W0920 19:00:34.867934 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:34.867971 653977 retry.go:31] will retry after 252.208741ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:34.967419 653977 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0920 19:00:34.967459 653977 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
W0920 19:00:35.046446 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.046487 653977 retry.go:31] will retry after 492.197471ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0920 19:00:35.046535 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.046550 653977 retry.go:31] will retry after 148.04313ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.054206 653977 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0920 19:00:35.054240 653977 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0920 19:00:35.085036 653977 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0920 19:00:35.085116 653977 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0920 19:00:35.119094 653977 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0920 19:00:35.119172 653977 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0920 19:00:35.120397 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0920 19:00:35.192259 653977 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0920 19:00:35.192336 653977 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0920 19:00:35.195659 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0920 19:00:35.252239 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0920 19:00:35.313446 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.313542 653977 retry.go:31] will retry after 229.08492ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0920 19:00:35.408922 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.409014 653977 retry.go:31] will retry after 425.599307ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0920 19:00:35.435759 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.435848 653977 retry.go:31] will retry after 177.893908ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.539107 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0920 19:00:35.543022 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0920 19:00:35.614587 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0920 19:00:35.686829 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.686886 653977 retry.go:31] will retry after 632.088543ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0920 19:00:35.749053 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.749120 653977 retry.go:31] will retry after 289.010206ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0920 19:00:35.791967 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.792016 653977 retry.go:31] will retry after 241.49281ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.835163 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0920 19:00:35.926688 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:35.926721 653977 retry.go:31] will retry after 586.254513ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:36.034061 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0920 19:00:36.038533 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0920 19:00:36.204391 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:36.204423 653977 retry.go:31] will retry after 708.369758ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0920 19:00:36.218583 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:36.218656 653977 retry.go:31] will retry after 681.59252ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:36.257223 653977 node_ready.go:53] error getting node "old-k8s-version-809747": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-809747": dial tcp 192.168.76.2:8443: connect: connection refused
I0920 19:00:36.319389 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0920 19:00:36.411828 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:36.411912 653977 retry.go:31] will retry after 500.160533ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:36.514059 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0920 19:00:36.616795 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:36.616879 653977 retry.go:31] will retry after 991.378116ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:36.900754 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0920 19:00:36.913168 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0920 19:00:36.913356 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0920 19:00:37.091066 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:37.091165 653977 retry.go:31] will retry after 714.861817ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0920 19:00:37.133174 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:37.133215 653977 retry.go:31] will retry after 1.136574144s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0920 19:00:37.133286 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:37.133302 653977 retry.go:31] will retry after 520.089132ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:37.609338 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0920 19:00:37.653755 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0920 19:00:37.750766 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:37.750828 653977 retry.go:31] will retry after 1.571580874s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0920 19:00:37.802152 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:37.802198 653977 retry.go:31] will retry after 1.845038058s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:37.806295 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0920 19:00:37.913625 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:37.913671 653977 retry.go:31] will retry after 2.139714448s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:38.270434 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0920 19:00:38.392679 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:38.392714 653977 retry.go:31] will retry after 1.156435371s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:38.756567 653977 node_ready.go:53] error getting node "old-k8s-version-809747": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-809747": dial tcp 192.168.76.2:8443: connect: connection refused
I0920 19:00:39.323521 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0920 19:00:39.415463 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:39.415501 653977 retry.go:31] will retry after 2.58842347s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:39.549817 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0920 19:00:39.629281 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:39.629323 653977 retry.go:31] will retry after 2.93109741s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:39.647422 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0920 19:00:39.722484 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:39.722519 653977 retry.go:31] will retry after 968.998158ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:40.054150 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0920 19:00:40.140044 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:40.140095 653977 retry.go:31] will retry after 3.740833405s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:40.692417 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0920 19:00:40.757143 653977 node_ready.go:53] error getting node "old-k8s-version-809747": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-809747": dial tcp 192.168.76.2:8443: connect: connection refused
W0920 19:00:40.767413 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:40.767450 653977 retry.go:31] will retry after 2.752304306s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:42.004149 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0920 19:00:42.126808 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:42.126857 653977 retry.go:31] will retry after 2.333345578s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:42.561621 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0920 19:00:42.803510 653977 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:42.803538 653977 retry.go:31] will retry after 3.797815098s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0920 19:00:43.256164 653977 node_ready.go:53] error getting node "old-k8s-version-809747": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-809747": dial tcp 192.168.76.2:8443: connect: connection refused
I0920 19:00:43.520687 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0920 19:00:43.881056 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0920 19:00:44.461334 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0920 19:00:46.601835 653977 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0920 19:00:52.536437 653977 node_ready.go:49] node "old-k8s-version-809747" has status "Ready":"True"
I0920 19:00:52.536460 653977 node_ready.go:38] duration metric: took 18.280780879s for node "old-k8s-version-809747" to be "Ready" ...
I0920 19:00:52.536470 653977 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0920 19:00:52.728957 653977 pod_ready.go:79] waiting up to 6m0s for pod "coredns-74ff55c5b-682lc" in "kube-system" namespace to be "Ready" ...
I0920 19:00:52.839880 653977 pod_ready.go:93] pod "coredns-74ff55c5b-682lc" in "kube-system" namespace has status "Ready":"True"
I0920 19:00:52.839949 653977 pod_ready.go:82] duration metric: took 110.911205ms for pod "coredns-74ff55c5b-682lc" in "kube-system" namespace to be "Ready" ...
I0920 19:00:52.839980 653977 pod_ready.go:79] waiting up to 6m0s for pod "etcd-old-k8s-version-809747" in "kube-system" namespace to be "Ready" ...
I0920 19:00:52.869197 653977 pod_ready.go:93] pod "etcd-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"True"
I0920 19:00:52.869277 653977 pod_ready.go:82] duration metric: took 29.275216ms for pod "etcd-old-k8s-version-809747" in "kube-system" namespace to be "Ready" ...
I0920 19:00:52.869308 653977 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-809747" in "kube-system" namespace to be "Ready" ...
I0920 19:00:52.892193 653977 pod_ready.go:93] pod "kube-apiserver-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"True"
I0920 19:00:52.892270 653977 pod_ready.go:82] duration metric: took 22.926823ms for pod "kube-apiserver-old-k8s-version-809747" in "kube-system" namespace to be "Ready" ...
I0920 19:00:52.892297 653977 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-809747" in "kube-system" namespace to be "Ready" ...
I0920 19:00:52.923907 653977 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"True"
I0920 19:00:52.923982 653977 pod_ready.go:82] duration metric: took 31.662266ms for pod "kube-controller-manager-old-k8s-version-809747" in "kube-system" namespace to be "Ready" ...
I0920 19:00:52.924009 653977 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-tczmb" in "kube-system" namespace to be "Ready" ...
I0920 19:00:52.931551 653977 pod_ready.go:93] pod "kube-proxy-tczmb" in "kube-system" namespace has status "Ready":"True"
I0920 19:00:52.931626 653977 pod_ready.go:82] duration metric: took 7.594275ms for pod "kube-proxy-tczmb" in "kube-system" namespace to be "Ready" ...
I0920 19:00:52.931654 653977 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace to be "Ready" ...
I0920 19:00:53.662541 653977 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (10.141795624s)
I0920 19:00:53.662833 653977 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (9.781743549s)
I0920 19:00:53.663013 653977 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (9.201643979s)
I0920 19:00:53.663048 653977 addons.go:475] Verifying addon metrics-server=true in "old-k8s-version-809747"
I0920 19:00:53.663136 653977 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (7.061280112s)
I0920 19:00:53.665669 653977 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-809747 addons enable metrics-server
I0920 19:00:53.676742 653977 out.go:177] * Enabled addons: metrics-server, storage-provisioner, dashboard, default-storageclass
I0920 19:00:53.678513 653977 addons.go:510] duration metric: took 19.699917763s for enable addons: enabled=[metrics-server storage-provisioner dashboard default-storageclass]
I0920 19:00:54.939896 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:00:57.447927 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:00:59.939374 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:01.939611 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:04.437515 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:06.438421 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:08.937500 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:10.949912 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:13.439383 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:15.451209 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:17.470631 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:19.939624 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:22.438287 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:24.438499 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:26.438651 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:28.439050 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:30.937439 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:32.937693 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:34.938249 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:37.438833 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:39.938527 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:41.939457 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:44.438113 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:46.440272 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:48.442691 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:50.941724 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:53.440294 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:55.938941 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:01:58.437924 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:00.487844 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:02.937913 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:04.938688 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:06.945809 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:09.458130 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:11.937956 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:14.442423 653977 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:14.937945 653977 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace has status "Ready":"True"
I0920 19:02:14.938015 653977 pod_ready.go:82] duration metric: took 1m22.006338563s for pod "kube-scheduler-old-k8s-version-809747" in "kube-system" namespace to be "Ready" ...
I0920 19:02:14.938035 653977 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace to be "Ready" ...
I0920 19:02:16.945061 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:19.444950 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:21.944950 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:23.945119 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:26.444388 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:28.446037 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:30.447386 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:32.944913 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:35.444452 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:37.445194 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:39.944849 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:41.951696 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:44.444379 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:46.944741 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:49.444620 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:51.444828 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:53.445196 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:55.945024 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:02:57.945504 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:00.445583 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:02.944534 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:04.944836 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:06.945276 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:09.444417 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:11.444652 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:13.945525 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:16.444243 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:18.944428 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:20.944873 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:22.946834 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:25.445237 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:27.945266 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:29.945583 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:32.443920 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:34.444977 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:36.944307 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:39.445108 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:41.445485 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:43.448412 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:45.945487 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:48.444027 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:50.444433 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:52.444954 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:54.445149 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:56.944770 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:03:58.955345 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:01.445920 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:03.944493 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:05.944570 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:07.945024 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:09.945391 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:12.444159 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:14.951303 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:17.444923 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:19.445237 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:21.944054 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:24.444213 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:26.945720 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:29.443834 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:31.443892 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:33.444779 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:35.945420 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:38.445334 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:40.944760 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:43.444753 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:45.444792 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:47.945457 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:50.444771 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:52.944773 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:55.444132 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:57.944283 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:04:59.945310 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:01.945617 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:04.444334 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:06.444682 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:08.445379 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:10.945058 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:13.446169 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:15.944376 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:17.944493 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:19.945041 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:22.444104 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:24.445307 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:26.945125 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:28.946115 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:31.444732 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:33.445103 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:35.445219 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:37.945636 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:39.950096 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:41.950671 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:44.503978 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:46.945602 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:49.444647 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:51.950151 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:54.445777 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:56.945405 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:05:59.445012 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:06:01.445966 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:06:03.944592 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:06:05.944734 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:06:07.945215 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:06:10.444198 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:06:12.445163 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:06:14.445421 653977 pod_ready.go:103] pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace has status "Ready":"False"
I0920 19:06:14.944609 653977 pod_ready.go:82] duration metric: took 4m0.00655808s for pod "metrics-server-9975d5f86-bx26v" in "kube-system" namespace to be "Ready" ...
E0920 19:06:14.944634 653977 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0920 19:06:14.944683 653977 pod_ready.go:39] duration metric: took 5m22.408163156s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0920 19:06:14.944705 653977 api_server.go:52] waiting for apiserver process to appear ...
I0920 19:06:14.944736 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0920 19:06:14.944807 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0920 19:06:14.984507 653977 cri.go:89] found id: "d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7"
I0920 19:06:14.984579 653977 cri.go:89] found id: "fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4"
I0920 19:06:14.984591 653977 cri.go:89] found id: ""
I0920 19:06:14.984601 653977 logs.go:276] 2 containers: [d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7 fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4]
I0920 19:06:14.984660 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:14.988230 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:14.991662 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0920 19:06:14.991757 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0920 19:06:15.056844 653977 cri.go:89] found id: "085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70"
I0920 19:06:15.056930 653977 cri.go:89] found id: "ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5"
I0920 19:06:15.056964 653977 cri.go:89] found id: ""
I0920 19:06:15.057009 653977 logs.go:276] 2 containers: [085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70 ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5]
I0920 19:06:15.057102 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.061726 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.066115 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0920 19:06:15.066220 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0920 19:06:15.133035 653977 cri.go:89] found id: "812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327"
I0920 19:06:15.133072 653977 cri.go:89] found id: "4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada"
I0920 19:06:15.133079 653977 cri.go:89] found id: ""
I0920 19:06:15.133088 653977 logs.go:276] 2 containers: [812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327 4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada]
I0920 19:06:15.133156 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.139228 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.144195 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0920 19:06:15.144326 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0920 19:06:15.201858 653977 cri.go:89] found id: "2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677"
I0920 19:06:15.201884 653977 cri.go:89] found id: "7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043"
I0920 19:06:15.201892 653977 cri.go:89] found id: ""
I0920 19:06:15.201901 653977 logs.go:276] 2 containers: [2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677 7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043]
I0920 19:06:15.201992 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.206566 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.210255 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0920 19:06:15.210403 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0920 19:06:15.255919 653977 cri.go:89] found id: "e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec"
I0920 19:06:15.255996 653977 cri.go:89] found id: "899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d"
I0920 19:06:15.256018 653977 cri.go:89] found id: ""
I0920 19:06:15.256050 653977 logs.go:276] 2 containers: [e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec 899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d]
I0920 19:06:15.256139 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.259706 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.263193 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0920 19:06:15.263319 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0920 19:06:15.306689 653977 cri.go:89] found id: "fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548"
I0920 19:06:15.306759 653977 cri.go:89] found id: "dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32"
I0920 19:06:15.306782 653977 cri.go:89] found id: ""
I0920 19:06:15.306798 653977 logs.go:276] 2 containers: [fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548 dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32]
I0920 19:06:15.306856 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.310982 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.314414 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0920 19:06:15.314485 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0920 19:06:15.352773 653977 cri.go:89] found id: "ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de"
I0920 19:06:15.352796 653977 cri.go:89] found id: "6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c"
I0920 19:06:15.352801 653977 cri.go:89] found id: ""
I0920 19:06:15.352808 653977 logs.go:276] 2 containers: [ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de 6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c]
I0920 19:06:15.352867 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.356635 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.360379 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0920 19:06:15.360473 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0920 19:06:15.402427 653977 cri.go:89] found id: "a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd"
I0920 19:06:15.402457 653977 cri.go:89] found id: ""
I0920 19:06:15.402465 653977 logs.go:276] 1 containers: [a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd]
I0920 19:06:15.402548 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.406374 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0920 19:06:15.406479 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0920 19:06:15.467757 653977 cri.go:89] found id: "78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6"
I0920 19:06:15.467777 653977 cri.go:89] found id: "558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f"
I0920 19:06:15.467782 653977 cri.go:89] found id: ""
I0920 19:06:15.467788 653977 logs.go:276] 2 containers: [78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6 558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f]
I0920 19:06:15.467844 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.473338 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:15.476870 653977 logs.go:123] Gathering logs for kube-proxy [899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d] ...
I0920 19:06:15.476944 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d"
I0920 19:06:15.522385 653977 logs.go:123] Gathering logs for kindnet [ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de] ...
I0920 19:06:15.522416 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de"
I0920 19:06:15.579079 653977 logs.go:123] Gathering logs for kindnet [6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c] ...
I0920 19:06:15.579113 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c"
I0920 19:06:15.628786 653977 logs.go:123] Gathering logs for etcd [ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5] ...
I0920 19:06:15.628819 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5"
I0920 19:06:15.687528 653977 logs.go:123] Gathering logs for kube-proxy [e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec] ...
I0920 19:06:15.687557 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec"
I0920 19:06:15.736648 653977 logs.go:123] Gathering logs for coredns [4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada] ...
I0920 19:06:15.736676 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada"
I0920 19:06:15.783007 653977 logs.go:123] Gathering logs for kube-scheduler [2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677] ...
I0920 19:06:15.783038 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677"
I0920 19:06:15.829656 653977 logs.go:123] Gathering logs for kube-scheduler [7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043] ...
I0920 19:06:15.829684 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043"
I0920 19:06:15.878134 653977 logs.go:123] Gathering logs for kube-controller-manager [dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32] ...
I0920 19:06:15.878172 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32"
I0920 19:06:15.946804 653977 logs.go:123] Gathering logs for storage-provisioner [78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6] ...
I0920 19:06:15.946844 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6"
I0920 19:06:15.988529 653977 logs.go:123] Gathering logs for storage-provisioner [558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f] ...
I0920 19:06:15.988560 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f"
I0920 19:06:16.034644 653977 logs.go:123] Gathering logs for kubelet ...
I0920 19:06:16.034680 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0920 19:06:16.100049 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302392 664 reflector.go:138] object-"kube-system"/"kindnet-token-ll598": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-ll598" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:16.100305 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302467 664 reflector.go:138] object-"kube-system"/"storage-provisioner-token-5rgk7": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-5rgk7" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:16.100627 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302516 664 reflector.go:138] object-"kube-system"/"kube-proxy-token-2m84d": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-2m84d" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:16.100849 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302566 664 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:16.101064 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302642 664 reflector.go:138] object-"kube-system"/"coredns-token-tqf8k": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-tqf8k" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:16.101268 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302685 664 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:16.101492 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302728 664 reflector.go:138] object-"kube-system"/"metrics-server-token-fk86c": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-fk86c" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:16.101705 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302769 664 reflector.go:138] object-"default"/"default-token-gqwtk": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-gqwtk" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:16.110802 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:56 old-k8s-version-809747 kubelet[664]: E0920 19:00:56.277740 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:16.111004 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:57 old-k8s-version-809747 kubelet[664]: E0920 19:00:57.075859 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.115127 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:08 old-k8s-version-809747 kubelet[664]: E0920 19:01:08.853318 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:16.117924 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:19 old-k8s-version-809747 kubelet[664]: E0920 19:01:19.861009 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.121173 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:22 old-k8s-version-809747 kubelet[664]: E0920 19:01:22.209550 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.121986 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:23 old-k8s-version-809747 kubelet[664]: E0920 19:01:23.208449 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.122473 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:24 old-k8s-version-809747 kubelet[664]: E0920 19:01:24.374198 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.123188 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:25 old-k8s-version-809747 kubelet[664]: E0920 19:01:25.231977 664 pod_workers.go:191] Error syncing pod fd544a9f-bcfa-4019-9bf3-71e1e9a2d2ae ("storage-provisioner_kube-system(fd544a9f-bcfa-4019-9bf3-71e1e9a2d2ae)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(fd544a9f-bcfa-4019-9bf3-71e1e9a2d2ae)"
W0920 19:06:16.126691 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:30 old-k8s-version-809747 kubelet[664]: E0920 19:01:30.851954 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:16.128767 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:38 old-k8s-version-809747 kubelet[664]: E0920 19:01:38.279749 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.129446 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:42 old-k8s-version-809747 kubelet[664]: E0920 19:01:42.842501 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.129840 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:44 old-k8s-version-809747 kubelet[664]: E0920 19:01:44.374858 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.134573 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:55 old-k8s-version-809747 kubelet[664]: E0920 19:01:55.842879 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.134786 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:56 old-k8s-version-809747 kubelet[664]: E0920 19:01:56.842908 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.135111 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:08 old-k8s-version-809747 kubelet[664]: E0920 19:02:08.844439 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.135580 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:10 old-k8s-version-809747 kubelet[664]: E0920 19:02:10.389468 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.135919 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:14 old-k8s-version-809747 kubelet[664]: E0920 19:02:14.374714 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.138430 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:20 old-k8s-version-809747 kubelet[664]: E0920 19:02:20.851952 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:16.138769 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:26 old-k8s-version-809747 kubelet[664]: E0920 19:02:26.841902 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.138962 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:31 old-k8s-version-809747 kubelet[664]: E0920 19:02:31.842985 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.139296 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:41 old-k8s-version-809747 kubelet[664]: E0920 19:02:41.843033 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.139484 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:44 old-k8s-version-809747 kubelet[664]: E0920 19:02:44.846601 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.140079 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:54 old-k8s-version-809747 kubelet[664]: E0920 19:02:54.515666 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.140266 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:59 old-k8s-version-809747 kubelet[664]: E0920 19:02:59.842434 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.140598 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:04 old-k8s-version-809747 kubelet[664]: E0920 19:03:04.374783 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.140786 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:11 old-k8s-version-809747 kubelet[664]: E0920 19:03:11.842669 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.141116 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:16 old-k8s-version-809747 kubelet[664]: E0920 19:03:16.841791 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.141304 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:22 old-k8s-version-809747 kubelet[664]: E0920 19:03:22.842284 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.141636 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:28 old-k8s-version-809747 kubelet[664]: E0920 19:03:28.842043 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.141822 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:33 old-k8s-version-809747 kubelet[664]: E0920 19:03:33.843004 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.142154 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:42 old-k8s-version-809747 kubelet[664]: E0920 19:03:42.841902 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.144722 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:48 old-k8s-version-809747 kubelet[664]: E0920 19:03:48.851175 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:16.145056 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:54 old-k8s-version-809747 kubelet[664]: E0920 19:03:54.841890 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.145247 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:01 old-k8s-version-809747 kubelet[664]: E0920 19:04:01.843066 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.145579 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:08 old-k8s-version-809747 kubelet[664]: E0920 19:04:08.841968 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.145765 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:15 old-k8s-version-809747 kubelet[664]: E0920 19:04:15.842571 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.146365 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:21 old-k8s-version-809747 kubelet[664]: E0920 19:04:21.744371 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.146700 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:24 old-k8s-version-809747 kubelet[664]: E0920 19:04:24.377802 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.146902 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:29 old-k8s-version-809747 kubelet[664]: E0920 19:04:29.842421 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.147233 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:36 old-k8s-version-809747 kubelet[664]: E0920 19:04:36.841921 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.147420 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:44 old-k8s-version-809747 kubelet[664]: E0920 19:04:44.842363 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.147756 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:51 old-k8s-version-809747 kubelet[664]: E0920 19:04:51.843005 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.147942 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:59 old-k8s-version-809747 kubelet[664]: E0920 19:04:59.842361 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.148272 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:02 old-k8s-version-809747 kubelet[664]: E0920 19:05:02.842297 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.148603 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:13 old-k8s-version-809747 kubelet[664]: E0920 19:05:13.842057 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.148791 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:14 old-k8s-version-809747 kubelet[664]: E0920 19:05:14.842283 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.149129 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:24 old-k8s-version-809747 kubelet[664]: E0920 19:05:24.842121 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.149319 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:28 old-k8s-version-809747 kubelet[664]: E0920 19:05:28.842363 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.149649 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:37 old-k8s-version-809747 kubelet[664]: E0920 19:05:37.842390 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.149834 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:43 old-k8s-version-809747 kubelet[664]: E0920 19:05:43.842490 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.150164 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:50 old-k8s-version-809747 kubelet[664]: E0920 19:05:50.841875 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.150358 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:58 old-k8s-version-809747 kubelet[664]: E0920 19:05:58.842395 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.150691 653977 logs.go:138] Found kubelet problem: Sep 20 19:06:04 old-k8s-version-809747 kubelet[664]: E0920 19:06:04.841931 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.150880 653977 logs.go:138] Found kubelet problem: Sep 20 19:06:09 old-k8s-version-809747 kubelet[664]: E0920 19:06:09.842365 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0920 19:06:16.150890 653977 logs.go:123] Gathering logs for kube-apiserver [d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7] ...
I0920 19:06:16.150905 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7"
I0920 19:06:16.225582 653977 logs.go:123] Gathering logs for containerd ...
I0920 19:06:16.225634 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0920 19:06:16.292587 653977 logs.go:123] Gathering logs for coredns [812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327] ...
I0920 19:06:16.292628 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327"
I0920 19:06:16.337773 653977 logs.go:123] Gathering logs for kube-controller-manager [fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548] ...
I0920 19:06:16.337804 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548"
I0920 19:06:16.402127 653977 logs.go:123] Gathering logs for kubernetes-dashboard [a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd] ...
I0920 19:06:16.402169 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd"
I0920 19:06:16.452253 653977 logs.go:123] Gathering logs for dmesg ...
I0920 19:06:16.452283 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0920 19:06:16.471492 653977 logs.go:123] Gathering logs for describe nodes ...
I0920 19:06:16.471570 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0920 19:06:16.635700 653977 logs.go:123] Gathering logs for container status ...
I0920 19:06:16.635739 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0920 19:06:16.681697 653977 logs.go:123] Gathering logs for kube-apiserver [fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4] ...
I0920 19:06:16.681730 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4"
I0920 19:06:16.779845 653977 logs.go:123] Gathering logs for etcd [085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70] ...
I0920 19:06:16.779931 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70"
I0920 19:06:16.833254 653977 out.go:358] Setting ErrFile to fd 2...
I0920 19:06:16.833407 653977 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0920 19:06:16.833496 653977 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0920 19:06:16.833542 653977 out.go:270] Sep 20 19:05:43 old-k8s-version-809747 kubelet[664]: E0920 19:05:43.842490 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:05:43 old-k8s-version-809747 kubelet[664]: E0920 19:05:43.842490 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.833604 653977 out.go:270] Sep 20 19:05:50 old-k8s-version-809747 kubelet[664]: E0920 19:05:50.841875 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
Sep 20 19:05:50 old-k8s-version-809747 kubelet[664]: E0920 19:05:50.841875 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.833657 653977 out.go:270] Sep 20 19:05:58 old-k8s-version-809747 kubelet[664]: E0920 19:05:58.842395 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:05:58 old-k8s-version-809747 kubelet[664]: E0920 19:05:58.842395 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:16.833695 653977 out.go:270] Sep 20 19:06:04 old-k8s-version-809747 kubelet[664]: E0920 19:06:04.841931 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
Sep 20 19:06:04 old-k8s-version-809747 kubelet[664]: E0920 19:06:04.841931 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:16.833727 653977 out.go:270] Sep 20 19:06:09 old-k8s-version-809747 kubelet[664]: E0920 19:06:09.842365 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:06:09 old-k8s-version-809747 kubelet[664]: E0920 19:06:09.842365 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0920 19:06:16.833758 653977 out.go:358] Setting ErrFile to fd 2...
I0920 19:06:16.833779 653977 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0920 19:06:26.835053 653977 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0920 19:06:26.849195 653977 api_server.go:72] duration metric: took 5m52.870971077s to wait for apiserver process to appear ...
I0920 19:06:26.849217 653977 api_server.go:88] waiting for apiserver healthz status ...
I0920 19:06:26.849253 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0920 19:06:26.849308 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0920 19:06:26.897788 653977 cri.go:89] found id: "d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7"
I0920 19:06:26.897808 653977 cri.go:89] found id: "fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4"
I0920 19:06:26.897813 653977 cri.go:89] found id: ""
I0920 19:06:26.897821 653977 logs.go:276] 2 containers: [d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7 fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4]
I0920 19:06:26.897878 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:26.902338 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:26.906548 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0920 19:06:26.906623 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0920 19:06:26.958378 653977 cri.go:89] found id: "085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70"
I0920 19:06:26.958398 653977 cri.go:89] found id: "ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5"
I0920 19:06:26.958403 653977 cri.go:89] found id: ""
I0920 19:06:26.958411 653977 logs.go:276] 2 containers: [085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70 ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5]
I0920 19:06:26.958470 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:26.962667 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:26.966536 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0920 19:06:26.966653 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0920 19:06:27.027915 653977 cri.go:89] found id: "812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327"
I0920 19:06:27.027936 653977 cri.go:89] found id: "4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada"
I0920 19:06:27.027941 653977 cri.go:89] found id: ""
I0920 19:06:27.027948 653977 logs.go:276] 2 containers: [812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327 4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada]
I0920 19:06:27.028013 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.033278 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.037758 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0920 19:06:27.037875 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0920 19:06:27.093653 653977 cri.go:89] found id: "2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677"
I0920 19:06:27.093726 653977 cri.go:89] found id: "7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043"
I0920 19:06:27.093746 653977 cri.go:89] found id: ""
I0920 19:06:27.093775 653977 logs.go:276] 2 containers: [2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677 7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043]
I0920 19:06:27.093857 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.098536 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.102887 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0920 19:06:27.103005 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0920 19:06:27.176029 653977 cri.go:89] found id: "e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec"
I0920 19:06:27.176091 653977 cri.go:89] found id: "899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d"
I0920 19:06:27.176123 653977 cri.go:89] found id: ""
I0920 19:06:27.176152 653977 logs.go:276] 2 containers: [e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec 899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d]
I0920 19:06:27.176226 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.181026 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.187926 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0920 19:06:27.188048 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0920 19:06:27.235096 653977 cri.go:89] found id: "fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548"
I0920 19:06:27.235167 653977 cri.go:89] found id: "dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32"
I0920 19:06:27.235187 653977 cri.go:89] found id: ""
I0920 19:06:27.235212 653977 logs.go:276] 2 containers: [fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548 dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32]
I0920 19:06:27.235293 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.239445 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.243444 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0920 19:06:27.243570 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0920 19:06:27.296352 653977 cri.go:89] found id: "ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de"
I0920 19:06:27.296445 653977 cri.go:89] found id: "6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c"
I0920 19:06:27.296469 653977 cri.go:89] found id: ""
I0920 19:06:27.296495 653977 logs.go:276] 2 containers: [ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de 6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c]
I0920 19:06:27.296576 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.300837 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.304814 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0920 19:06:27.304929 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0920 19:06:27.366905 653977 cri.go:89] found id: "a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd"
I0920 19:06:27.366980 653977 cri.go:89] found id: ""
I0920 19:06:27.367003 653977 logs.go:276] 1 containers: [a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd]
I0920 19:06:27.367084 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.371726 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0920 19:06:27.371855 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0920 19:06:27.419397 653977 cri.go:89] found id: "78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6"
I0920 19:06:27.419472 653977 cri.go:89] found id: "558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f"
I0920 19:06:27.419518 653977 cri.go:89] found id: ""
I0920 19:06:27.419545 653977 logs.go:276] 2 containers: [78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6 558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f]
I0920 19:06:27.419629 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.423969 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.430101 653977 logs.go:123] Gathering logs for kube-proxy [e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec] ...
I0920 19:06:27.430174 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec"
I0920 19:06:27.488295 653977 logs.go:123] Gathering logs for kube-controller-manager [dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32] ...
I0920 19:06:27.488370 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32"
I0920 19:06:27.567691 653977 logs.go:123] Gathering logs for kube-apiserver [d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7] ...
I0920 19:06:27.567769 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7"
I0920 19:06:27.637441 653977 logs.go:123] Gathering logs for coredns [812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327] ...
I0920 19:06:27.637485 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327"
I0920 19:06:27.712024 653977 logs.go:123] Gathering logs for coredns [4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada] ...
I0920 19:06:27.712059 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada"
I0920 19:06:27.761492 653977 logs.go:123] Gathering logs for kube-scheduler [2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677] ...
I0920 19:06:27.761548 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677"
I0920 19:06:27.815260 653977 logs.go:123] Gathering logs for kube-proxy [899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d] ...
I0920 19:06:27.815292 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d"
I0920 19:06:27.868976 653977 logs.go:123] Gathering logs for kube-controller-manager [fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548] ...
I0920 19:06:27.869008 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548"
I0920 19:06:27.955237 653977 logs.go:123] Gathering logs for kindnet [6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c] ...
I0920 19:06:27.955275 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c"
I0920 19:06:28.009037 653977 logs.go:123] Gathering logs for kubernetes-dashboard [a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd] ...
I0920 19:06:28.009074 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd"
I0920 19:06:28.059987 653977 logs.go:123] Gathering logs for kube-apiserver [fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4] ...
I0920 19:06:28.060019 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4"
I0920 19:06:28.165966 653977 logs.go:123] Gathering logs for etcd [ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5] ...
I0920 19:06:28.166007 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5"
I0920 19:06:28.221462 653977 logs.go:123] Gathering logs for container status ...
I0920 19:06:28.221495 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0920 19:06:28.317786 653977 logs.go:123] Gathering logs for storage-provisioner [558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f] ...
I0920 19:06:28.317819 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f"
I0920 19:06:28.363443 653977 logs.go:123] Gathering logs for containerd ...
I0920 19:06:28.363472 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0920 19:06:28.429304 653977 logs.go:123] Gathering logs for kube-scheduler [7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043] ...
I0920 19:06:28.429342 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043"
I0920 19:06:28.511550 653977 logs.go:123] Gathering logs for dmesg ...
I0920 19:06:28.511585 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0920 19:06:28.530657 653977 logs.go:123] Gathering logs for etcd [085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70] ...
I0920 19:06:28.530687 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70"
I0920 19:06:28.612728 653977 logs.go:123] Gathering logs for kindnet [ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de] ...
I0920 19:06:28.612765 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de"
I0920 19:06:28.688022 653977 logs.go:123] Gathering logs for storage-provisioner [78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6] ...
I0920 19:06:28.688058 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6"
I0920 19:06:28.738857 653977 logs.go:123] Gathering logs for kubelet ...
I0920 19:06:28.738886 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0920 19:06:28.797990 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302392 664 reflector.go:138] object-"kube-system"/"kindnet-token-ll598": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-ll598" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.798246 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302467 664 reflector.go:138] object-"kube-system"/"storage-provisioner-token-5rgk7": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-5rgk7" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.798535 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302516 664 reflector.go:138] object-"kube-system"/"kube-proxy-token-2m84d": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-2m84d" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.798750 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302566 664 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.798966 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302642 664 reflector.go:138] object-"kube-system"/"coredns-token-tqf8k": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-tqf8k" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.799170 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302685 664 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.799394 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302728 664 reflector.go:138] object-"kube-system"/"metrics-server-token-fk86c": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-fk86c" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.799604 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302769 664 reflector.go:138] object-"default"/"default-token-gqwtk": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-gqwtk" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.808631 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:56 old-k8s-version-809747 kubelet[664]: E0920 19:00:56.277740 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:28.808831 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:57 old-k8s-version-809747 kubelet[664]: E0920 19:00:57.075859 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.811689 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:08 old-k8s-version-809747 kubelet[664]: E0920 19:01:08.853318 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:28.813374 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:19 old-k8s-version-809747 kubelet[664]: E0920 19:01:19.861009 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.814320 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:22 old-k8s-version-809747 kubelet[664]: E0920 19:01:22.209550 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.814660 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:23 old-k8s-version-809747 kubelet[664]: E0920 19:01:23.208449 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.814996 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:24 old-k8s-version-809747 kubelet[664]: E0920 19:01:24.374198 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.815437 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:25 old-k8s-version-809747 kubelet[664]: E0920 19:01:25.231977 664 pod_workers.go:191] Error syncing pod fd544a9f-bcfa-4019-9bf3-71e1e9a2d2ae ("storage-provisioner_kube-system(fd544a9f-bcfa-4019-9bf3-71e1e9a2d2ae)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(fd544a9f-bcfa-4019-9bf3-71e1e9a2d2ae)"
W0920 19:06:28.817920 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:30 old-k8s-version-809747 kubelet[664]: E0920 19:01:30.851954 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:28.819220 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:38 old-k8s-version-809747 kubelet[664]: E0920 19:01:38.279749 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.819409 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:42 old-k8s-version-809747 kubelet[664]: E0920 19:01:42.842501 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.819748 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:44 old-k8s-version-809747 kubelet[664]: E0920 19:01:44.374858 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.820076 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:55 old-k8s-version-809747 kubelet[664]: E0920 19:01:55.842879 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.820260 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:56 old-k8s-version-809747 kubelet[664]: E0920 19:01:56.842908 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.820577 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:08 old-k8s-version-809747 kubelet[664]: E0920 19:02:08.844439 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.821043 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:10 old-k8s-version-809747 kubelet[664]: E0920 19:02:10.389468 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.821373 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:14 old-k8s-version-809747 kubelet[664]: E0920 19:02:14.374714 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.823922 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:20 old-k8s-version-809747 kubelet[664]: E0920 19:02:20.851952 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:28.824256 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:26 old-k8s-version-809747 kubelet[664]: E0920 19:02:26.841902 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.824495 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:31 old-k8s-version-809747 kubelet[664]: E0920 19:02:31.842985 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.824826 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:41 old-k8s-version-809747 kubelet[664]: E0920 19:02:41.843033 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.825012 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:44 old-k8s-version-809747 kubelet[664]: E0920 19:02:44.846601 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.825614 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:54 old-k8s-version-809747 kubelet[664]: E0920 19:02:54.515666 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.825802 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:59 old-k8s-version-809747 kubelet[664]: E0920 19:02:59.842434 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.826131 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:04 old-k8s-version-809747 kubelet[664]: E0920 19:03:04.374783 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.826352 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:11 old-k8s-version-809747 kubelet[664]: E0920 19:03:11.842669 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.826681 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:16 old-k8s-version-809747 kubelet[664]: E0920 19:03:16.841791 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.826866 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:22 old-k8s-version-809747 kubelet[664]: E0920 19:03:22.842284 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.827271 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:28 old-k8s-version-809747 kubelet[664]: E0920 19:03:28.842043 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.827472 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:33 old-k8s-version-809747 kubelet[664]: E0920 19:03:33.843004 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.827805 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:42 old-k8s-version-809747 kubelet[664]: E0920 19:03:42.841902 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.830249 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:48 old-k8s-version-809747 kubelet[664]: E0920 19:03:48.851175 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:28.830593 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:54 old-k8s-version-809747 kubelet[664]: E0920 19:03:54.841890 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.830778 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:01 old-k8s-version-809747 kubelet[664]: E0920 19:04:01.843066 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.831103 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:08 old-k8s-version-809747 kubelet[664]: E0920 19:04:08.841968 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.831288 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:15 old-k8s-version-809747 kubelet[664]: E0920 19:04:15.842571 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.831935 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:21 old-k8s-version-809747 kubelet[664]: E0920 19:04:21.744371 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.832267 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:24 old-k8s-version-809747 kubelet[664]: E0920 19:04:24.377802 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.832453 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:29 old-k8s-version-809747 kubelet[664]: E0920 19:04:29.842421 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.832781 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:36 old-k8s-version-809747 kubelet[664]: E0920 19:04:36.841921 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.832965 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:44 old-k8s-version-809747 kubelet[664]: E0920 19:04:44.842363 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.833331 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:51 old-k8s-version-809747 kubelet[664]: E0920 19:04:51.843005 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.833519 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:59 old-k8s-version-809747 kubelet[664]: E0920 19:04:59.842361 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.833846 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:02 old-k8s-version-809747 kubelet[664]: E0920 19:05:02.842297 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.834176 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:13 old-k8s-version-809747 kubelet[664]: E0920 19:05:13.842057 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.834367 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:14 old-k8s-version-809747 kubelet[664]: E0920 19:05:14.842283 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.834699 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:24 old-k8s-version-809747 kubelet[664]: E0920 19:05:24.842121 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.834882 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:28 old-k8s-version-809747 kubelet[664]: E0920 19:05:28.842363 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.835207 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:37 old-k8s-version-809747 kubelet[664]: E0920 19:05:37.842390 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.835394 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:43 old-k8s-version-809747 kubelet[664]: E0920 19:05:43.842490 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.835726 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:50 old-k8s-version-809747 kubelet[664]: E0920 19:05:50.841875 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.835911 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:58 old-k8s-version-809747 kubelet[664]: E0920 19:05:58.842395 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.836240 653977 logs.go:138] Found kubelet problem: Sep 20 19:06:04 old-k8s-version-809747 kubelet[664]: E0920 19:06:04.841931 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.836424 653977 logs.go:138] Found kubelet problem: Sep 20 19:06:09 old-k8s-version-809747 kubelet[664]: E0920 19:06:09.842365 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.836754 653977 logs.go:138] Found kubelet problem: Sep 20 19:06:17 old-k8s-version-809747 kubelet[664]: E0920 19:06:17.842782 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.836938 653977 logs.go:138] Found kubelet problem: Sep 20 19:06:23 old-k8s-version-809747 kubelet[664]: E0920 19:06:23.842473 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0920 19:06:28.836948 653977 logs.go:123] Gathering logs for describe nodes ...
I0920 19:06:28.836964 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0920 19:06:29.018737 653977 out.go:358] Setting ErrFile to fd 2...
I0920 19:06:29.018770 653977 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0920 19:06:29.018819 653977 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0920 19:06:29.018834 653977 out.go:270] Sep 20 19:05:58 old-k8s-version-809747 kubelet[664]: E0920 19:05:58.842395 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:05:58 old-k8s-version-809747 kubelet[664]: E0920 19:05:58.842395 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:29.018842 653977 out.go:270] Sep 20 19:06:04 old-k8s-version-809747 kubelet[664]: E0920 19:06:04.841931 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
Sep 20 19:06:04 old-k8s-version-809747 kubelet[664]: E0920 19:06:04.841931 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:29.018857 653977 out.go:270] Sep 20 19:06:09 old-k8s-version-809747 kubelet[664]: E0920 19:06:09.842365 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:06:09 old-k8s-version-809747 kubelet[664]: E0920 19:06:09.842365 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:29.018863 653977 out.go:270] Sep 20 19:06:17 old-k8s-version-809747 kubelet[664]: E0920 19:06:17.842782 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
Sep 20 19:06:17 old-k8s-version-809747 kubelet[664]: E0920 19:06:17.842782 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:29.018869 653977 out.go:270] Sep 20 19:06:23 old-k8s-version-809747 kubelet[664]: E0920 19:06:23.842473 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:06:23 old-k8s-version-809747 kubelet[664]: E0920 19:06:23.842473 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0920 19:06:29.018875 653977 out.go:358] Setting ErrFile to fd 2...
I0920 19:06:29.018889 653977 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0920 19:06:39.019670 653977 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 19:06:39.029993 653977 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0920 19:06:39.032174 653977 out.go:201]
W0920 19:06:39.034164 653977 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0920 19:06:39.034205 653977 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0920 19:06:39.034237 653977 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0920 19:06:39.034250 653977 out.go:270] *
*
W0920 19:06:39.035133 653977 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0920 19:06:39.038378 653977 out.go:201]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-809747 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-809747
helpers_test.go:235: (dbg) docker inspect old-k8s-version-809747:
-- stdout --
[
{
"Id": "1cd1d88cd1bb5c3b9f55eece743bd024009a64cbd1190993de0e8ccae96ae693",
"Created": "2024-09-20T18:57:26.973312319Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 654182,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-09-20T19:00:25.566123858Z",
"FinishedAt": "2024-09-20T19:00:24.232284705Z"
},
"Image": "sha256:f8be4f9f9351784955e36c0e64d55ad19451839d9f6d0c057285eb8f9072963b",
"ResolvConfPath": "/var/lib/docker/containers/1cd1d88cd1bb5c3b9f55eece743bd024009a64cbd1190993de0e8ccae96ae693/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/1cd1d88cd1bb5c3b9f55eece743bd024009a64cbd1190993de0e8ccae96ae693/hostname",
"HostsPath": "/var/lib/docker/containers/1cd1d88cd1bb5c3b9f55eece743bd024009a64cbd1190993de0e8ccae96ae693/hosts",
"LogPath": "/var/lib/docker/containers/1cd1d88cd1bb5c3b9f55eece743bd024009a64cbd1190993de0e8ccae96ae693/1cd1d88cd1bb5c3b9f55eece743bd024009a64cbd1190993de0e8ccae96ae693-json.log",
"Name": "/old-k8s-version-809747",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"old-k8s-version-809747:/var",
"/lib/modules:/lib/modules:ro"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-809747",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/961870f7bf9df920505510a2445372c668a086820841b60d6db77a5ff75531c4-init/diff:/var/lib/docker/overlay2/3aa0f15c41477a99e99dc1a77b5fdd60c51e1433d51cff06d0a41fe51ac2c7c3/diff",
"MergedDir": "/var/lib/docker/overlay2/961870f7bf9df920505510a2445372c668a086820841b60d6db77a5ff75531c4/merged",
"UpperDir": "/var/lib/docker/overlay2/961870f7bf9df920505510a2445372c668a086820841b60d6db77a5ff75531c4/diff",
"WorkDir": "/var/lib/docker/overlay2/961870f7bf9df920505510a2445372c668a086820841b60d6db77a5ff75531c4/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "old-k8s-version-809747",
"Source": "/var/lib/docker/volumes/old-k8s-version-809747/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
},
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "old-k8s-version-809747",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-809747",
"name.minikube.sigs.k8s.io": "old-k8s-version-809747",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "bb4a6ab0d0c1ca3e076d177b990f1af0626b75394d43bae7cb12b213c53f4ae6",
"SandboxKey": "/var/run/docker/netns/bb4a6ab0d0c1",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33063"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33064"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33067"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33065"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33066"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-809747": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null,
"NetworkID": "d24ef9c80fc9fb9721b21336a3a800d8be9a6d5cab6fa0741ec759024df649cc",
"EndpointID": "f50f6dfb338d4f827c1d6760366eb7c3ca7333bf4c085bae72e42d3a74e8bd57",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"old-k8s-version-809747",
"1cd1d88cd1bb"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-809747 -n old-k8s-version-809747
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-809747 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-809747 logs -n 25: (2.595011753s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
| start | -p cert-expiration-735719 | cert-expiration-735719 | jenkins | v1.34.0 | 20 Sep 24 18:56 UTC | 20 Sep 24 18:56 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | force-systemd-env-042522 | force-systemd-env-042522 | jenkins | v1.34.0 | 20 Sep 24 18:56 UTC | 20 Sep 24 18:56 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| delete | -p force-systemd-env-042522 | force-systemd-env-042522 | jenkins | v1.34.0 | 20 Sep 24 18:56 UTC | 20 Sep 24 18:56 UTC |
| start | -p cert-options-257492 | cert-options-257492 | jenkins | v1.34.0 | 20 Sep 24 18:56 UTC | 20 Sep 24 18:57 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | cert-options-257492 ssh | cert-options-257492 | jenkins | v1.34.0 | 20 Sep 24 18:57 UTC | 20 Sep 24 18:57 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-257492 -- sudo | cert-options-257492 | jenkins | v1.34.0 | 20 Sep 24 18:57 UTC | 20 Sep 24 18:57 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-257492 | cert-options-257492 | jenkins | v1.34.0 | 20 Sep 24 18:57 UTC | 20 Sep 24 18:57 UTC |
| start | -p old-k8s-version-809747 | old-k8s-version-809747 | jenkins | v1.34.0 | 20 Sep 24 18:57 UTC | 20 Sep 24 19:00 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-735719 | cert-expiration-735719 | jenkins | v1.34.0 | 20 Sep 24 18:59 UTC | 20 Sep 24 18:59 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p cert-expiration-735719 | cert-expiration-735719 | jenkins | v1.34.0 | 20 Sep 24 18:59 UTC | 20 Sep 24 18:59 UTC |
| start | -p no-preload-851913 | no-preload-851913 | jenkins | v1.34.0 | 20 Sep 24 18:59 UTC | 20 Sep 24 19:01 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
| addons | enable metrics-server -p old-k8s-version-809747 | old-k8s-version-809747 | jenkins | v1.34.0 | 20 Sep 24 19:00 UTC | 20 Sep 24 19:00 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-809747 | old-k8s-version-809747 | jenkins | v1.34.0 | 20 Sep 24 19:00 UTC | 20 Sep 24 19:00 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-809747 | old-k8s-version-809747 | jenkins | v1.34.0 | 20 Sep 24 19:00 UTC | 20 Sep 24 19:00 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-809747 | old-k8s-version-809747 | jenkins | v1.34.0 | 20 Sep 24 19:00 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-851913 | no-preload-851913 | jenkins | v1.34.0 | 20 Sep 24 19:01 UTC | 20 Sep 24 19:01 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-851913 | no-preload-851913 | jenkins | v1.34.0 | 20 Sep 24 19:01 UTC | 20 Sep 24 19:01 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-851913 | no-preload-851913 | jenkins | v1.34.0 | 20 Sep 24 19:01 UTC | 20 Sep 24 19:01 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-851913 | no-preload-851913 | jenkins | v1.34.0 | 20 Sep 24 19:01 UTC | 20 Sep 24 19:06 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
| image | no-preload-851913 image list | no-preload-851913 | jenkins | v1.34.0 | 20 Sep 24 19:06 UTC | 20 Sep 24 19:06 UTC |
| | --format=json | | | | | |
| pause | -p no-preload-851913 | no-preload-851913 | jenkins | v1.34.0 | 20 Sep 24 19:06 UTC | 20 Sep 24 19:06 UTC |
| | --alsologtostderr -v=1 | | | | | |
| unpause | -p no-preload-851913 | no-preload-851913 | jenkins | v1.34.0 | 20 Sep 24 19:06 UTC | 20 Sep 24 19:06 UTC |
| | --alsologtostderr -v=1 | | | | | |
| delete | -p no-preload-851913 | no-preload-851913 | jenkins | v1.34.0 | 20 Sep 24 19:06 UTC | 20 Sep 24 19:06 UTC |
| delete | -p no-preload-851913 | no-preload-851913 | jenkins | v1.34.0 | 20 Sep 24 19:06 UTC | 20 Sep 24 19:06 UTC |
| start | -p embed-certs-208780 | embed-certs-208780 | jenkins | v1.34.0 | 20 Sep 24 19:06 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --embed-certs --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/09/20 19:06:24
Running on machine: ip-172-31-30-239
Binary: Built with gc go1.23.0 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0920 19:06:24.024120 664902 out.go:345] Setting OutFile to fd 1 ...
I0920 19:06:24.024270 664902 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0920 19:06:24.024283 664902 out.go:358] Setting ErrFile to fd 2...
I0920 19:06:24.024288 664902 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0920 19:06:24.024585 664902 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19679-440039/.minikube/bin
I0920 19:06:24.025076 664902 out.go:352] Setting JSON to false
I0920 19:06:24.028287 664902 start.go:129] hostinfo: {"hostname":"ip-172-31-30-239","uptime":10135,"bootTime":1726849049,"procs":230,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1070-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"92f46a7d-c249-4c12-924a-77f64874c910"}
I0920 19:06:24.028380 664902 start.go:139] virtualization:
I0920 19:06:24.031266 664902 out.go:177] * [embed-certs-208780] minikube v1.34.0 on Ubuntu 20.04 (arm64)
I0920 19:06:24.034361 664902 out.go:177] - MINIKUBE_LOCATION=19679
I0920 19:06:24.034488 664902 notify.go:220] Checking for updates...
I0920 19:06:24.038338 664902 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0920 19:06:24.040807 664902 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19679-440039/kubeconfig
I0920 19:06:24.042982 664902 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19679-440039/.minikube
I0920 19:06:24.045179 664902 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0920 19:06:24.047192 664902 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0920 19:06:24.049984 664902 config.go:182] Loaded profile config "old-k8s-version-809747": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0920 19:06:24.050179 664902 driver.go:394] Setting default libvirt URI to qemu:///system
I0920 19:06:24.089167 664902 docker.go:123] docker version: linux-27.3.1:Docker Engine - Community
I0920 19:06:24.089390 664902 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0920 19:06:24.156210 664902 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:35 OomKillDisable:true NGoroutines:53 SystemTime:2024-09-20 19:06:24.142873751 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0920 19:06:24.156324 664902 docker.go:318] overlay module found
I0920 19:06:24.158639 664902 out.go:177] * Using the docker driver based on user configuration
I0920 19:06:24.160998 664902 start.go:297] selected driver: docker
I0920 19:06:24.161025 664902 start.go:901] validating driver "docker" against <nil>
I0920 19:06:24.161040 664902 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0920 19:06:24.161745 664902 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0920 19:06:24.245711 664902 info.go:266] docker info: {ID:6ZPO:QZND:VNGE:LUKL:4Y3K:XELL:AAX4:2GTK:E6LM:MPRN:3ZXR:TTMR Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:35 OomKillDisable:true NGoroutines:53 SystemTime:2024-09-20 19:06:24.235109654 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-30-239 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0920 19:06:24.245906 664902 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0920 19:06:24.246125 664902 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0920 19:06:24.248524 664902 out.go:177] * Using Docker driver with root privileges
I0920 19:06:24.250629 664902 cni.go:84] Creating CNI manager for ""
I0920 19:06:24.250700 664902 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0920 19:06:24.250715 664902 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni
I0920 19:06:24.250801 664902 start.go:340] cluster config:
{Name:embed-certs-208780 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:embed-certs-208780 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Contain
erRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: Stati
cIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0920 19:06:24.253504 664902 out.go:177] * Starting "embed-certs-208780" primary control-plane node in "embed-certs-208780" cluster
I0920 19:06:24.255960 664902 cache.go:121] Beginning downloading kic base image for docker with containerd
I0920 19:06:24.258450 664902 out.go:177] * Pulling base image v0.0.45-1726589491-19662 ...
I0920 19:06:24.260698 664902 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0920 19:06:24.260757 664902 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19679-440039/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4
I0920 19:06:24.260767 664902 cache.go:56] Caching tarball of preloaded images
I0920 19:06:24.260859 664902 preload.go:172] Found /home/jenkins/minikube-integration/19679-440039/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0920 19:06:24.260867 664902 cache.go:59] Finished verifying existence of preloaded tar for v1.31.1 on containerd
I0920 19:06:24.260979 664902 profile.go:143] Saving config to /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/config.json ...
I0920 19:06:24.260996 664902 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/config.json: {Name:mk8dc3839c5a5d6438c6e752bef073719468e07d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:06:24.261085 664902 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 in local docker daemon
W0920 19:06:24.282706 664902 image.go:95] image gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 is of wrong architecture
I0920 19:06:24.282730 664902 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 to local cache
I0920 19:06:24.282830 664902 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 in local cache directory
I0920 19:06:24.282854 664902 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 in local cache directory, skipping pull
I0920 19:06:24.282860 664902 image.go:135] gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 exists in cache, skipping pull
I0920 19:06:24.282868 664902 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 as a tarball
I0920 19:06:24.282877 664902 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 from local cache
I0920 19:06:24.415846 664902 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 from cached tarball
I0920 19:06:24.415891 664902 cache.go:194] Successfully downloaded all kic artifacts
I0920 19:06:24.415921 664902 start.go:360] acquireMachinesLock for embed-certs-208780: {Name:mke615d6ad4ebf4a07dd1444d21acc29cd55def8 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0920 19:06:24.416532 664902 start.go:364] duration metric: took 586.007µs to acquireMachinesLock for "embed-certs-208780"
I0920 19:06:24.416571 664902 start.go:93] Provisioning new machine with config: &{Name:embed-certs-208780 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:embed-certs-208780 Namespace:default APIServerHAVIP: APIServe
rName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableM
etrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0920 19:06:24.416666 664902 start.go:125] createHost starting for "" (driver="docker")
I0920 19:06:24.419389 664902 out.go:235] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0920 19:06:24.419623 664902 start.go:159] libmachine.API.Create for "embed-certs-208780" (driver="docker")
I0920 19:06:24.419655 664902 client.go:168] LocalClient.Create starting
I0920 19:06:24.419729 664902 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca.pem
I0920 19:06:24.419772 664902 main.go:141] libmachine: Decoding PEM data...
I0920 19:06:24.419792 664902 main.go:141] libmachine: Parsing certificate...
I0920 19:06:24.419848 664902 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19679-440039/.minikube/certs/cert.pem
I0920 19:06:24.419876 664902 main.go:141] libmachine: Decoding PEM data...
I0920 19:06:24.419892 664902 main.go:141] libmachine: Parsing certificate...
I0920 19:06:24.420263 664902 cli_runner.go:164] Run: docker network inspect embed-certs-208780 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0920 19:06:24.436364 664902 cli_runner.go:211] docker network inspect embed-certs-208780 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0920 19:06:24.436465 664902 network_create.go:284] running [docker network inspect embed-certs-208780] to gather additional debugging logs...
I0920 19:06:24.436485 664902 cli_runner.go:164] Run: docker network inspect embed-certs-208780
W0920 19:06:24.451939 664902 cli_runner.go:211] docker network inspect embed-certs-208780 returned with exit code 1
I0920 19:06:24.451981 664902 network_create.go:287] error running [docker network inspect embed-certs-208780]: docker network inspect embed-certs-208780: exit status 1
stdout:
[]
stderr:
Error response from daemon: network embed-certs-208780 not found
I0920 19:06:24.451994 664902 network_create.go:289] output of [docker network inspect embed-certs-208780]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network embed-certs-208780 not found
** /stderr **
I0920 19:06:24.452127 664902 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0920 19:06:24.469274 664902 network.go:211] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName:br-cc05dbef80c5 IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:02:42:d9:4b:c6:6a} reservation:<nil>}
I0920 19:06:24.469705 664902 network.go:211] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName:br-fb0bc5b904d0 IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:02:42:6a:ff:0d:2f} reservation:<nil>}
I0920 19:06:24.470066 664902 network.go:211] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName:br-a2acba418518 IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:02:42:6d:1c:20:df} reservation:<nil>}
I0920 19:06:24.470562 664902 network.go:211] skipping subnet 192.168.76.0/24 that is taken: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName:br-d24ef9c80fc9 IfaceIPv4:192.168.76.1 IfaceMTU:1500 IfaceMAC:02:42:d6:2f:3a:7a} reservation:<nil>}
I0920 19:06:24.471075 664902 network.go:206] using free private subnet 192.168.85.0/24: &{IP:192.168.85.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.85.0/24 Gateway:192.168.85.1 ClientMin:192.168.85.2 ClientMax:192.168.85.254 Broadcast:192.168.85.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0x400189eab0}
I0920 19:06:24.471100 664902 network_create.go:124] attempt to create docker network embed-certs-208780 192.168.85.0/24 with gateway 192.168.85.1 and MTU of 1500 ...
I0920 19:06:24.471160 664902 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.85.0/24 --gateway=192.168.85.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=embed-certs-208780 embed-certs-208780
I0920 19:06:24.548862 664902 network_create.go:108] docker network embed-certs-208780 192.168.85.0/24 created
I0920 19:06:24.548899 664902 kic.go:121] calculated static IP "192.168.85.2" for the "embed-certs-208780" container
I0920 19:06:24.548988 664902 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0920 19:06:24.565515 664902 cli_runner.go:164] Run: docker volume create embed-certs-208780 --label name.minikube.sigs.k8s.io=embed-certs-208780 --label created_by.minikube.sigs.k8s.io=true
I0920 19:06:24.584142 664902 oci.go:103] Successfully created a docker volume embed-certs-208780
I0920 19:06:24.584232 664902 cli_runner.go:164] Run: docker run --rm --name embed-certs-208780-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-208780 --entrypoint /usr/bin/test -v embed-certs-208780:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 -d /var/lib
I0920 19:06:25.241011 664902 oci.go:107] Successfully prepared a docker volume embed-certs-208780
I0920 19:06:25.241080 664902 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0920 19:06:25.241102 664902 kic.go:194] Starting extracting preloaded images to volume ...
I0920 19:06:25.241185 664902 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/19679-440039/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-208780:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 -I lz4 -xf /preloaded.tar -C /extractDir
I0920 19:06:26.835053 653977 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0920 19:06:26.849195 653977 api_server.go:72] duration metric: took 5m52.870971077s to wait for apiserver process to appear ...
I0920 19:06:26.849217 653977 api_server.go:88] waiting for apiserver healthz status ...
I0920 19:06:26.849253 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0920 19:06:26.849308 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0920 19:06:26.897788 653977 cri.go:89] found id: "d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7"
I0920 19:06:26.897808 653977 cri.go:89] found id: "fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4"
I0920 19:06:26.897813 653977 cri.go:89] found id: ""
I0920 19:06:26.897821 653977 logs.go:276] 2 containers: [d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7 fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4]
I0920 19:06:26.897878 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:26.902338 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:26.906548 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0920 19:06:26.906623 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0920 19:06:26.958378 653977 cri.go:89] found id: "085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70"
I0920 19:06:26.958398 653977 cri.go:89] found id: "ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5"
I0920 19:06:26.958403 653977 cri.go:89] found id: ""
I0920 19:06:26.958411 653977 logs.go:276] 2 containers: [085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70 ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5]
I0920 19:06:26.958470 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:26.962667 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:26.966536 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0920 19:06:26.966653 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0920 19:06:27.027915 653977 cri.go:89] found id: "812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327"
I0920 19:06:27.027936 653977 cri.go:89] found id: "4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada"
I0920 19:06:27.027941 653977 cri.go:89] found id: ""
I0920 19:06:27.027948 653977 logs.go:276] 2 containers: [812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327 4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada]
I0920 19:06:27.028013 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.033278 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.037758 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0920 19:06:27.037875 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0920 19:06:27.093653 653977 cri.go:89] found id: "2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677"
I0920 19:06:27.093726 653977 cri.go:89] found id: "7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043"
I0920 19:06:27.093746 653977 cri.go:89] found id: ""
I0920 19:06:27.093775 653977 logs.go:276] 2 containers: [2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677 7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043]
I0920 19:06:27.093857 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.098536 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.102887 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0920 19:06:27.103005 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0920 19:06:27.176029 653977 cri.go:89] found id: "e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec"
I0920 19:06:27.176091 653977 cri.go:89] found id: "899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d"
I0920 19:06:27.176123 653977 cri.go:89] found id: ""
I0920 19:06:27.176152 653977 logs.go:276] 2 containers: [e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec 899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d]
I0920 19:06:27.176226 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.181026 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.187926 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0920 19:06:27.188048 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0920 19:06:27.235096 653977 cri.go:89] found id: "fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548"
I0920 19:06:27.235167 653977 cri.go:89] found id: "dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32"
I0920 19:06:27.235187 653977 cri.go:89] found id: ""
I0920 19:06:27.235212 653977 logs.go:276] 2 containers: [fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548 dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32]
I0920 19:06:27.235293 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.239445 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.243444 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0920 19:06:27.243570 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0920 19:06:27.296352 653977 cri.go:89] found id: "ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de"
I0920 19:06:27.296445 653977 cri.go:89] found id: "6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c"
I0920 19:06:27.296469 653977 cri.go:89] found id: ""
I0920 19:06:27.296495 653977 logs.go:276] 2 containers: [ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de 6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c]
I0920 19:06:27.296576 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.300837 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.304814 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0920 19:06:27.304929 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0920 19:06:27.366905 653977 cri.go:89] found id: "a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd"
I0920 19:06:27.366980 653977 cri.go:89] found id: ""
I0920 19:06:27.367003 653977 logs.go:276] 1 containers: [a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd]
I0920 19:06:27.367084 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.371726 653977 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0920 19:06:27.371855 653977 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0920 19:06:27.419397 653977 cri.go:89] found id: "78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6"
I0920 19:06:27.419472 653977 cri.go:89] found id: "558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f"
I0920 19:06:27.419518 653977 cri.go:89] found id: ""
I0920 19:06:27.419545 653977 logs.go:276] 2 containers: [78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6 558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f]
I0920 19:06:27.419629 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.423969 653977 ssh_runner.go:195] Run: which crictl
I0920 19:06:27.430101 653977 logs.go:123] Gathering logs for kube-proxy [e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec] ...
I0920 19:06:27.430174 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec"
I0920 19:06:27.488295 653977 logs.go:123] Gathering logs for kube-controller-manager [dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32] ...
I0920 19:06:27.488370 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32"
I0920 19:06:27.567691 653977 logs.go:123] Gathering logs for kube-apiserver [d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7] ...
I0920 19:06:27.567769 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7"
I0920 19:06:27.637441 653977 logs.go:123] Gathering logs for coredns [812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327] ...
I0920 19:06:27.637485 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327"
I0920 19:06:27.712024 653977 logs.go:123] Gathering logs for coredns [4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada] ...
I0920 19:06:27.712059 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada"
I0920 19:06:27.761492 653977 logs.go:123] Gathering logs for kube-scheduler [2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677] ...
I0920 19:06:27.761548 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677"
I0920 19:06:27.815260 653977 logs.go:123] Gathering logs for kube-proxy [899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d] ...
I0920 19:06:27.815292 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d"
I0920 19:06:27.868976 653977 logs.go:123] Gathering logs for kube-controller-manager [fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548] ...
I0920 19:06:27.869008 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548"
I0920 19:06:27.955237 653977 logs.go:123] Gathering logs for kindnet [6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c] ...
I0920 19:06:27.955275 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c"
I0920 19:06:28.009037 653977 logs.go:123] Gathering logs for kubernetes-dashboard [a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd] ...
I0920 19:06:28.009074 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd"
I0920 19:06:28.059987 653977 logs.go:123] Gathering logs for kube-apiserver [fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4] ...
I0920 19:06:28.060019 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4"
I0920 19:06:28.165966 653977 logs.go:123] Gathering logs for etcd [ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5] ...
I0920 19:06:28.166007 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5"
I0920 19:06:28.221462 653977 logs.go:123] Gathering logs for container status ...
I0920 19:06:28.221495 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0920 19:06:28.317786 653977 logs.go:123] Gathering logs for storage-provisioner [558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f] ...
I0920 19:06:28.317819 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f"
I0920 19:06:28.363443 653977 logs.go:123] Gathering logs for containerd ...
I0920 19:06:28.363472 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0920 19:06:28.429304 653977 logs.go:123] Gathering logs for kube-scheduler [7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043] ...
I0920 19:06:28.429342 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043"
I0920 19:06:28.511550 653977 logs.go:123] Gathering logs for dmesg ...
I0920 19:06:28.511585 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0920 19:06:28.530657 653977 logs.go:123] Gathering logs for etcd [085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70] ...
I0920 19:06:28.530687 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70"
I0920 19:06:28.612728 653977 logs.go:123] Gathering logs for kindnet [ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de] ...
I0920 19:06:28.612765 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de"
I0920 19:06:28.688022 653977 logs.go:123] Gathering logs for storage-provisioner [78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6] ...
I0920 19:06:28.688058 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6"
I0920 19:06:28.738857 653977 logs.go:123] Gathering logs for kubelet ...
I0920 19:06:28.738886 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0920 19:06:28.797990 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302392 664 reflector.go:138] object-"kube-system"/"kindnet-token-ll598": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-ll598" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.798246 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302467 664 reflector.go:138] object-"kube-system"/"storage-provisioner-token-5rgk7": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-5rgk7" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.798535 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302516 664 reflector.go:138] object-"kube-system"/"kube-proxy-token-2m84d": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-2m84d" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.798750 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302566 664 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.798966 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302642 664 reflector.go:138] object-"kube-system"/"coredns-token-tqf8k": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-tqf8k" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.799170 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302685 664 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.799394 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302728 664 reflector.go:138] object-"kube-system"/"metrics-server-token-fk86c": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-fk86c" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.799604 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:52 old-k8s-version-809747 kubelet[664]: E0920 19:00:52.302769 664 reflector.go:138] object-"default"/"default-token-gqwtk": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-gqwtk" is forbidden: User "system:node:old-k8s-version-809747" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-809747' and this object
W0920 19:06:28.808631 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:56 old-k8s-version-809747 kubelet[664]: E0920 19:00:56.277740 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:28.808831 653977 logs.go:138] Found kubelet problem: Sep 20 19:00:57 old-k8s-version-809747 kubelet[664]: E0920 19:00:57.075859 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.811689 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:08 old-k8s-version-809747 kubelet[664]: E0920 19:01:08.853318 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:28.813374 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:19 old-k8s-version-809747 kubelet[664]: E0920 19:01:19.861009 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.814320 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:22 old-k8s-version-809747 kubelet[664]: E0920 19:01:22.209550 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.814660 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:23 old-k8s-version-809747 kubelet[664]: E0920 19:01:23.208449 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.814996 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:24 old-k8s-version-809747 kubelet[664]: E0920 19:01:24.374198 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.815437 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:25 old-k8s-version-809747 kubelet[664]: E0920 19:01:25.231977 664 pod_workers.go:191] Error syncing pod fd544a9f-bcfa-4019-9bf3-71e1e9a2d2ae ("storage-provisioner_kube-system(fd544a9f-bcfa-4019-9bf3-71e1e9a2d2ae)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(fd544a9f-bcfa-4019-9bf3-71e1e9a2d2ae)"
W0920 19:06:28.817920 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:30 old-k8s-version-809747 kubelet[664]: E0920 19:01:30.851954 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:28.819220 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:38 old-k8s-version-809747 kubelet[664]: E0920 19:01:38.279749 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.819409 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:42 old-k8s-version-809747 kubelet[664]: E0920 19:01:42.842501 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.819748 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:44 old-k8s-version-809747 kubelet[664]: E0920 19:01:44.374858 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.820076 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:55 old-k8s-version-809747 kubelet[664]: E0920 19:01:55.842879 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.820260 653977 logs.go:138] Found kubelet problem: Sep 20 19:01:56 old-k8s-version-809747 kubelet[664]: E0920 19:01:56.842908 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.820577 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:08 old-k8s-version-809747 kubelet[664]: E0920 19:02:08.844439 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.821043 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:10 old-k8s-version-809747 kubelet[664]: E0920 19:02:10.389468 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.821373 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:14 old-k8s-version-809747 kubelet[664]: E0920 19:02:14.374714 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.823922 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:20 old-k8s-version-809747 kubelet[664]: E0920 19:02:20.851952 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:28.824256 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:26 old-k8s-version-809747 kubelet[664]: E0920 19:02:26.841902 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.824495 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:31 old-k8s-version-809747 kubelet[664]: E0920 19:02:31.842985 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.824826 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:41 old-k8s-version-809747 kubelet[664]: E0920 19:02:41.843033 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.825012 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:44 old-k8s-version-809747 kubelet[664]: E0920 19:02:44.846601 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.825614 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:54 old-k8s-version-809747 kubelet[664]: E0920 19:02:54.515666 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.825802 653977 logs.go:138] Found kubelet problem: Sep 20 19:02:59 old-k8s-version-809747 kubelet[664]: E0920 19:02:59.842434 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.826131 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:04 old-k8s-version-809747 kubelet[664]: E0920 19:03:04.374783 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.826352 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:11 old-k8s-version-809747 kubelet[664]: E0920 19:03:11.842669 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.826681 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:16 old-k8s-version-809747 kubelet[664]: E0920 19:03:16.841791 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.826866 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:22 old-k8s-version-809747 kubelet[664]: E0920 19:03:22.842284 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.827271 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:28 old-k8s-version-809747 kubelet[664]: E0920 19:03:28.842043 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.827472 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:33 old-k8s-version-809747 kubelet[664]: E0920 19:03:33.843004 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.827805 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:42 old-k8s-version-809747 kubelet[664]: E0920 19:03:42.841902 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.830249 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:48 old-k8s-version-809747 kubelet[664]: E0920 19:03:48.851175 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0920 19:06:28.830593 653977 logs.go:138] Found kubelet problem: Sep 20 19:03:54 old-k8s-version-809747 kubelet[664]: E0920 19:03:54.841890 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.830778 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:01 old-k8s-version-809747 kubelet[664]: E0920 19:04:01.843066 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.831103 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:08 old-k8s-version-809747 kubelet[664]: E0920 19:04:08.841968 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.831288 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:15 old-k8s-version-809747 kubelet[664]: E0920 19:04:15.842571 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.831935 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:21 old-k8s-version-809747 kubelet[664]: E0920 19:04:21.744371 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.832267 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:24 old-k8s-version-809747 kubelet[664]: E0920 19:04:24.377802 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.832453 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:29 old-k8s-version-809747 kubelet[664]: E0920 19:04:29.842421 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.832781 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:36 old-k8s-version-809747 kubelet[664]: E0920 19:04:36.841921 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.832965 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:44 old-k8s-version-809747 kubelet[664]: E0920 19:04:44.842363 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.833331 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:51 old-k8s-version-809747 kubelet[664]: E0920 19:04:51.843005 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.833519 653977 logs.go:138] Found kubelet problem: Sep 20 19:04:59 old-k8s-version-809747 kubelet[664]: E0920 19:04:59.842361 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.833846 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:02 old-k8s-version-809747 kubelet[664]: E0920 19:05:02.842297 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.834176 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:13 old-k8s-version-809747 kubelet[664]: E0920 19:05:13.842057 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.834367 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:14 old-k8s-version-809747 kubelet[664]: E0920 19:05:14.842283 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.834699 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:24 old-k8s-version-809747 kubelet[664]: E0920 19:05:24.842121 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.834882 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:28 old-k8s-version-809747 kubelet[664]: E0920 19:05:28.842363 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.835207 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:37 old-k8s-version-809747 kubelet[664]: E0920 19:05:37.842390 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.835394 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:43 old-k8s-version-809747 kubelet[664]: E0920 19:05:43.842490 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.835726 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:50 old-k8s-version-809747 kubelet[664]: E0920 19:05:50.841875 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.835911 653977 logs.go:138] Found kubelet problem: Sep 20 19:05:58 old-k8s-version-809747 kubelet[664]: E0920 19:05:58.842395 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.836240 653977 logs.go:138] Found kubelet problem: Sep 20 19:06:04 old-k8s-version-809747 kubelet[664]: E0920 19:06:04.841931 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.836424 653977 logs.go:138] Found kubelet problem: Sep 20 19:06:09 old-k8s-version-809747 kubelet[664]: E0920 19:06:09.842365 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:28.836754 653977 logs.go:138] Found kubelet problem: Sep 20 19:06:17 old-k8s-version-809747 kubelet[664]: E0920 19:06:17.842782 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:28.836938 653977 logs.go:138] Found kubelet problem: Sep 20 19:06:23 old-k8s-version-809747 kubelet[664]: E0920 19:06:23.842473 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0920 19:06:28.836948 653977 logs.go:123] Gathering logs for describe nodes ...
I0920 19:06:28.836964 653977 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0920 19:06:29.018737 653977 out.go:358] Setting ErrFile to fd 2...
I0920 19:06:29.018770 653977 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0920 19:06:29.018819 653977 out.go:270] X Problems detected in kubelet:
W0920 19:06:29.018834 653977 out.go:270] Sep 20 19:05:58 old-k8s-version-809747 kubelet[664]: E0920 19:05:58.842395 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:29.018842 653977 out.go:270] Sep 20 19:06:04 old-k8s-version-809747 kubelet[664]: E0920 19:06:04.841931 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:29.018857 653977 out.go:270] Sep 20 19:06:09 old-k8s-version-809747 kubelet[664]: E0920 19:06:09.842365 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0920 19:06:29.018863 653977 out.go:270] Sep 20 19:06:17 old-k8s-version-809747 kubelet[664]: E0920 19:06:17.842782 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
W0920 19:06:29.018869 653977 out.go:270] Sep 20 19:06:23 old-k8s-version-809747 kubelet[664]: E0920 19:06:23.842473 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0920 19:06:29.018875 653977 out.go:358] Setting ErrFile to fd 2...
I0920 19:06:29.018889 653977 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0920 19:06:30.550575 664902 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/19679-440039/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-208780:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 -I lz4 -xf /preloaded.tar -C /extractDir: (5.309348894s)
I0920 19:06:30.550612 664902 kic.go:203] duration metric: took 5.309506497s to extract preloaded images to volume ...
W0920 19:06:30.550774 664902 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0920 19:06:30.550900 664902 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0920 19:06:30.623085 664902 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname embed-certs-208780 --name embed-certs-208780 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-208780 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=embed-certs-208780 --network embed-certs-208780 --ip 192.168.85.2 --volume embed-certs-208780:/var --security-opt apparmor=unconfined --memory=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4
I0920 19:06:30.973508 664902 cli_runner.go:164] Run: docker container inspect embed-certs-208780 --format={{.State.Running}}
I0920 19:06:31.004886 664902 cli_runner.go:164] Run: docker container inspect embed-certs-208780 --format={{.State.Status}}
I0920 19:06:31.034518 664902 cli_runner.go:164] Run: docker exec embed-certs-208780 stat /var/lib/dpkg/alternatives/iptables
I0920 19:06:31.119343 664902 oci.go:144] the created container "embed-certs-208780" has a running status.
I0920 19:06:31.119370 664902 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/19679-440039/.minikube/machines/embed-certs-208780/id_rsa...
I0920 19:06:31.545715 664902 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/19679-440039/.minikube/machines/embed-certs-208780/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0920 19:06:31.575719 664902 cli_runner.go:164] Run: docker container inspect embed-certs-208780 --format={{.State.Status}}
I0920 19:06:31.601327 664902 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0920 19:06:31.601348 664902 kic_runner.go:114] Args: [docker exec --privileged embed-certs-208780 chown docker:docker /home/docker/.ssh/authorized_keys]
I0920 19:06:31.685548 664902 cli_runner.go:164] Run: docker container inspect embed-certs-208780 --format={{.State.Status}}
I0920 19:06:31.713655 664902 machine.go:93] provisionDockerMachine start ...
I0920 19:06:31.713747 664902 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-208780
I0920 19:06:31.740395 664902 main.go:141] libmachine: Using SSH client type: native
I0920 19:06:31.740669 664902 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33073 <nil> <nil>}
I0920 19:06:31.740679 664902 main.go:141] libmachine: About to run SSH command:
hostname
I0920 19:06:31.930610 664902 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-208780
I0920 19:06:31.930686 664902 ubuntu.go:169] provisioning hostname "embed-certs-208780"
I0920 19:06:31.930778 664902 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-208780
I0920 19:06:31.947730 664902 main.go:141] libmachine: Using SSH client type: native
I0920 19:06:31.947989 664902 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33073 <nil> <nil>}
I0920 19:06:31.948007 664902 main.go:141] libmachine: About to run SSH command:
sudo hostname embed-certs-208780 && echo "embed-certs-208780" | sudo tee /etc/hostname
I0920 19:06:32.125174 664902 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-208780
I0920 19:06:32.125278 664902 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-208780
I0920 19:06:32.155835 664902 main.go:141] libmachine: Using SSH client type: native
I0920 19:06:32.156086 664902 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33073 <nil> <nil>}
I0920 19:06:32.156109 664902 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sembed-certs-208780' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 embed-certs-208780/g' /etc/hosts;
else
echo '127.0.1.1 embed-certs-208780' | sudo tee -a /etc/hosts;
fi
fi
I0920 19:06:32.326095 664902 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0920 19:06:32.326169 664902 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19679-440039/.minikube CaCertPath:/home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19679-440039/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19679-440039/.minikube}
I0920 19:06:32.326216 664902 ubuntu.go:177] setting up certificates
I0920 19:06:32.326254 664902 provision.go:84] configureAuth start
I0920 19:06:32.326386 664902 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-208780
I0920 19:06:32.343993 664902 provision.go:143] copyHostCerts
I0920 19:06:32.344057 664902 exec_runner.go:144] found /home/jenkins/minikube-integration/19679-440039/.minikube/ca.pem, removing ...
I0920 19:06:32.344068 664902 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19679-440039/.minikube/ca.pem
I0920 19:06:32.344138 664902 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19679-440039/.minikube/ca.pem (1082 bytes)
I0920 19:06:32.344258 664902 exec_runner.go:144] found /home/jenkins/minikube-integration/19679-440039/.minikube/cert.pem, removing ...
I0920 19:06:32.344263 664902 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19679-440039/.minikube/cert.pem
I0920 19:06:32.344291 664902 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19679-440039/.minikube/cert.pem (1123 bytes)
I0920 19:06:32.344340 664902 exec_runner.go:144] found /home/jenkins/minikube-integration/19679-440039/.minikube/key.pem, removing ...
I0920 19:06:32.344344 664902 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19679-440039/.minikube/key.pem
I0920 19:06:32.344368 664902 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19679-440039/.minikube/key.pem (1675 bytes)
I0920 19:06:32.344410 664902 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19679-440039/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca-key.pem org=jenkins.embed-certs-208780 san=[127.0.0.1 192.168.85.2 embed-certs-208780 localhost minikube]
I0920 19:06:32.597656 664902 provision.go:177] copyRemoteCerts
I0920 19:06:32.597749 664902 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0920 19:06:32.597817 664902 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-208780
I0920 19:06:32.625994 664902 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33073 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/embed-certs-208780/id_rsa Username:docker}
I0920 19:06:32.731551 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0920 19:06:32.757393 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0920 19:06:32.783130 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0920 19:06:32.809505 664902 provision.go:87] duration metric: took 483.217164ms to configureAuth
I0920 19:06:32.809539 664902 ubuntu.go:193] setting minikube options for container-runtime
I0920 19:06:32.809743 664902 config.go:182] Loaded profile config "embed-certs-208780": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0920 19:06:32.809757 664902 machine.go:96] duration metric: took 1.096084635s to provisionDockerMachine
I0920 19:06:32.809765 664902 client.go:171] duration metric: took 8.390104244s to LocalClient.Create
I0920 19:06:32.809786 664902 start.go:167] duration metric: took 8.390163804s to libmachine.API.Create "embed-certs-208780"
I0920 19:06:32.809800 664902 start.go:293] postStartSetup for "embed-certs-208780" (driver="docker")
I0920 19:06:32.809810 664902 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0920 19:06:32.809871 664902 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0920 19:06:32.809916 664902 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-208780
I0920 19:06:32.828510 664902 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33073 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/embed-certs-208780/id_rsa Username:docker}
I0920 19:06:32.935752 664902 ssh_runner.go:195] Run: cat /etc/os-release
I0920 19:06:32.939055 664902 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0920 19:06:32.939088 664902 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0920 19:06:32.939099 664902 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0920 19:06:32.939105 664902 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I0920 19:06:32.939116 664902 filesync.go:126] Scanning /home/jenkins/minikube-integration/19679-440039/.minikube/addons for local assets ...
I0920 19:06:32.939174 664902 filesync.go:126] Scanning /home/jenkins/minikube-integration/19679-440039/.minikube/files for local assets ...
I0920 19:06:32.939249 664902 filesync.go:149] local asset: /home/jenkins/minikube-integration/19679-440039/.minikube/files/etc/ssl/certs/4467832.pem -> 4467832.pem in /etc/ssl/certs
I0920 19:06:32.939380 664902 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0920 19:06:32.948169 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/files/etc/ssl/certs/4467832.pem --> /etc/ssl/certs/4467832.pem (1708 bytes)
I0920 19:06:32.973379 664902 start.go:296] duration metric: took 163.563799ms for postStartSetup
I0920 19:06:32.973762 664902 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-208780
I0920 19:06:32.994600 664902 profile.go:143] Saving config to /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/config.json ...
I0920 19:06:32.994905 664902 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0920 19:06:32.994949 664902 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-208780
I0920 19:06:33.025672 664902 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33073 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/embed-certs-208780/id_rsa Username:docker}
I0920 19:06:33.123599 664902 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0920 19:06:33.128294 664902 start.go:128] duration metric: took 8.711607888s to createHost
I0920 19:06:33.128318 664902 start.go:83] releasing machines lock for "embed-certs-208780", held for 8.711768946s
I0920 19:06:33.128395 664902 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-208780
I0920 19:06:33.146094 664902 ssh_runner.go:195] Run: cat /version.json
I0920 19:06:33.146108 664902 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0920 19:06:33.146178 664902 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-208780
I0920 19:06:33.146185 664902 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-208780
I0920 19:06:33.174212 664902 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33073 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/embed-certs-208780/id_rsa Username:docker}
I0920 19:06:33.175387 664902 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33073 SSHKeyPath:/home/jenkins/minikube-integration/19679-440039/.minikube/machines/embed-certs-208780/id_rsa Username:docker}
I0920 19:06:33.398827 664902 ssh_runner.go:195] Run: systemctl --version
I0920 19:06:33.403169 664902 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0920 19:06:33.407901 664902 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0920 19:06:33.443225 664902 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0920 19:06:33.443308 664902 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0920 19:06:33.474284 664902 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0920 19:06:33.474335 664902 start.go:495] detecting cgroup driver to use...
I0920 19:06:33.474370 664902 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0920 19:06:33.474457 664902 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0920 19:06:33.487291 664902 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0920 19:06:33.499720 664902 docker.go:217] disabling cri-docker service (if available) ...
I0920 19:06:33.499793 664902 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0920 19:06:33.513362 664902 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0920 19:06:33.528563 664902 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0920 19:06:33.626585 664902 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0920 19:06:33.727572 664902 docker.go:233] disabling docker service ...
I0920 19:06:33.727684 664902 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0920 19:06:33.749421 664902 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0920 19:06:33.761630 664902 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0920 19:06:33.862377 664902 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0920 19:06:33.967231 664902 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0920 19:06:33.979718 664902 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0920 19:06:34.005001 664902 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0920 19:06:34.017091 664902 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0920 19:06:34.030939 664902 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0920 19:06:34.031031 664902 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0920 19:06:34.042377 664902 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0920 19:06:34.053914 664902 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0920 19:06:34.067214 664902 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0920 19:06:34.085553 664902 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0920 19:06:34.095782 664902 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0920 19:06:34.107830 664902 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0920 19:06:34.119042 664902 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0920 19:06:34.130947 664902 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0920 19:06:34.141108 664902 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0920 19:06:34.150505 664902 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0920 19:06:34.248601 664902 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0920 19:06:34.415895 664902 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0920 19:06:34.416010 664902 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0920 19:06:34.420153 664902 start.go:563] Will wait 60s for crictl version
I0920 19:06:34.420230 664902 ssh_runner.go:195] Run: which crictl
I0920 19:06:34.423945 664902 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0920 19:06:34.463906 664902 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.22
RuntimeApiVersion: v1
I0920 19:06:34.464024 664902 ssh_runner.go:195] Run: containerd --version
I0920 19:06:34.498929 664902 ssh_runner.go:195] Run: containerd --version
I0920 19:06:34.527133 664902 out.go:177] * Preparing Kubernetes v1.31.1 on containerd 1.7.22 ...
I0920 19:06:34.529542 664902 cli_runner.go:164] Run: docker network inspect embed-certs-208780 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0920 19:06:34.545775 664902 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0920 19:06:34.549616 664902 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0920 19:06:34.561254 664902 kubeadm.go:883] updating cluster {Name:embed-certs-208780 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:embed-certs-208780 Namespace:default APIServerHAVIP: APIServerName:minikubeCA AP
IServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:
false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0920 19:06:34.561388 664902 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0920 19:06:34.561467 664902 ssh_runner.go:195] Run: sudo crictl images --output json
I0920 19:06:34.600774 664902 containerd.go:627] all images are preloaded for containerd runtime.
I0920 19:06:34.600801 664902 containerd.go:534] Images already preloaded, skipping extraction
I0920 19:06:34.600865 664902 ssh_runner.go:195] Run: sudo crictl images --output json
I0920 19:06:34.653722 664902 containerd.go:627] all images are preloaded for containerd runtime.
I0920 19:06:34.653746 664902 cache_images.go:84] Images are preloaded, skipping loading
I0920 19:06:34.653756 664902 kubeadm.go:934] updating node { 192.168.85.2 8443 v1.31.1 containerd true true} ...
I0920 19:06:34.653854 664902 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=embed-certs-208780 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.31.1 ClusterName:embed-certs-208780 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0920 19:06:34.653922 664902 ssh_runner.go:195] Run: sudo crictl info
I0920 19:06:34.697411 664902 cni.go:84] Creating CNI manager for ""
I0920 19:06:34.697442 664902 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0920 19:06:34.697452 664902 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0920 19:06:34.697475 664902 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.31.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:embed-certs-208780 NodeName:embed-certs-208780 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPo
dPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0920 19:06:34.697611 664902 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "embed-certs-208780"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.31.1
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0920 19:06:34.697688 664902 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.1
I0920 19:06:34.707470 664902 binaries.go:44] Found k8s binaries, skipping transfer
I0920 19:06:34.707565 664902 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0920 19:06:34.716451 664902 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (322 bytes)
I0920 19:06:34.734556 664902 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0920 19:06:34.753320 664902 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2172 bytes)
I0920 19:06:34.772982 664902 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0920 19:06:34.776399 664902 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0920 19:06:34.787662 664902 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0920 19:06:34.892726 664902 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0920 19:06:34.911215 664902 certs.go:68] Setting up /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780 for IP: 192.168.85.2
I0920 19:06:34.911239 664902 certs.go:194] generating shared ca certs ...
I0920 19:06:34.911256 664902 certs.go:226] acquiring lock for ca certs: {Name:mk3d7fcf9ade00248d7372a8cec4403eeffc64da Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:06:34.911428 664902 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19679-440039/.minikube/ca.key
I0920 19:06:34.911484 664902 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19679-440039/.minikube/proxy-client-ca.key
I0920 19:06:34.911496 664902 certs.go:256] generating profile certs ...
I0920 19:06:34.911556 664902 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/client.key
I0920 19:06:34.911590 664902 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/client.crt with IP's: []
I0920 19:06:35.789067 664902 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/client.crt ...
I0920 19:06:35.789101 664902 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/client.crt: {Name:mkaf13b24f6bb8e22c5eeddae26e8a8b238a9099 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:06:35.789301 664902 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/client.key ...
I0920 19:06:35.789314 664902 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/client.key: {Name:mk91ebc337326469b242489facd16cdb4c96eea8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:06:35.789902 664902 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.key.ba0ee4f5
I0920 19:06:35.789928 664902 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.crt.ba0ee4f5 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.85.2]
I0920 19:06:36.246174 664902 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.crt.ba0ee4f5 ...
I0920 19:06:36.246211 664902 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.crt.ba0ee4f5: {Name:mkfb38bdb14b7ec9e1e42499a3a7f1b92deb0dce Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:06:36.246943 664902 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.key.ba0ee4f5 ...
I0920 19:06:36.246968 664902 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.key.ba0ee4f5: {Name:mk1d524948ec0c93c36f6b0f68a56cc40f6ef017 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:06:36.247069 664902 certs.go:381] copying /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.crt.ba0ee4f5 -> /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.crt
I0920 19:06:36.247166 664902 certs.go:385] copying /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.key.ba0ee4f5 -> /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.key
I0920 19:06:36.247230 664902 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/proxy-client.key
I0920 19:06:36.247249 664902 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/proxy-client.crt with IP's: []
I0920 19:06:36.533763 664902 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/proxy-client.crt ...
I0920 19:06:36.533795 664902 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/proxy-client.crt: {Name:mk350e26d87c6bd3e26fa1d118d6669ba756ef4b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:06:36.534561 664902 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/proxy-client.key ...
I0920 19:06:36.534582 664902 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/proxy-client.key: {Name:mk36132daa9b5f9a8d4745aeeb0757a67c2faef8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0920 19:06:36.535347 664902 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/446783.pem (1338 bytes)
W0920 19:06:36.535398 664902 certs.go:480] ignoring /home/jenkins/minikube-integration/19679-440039/.minikube/certs/446783_empty.pem, impossibly tiny 0 bytes
I0920 19:06:36.535417 664902 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca-key.pem (1679 bytes)
I0920 19:06:36.535451 664902 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/ca.pem (1082 bytes)
I0920 19:06:36.535482 664902 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/cert.pem (1123 bytes)
I0920 19:06:36.535510 664902 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/certs/key.pem (1675 bytes)
I0920 19:06:36.535558 664902 certs.go:484] found cert: /home/jenkins/minikube-integration/19679-440039/.minikube/files/etc/ssl/certs/4467832.pem (1708 bytes)
I0920 19:06:36.536565 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0920 19:06:36.567269 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0920 19:06:36.594605 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0920 19:06:36.622897 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0920 19:06:36.668089 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1428 bytes)
I0920 19:06:36.695643 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0920 19:06:36.724428 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0920 19:06:36.749945 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/profiles/embed-certs-208780/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0920 19:06:36.775722 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/files/etc/ssl/certs/4467832.pem --> /usr/share/ca-certificates/4467832.pem (1708 bytes)
I0920 19:06:36.801489 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0920 19:06:36.828250 664902 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19679-440039/.minikube/certs/446783.pem --> /usr/share/ca-certificates/446783.pem (1338 bytes)
I0920 19:06:36.856100 664902 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0920 19:06:36.875764 664902 ssh_runner.go:195] Run: openssl version
I0920 19:06:36.887189 664902 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0920 19:06:36.896849 664902 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0920 19:06:36.900407 664902 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 20 18:10 /usr/share/ca-certificates/minikubeCA.pem
I0920 19:06:36.900501 664902 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0920 19:06:36.907841 664902 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0920 19:06:36.917192 664902 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/446783.pem && ln -fs /usr/share/ca-certificates/446783.pem /etc/ssl/certs/446783.pem"
I0920 19:06:36.926633 664902 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/446783.pem
I0920 19:06:36.930503 664902 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 20 18:20 /usr/share/ca-certificates/446783.pem
I0920 19:06:36.930604 664902 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/446783.pem
I0920 19:06:36.938375 664902 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/446783.pem /etc/ssl/certs/51391683.0"
I0920 19:06:36.948370 664902 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4467832.pem && ln -fs /usr/share/ca-certificates/4467832.pem /etc/ssl/certs/4467832.pem"
I0920 19:06:36.958229 664902 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/4467832.pem
I0920 19:06:36.961885 664902 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 20 18:20 /usr/share/ca-certificates/4467832.pem
I0920 19:06:36.961972 664902 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/4467832.pem
I0920 19:06:36.969068 664902 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/4467832.pem /etc/ssl/certs/3ec20f2e.0"
I0920 19:06:36.978796 664902 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0920 19:06:36.982267 664902 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0920 19:06:36.982381 664902 kubeadm.go:392] StartCluster: {Name:embed-certs-208780 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726589491-19662@sha256:6370b9fec173944088c2d87d44b01819c0ec611a83d9e2f38d36352dff8121a4 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:embed-certs-208780 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISe
rverNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:fal
se CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0920 19:06:36.982480 664902 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0920 19:06:36.982550 664902 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0920 19:06:37.036302 664902 cri.go:89] found id: ""
I0920 19:06:37.036436 664902 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0920 19:06:37.055766 664902 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0920 19:06:37.066495 664902 kubeadm.go:214] ignoring SystemVerification for kubeadm because of docker driver
I0920 19:06:37.066616 664902 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0920 19:06:37.082378 664902 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0920 19:06:37.082398 664902 kubeadm.go:157] found existing configuration files:
I0920 19:06:37.082451 664902 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0920 19:06:37.092289 664902 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0920 19:06:37.092374 664902 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0920 19:06:37.101310 664902 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0920 19:06:37.110809 664902 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0920 19:06:37.110883 664902 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0920 19:06:37.119690 664902 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0920 19:06:37.128499 664902 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0920 19:06:37.128575 664902 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0920 19:06:37.137584 664902 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0920 19:06:37.147315 664902 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0920 19:06:37.147407 664902 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0920 19:06:37.158144 664902 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.31.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0920 19:06:37.209270 664902 kubeadm.go:310] [init] Using Kubernetes version: v1.31.1
I0920 19:06:37.209552 664902 kubeadm.go:310] [preflight] Running pre-flight checks
I0920 19:06:37.231267 664902 kubeadm.go:310] [preflight] The system verification failed. Printing the output from the verification:
I0920 19:06:37.231381 664902 kubeadm.go:310] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1070-aws[0m
I0920 19:06:37.231434 664902 kubeadm.go:310] [0;37mOS[0m: [0;32mLinux[0m
I0920 19:06:37.231505 664902 kubeadm.go:310] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0920 19:06:37.231574 664902 kubeadm.go:310] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0920 19:06:37.231646 664902 kubeadm.go:310] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0920 19:06:37.231716 664902 kubeadm.go:310] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0920 19:06:37.231785 664902 kubeadm.go:310] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0920 19:06:37.231851 664902 kubeadm.go:310] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0920 19:06:37.231911 664902 kubeadm.go:310] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0920 19:06:37.231982 664902 kubeadm.go:310] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0920 19:06:37.232051 664902 kubeadm.go:310] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0920 19:06:37.295290 664902 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0920 19:06:37.295484 664902 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0920 19:06:37.295633 664902 kubeadm.go:310] [preflight] You can also perform this action beforehand using 'kubeadm config images pull'
I0920 19:06:37.301442 664902 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0920 19:06:37.306077 664902 out.go:235] - Generating certificates and keys ...
I0920 19:06:37.306196 664902 kubeadm.go:310] [certs] Using existing ca certificate authority
I0920 19:06:37.306280 664902 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0920 19:06:38.111076 664902 kubeadm.go:310] [certs] Generating "apiserver-kubelet-client" certificate and key
I0920 19:06:39.019670 653977 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0920 19:06:39.029993 653977 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0920 19:06:39.032174 653977 out.go:201]
W0920 19:06:39.034164 653977 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0920 19:06:39.034205 653977 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0920 19:06:39.034237 653977 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0920 19:06:39.034250 653977 out.go:270] *
W0920 19:06:39.035133 653977 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0920 19:06:39.038378 653977 out.go:201]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
0c98e7f3d0165 523cad1a4df73 2 minutes ago Exited dashboard-metrics-scraper 5 8cc227382c509 dashboard-metrics-scraper-8d5bb5db8-z27f2
78dbadd374e40 ba04bb24b9575 5 minutes ago Running storage-provisioner 2 c018a1a078bd2 storage-provisioner
a29e29bd825c6 20b332c9a70d8 5 minutes ago Running kubernetes-dashboard 0 0892bdde55047 kubernetes-dashboard-cd95d586-xd568
e7bf621eab17a 25a5233254979 5 minutes ago Running kube-proxy 1 0542638a6c1df kube-proxy-tczmb
66c9324267027 1611cd07b61d5 5 minutes ago Running busybox 1 34d3dc3be77ec busybox
558af720d1a83 ba04bb24b9575 5 minutes ago Exited storage-provisioner 1 c018a1a078bd2 storage-provisioner
ad9741225d44e 6a23fa8fd2b78 5 minutes ago Running kindnet-cni 1 c2eaa1091bbec kindnet-jz4sz
812bd46cee424 db91994f4ee8f 5 minutes ago Running coredns 1 f5bbeb3f1aedd coredns-74ff55c5b-682lc
d106efe91d320 2c08bbbc02d3a 5 minutes ago Running kube-apiserver 1 4f4d1de53f75c kube-apiserver-old-k8s-version-809747
2097a062acf82 e7605f88f17d6 5 minutes ago Running kube-scheduler 1 27cd1cb2c4322 kube-scheduler-old-k8s-version-809747
fc97c68baab83 1df8a2b116bd1 5 minutes ago Running kube-controller-manager 1 1a6665d996978 kube-controller-manager-old-k8s-version-809747
085be7ace136b 05b738aa1bc63 5 minutes ago Running etcd 1 fbbd0651c1fd5 etcd-old-k8s-version-809747
1b2fc193df81b 1611cd07b61d5 6 minutes ago Exited busybox 0 2b6e75cb48369 busybox
4142a2b627abf db91994f4ee8f 8 minutes ago Exited coredns 0 32f01c3ad0248 coredns-74ff55c5b-682lc
6451e639fa627 6a23fa8fd2b78 8 minutes ago Exited kindnet-cni 0 7da1fe9063ba6 kindnet-jz4sz
899a291eb59c3 25a5233254979 8 minutes ago Exited kube-proxy 0 b59b66a8f5546 kube-proxy-tczmb
ffeafa6ea9046 05b738aa1bc63 8 minutes ago Exited etcd 0 02797dcea4028 etcd-old-k8s-version-809747
dc58bda86f634 1df8a2b116bd1 8 minutes ago Exited kube-controller-manager 0 0b3190b6a78c4 kube-controller-manager-old-k8s-version-809747
7407be95357aa e7605f88f17d6 8 minutes ago Exited kube-scheduler 0 f054b3622c6e4 kube-scheduler-old-k8s-version-809747
fb7f2e4033f4a 2c08bbbc02d3a 8 minutes ago Exited kube-apiserver 0 8d1f1f6b3d8b1 kube-apiserver-old-k8s-version-809747
==> containerd <==
Sep 20 19:02:53 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:02:53.864136338Z" level=info msg="CreateContainer within sandbox \"8cc227382c50915e7f055d439af58609e29f9eb1eb4c3361bf5d28b10af94b2d\" for name:\"dashboard-metrics-scraper\" attempt:4 returns container id \"c3b46354ae0e4139240c56942b5c41741ba0ec9f8b25200c2ed7d08557ef6059\""
Sep 20 19:02:53 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:02:53.866262159Z" level=info msg="StartContainer for \"c3b46354ae0e4139240c56942b5c41741ba0ec9f8b25200c2ed7d08557ef6059\""
Sep 20 19:02:53 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:02:53.931454539Z" level=info msg="StartContainer for \"c3b46354ae0e4139240c56942b5c41741ba0ec9f8b25200c2ed7d08557ef6059\" returns successfully"
Sep 20 19:02:53 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:02:53.972353728Z" level=info msg="shim disconnected" id=c3b46354ae0e4139240c56942b5c41741ba0ec9f8b25200c2ed7d08557ef6059 namespace=k8s.io
Sep 20 19:02:53 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:02:53.972414010Z" level=warning msg="cleaning up after shim disconnected" id=c3b46354ae0e4139240c56942b5c41741ba0ec9f8b25200c2ed7d08557ef6059 namespace=k8s.io
Sep 20 19:02:53 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:02:53.972424439Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 20 19:02:54 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:02:54.516969943Z" level=info msg="RemoveContainer for \"95c2e7b27182cdc6e01c35a85eba6e2564708544309aea715b3cf36f516b1b84\""
Sep 20 19:02:54 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:02:54.523495193Z" level=info msg="RemoveContainer for \"95c2e7b27182cdc6e01c35a85eba6e2564708544309aea715b3cf36f516b1b84\" returns successfully"
Sep 20 19:03:48 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:03:48.842887541Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:03:48 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:03:48.848957132Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" host=fake.domain
Sep 20 19:03:48 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:03:48.850587259Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Sep 20 19:03:48 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:03:48.850723332Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
Sep 20 19:04:20 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:04:20.843997888Z" level=info msg="CreateContainer within sandbox \"8cc227382c50915e7f055d439af58609e29f9eb1eb4c3361bf5d28b10af94b2d\" for container name:\"dashboard-metrics-scraper\" attempt:5"
Sep 20 19:04:20 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:04:20.867116428Z" level=info msg="CreateContainer within sandbox \"8cc227382c50915e7f055d439af58609e29f9eb1eb4c3361bf5d28b10af94b2d\" for name:\"dashboard-metrics-scraper\" attempt:5 returns container id \"0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f\""
Sep 20 19:04:20 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:04:20.867924957Z" level=info msg="StartContainer for \"0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f\""
Sep 20 19:04:20 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:04:20.947707575Z" level=info msg="StartContainer for \"0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f\" returns successfully"
Sep 20 19:04:20 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:04:20.973908303Z" level=info msg="shim disconnected" id=0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f namespace=k8s.io
Sep 20 19:04:20 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:04:20.973994522Z" level=warning msg="cleaning up after shim disconnected" id=0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f namespace=k8s.io
Sep 20 19:04:20 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:04:20.974009661Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 20 19:04:21 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:04:21.745976313Z" level=info msg="RemoveContainer for \"c3b46354ae0e4139240c56942b5c41741ba0ec9f8b25200c2ed7d08557ef6059\""
Sep 20 19:04:21 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:04:21.752798180Z" level=info msg="RemoveContainer for \"c3b46354ae0e4139240c56942b5c41741ba0ec9f8b25200c2ed7d08557ef6059\" returns successfully"
Sep 20 19:06:34 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:06:34.842969744Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:06:34 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:06:34.885764031Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" host=fake.domain
Sep 20 19:06:34 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:06:34.887397319Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Sep 20 19:06:34 old-k8s-version-809747 containerd[571]: time="2024-09-20T19:06:34.887543115Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
==> coredns [4142a2b627abf853d590ea7ce75f7deda0b722cae15d402d8283d5eba06bcada] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:58376 - 31684 "HINFO IN 3451975236257540435.8144399278605540589. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.010957265s
==> coredns [812bd46cee424729eb0e399a232c5b3f90764d593c9aca7c0f74a4afe099a327] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:40062 - 64181 "HINFO IN 6255402163476102179.5470227237908667855. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.011273756s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
I0920 19:01:24.567685 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-20 19:00:54.56723247 +0000 UTC m=+0.021312403) (total time: 30.000351087s):
Trace[2019727887]: [30.000351087s] [30.000351087s] END
E0920 19:01:24.567721 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0920 19:01:24.568109 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-20 19:00:54.566641065 +0000 UTC m=+0.020720998) (total time: 30.001451666s):
Trace[939984059]: [30.001451666s] [30.001451666s] END
E0920 19:01:24.568162 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0920 19:01:24.568290 1 trace.go:116] Trace[911902081]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-20 19:00:54.56765874 +0000 UTC m=+0.021738673) (total time: 30.00061547s):
Trace[911902081]: [30.00061547s] [30.00061547s] END
E0920 19:01:24.568305 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
==> describe nodes <==
Name: old-k8s-version-809747
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-809747
kubernetes.io/os=linux
minikube.k8s.io/commit=35d0eeb96573bd708dfd5c070da844e6f0fad78a
minikube.k8s.io/name=old-k8s-version-809747
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_09_20T18_58_10_0700
minikube.k8s.io/version=v1.34.0
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Fri, 20 Sep 2024 18:58:06 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-809747
AcquireTime: <unset>
RenewTime: Fri, 20 Sep 2024 19:06:34 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Fri, 20 Sep 2024 19:01:42 +0000 Fri, 20 Sep 2024 18:57:58 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Fri, 20 Sep 2024 19:01:42 +0000 Fri, 20 Sep 2024 18:57:58 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Fri, 20 Sep 2024 19:01:42 +0000 Fri, 20 Sep 2024 18:57:58 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Fri, 20 Sep 2024 19:01:42 +0000 Fri, 20 Sep 2024 18:58:25 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.76.2
Hostname: old-k8s-version-809747
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022296Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022296Ki
pods: 110
System Info:
Machine ID: f17185a013eb45b9ad4ddc75ddfefa8a
System UUID: b386f02c-96dd-4c82-b396-1fb15cc3eff5
Boot ID: cfeac633-1b4b-4878-a7d1-bdd76da68a0f
Kernel Version: 5.15.0-1070-aws
OS Image: Ubuntu 22.04.5 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.7.22
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m40s
kube-system coredns-74ff55c5b-682lc 100m (5%) 0 (0%) 70Mi (0%) 170Mi (2%) 8m16s
kube-system etcd-old-k8s-version-809747 100m (5%) 0 (0%) 100Mi (1%) 0 (0%) 8m23s
kube-system kindnet-jz4sz 100m (5%) 100m (5%) 50Mi (0%) 50Mi (0%) 8m16s
kube-system kube-apiserver-old-k8s-version-809747 250m (12%) 0 (0%) 0 (0%) 0 (0%) 8m23s
kube-system kube-controller-manager-old-k8s-version-809747 200m (10%) 0 (0%) 0 (0%) 0 (0%) 8m23s
kube-system kube-proxy-tczmb 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m16s
kube-system kube-scheduler-old-k8s-version-809747 100m (5%) 0 (0%) 0 (0%) 0 (0%) 8m23s
kube-system metrics-server-9975d5f86-bx26v 100m (5%) 0 (0%) 200Mi (2%) 0 (0%) 6m29s
kube-system storage-provisioner 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m15s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-z27f2 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m31s
kubernetes-dashboard kubernetes-dashboard-cd95d586-xd568 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m31s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%) 100m (5%)
memory 420Mi (5%) 220Mi (2%)
ephemeral-storage 100Mi (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
hugepages-32Mi 0 (0%) 0 (0%)
hugepages-64Ki 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m44s (x5 over 8m44s) kubelet Node old-k8s-version-809747 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m44s (x5 over 8m44s) kubelet Node old-k8s-version-809747 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m44s (x4 over 8m44s) kubelet Node old-k8s-version-809747 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m44s kubelet Updated Node Allocatable limit across pods
Normal Starting 8m23s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m23s kubelet Node old-k8s-version-809747 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m23s kubelet Node old-k8s-version-809747 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m23s kubelet Node old-k8s-version-809747 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m23s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m16s kubelet Node old-k8s-version-809747 status is now: NodeReady
Normal Starting 8m15s kube-proxy Starting kube-proxy.
Normal Starting 6m kubelet Starting kubelet.
Normal NodeHasSufficientMemory 6m (x7 over 6m) kubelet Node old-k8s-version-809747 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 6m (x8 over 6m) kubelet Node old-k8s-version-809747 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 6m (x8 over 6m) kubelet Node old-k8s-version-809747 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 6m kubelet Updated Node Allocatable limit across pods
Normal Starting 5m45s kube-proxy Starting kube-proxy.
==> dmesg <==
[Sep20 17:41] kmem.limit_in_bytes is deprecated and will be removed. Please report your usecase to linux-mm@kvack.org if you depend on this functionality.
[Sep20 17:43] overlayfs: '/var/lib/docker/overlay2/l/WCSESTVH3U25P3IZ5LAJCY2BWZ' not a directory
[ +0.012326] overlayfs: '/var/lib/docker/overlay2/l/WCSESTVH3U25P3IZ5LAJCY2BWZ' not a directory
[ +0.005861] overlayfs: '/var/lib/docker/overlay2/l/WCSESTVH3U25P3IZ5LAJCY2BWZ' not a directory
[ +0.189191] overlayfs: '/var/lib/docker/overlay2/l/WCSESTVH3U25P3IZ5LAJCY2BWZ' not a directory
[Sep20 18:22] FS-Cache: Duplicate cookie detected
[ +0.000753] FS-Cache: O-cookie c=00000025 [p=00000002 fl=222 nc=0 na=1]
[ +0.001031] FS-Cache: O-cookie d=000000007b04e949{9P.session} n=00000000fd4f4036
[ +0.001114] FS-Cache: O-key=[10] '34323936373734333137'
[ +0.000820] FS-Cache: N-cookie c=00000026 [p=00000002 fl=2 nc=0 na=1]
[ +0.000956] FS-Cache: N-cookie d=000000007b04e949{9P.session} n=00000000b2f1fccb
[ +0.001112] FS-Cache: N-key=[10] '34323936373734333137'
==> etcd [085be7ace136b801ec32172b7f7a4a18032b49fc30e4fc31c9e16a3aeb8fcf70] <==
2024-09-20 19:02:38.463396 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:02:48.463039 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:02:58.463234 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:03:08.463436 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:03:18.463109 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:03:28.463341 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:03:38.463037 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:03:48.463049 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:03:58.463212 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:04:08.463047 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:04:18.463265 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:04:28.463063 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:04:38.463080 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:04:48.463058 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:04:58.463158 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:05:08.463227 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:05:18.463221 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:05:28.463039 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:05:38.463084 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:05:48.463153 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:05:58.463130 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:06:08.463227 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:06:18.463358 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:06:28.463920 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:06:38.463269 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [ffeafa6ea90467e9aee01c15062a10d7e7d8deb2522a37b559544d399360d4c5] <==
2024-09-20 18:57:58.319664 I | etcdserver/membership: added member ea7e25599daad906 [https://192.168.76.2:2380] to cluster 6f20f2c4b2fb5f8a
raft2024/09/20 18:57:58 INFO: ea7e25599daad906 is starting a new election at term 1
raft2024/09/20 18:57:58 INFO: ea7e25599daad906 became candidate at term 2
raft2024/09/20 18:57:58 INFO: ea7e25599daad906 received MsgVoteResp from ea7e25599daad906 at term 2
raft2024/09/20 18:57:58 INFO: ea7e25599daad906 became leader at term 2
raft2024/09/20 18:57:58 INFO: raft.node: ea7e25599daad906 elected leader ea7e25599daad906 at term 2
2024-09-20 18:57:58.459115 I | etcdserver: setting up the initial cluster version to 3.4
2024-09-20 18:57:58.462504 N | etcdserver/membership: set the initial cluster version to 3.4
2024-09-20 18:57:58.462713 I | etcdserver/api: enabled capabilities for version 3.4
2024-09-20 18:57:58.462815 I | etcdserver: published {Name:old-k8s-version-809747 ClientURLs:[https://192.168.76.2:2379]} to cluster 6f20f2c4b2fb5f8a
2024-09-20 18:57:58.463075 I | embed: ready to serve client requests
2024-09-20 18:57:58.464480 I | embed: serving client requests on 192.168.76.2:2379
2024-09-20 18:57:58.467163 I | embed: ready to serve client requests
2024-09-20 18:57:58.474415 I | embed: serving client requests on 127.0.0.1:2379
2024-09-20 18:58:26.366339 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 18:58:35.058489 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 18:58:45.059599 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 18:58:55.058669 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 18:59:05.058396 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 18:59:15.059362 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 18:59:25.058986 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 18:59:35.058458 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 18:59:45.060014 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 18:59:55.058412 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-20 19:00:05.058623 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
19:06:41 up 2:49, 0 users, load average: 1.47, 1.91, 2.45
Linux old-k8s-version-809747 5.15.0-1070-aws #76~20.04.1-Ubuntu SMP Mon Sep 2 12:20:48 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.5 LTS"
==> kindnet [6451e639fa627b6f7eb0c0d6d43b8de7a1c9297b75f90eba2d66578a806ba13c] <==
I0920 18:58:28.919196 1 controller.go:338] Waiting for informer caches to sync
I0920 18:58:28.919202 1 shared_informer.go:313] Waiting for caches to sync for kube-network-policies
I0920 18:58:29.119771 1 shared_informer.go:320] Caches are synced for kube-network-policies
I0920 18:58:29.119938 1 metrics.go:61] Registering metrics
I0920 18:58:29.120078 1 controller.go:374] Syncing nftables rules
I0920 18:58:38.923134 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 18:58:38.923219 1 main.go:299] handling current node
I0920 18:58:48.919500 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 18:58:48.919660 1 main.go:299] handling current node
I0920 18:58:58.923150 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 18:58:58.923187 1 main.go:299] handling current node
I0920 18:59:08.923788 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 18:59:08.923821 1 main.go:299] handling current node
I0920 18:59:18.919510 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 18:59:18.919544 1 main.go:299] handling current node
I0920 18:59:28.919214 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 18:59:28.919287 1 main.go:299] handling current node
I0920 18:59:38.920547 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 18:59:38.920610 1 main.go:299] handling current node
I0920 18:59:48.919128 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 18:59:48.919161 1 main.go:299] handling current node
I0920 18:59:58.919340 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 18:59:58.919374 1 main.go:299] handling current node
I0920 19:00:08.920547 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:00:08.920595 1 main.go:299] handling current node
==> kindnet [ad9741225d44ef3cd93a867e4aedc84a8ed23877c1b05cd8229cfb1260ced7de] <==
I0920 19:04:35.326438 1 main.go:299] handling current node
I0920 19:04:45.327348 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:04:45.327389 1 main.go:299] handling current node
I0920 19:04:55.319419 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:04:55.319516 1 main.go:299] handling current node
I0920 19:05:05.326486 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:05:05.326523 1 main.go:299] handling current node
I0920 19:05:15.327774 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:05:15.327811 1 main.go:299] handling current node
I0920 19:05:25.321140 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:05:25.321174 1 main.go:299] handling current node
I0920 19:05:35.323845 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:05:35.323887 1 main.go:299] handling current node
I0920 19:05:45.328019 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:05:45.328066 1 main.go:299] handling current node
I0920 19:05:55.319819 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:05:55.319859 1 main.go:299] handling current node
I0920 19:06:05.322564 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:06:05.322605 1 main.go:299] handling current node
I0920 19:06:15.328439 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:06:15.328479 1 main.go:299] handling current node
I0920 19:06:25.327952 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:06:25.327985 1 main.go:299] handling current node
I0920 19:06:35.322385 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0920 19:06:35.322622 1 main.go:299] handling current node
==> kube-apiserver [d106efe91d320b3b45d7444b13bc8682a69b480fd3d2c46ef29ca1522cc0dba7] <==
I0920 19:03:12.110812 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0920 19:03:12.110822 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0920 19:03:44.428924 1 client.go:360] parsed scheme: "passthrough"
I0920 19:03:44.428985 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0920 19:03:44.428997 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0920 19:03:55.142660 1 handler_proxy.go:102] no RequestInfo found in the context
E0920 19:03:55.142738 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0920 19:03:55.142750 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0920 19:04:19.661847 1 client.go:360] parsed scheme: "passthrough"
I0920 19:04:19.661917 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0920 19:04:19.661927 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0920 19:04:52.894511 1 client.go:360] parsed scheme: "passthrough"
I0920 19:04:52.894559 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0920 19:04:52.894568 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0920 19:05:35.123321 1 client.go:360] parsed scheme: "passthrough"
I0920 19:05:35.123366 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0920 19:05:35.123376 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0920 19:05:53.517025 1 handler_proxy.go:102] no RequestInfo found in the context
E0920 19:05:53.517256 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0920 19:05:53.517338 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0920 19:06:12.377995 1 client.go:360] parsed scheme: "passthrough"
I0920 19:06:12.378036 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0920 19:06:12.378043 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-apiserver [fb7f2e4033f4a4873ebbbd7a1613734b2ca70f9f331577110431f7b00029efa4] <==
I0920 18:58:07.575267 1 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
I0920 18:58:07.583589 1 storage_scheduling.go:132] created PriorityClass system-node-critical with value 2000001000
I0920 18:58:07.588477 1 storage_scheduling.go:132] created PriorityClass system-cluster-critical with value 2000000000
I0920 18:58:07.588505 1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.
I0920 18:58:08.108242 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0920 18:58:08.155074 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W0920 18:58:08.270414 1 lease.go:233] Resetting endpoints for master service "kubernetes" to [192.168.76.2]
I0920 18:58:08.271917 1 controller.go:606] quota admission added evaluator for: endpoints
I0920 18:58:08.281280 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0920 18:58:09.288209 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0920 18:58:09.856161 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0920 18:58:09.922591 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0920 18:58:18.336329 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I0920 18:58:25.263178 1 controller.go:606] quota admission added evaluator for: replicasets.apps
I0920 18:58:25.343867 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0920 18:58:37.185875 1 client.go:360] parsed scheme: "passthrough"
I0920 18:58:37.185919 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0920 18:58:37.185927 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0920 18:59:10.862907 1 client.go:360] parsed scheme: "passthrough"
I0920 18:59:10.862967 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0920 18:59:10.862985 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0920 18:59:51.032066 1 client.go:360] parsed scheme: "passthrough"
I0920 18:59:51.032353 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0920 18:59:51.032400 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
E0920 19:00:12.020370 1 available_controller.go:508] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
==> kube-controller-manager [dc58bda86f6344646d9480f42703769ac45b07cf183c47ae95ac99cf89959d32] <==
I0920 18:58:25.322989 1 event.go:291] "Event occurred" object="old-k8s-version-809747" kind="Node" apiVersion="v1" type="Normal" reason="RegisteredNode" message="Node old-k8s-version-809747 event: Registered Node old-k8s-version-809747 in Controller"
I0920 18:58:25.305436 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set coredns-74ff55c5b to 2"
I0920 18:58:25.306489 1 shared_informer.go:247] Caches are synced for daemon sets
I0920 18:58:25.323322 1 taint_manager.go:187] Starting NoExecuteTaintManager
I0920 18:58:25.306508 1 shared_informer.go:247] Caches are synced for ReplicaSet
I0920 18:58:25.306520 1 shared_informer.go:247] Caches are synced for PVC protection
I0920 18:58:25.353340 1 shared_informer.go:247] Caches are synced for persistent volume
I0920 18:58:25.379330 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-w5hf6"
I0920 18:58:25.435871 1 shared_informer.go:247] Caches are synced for disruption
I0920 18:58:25.436981 1 disruption.go:339] Sending events to api server.
I0920 18:58:25.443906 1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-tczmb"
I0920 18:58:25.447207 1 event.go:291] "Event occurred" object="kube-system/kindnet" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kindnet-jz4sz"
I0920 18:58:25.448088 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-682lc"
I0920 18:58:25.459430 1 shared_informer.go:247] Caches are synced for resource quota
I0920 18:58:25.467368 1 shared_informer.go:247] Caches are synced for resource quota
I0920 18:58:25.511897 1 shared_informer.go:247] Caches are synced for stateful set
I0920 18:58:25.631042 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
I0920 18:58:25.929062 1 shared_informer.go:247] Caches are synced for garbage collector
I0920 18:58:25.929084 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0920 18:58:25.931458 1 shared_informer.go:247] Caches are synced for garbage collector
I0920 18:58:27.051482 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0920 18:58:27.064304 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-w5hf6"
I0920 19:00:11.712339 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
I0920 19:00:11.754378 1 event.go:291] "Event occurred" object="kube-system/metrics-server-9975d5f86" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"metrics-server-9975d5f86-\" is forbidden: error looking up service account kube-system/metrics-server: serviceaccount \"metrics-server\" not found"
E0920 19:00:11.795508 1 replica_set.go:532] sync "kube-system/metrics-server-9975d5f86" failed with pods "metrics-server-9975d5f86-" is forbidden: error looking up service account kube-system/metrics-server: serviceaccount "metrics-server" not found
==> kube-controller-manager [fc97c68baab83ca4a8e34d1add491da97117d8436842e907b6c7ef5194a66548] <==
W0920 19:02:16.390635 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0920 19:02:42.442256 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0920 19:02:48.041136 1 request.go:655] Throttling request took 1.048520946s, request: GET:https://192.168.76.2:8443/apis/scheduling.k8s.io/v1?timeout=32s
W0920 19:02:48.892555 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0920 19:03:12.945151 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0920 19:03:20.542944 1 request.go:655] Throttling request took 1.048457213s, request: GET:https://192.168.76.2:8443/apis/authentication.k8s.io/v1beta1?timeout=32s
W0920 19:03:21.394510 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0920 19:03:43.447456 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0920 19:03:53.045095 1 request.go:655] Throttling request took 1.0483931s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0920 19:03:53.896534 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0920 19:04:13.949354 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0920 19:04:25.547186 1 request.go:655] Throttling request took 1.044016943s, request: GET:https://192.168.76.2:8443/apis/authentication.k8s.io/v1?timeout=32s
W0920 19:04:26.398779 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0920 19:04:44.451279 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0920 19:04:58.049356 1 request.go:655] Throttling request took 1.045209184s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0920 19:04:58.900909 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0920 19:05:14.953054 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0920 19:05:30.551377 1 request.go:655] Throttling request took 1.048296387s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0920 19:05:31.402942 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0920 19:05:45.455223 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0920 19:06:03.053502 1 request.go:655] Throttling request took 1.047596345s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0920 19:06:03.905223 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0920 19:06:15.957171 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0920 19:06:35.554918 1 request.go:655] Throttling request took 1.047976435s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0920 19:06:36.406840 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-proxy [899a291eb59c3b5f9d6b0939098b577edab6865605118eb9e26163895e023c2d] <==
I0920 18:58:26.463955 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0920 18:58:26.464047 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0920 18:58:26.538422 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0920 18:58:26.538524 1 server_others.go:185] Using iptables Proxier.
I0920 18:58:26.538750 1 server.go:650] Version: v1.20.0
I0920 18:58:26.539238 1 config.go:315] Starting service config controller
I0920 18:58:26.539251 1 shared_informer.go:240] Waiting for caches to sync for service config
I0920 18:58:26.541771 1 config.go:224] Starting endpoint slice config controller
I0920 18:58:26.541784 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0920 18:58:26.639574 1 shared_informer.go:247] Caches are synced for service config
I0920 18:58:26.642813 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-proxy [e7bf621eab17a86a9873dcbf28cace6f79a68a72ffa3d75afc890ecc389a86ec] <==
I0920 19:00:56.431360 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0920 19:00:56.431437 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0920 19:00:56.465636 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0920 19:00:56.465740 1 server_others.go:185] Using iptables Proxier.
I0920 19:00:56.466087 1 server.go:650] Version: v1.20.0
I0920 19:00:56.466680 1 config.go:224] Starting endpoint slice config controller
I0920 19:00:56.466698 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0920 19:00:56.466851 1 config.go:315] Starting service config controller
I0920 19:00:56.466884 1 shared_informer.go:240] Waiting for caches to sync for service config
I0920 19:00:56.566903 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0920 19:00:56.567101 1 shared_informer.go:247] Caches are synced for service config
==> kube-scheduler [2097a062acf828599a57c98a30c658af273f5825500a3bf75e77de707c622677] <==
I0920 19:00:48.717942 1 serving.go:331] Generated self-signed cert in-memory
W0920 19:00:52.119136 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0920 19:00:52.119452 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0920 19:00:52.119526 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0920 19:00:52.119553 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0920 19:00:52.375891 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0920 19:00:52.378959 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0920 19:00:52.378977 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0920 19:00:52.378994 1 tlsconfig.go:240] Starting DynamicServingCertificateController
E0920 19:00:52.522247 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0920 19:00:52.522379 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0920 19:00:52.522449 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0920 19:00:52.522516 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0920 19:00:52.522583 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0920 19:00:52.522648 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0920 19:00:52.522715 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0920 19:00:52.522768 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0920 19:00:52.522815 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0920 19:00:52.522874 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0920 19:00:52.522937 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0920 19:00:52.537023 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
I0920 19:00:54.179171 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [7407be95357aafd1f43bd30ec343dcf522e2b4e01f1263ebcbc21b337b0c8043] <==
W0920 18:58:06.791359 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0920 18:58:06.791365 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0920 18:58:06.853278 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
E0920 18:58:06.857116 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0920 18:58:06.859822 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0920 18:58:06.863002 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0920 18:58:06.863122 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
I0920 18:58:06.863258 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0920 18:58:06.863319 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0920 18:58:06.869065 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
E0920 18:58:06.866708 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0920 18:58:06.866849 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0920 18:58:06.866976 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0920 18:58:06.867098 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0920 18:58:06.867207 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0920 18:58:06.867341 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0920 18:58:06.871143 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0920 18:58:06.871617 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0920 18:58:07.702192 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0920 18:58:07.827774 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0920 18:58:07.871471 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0920 18:58:07.879312 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0920 18:58:07.915507 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0920 18:58:08.014284 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
I0920 18:58:10.169290 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Sep 20 19:05:13 old-k8s-version-809747 kubelet[664]: E0920 19:05:13.842057 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
Sep 20 19:05:14 old-k8s-version-809747 kubelet[664]: E0920 19:05:14.842283 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:05:24 old-k8s-version-809747 kubelet[664]: I0920 19:05:24.841742 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f
Sep 20 19:05:24 old-k8s-version-809747 kubelet[664]: E0920 19:05:24.842121 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
Sep 20 19:05:28 old-k8s-version-809747 kubelet[664]: E0920 19:05:28.842363 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:05:37 old-k8s-version-809747 kubelet[664]: I0920 19:05:37.841544 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f
Sep 20 19:05:37 old-k8s-version-809747 kubelet[664]: E0920 19:05:37.842390 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
Sep 20 19:05:43 old-k8s-version-809747 kubelet[664]: E0920 19:05:43.842490 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:05:50 old-k8s-version-809747 kubelet[664]: I0920 19:05:50.841534 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f
Sep 20 19:05:50 old-k8s-version-809747 kubelet[664]: E0920 19:05:50.841875 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
Sep 20 19:05:58 old-k8s-version-809747 kubelet[664]: E0920 19:05:58.842395 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:06:04 old-k8s-version-809747 kubelet[664]: I0920 19:06:04.841549 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f
Sep 20 19:06:04 old-k8s-version-809747 kubelet[664]: E0920 19:06:04.841931 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
Sep 20 19:06:09 old-k8s-version-809747 kubelet[664]: E0920 19:06:09.842365 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:06:17 old-k8s-version-809747 kubelet[664]: I0920 19:06:17.841709 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f
Sep 20 19:06:17 old-k8s-version-809747 kubelet[664]: E0920 19:06:17.842782 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
Sep 20 19:06:23 old-k8s-version-809747 kubelet[664]: E0920 19:06:23.842473 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 20 19:06:28 old-k8s-version-809747 kubelet[664]: I0920 19:06:28.841617 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f
Sep 20 19:06:28 old-k8s-version-809747 kubelet[664]: E0920 19:06:28.842755 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
Sep 20 19:06:34 old-k8s-version-809747 kubelet[664]: E0920 19:06:34.887923 664 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Sep 20 19:06:34 old-k8s-version-809747 kubelet[664]: E0920 19:06:34.887985 664 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Sep 20 19:06:34 old-k8s-version-809747 kubelet[664]: E0920 19:06:34.888127 664 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-fk86c,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exec
:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-bx26v_kube-system(c41b732
f-de87-4ddf-ba27-2a549da6b22f): ErrImagePull: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Sep 20 19:06:34 old-k8s-version-809747 kubelet[664]: E0920 19:06:34.888172 664 pod_workers.go:191] Error syncing pod c41b732f-de87-4ddf-ba27-2a549da6b22f ("metrics-server-9975d5f86-bx26v_kube-system(c41b732f-de87-4ddf-ba27-2a549da6b22f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Sep 20 19:06:39 old-k8s-version-809747 kubelet[664]: I0920 19:06:39.841643 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 0c98e7f3d016526c516965888a4391a3ed700f52c82b52dbf151c403a9d21a1f
Sep 20 19:06:39 old-k8s-version-809747 kubelet[664]: E0920 19:06:39.842231 664 pod_workers.go:191] Error syncing pod fdb51bf1-6934-4d9e-88fc-19e126a6cdf4 ("dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z27f2_kubernetes-dashboard(fdb51bf1-6934-4d9e-88fc-19e126a6cdf4)"
==> kubernetes-dashboard [a29e29bd825c67d29d71f1b611800e16571bd7e880bbb8077e23f09e4d6b05fd] <==
2024/09/20 19:01:15 Using namespace: kubernetes-dashboard
2024/09/20 19:01:15 Using in-cluster config to connect to apiserver
2024/09/20 19:01:15 Using secret token for csrf signing
2024/09/20 19:01:15 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/09/20 19:01:15 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/09/20 19:01:15 Successful initial request to the apiserver, version: v1.20.0
2024/09/20 19:01:15 Generating JWE encryption key
2024/09/20 19:01:15 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/09/20 19:01:15 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/09/20 19:01:16 Initializing JWE encryption key from synchronized object
2024/09/20 19:01:16 Creating in-cluster Sidecar client
2024/09/20 19:01:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/20 19:01:16 Serving insecurely on HTTP port: 9090
2024/09/20 19:01:46 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/20 19:02:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/20 19:02:46 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/20 19:03:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/20 19:03:46 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/20 19:04:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/20 19:04:46 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/20 19:05:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/20 19:05:46 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/20 19:06:16 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/20 19:01:15 Starting overwatch
==> storage-provisioner [558af720d1a83df5088daec078428c689fe95a5e71a61e7c6b36ca73bcbd321f] <==
I0920 19:00:54.775241 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0920 19:01:24.777314 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
==> storage-provisioner [78dbadd374e4006947e4c0a4b40642cfe3d16108cbdaad7fa8a335c2eadfccc6] <==
I0920 19:01:38.147440 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0920 19:01:38.203047 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0920 19:01:38.203364 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0920 19:01:55.695986 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0920 19:01:55.696652 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"f49aae3a-f8f8-49ab-a59f-7ad69dbaa272", APIVersion:"v1", ResourceVersion:"861", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-809747_269b0b69-c519-4c8c-9a14-93e6f0109a84 became leader
I0920 19:01:55.697220 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-809747_269b0b69-c519-4c8c-9a14-93e6f0109a84!
I0920 19:01:55.797845 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-809747_269b0b69-c519-4c8c-9a14-93e6f0109a84!
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-809747 -n old-k8s-version-809747
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-809747 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-bx26v
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-809747 describe pod metrics-server-9975d5f86-bx26v
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-809747 describe pod metrics-server-9975d5f86-bx26v: exit status 1 (119.78791ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-bx26v" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-809747 describe pod metrics-server-9975d5f86-bx26v: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (378.31s)