=== RUN TestMultiNode/serial/ValidateNameConflict
multinode_test.go:441: (dbg) Run: out/minikube-linux-amd64 node list -p multinode-20220630203821-10507
multinode_test.go:450: (dbg) Run: out/minikube-linux-amd64 start -p multinode-20220630203821-10507-m02 --driver=kvm2
multinode_test.go:450: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p multinode-20220630203821-10507-m02 --driver=kvm2 : exit status 14 (89.234039ms)
-- stdout --
* [multinode-20220630203821-10507-m02] minikube v1.26.0 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=14420
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
-- /stdout --
** stderr **
! Profile name 'multinode-20220630203821-10507-m02' is duplicated with machine name 'multinode-20220630203821-10507-m02' in profile 'multinode-20220630203821-10507'
X Exiting due to MK_USAGE: Profile name should be unique
** /stderr **
multinode_test.go:458: (dbg) Run: out/minikube-linux-amd64 start -p multinode-20220630203821-10507-m03 --driver=kvm2
E0630 21:08:14.193274 10507 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/addons-20220630202043-10507/client.crt: no such file or directory
multinode_test.go:458: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p multinode-20220630203821-10507-m03 --driver=kvm2 : signal: killed (28.287255648s)
-- stdout --
* [multinode-20220630203821-10507-m03] minikube v1.26.0 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=14420
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
* Using the kvm2 driver based on user configuration
* Starting control plane node multinode-20220630203821-10507-m03 in cluster multinode-20220630203821-10507-m03
* Creating kvm2 VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
-- /stdout --
multinode_test.go:460: failed to start profile. args "out/minikube-linux-amd64 start -p multinode-20220630203821-10507-m03 --driver=kvm2 " : signal: killed
multinode_test.go:465: (dbg) Run: out/minikube-linux-amd64 node add -p multinode-20220630203821-10507
multinode_test.go:465: (dbg) Non-zero exit: out/minikube-linux-amd64 node add -p multinode-20220630203821-10507: context deadline exceeded (1.098µs)
multinode_test.go:470: (dbg) Run: out/minikube-linux-amd64 delete -p multinode-20220630203821-10507-m03
multinode_test.go:470: (dbg) Non-zero exit: out/minikube-linux-amd64 delete -p multinode-20220630203821-10507-m03: context deadline exceeded (144ns)
multinode_test.go:472: failed to clean temporary profile. args "out/minikube-linux-amd64 delete -p multinode-20220630203821-10507-m03" : context deadline exceeded
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p multinode-20220630203821-10507 -n multinode-20220630203821-10507
helpers_test.go:244: <<< TestMultiNode/serial/ValidateNameConflict FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestMultiNode/serial/ValidateNameConflict]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-amd64 -p multinode-20220630203821-10507 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-amd64 -p multinode-20220630203821-10507 logs -n 25: (1.408472675s)
helpers_test.go:252: TestMultiNode/serial/ValidateNameConflict logs:
-- stdout --
*
* ==> Audit <==
* |---------|-----------------------------------------------------------------------------------------------------------------------------------|----------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|-----------------------------------------------------------------------------------------------------------------------------------|----------|---------|---------|---------------------|---------------------|
| cp | multinode-20220630203821-10507 cp multinode-20220630203821-10507-m02:/home/docker/cp-test.txt | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | multinode-20220630203821-10507-m03:/home/docker/cp-test_multinode-20220630203821-10507-m02_multinode-20220630203821-10507-m03.txt | | | | | |
| ssh | multinode-20220630203821-10507 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | ssh -n | | | | | |
| | multinode-20220630203821-10507-m02 | | | | | |
| | sudo cat /home/docker/cp-test.txt | | | | | |
| ssh | multinode-20220630203821-10507 ssh -n multinode-20220630203821-10507-m03 sudo cat | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | /home/docker/cp-test_multinode-20220630203821-10507-m02_multinode-20220630203821-10507-m03.txt | | | | | |
| cp | multinode-20220630203821-10507 cp testdata/cp-test.txt | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | multinode-20220630203821-10507-m03:/home/docker/cp-test.txt | | | | | |
| ssh | multinode-20220630203821-10507 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | ssh -n | | | | | |
| | multinode-20220630203821-10507-m03 | | | | | |
| | sudo cat /home/docker/cp-test.txt | | | | | |
| cp | multinode-20220630203821-10507 cp | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | multinode-20220630203821-10507-m03:/home/docker/cp-test.txt | | | | | |
| | /tmp/TestMultiNodeserialCopyFile699734933/001/cp-test_multinode-20220630203821-10507-m03.txt | | | | | |
| ssh | multinode-20220630203821-10507 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | ssh -n | | | | | |
| | multinode-20220630203821-10507-m03 | | | | | |
| | sudo cat /home/docker/cp-test.txt | | | | | |
| cp | multinode-20220630203821-10507 cp multinode-20220630203821-10507-m03:/home/docker/cp-test.txt | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | multinode-20220630203821-10507:/home/docker/cp-test_multinode-20220630203821-10507-m03_multinode-20220630203821-10507.txt | | | | | |
| ssh | multinode-20220630203821-10507 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | ssh -n | | | | | |
| | multinode-20220630203821-10507-m03 | | | | | |
| | sudo cat /home/docker/cp-test.txt | | | | | |
| ssh | multinode-20220630203821-10507 ssh -n multinode-20220630203821-10507 sudo cat | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | /home/docker/cp-test_multinode-20220630203821-10507-m03_multinode-20220630203821-10507.txt | | | | | |
| cp | multinode-20220630203821-10507 cp multinode-20220630203821-10507-m03:/home/docker/cp-test.txt | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | multinode-20220630203821-10507-m02:/home/docker/cp-test_multinode-20220630203821-10507-m03_multinode-20220630203821-10507-m02.txt | | | | | |
| ssh | multinode-20220630203821-10507 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | ssh -n | | | | | |
| | multinode-20220630203821-10507-m03 | | | | | |
| | sudo cat /home/docker/cp-test.txt | | | | | |
| ssh | multinode-20220630203821-10507 ssh -n multinode-20220630203821-10507-m02 sudo cat | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | /home/docker/cp-test_multinode-20220630203821-10507-m03_multinode-20220630203821-10507-m02.txt | | | | | |
| node | multinode-20220630203821-10507 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | node stop m03 | | | | | |
| node | multinode-20220630203821-10507 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:42 UTC |
| | node start m03 | | | | | |
| | --alsologtostderr | | | | | |
| node | list -p | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | |
| | multinode-20220630203821-10507 | | | | | |
| stop | -p | minikube | jenkins | v1.26.0 | 30 Jun 22 20:42 UTC | 30 Jun 22 20:43 UTC |
| | multinode-20220630203821-10507 | | | | | |
| start | -p | minikube | jenkins | v1.26.0 | 30 Jun 22 20:43 UTC | 30 Jun 22 20:57 UTC |
| | multinode-20220630203821-10507 | | | | | |
| | --wait=true -v=8 | | | | | |
| | --alsologtostderr | | | | | |
| node | list -p | minikube | jenkins | v1.26.0 | 30 Jun 22 20:57 UTC | |
| | multinode-20220630203821-10507 | | | | | |
| node | multinode-20220630203821-10507 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:57 UTC | 30 Jun 22 20:57 UTC |
| | node delete m03 | | | | | |
| stop | multinode-20220630203821-10507 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:57 UTC | 30 Jun 22 20:57 UTC |
| | stop | | | | | |
| start | -p | minikube | jenkins | v1.26.0 | 30 Jun 22 20:57 UTC | 30 Jun 22 21:07 UTC |
| | multinode-20220630203821-10507 | | | | | |
| | --wait=true -v=8 | | | | | |
| | --alsologtostderr | | | | | |
| | --driver=kvm2 | | | | | |
| node | list -p | minikube | jenkins | v1.26.0 | 30 Jun 22 21:07 UTC | |
| | multinode-20220630203821-10507 | | | | | |
| start | -p | minikube | jenkins | v1.26.0 | 30 Jun 22 21:07 UTC | |
| | multinode-20220630203821-10507-m02 | | | | | |
| | --driver=kvm2 | | | | | |
| start | -p | minikube | jenkins | v1.26.0 | 30 Jun 22 21:07 UTC | |
| | multinode-20220630203821-10507-m03 | | | | | |
| | --driver=kvm2 | | | | | |
|---------|-----------------------------------------------------------------------------------------------------------------------------------|----------|---------|---------|---------------------|---------------------|
*
* ==> Last Start <==
* Log file created at: 2022/06/30 21:07:52
Running on machine: ubuntu-20-agent-4
Binary: Built with gc go1.18.3 for linux/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0630 21:07:52.914508 22302 out.go:296] Setting OutFile to fd 1 ...
I0630 21:07:52.914830 22302 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 21:07:52.914837 22302 out.go:309] Setting ErrFile to fd 2...
I0630 21:07:52.914842 22302 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 21:07:52.915288 22302 root.go:329] Updating PATH: /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/bin
I0630 21:07:52.915535 22302 out.go:303] Setting JSON to false
I0630 21:07:52.916392 22302 start.go:115] hostinfo: {"hostname":"ubuntu-20-agent-4","uptime":3026,"bootTime":1656620247,"procs":189,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1012-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0630 21:07:52.916445 22302 start.go:125] virtualization: kvm guest
I0630 21:07:52.919295 22302 out.go:177] * [multinode-20220630203821-10507-m03] minikube v1.26.0 on Ubuntu 20.04 (kvm/amd64)
I0630 21:07:52.921195 22302 out.go:177] - MINIKUBE_LOCATION=14420
I0630 21:07:52.921138 22302 notify.go:193] Checking for updates...
I0630 21:07:52.922914 22302 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0630 21:07:52.924714 22302 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig
I0630 21:07:52.926483 22302 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube
I0630 21:07:52.928127 22302 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0630 21:07:52.930208 22302 config.go:178] Loaded profile config "multinode-20220630203821-10507": Driver=kvm2, ContainerRuntime=docker, KubernetesVersion=v1.24.2
I0630 21:07:52.930268 22302 driver.go:360] Setting default libvirt URI to qemu:///system
I0630 21:07:52.965234 22302 out.go:177] * Using the kvm2 driver based on user configuration
I0630 21:07:52.966827 22302 start.go:284] selected driver: kvm2
I0630 21:07:52.966840 22302 start.go:808] validating driver "kvm2" against <nil>
I0630 21:07:52.966870 22302 start.go:819] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0630 21:07:52.967220 22302 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0630 21:07:52.967415 22302 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/bin:/home/jenkins/workspace/KVM_Linux_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0630 21:07:52.981943 22302 install.go:137] /home/jenkins/workspace/KVM_Linux_integration/out/docker-machine-driver-kvm2 version is 1.26.0
I0630 21:07:52.982002 22302 start_flags.go:296] no existing cluster config was found, will generate one from the flags
I0630 21:07:52.982478 22302 start_flags.go:377] Using suggested 6000MB memory alloc based on sys=32103MB, container=0MB
I0630 21:07:52.982573 22302 start_flags.go:835] Wait components to verify : map[apiserver:true system_pods:true]
I0630 21:07:52.982587 22302 cni.go:95] Creating CNI manager for ""
I0630 21:07:52.982595 22302 cni.go:169] CNI unnecessary in this configuration, recommending no CNI
I0630 21:07:52.982601 22302 start_flags.go:310] config:
{Name:multinode-20220630203821-10507-m03 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e Memory:6000 CPUs:2 DiskSize:20000 VMDriver: Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.2 ClusterName:multinode-20220630203821-10507-m03 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain
:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0630 21:07:52.982703 22302 iso.go:128] acquiring lock: {Name:mkeebf3e76072464738f18646c09ed1445709013 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0630 21:07:52.984864 22302 out.go:177] * Starting control plane node multinode-20220630203821-10507-m03 in cluster multinode-20220630203821-10507-m03
I0630 21:07:52.986382 22302 preload.go:132] Checking if preload exists for k8s version v1.24.2 and runtime docker
I0630 21:07:52.986410 22302 preload.go:148] Found local preload: /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.2-docker-overlay2-amd64.tar.lz4
I0630 21:07:52.986423 22302 cache.go:57] Caching tarball of preloaded images
I0630 21:07:52.986529 22302 preload.go:174] Found /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0630 21:07:52.986542 22302 cache.go:60] Finished verifying existence of preloaded tar for v1.24.2 on docker
I0630 21:07:52.986625 22302 profile.go:148] Saving config to /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/multinode-20220630203821-10507-m03/config.json ...
I0630 21:07:52.986640 22302 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/multinode-20220630203821-10507-m03/config.json: {Name:mk62a0fcc676a67ce16f8d27a6927c1fa015aced Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 21:07:52.986767 22302 cache.go:208] Successfully downloaded all kic artifacts
I0630 21:07:52.986780 22302 start.go:352] acquiring machines lock for multinode-20220630203821-10507-m03: {Name:mkd6ffa4c52126a32ce14acade16a78767c09fed Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0630 21:07:52.986819 22302 start.go:356] acquired machines lock for "multinode-20220630203821-10507-m03" in 30.968µs
I0630 21:07:52.986828 22302 start.go:91] Provisioning new machine with config: &{Name:multinode-20220630203821-10507-m03 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/14420/minikube-v1.26.0-1656448385-14420-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e Memory:6000 CPUs:2 DiskSize:20000 VMDriver: Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPor
t:22 KubernetesConfig:{KubernetesVersion:v1.24.2 ClusterName:multinode-20220630203821-10507-m03 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.24.2 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType
:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:} &{Name: IP: Port:8443 KubernetesVersion:v1.24.2 ContainerRuntime:docker ControlPlane:true Worker:true}
I0630 21:07:52.986884 22302 start.go:131] createHost starting for "" (driver="kvm2")
I0630 21:07:52.990111 22302 out.go:204] * Creating kvm2 VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
I0630 21:07:52.990319 22302 main.go:134] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_integration/out/docker-machine-driver-kvm2
I0630 21:07:52.990360 22302 main.go:134] libmachine: Launching plugin server for driver kvm2
I0630 21:07:53.004545 22302 main.go:134] libmachine: Plugin server listening at address 127.0.0.1:43639
I0630 21:07:53.005004 22302 main.go:134] libmachine: () Calling .GetVersion
I0630 21:07:53.005543 22302 main.go:134] libmachine: Using API Version 1
I0630 21:07:53.005553 22302 main.go:134] libmachine: () Calling .SetConfigRaw
I0630 21:07:53.005922 22302 main.go:134] libmachine: () Calling .GetMachineName
I0630 21:07:53.006132 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Calling .GetMachineName
I0630 21:07:53.006311 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Calling .DriverName
I0630 21:07:53.006470 22302 start.go:165] libmachine.API.Create for "multinode-20220630203821-10507-m03" (driver="kvm2")
I0630 21:07:53.006494 22302 client.go:168] LocalClient.Create starting
I0630 21:07:53.006530 22302 main.go:134] libmachine: Reading certificate data from /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem
I0630 21:07:53.006558 22302 main.go:134] libmachine: Decoding PEM data...
I0630 21:07:53.006569 22302 main.go:134] libmachine: Parsing certificate...
I0630 21:07:53.006619 22302 main.go:134] libmachine: Reading certificate data from /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/cert.pem
I0630 21:07:53.006631 22302 main.go:134] libmachine: Decoding PEM data...
I0630 21:07:53.006639 22302 main.go:134] libmachine: Parsing certificate...
I0630 21:07:53.006659 22302 main.go:134] libmachine: Running pre-create checks...
I0630 21:07:53.006665 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Calling .PreCreateCheck
I0630 21:07:53.006979 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Calling .GetConfigRaw
I0630 21:07:53.007384 22302 main.go:134] libmachine: Creating machine...
I0630 21:07:53.007394 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Calling .Create
I0630 21:07:53.007537 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Creating KVM machine...
I0630 21:07:53.008737 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | found existing default KVM network
I0630 21:07:53.009657 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:53.009520 22325 network.go:240] skipping subnet 192.168.39.0/24 that is taken: &{IP:192.168.39.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.39.0/24 Gateway:192.168.39.1 ClientMin:192.168.39.2 ClientMax:192.168.39.254 Broadcast:192.168.39.255 Interface:{IfaceName:virbr1 IfaceIPv4:192.168.39.1 IfaceMTU:1500 IfaceMAC:52:54:00:7e:ac:e3}}
I0630 21:07:53.010331 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:53.010230 22325 network.go:288] reserving subnet 192.168.50.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[] amended:true}} dirty:map[192.168.50.0:0xc000188600] misses:0}
I0630 21:07:53.010368 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:53.010266 22325 network.go:235] using free private subnet 192.168.50.0/24: &{IP:192.168.50.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.50.0/24 Gateway:192.168.50.1 ClientMin:192.168.50.2 ClientMax:192.168.50.254 Broadcast:192.168.50.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0630 21:07:53.016243 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | trying to create private KVM network mk-multinode-20220630203821-10507-m03 192.168.50.0/24...
I0630 21:07:53.092588 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | private KVM network mk-multinode-20220630203821-10507-m03 192.168.50.0/24 created
I0630 21:07:53.092626 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:53.092572 22325 common.go:107] Making disk image using store path: /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube
I0630 21:07:53.092638 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Setting up store path in /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03 ...
I0630 21:07:53.092656 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Building disk image from file:///home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/iso/amd64/minikube-v1.26.0-1656448385-14420-amd64.iso
I0630 21:07:53.092712 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Downloading /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/iso/amd64/minikube-v1.26.0-1656448385-14420-amd64.iso...
I0630 21:07:53.273789 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:53.273637 22325 common.go:114] Creating ssh key: /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03/id_rsa...
I0630 21:07:53.646184 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:53.646043 22325 common.go:120] Creating raw disk image: /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03/multinode-20220630203821-10507-m03.rawdisk...
I0630 21:07:53.646218 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Writing magic tar header
I0630 21:07:53.646234 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Writing SSH key tar header
I0630 21:07:53.646255 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:53.646190 22325 common.go:134] Fixing permissions on /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03 ...
I0630 21:07:53.646339 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03
I0630 21:07:53.646369 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Setting executable bit set on /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03 (perms=drwx------)
I0630 21:07:53.646386 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines
I0630 21:07:53.646405 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube
I0630 21:07:53.646417 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Checking permissions on dir: /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44
I0630 21:07:53.646428 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Checking permissions on dir: /home/jenkins/minikube-integration
I0630 21:07:53.646451 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Checking permissions on dir: /home/jenkins
I0630 21:07:53.646465 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Setting executable bit set on /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines (perms=drwxrwxr-x)
I0630 21:07:53.646480 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Setting executable bit set on /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube (perms=drwxr-xr-x)
I0630 21:07:53.646494 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Setting executable bit set on /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44 (perms=drwxrwxr-x)
I0630 21:07:53.646520 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Setting executable bit set on /home/jenkins/minikube-integration (perms=drwxrwxr-x)
I0630 21:07:53.646531 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Checking permissions on dir: /home
I0630 21:07:53.646545 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Setting executable bit set on /home/jenkins (perms=drwxr-xr-x)
I0630 21:07:53.646556 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Creating domain...
I0630 21:07:53.646563 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Skipping /home - not owner
I0630 21:07:53.647793 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) define libvirt domain using xml:
I0630 21:07:53.647813 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <domain type='kvm'>
I0630 21:07:53.647825 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <name>multinode-20220630203821-10507-m03</name>
I0630 21:07:53.647839 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <memory unit='MiB'>6000</memory>
I0630 21:07:53.647849 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <vcpu>2</vcpu>
I0630 21:07:53.647857 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <features>
I0630 21:07:53.647876 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <acpi/>
I0630 21:07:53.647887 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <apic/>
I0630 21:07:53.647893 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <pae/>
I0630 21:07:53.647897 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03)
I0630 21:07:53.647902 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </features>
I0630 21:07:53.647907 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <cpu mode='host-passthrough'>
I0630 21:07:53.647912 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03)
I0630 21:07:53.647920 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </cpu>
I0630 21:07:53.647925 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <os>
I0630 21:07:53.647931 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <type>hvm</type>
I0630 21:07:53.647940 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <boot dev='cdrom'/>
I0630 21:07:53.647949 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <boot dev='hd'/>
I0630 21:07:53.647959 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <bootmenu enable='no'/>
I0630 21:07:53.647967 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </os>
I0630 21:07:53.647992 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <devices>
I0630 21:07:53.648012 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <disk type='file' device='cdrom'>
I0630 21:07:53.648033 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <source file='/home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03/boot2docker.iso'/>
I0630 21:07:53.648048 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <target dev='hdc' bus='scsi'/>
I0630 21:07:53.648057 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <readonly/>
I0630 21:07:53.648065 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </disk>
I0630 21:07:53.648074 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <disk type='file' device='disk'>
I0630 21:07:53.648085 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <driver name='qemu' type='raw' cache='default' io='threads' />
I0630 21:07:53.648132 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <source file='/home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03/multinode-20220630203821-10507-m03.rawdisk'/>
I0630 21:07:53.648152 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <target dev='hda' bus='virtio'/>
I0630 21:07:53.648165 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </disk>
I0630 21:07:53.648173 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <interface type='network'>
I0630 21:07:53.648183 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <source network='mk-multinode-20220630203821-10507-m03'/>
I0630 21:07:53.648199 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <model type='virtio'/>
I0630 21:07:53.648209 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </interface>
I0630 21:07:53.648217 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <interface type='network'>
I0630 21:07:53.648228 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <source network='default'/>
I0630 21:07:53.648237 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <model type='virtio'/>
I0630 21:07:53.648246 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </interface>
I0630 21:07:53.648259 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <serial type='pty'>
I0630 21:07:53.648270 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <target port='0'/>
I0630 21:07:53.648282 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </serial>
I0630 21:07:53.648292 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <console type='pty'>
I0630 21:07:53.648302 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <target type='serial' port='0'/>
I0630 21:07:53.648311 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </console>
I0630 21:07:53.648319 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <rng model='virtio'>
I0630 21:07:53.648331 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) <backend model='random'>/dev/random</backend>
I0630 21:07:53.648340 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </rng>
I0630 21:07:53.648348 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03)
I0630 21:07:53.648356 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03)
I0630 21:07:53.648366 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </devices>
I0630 21:07:53.648374 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) </domain>
I0630 21:07:53.648387 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03)
I0630 21:07:53.653194 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:94:79:2e in network default
I0630 21:07:53.653782 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Ensuring networks are active...
I0630 21:07:53.653801 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:07:53.654502 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Ensuring network default is active
I0630 21:07:53.654805 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Ensuring network mk-multinode-20220630203821-10507-m03 is active
I0630 21:07:53.655313 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Getting domain xml...
I0630 21:07:53.656040 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Creating domain...
I0630 21:07:54.046751 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Waiting to get IP...
I0630 21:07:54.047571 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:07:54.047989 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:07:54.048002 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:54.047965 22325 retry.go:31] will retry after 263.082536ms: waiting for machine to come up
I0630 21:07:54.312221 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:07:54.312731 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:07:54.312752 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:54.312683 22325 retry.go:31] will retry after 381.329545ms: waiting for machine to come up
I0630 21:07:54.695357 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:07:54.695822 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:07:54.695849 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:54.695765 22325 retry.go:31] will retry after 422.765636ms: waiting for machine to come up
I0630 21:07:55.120311 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:07:55.120724 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:07:55.120747 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:55.120685 22325 retry.go:31] will retry after 473.074753ms: waiting for machine to come up
I0630 21:07:55.595401 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:07:55.595863 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:07:55.595903 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:55.595808 22325 retry.go:31] will retry after 587.352751ms: waiting for machine to come up
I0630 21:07:56.184766 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:07:56.185324 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:07:56.185344 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:56.185269 22325 retry.go:31] will retry after 834.206799ms: waiting for machine to come up
I0630 21:07:57.024065 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:07:57.024588 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:07:57.024616 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:57.024529 22325 retry.go:31] will retry after 746.553905ms: waiting for machine to come up
I0630 21:07:57.773076 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:07:57.773452 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:07:57.773475 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:57.773401 22325 retry.go:31] will retry after 987.362415ms: waiting for machine to come up
I0630 21:07:58.761998 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:07:58.762481 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:07:58.762498 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:58.762447 22325 retry.go:31] will retry after 1.189835008s: waiting for machine to come up
I0630 21:07:59.953746 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:07:59.954071 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:07:59.954094 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:07:59.954021 22325 retry.go:31] will retry after 1.677229867s: waiting for machine to come up
I0630 21:08:01.632566 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:08:01.633037 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:08:01.633109 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:08:01.632989 22325 retry.go:31] will retry after 2.346016261s: waiting for machine to come up
I0630 21:08:03.980548 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:08:03.980963 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:08:03.980995 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:08:03.980908 22325 retry.go:31] will retry after 3.36678925s: waiting for machine to come up
I0630 21:08:07.350204 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:08:07.350661 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:08:07.350688 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:08:07.350602 22325 retry.go:31] will retry after 3.11822781s: waiting for machine to come up
I0630 21:08:10.470236 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:08:10.470644 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find current IP address of domain multinode-20220630203821-10507-m03 in network mk-multinode-20220630203821-10507-m03
I0630 21:08:10.470670 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | I0630 21:08:10.470594 22325 retry.go:31] will retry after 4.276119362s: waiting for machine to come up
I0630 21:08:14.749595 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:08:14.750010 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Found IP for machine: 192.168.50.54
I0630 21:08:14.750034 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has current primary IP address 192.168.50.54 and MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:08:14.750043 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Reserving static IP address...
I0630 21:08:14.750384 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find host DHCP lease matching {name: "multinode-20220630203821-10507-m03", mac: "52:54:00:a7:bf:5e", ip: "192.168.50.54"} in network mk-multinode-20220630203821-10507-m03
I0630 21:08:14.822536 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Getting to WaitForSSH function...
I0630 21:08:14.822565 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Reserved static IP address: 192.168.50.54
I0630 21:08:14.822578 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) Waiting for SSH to be available...
I0630 21:08:14.825239 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:08:14.825534 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find host DHCP lease matching {name: "", mac: "52:54:00:a7:bf:5e", ip: ""} in network mk-multinode-20220630203821-10507-m03
I0630 21:08:14.825559 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | unable to find defined IP address of network mk-multinode-20220630203821-10507-m03 interface with MAC address 52:54:00:a7:bf:5e
I0630 21:08:14.825851 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Using SSH client type: external
I0630 21:08:14.825881 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Using SSH private key: /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03/id_rsa (-rw-------)
I0630 21:08:14.825920 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@ -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03/id_rsa -p 22] /usr/bin/ssh <nil>}
I0630 21:08:14.825940 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | About to run SSH command:
I0630 21:08:14.825950 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | exit 0
I0630 21:08:14.829667 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | SSH cmd err, output: exit status 255:
I0630 21:08:14.829683 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Error getting ssh command 'exit 0' : ssh command error:
I0630 21:08:14.829690 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | command : exit 0
I0630 21:08:14.829705 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | err : exit status 255
I0630 21:08:14.829713 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | output :
I0630 21:08:17.830348 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Getting to WaitForSSH function...
I0630 21:08:17.832845 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:08:17.833242 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:a7:bf:5e", ip: ""} in network mk-multinode-20220630203821-10507-m03: {Iface:virbr2 ExpiryTime:2022-06-30 22:08:07 +0000 UTC Type:0 Mac:52:54:00:a7:bf:5e Iaid: IPaddr:192.168.50.54 Prefix:24 Hostname:multinode-20220630203821-10507-m03 Clientid:01:52:54:00:a7:bf:5e}
I0630 21:08:17.833264 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | domain multinode-20220630203821-10507-m03 has defined IP address 192.168.50.54 and MAC address 52:54:00:a7:bf:5e in network mk-multinode-20220630203821-10507-m03
I0630 21:08:17.833405 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Using SSH client type: external
I0630 21:08:17.833435 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | Using SSH private key: /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03/id_rsa (-rw-------)
I0630 21:08:17.833473 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | &{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker@192.168.50.54 -o IdentitiesOnly=yes -i /home/jenkins/minikube-integration/linux-amd64-kvm2--14420-3582-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/multinode-20220630203821-10507-m03/id_rsa -p 22] /usr/bin/ssh <nil>}
I0630 21:08:17.833487 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | About to run SSH command:
I0630 21:08:17.833495 22302 main.go:134] libmachine: (multinode-20220630203821-10507-m03) DBG | exit 0
*
* ==> Docker <==
* -- Journal begins at Thu 2022-06-30 20:58:08 UTC, ends at Thu 2022-06-30 21:08:21 UTC. --
Jun 30 20:58:34 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:34.778394827Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Jun 30 20:58:34 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:34.778483594Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Jun 30 20:58:34 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:34.778494779Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Jun 30 20:58:34 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:34.778766141Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/fd01112af6cdf8460e9b4158d6aceecb58f55e79016bfc56b1f6d68c08aea2b2 pid=1994 runtime=io.containerd.runc.v2
Jun 30 20:58:35 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:35.281321643Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Jun 30 20:58:35 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:35.281377669Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Jun 30 20:58:35 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:35.281386965Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Jun 30 20:58:35 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:35.281884191Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/625920e8cb8d8d789980863b7130f0ca0a2b7148cc9001995062c55574ac2cac pid=2043 runtime=io.containerd.runc.v2
Jun 30 20:58:37 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:37.983847863Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Jun 30 20:58:37 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:37.984218175Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Jun 30 20:58:37 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:37.984461751Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Jun 30 20:58:37 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:37.981854515Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Jun 30 20:58:37 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:37.984677871Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Jun 30 20:58:37 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:37.984691687Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Jun 30 20:58:37 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:37.984883679Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/377511497dd368b87ae4c20feed8e7e1f168537d8290b7a20fbc96b81fb4b535 pid=2199 runtime=io.containerd.runc.v2
Jun 30 20:58:37 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:58:37.985163808Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/ee5a3e8296ffa8d8dcdb73cad697d0201826a7a0ddf604388a75bece9ed90e82 pid=2211 runtime=io.containerd.runc.v2
Jun 30 20:59:05 multinode-20220630203821-10507 dockerd[831]: time="2022-06-30T20:59:05.868489126Z" level=info msg="ignoring event" container=625920e8cb8d8d789980863b7130f0ca0a2b7148cc9001995062c55574ac2cac module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Jun 30 20:59:05 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:59:05.868959826Z" level=info msg="shim disconnected" id=625920e8cb8d8d789980863b7130f0ca0a2b7148cc9001995062c55574ac2cac
Jun 30 20:59:05 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:59:05.870347384Z" level=warning msg="cleaning up after shim disconnected" id=625920e8cb8d8d789980863b7130f0ca0a2b7148cc9001995062c55574ac2cac namespace=moby
Jun 30 20:59:05 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:59:05.870819526Z" level=info msg="cleaning up dead shim"
Jun 30 20:59:05 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:59:05.887299677Z" level=warning msg="cleanup warnings time=\"2022-06-30T20:59:05Z\" level=info msg=\"starting signal loop\" namespace=moby pid=2742 runtime=io.containerd.runc.v2\n"
Jun 30 20:59:20 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:59:20.734761781Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Jun 30 20:59:20 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:59:20.735017287Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Jun 30 20:59:20 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:59:20.735029231Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Jun 30 20:59:20 multinode-20220630203821-10507 dockerd[838]: time="2022-06-30T20:59:20.735640248Z" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/ba92dd87a7d921882270d52404dca8fbb06acca34e1e332b50fdbb90f16854e8 pid=2921 runtime=io.containerd.runc.v2
*
* ==> container status <==
* CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
ba92dd87a7d92 6e38f40d628db 9 minutes ago Running storage-provisioner 4 fd01112af6cdf
ee5a3e8296ffa a634548d10b03 9 minutes ago Running kube-proxy 2 f467d83ad0237
377511497dd36 6fb66cd78abfe 9 minutes ago Running kindnet-cni 2 e4f22a8ef8b32
625920e8cb8d8 6e38f40d628db 9 minutes ago Exited storage-provisioner 3 fd01112af6cdf
188e88dee6d95 aebe758cef4cd 9 minutes ago Running etcd 2 74fa3d21e3e09
dd8b8a8bcef67 5d725196c1f47 9 minutes ago Running kube-scheduler 2 2bf0251155467
65645e95496f7 34cdf99b1bb3b 9 minutes ago Running kube-controller-manager 2 ea70d98cf7ab9
3e52c381f83fe d3377ffb7177c 9 minutes ago Running kube-apiserver 2 006fc83ccd8f1
ec113fc966602 6fb66cd78abfe 24 minutes ago Exited kindnet-cni 1 b8fa68d341428
0fe08b34ff2cf a634548d10b03 24 minutes ago Exited kube-proxy 1 a65e872343c8f
0490c13412f49 5d725196c1f47 24 minutes ago Exited kube-scheduler 1 40a921c067ad3
64452406f72b3 aebe758cef4cd 24 minutes ago Exited etcd 1 a8da6f3ea4400
8e6a4956b8c44 d3377ffb7177c 24 minutes ago Exited kube-apiserver 1 82a0b83338f02
2807bd0bc3ef3 34cdf99b1bb3b 24 minutes ago Exited kube-controller-manager 1 17e8cd7dc2593
a123aead1039c gcr.io/k8s-minikube/busybox@sha256:9afb80db71730dbb303fe00765cbf34bddbdc6b66e49897fc2e1861967584b12 27 minutes ago Exited busybox 0 ac8b13f76a9b1
dc78430f93146 a4ca41631cc7a 28 minutes ago Exited coredns 0 3a106f36b825a
*
* ==> coredns [dc78430f9314] <==
* .:53
[INFO] plugin/reload: Running configuration MD5 = 8f51b271a18f2ce6fcaee5f1cfda3ed0
CoreDNS-1.8.6
linux/amd64, go1.17.1, 13a9191
[INFO] SIGTERM: Shutting down servers then terminating
[INFO] plugin/health: Going into lameduck mode for 5s
*
* ==> describe nodes <==
* Name: multinode-20220630203821-10507
Roles: control-plane
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=multinode-20220630203821-10507
kubernetes.io/os=linux
minikube.k8s.io/commit=251ffcec0e4c997cd7732864187747cc579943c9
minikube.k8s.io/name=multinode-20220630203821-10507
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2022_06_30T20_39_16_0700
minikube.k8s.io/version=v1.26.0
node-role.kubernetes.io/control-plane=
node.kubernetes.io/exclude-from-external-load-balancers=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Thu, 30 Jun 2022 20:39:12 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: multinode-20220630203821-10507
AcquireTime: <unset>
RenewTime: Thu, 30 Jun 2022 21:08:15 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Thu, 30 Jun 2022 21:04:00 +0000 Thu, 30 Jun 2022 20:39:09 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Thu, 30 Jun 2022 21:04:00 +0000 Thu, 30 Jun 2022 20:39:09 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Thu, 30 Jun 2022 21:04:00 +0000 Thu, 30 Jun 2022 20:39:09 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Thu, 30 Jun 2022 21:04:00 +0000 Thu, 30 Jun 2022 20:58:53 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.39.215
Hostname: multinode-20220630203821-10507
Capacity:
cpu: 2
ephemeral-storage: 17784752Ki
hugepages-2Mi: 0
memory: 2165916Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 17784752Ki
hugepages-2Mi: 0
memory: 2165916Ki
pods: 110
System Info:
Machine ID: 415a934ef3004daea70d7f4c52780a5f
System UUID: 415a934e-f300-4dae-a70d-7f4c52780a5f
Boot ID: ed0175b9-f3a7-4c7f-8cb2-9530b520124b
Kernel Version: 5.10.57
OS Image: Buildroot 2021.02.12
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://20.10.16
Kubelet Version: v1.24.2
Kube-Proxy Version: v1.24.2
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (9 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox-d46db594c-dwpcw 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 27m
kube-system coredns-6d4b75cb6d-qjdcj 100m (5%!)(MISSING) 0 (0%!)(MISSING) 70Mi (3%!)(MISSING) 170Mi (8%!)(MISSING) 28m
kube-system etcd-multinode-20220630203821-10507 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (4%!)(MISSING) 0 (0%!)(MISSING) 29m
kube-system kindnet-fq2vn 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (2%!)(MISSING) 50Mi (2%!)(MISSING) 28m
kube-system kube-apiserver-multinode-20220630203821-10507 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 29m
kube-system kube-controller-manager-multinode-20220630203821-10507 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 29m
kube-system kube-proxy-gfvjt 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 28m
kube-system kube-scheduler-multinode-20220630203821-10507 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 29m
kube-system storage-provisioner 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 28m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 850m (42%!)(MISSING) 100m (5%!)(MISSING)
memory 220Mi (10%!)(MISSING) 220Mi (10%!)(MISSING)
ephemeral-storage 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 28m kube-proxy
Normal Starting 24m kube-proxy
Normal Starting 9m43s kube-proxy
Normal NodeHasSufficientMemory 29m (x5 over 29m) kubelet Node multinode-20220630203821-10507 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 29m (x4 over 29m) kubelet Node multinode-20220630203821-10507 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 29m (x4 over 29m) kubelet Node multinode-20220630203821-10507 status is now: NodeHasSufficientPID
Normal Starting 29m kubelet Starting kubelet.
Normal NodeHasSufficientMemory 29m kubelet Node multinode-20220630203821-10507 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 29m kubelet Node multinode-20220630203821-10507 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 29m kubelet Node multinode-20220630203821-10507 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 29m kubelet Updated Node Allocatable limit across pods
Normal RegisteredNode 28m node-controller Node multinode-20220630203821-10507 event: Registered Node multinode-20220630203821-10507 in Controller
Normal NodeReady 28m kubelet Node multinode-20220630203821-10507 status is now: NodeReady
Normal Starting 24m kubelet Starting kubelet.
Normal NodeHasSufficientMemory 24m (x8 over 24m) kubelet Node multinode-20220630203821-10507 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 24m (x8 over 24m) kubelet Node multinode-20220630203821-10507 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 24m (x7 over 24m) kubelet Node multinode-20220630203821-10507 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 24m kubelet Updated Node Allocatable limit across pods
Normal RegisteredNode 24m node-controller Node multinode-20220630203821-10507 event: Registered Node multinode-20220630203821-10507 in Controller
Normal Starting 9m57s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 9m57s (x8 over 9m57s) kubelet Node multinode-20220630203821-10507 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 9m57s (x8 over 9m57s) kubelet Node multinode-20220630203821-10507 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 9m57s (x7 over 9m57s) kubelet Node multinode-20220630203821-10507 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 9m57s kubelet Updated Node Allocatable limit across pods
Normal RegisteredNode 9m36s node-controller Node multinode-20220630203821-10507 event: Registered Node multinode-20220630203821-10507 in Controller
Name: multinode-20220630203821-10507-m02
Roles: <none>
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=multinode-20220630203821-10507-m02
kubernetes.io/os=linux
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /var/run/cri-dockerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Thu, 30 Jun 2022 21:03:29 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: multinode-20220630203821-10507-m02
AcquireTime: <unset>
RenewTime: Thu, 30 Jun 2022 21:08:15 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Thu, 30 Jun 2022 21:03:50 +0000 Thu, 30 Jun 2022 21:03:29 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Thu, 30 Jun 2022 21:03:50 +0000 Thu, 30 Jun 2022 21:03:29 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Thu, 30 Jun 2022 21:03:50 +0000 Thu, 30 Jun 2022 21:03:29 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Thu, 30 Jun 2022 21:03:50 +0000 Thu, 30 Jun 2022 21:03:50 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.39.124
Hostname: multinode-20220630203821-10507-m02
Capacity:
cpu: 2
ephemeral-storage: 17784752Ki
hugepages-2Mi: 0
memory: 2165916Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 17784752Ki
hugepages-2Mi: 0
memory: 2165916Ki
pods: 110
System Info:
Machine ID: 035a7d36703c489d87e3f3d0badd94d2
System UUID: 035a7d36-703c-489d-87e3-f3d0badd94d2
Boot ID: d318f625-ddee-495c-abb3-37ae84959d23
Kernel Version: 5.10.57
OS Image: Buildroot 2021.02.12
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://20.10.16
Kubelet Version: v1.24.2
Kube-Proxy Version: v1.24.2
PodCIDR: 10.244.1.0/24
PodCIDRs: 10.244.1.0/24
Non-terminated Pods: (3 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox-d46db594c-wnlkb 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 27m
kube-system kindnet-jjgwh 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (2%!)(MISSING) 50Mi (2%!)(MISSING) 27m
kube-system kube-proxy-gmc55 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 27m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 100m (5%!)(MISSING) 100m (5%!)(MISSING)
memory 50Mi (2%!)(MISSING) 50Mi (2%!)(MISSING)
ephemeral-storage 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 19m kube-proxy
Normal Starting 27m kube-proxy
Normal Starting 4m49s kube-proxy
Normal NodeHasNoDiskPressure 27m (x8 over 28m) kubelet Node multinode-20220630203821-10507-m02 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientMemory 27m (x8 over 28m) kubelet Node multinode-20220630203821-10507-m02 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 19m (x2 over 19m) kubelet Node multinode-20220630203821-10507-m02 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 19m (x2 over 19m) kubelet Node multinode-20220630203821-10507-m02 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 19m kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 19m (x2 over 19m) kubelet Node multinode-20220630203821-10507-m02 status is now: NodeHasSufficientMemory
Normal Starting 19m kubelet Starting kubelet.
Normal NodeReady 19m kubelet Node multinode-20220630203821-10507-m02 status is now: NodeReady
Normal Starting 4m53s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 4m53s (x2 over 4m53s) kubelet Node multinode-20220630203821-10507-m02 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 4m53s (x2 over 4m53s) kubelet Node multinode-20220630203821-10507-m02 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 4m53s (x2 over 4m53s) kubelet Node multinode-20220630203821-10507-m02 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 4m53s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 4m32s kubelet Node multinode-20220630203821-10507-m02 status is now: NodeReady
*
* ==> dmesg <==
* [Jun30 20:58] You have booted with nomodeset. This means your GPU drivers are DISABLED
[ +0.000000] Any video related functionality will be severely degraded, and you may not even be able to suspend the system properly
[ +0.000001] Unless you actually understand what nomodeset does, you should reboot without enabling it
[ +0.069298] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ +3.908263] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[ +2.985835] systemd-fstab-generator[114]: Ignoring "noauto" for root device
[ +0.133227] systemd[1]: systemd-journald.service: unit configures an IP firewall, but the local system does not support BPF/cgroup firewalling.
[ +0.000002] systemd[1]: (This warning is only shown for the first unit using IP firewalling.)
[ +2.407009] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[ +0.000007] NFSD: unable to find recovery directory /var/lib/nfs/v4recovery
[ +0.000001] NFSD: Unable to initialize client recovery tracking! (-2)
[ +6.038181] systemd-fstab-generator[512]: Ignoring "noauto" for root device
[ +0.099733] systemd-fstab-generator[523]: Ignoring "noauto" for root device
[ +1.037441] systemd-fstab-generator[745]: Ignoring "noauto" for root device
[ +0.300342] systemd-fstab-generator[800]: Ignoring "noauto" for root device
[ +0.098404] systemd-fstab-generator[811]: Ignoring "noauto" for root device
[ +0.109757] systemd-fstab-generator[822]: Ignoring "noauto" for root device
[ +1.669098] systemd-fstab-generator[999]: Ignoring "noauto" for root device
[ +0.102950] systemd-fstab-generator[1010]: Ignoring "noauto" for root device
[ +5.283903] systemd-fstab-generator[1217]: Ignoring "noauto" for root device
[ +0.280394] kauditd_printk_skb: 67 callbacks suppressed
[ +12.880748] kauditd_printk_skb: 7 callbacks suppressed
*
* ==> etcd [188e88dee6d9] <==
* {"level":"info","ts":"2022-06-30T20:58:29.632Z","caller":"etcdserver/server.go:851","msg":"starting etcd server","local-member-id":"ce9e8f286885b37e","local-server-version":"3.5.3","cluster-version":"to_be_decided"}
{"level":"info","ts":"2022-06-30T20:58:29.634Z","caller":"etcdserver/server.go:752","msg":"starting initial election tick advance","election-ticks":10}
{"level":"info","ts":"2022-06-30T20:58:29.638Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e switched to configuration voters=(14888494821848494974)"}
{"level":"info","ts":"2022-06-30T20:58:29.642Z","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"4cd5d1376c5e8c88","local-member-id":"ce9e8f286885b37e","added-peer-id":"ce9e8f286885b37e","added-peer-peer-urls":["https://192.168.39.215:2380"]}
{"level":"info","ts":"2022-06-30T20:58:29.643Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"4cd5d1376c5e8c88","local-member-id":"ce9e8f286885b37e","cluster-version":"3.5"}
{"level":"info","ts":"2022-06-30T20:58:29.643Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}
{"level":"info","ts":"2022-06-30T20:58:29.660Z","caller":"embed/etcd.go:688","msg":"starting with client TLS","tls-info":"cert = /var/lib/minikube/certs/etcd/server.crt, key = /var/lib/minikube/certs/etcd/server.key, client-cert=, client-key=, trusted-ca = /var/lib/minikube/certs/etcd/ca.crt, client-cert-auth = true, crl-file = ","cipher-suites":[]}
{"level":"info","ts":"2022-06-30T20:58:29.663Z","caller":"embed/etcd.go:277","msg":"now serving peer/client/metrics","local-member-id":"ce9e8f286885b37e","initial-advertise-peer-urls":["https://192.168.39.215:2380"],"listen-peer-urls":["https://192.168.39.215:2380"],"advertise-client-urls":["https://192.168.39.215:2379"],"listen-client-urls":["https://127.0.0.1:2379","https://192.168.39.215:2379"],"listen-metrics-urls":["http://127.0.0.1:2381"]}
{"level":"info","ts":"2022-06-30T20:58:29.664Z","caller":"embed/etcd.go:763","msg":"serving metrics","address":"http://127.0.0.1:2381"}
{"level":"info","ts":"2022-06-30T20:58:29.660Z","caller":"embed/etcd.go:581","msg":"serving peer traffic","address":"192.168.39.215:2380"}
{"level":"info","ts":"2022-06-30T20:58:29.668Z","caller":"embed/etcd.go:553","msg":"cmux::serve","address":"192.168.39.215:2380"}
{"level":"info","ts":"2022-06-30T20:58:30.691Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e is starting a new election at term 3"}
{"level":"info","ts":"2022-06-30T20:58:30.691Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e became pre-candidate at term 3"}
{"level":"info","ts":"2022-06-30T20:58:30.691Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e received MsgPreVoteResp from ce9e8f286885b37e at term 3"}
{"level":"info","ts":"2022-06-30T20:58:30.691Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e became candidate at term 4"}
{"level":"info","ts":"2022-06-30T20:58:30.691Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e received MsgVoteResp from ce9e8f286885b37e at term 4"}
{"level":"info","ts":"2022-06-30T20:58:30.691Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e became leader at term 4"}
{"level":"info","ts":"2022-06-30T20:58:30.691Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: ce9e8f286885b37e elected leader ce9e8f286885b37e at term 4"}
{"level":"info","ts":"2022-06-30T20:58:30.694Z","caller":"etcdserver/server.go:2042","msg":"published local member to cluster through raft","local-member-id":"ce9e8f286885b37e","local-member-attributes":"{Name:multinode-20220630203821-10507 ClientURLs:[https://192.168.39.215:2379]}","request-path":"/0/members/ce9e8f286885b37e/attributes","cluster-id":"4cd5d1376c5e8c88","publish-timeout":"7s"}
{"level":"info","ts":"2022-06-30T20:58:30.695Z","caller":"embed/serve.go:98","msg":"ready to serve client requests"}
{"level":"info","ts":"2022-06-30T20:58:30.696Z","caller":"embed/serve.go:98","msg":"ready to serve client requests"}
{"level":"info","ts":"2022-06-30T20:58:30.697Z","caller":"embed/serve.go:188","msg":"serving client traffic securely","address":"127.0.0.1:2379"}
{"level":"info","ts":"2022-06-30T20:58:30.698Z","caller":"embed/serve.go:188","msg":"serving client traffic securely","address":"192.168.39.215:2379"}
{"level":"info","ts":"2022-06-30T20:58:30.699Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}
{"level":"info","ts":"2022-06-30T20:58:30.699Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}
*
* ==> etcd [64452406f72b] <==
* {"level":"info","ts":"2022-06-30T20:43:46.314Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e switched to configuration voters=(14888494821848494974)"}
{"level":"info","ts":"2022-06-30T20:43:46.314Z","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"4cd5d1376c5e8c88","local-member-id":"ce9e8f286885b37e","added-peer-id":"ce9e8f286885b37e","added-peer-peer-urls":["https://192.168.39.215:2380"]}
{"level":"info","ts":"2022-06-30T20:43:46.315Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"4cd5d1376c5e8c88","local-member-id":"ce9e8f286885b37e","cluster-version":"3.5"}
{"level":"info","ts":"2022-06-30T20:43:46.317Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}
{"level":"info","ts":"2022-06-30T20:43:46.331Z","caller":"embed/etcd.go:688","msg":"starting with client TLS","tls-info":"cert = /var/lib/minikube/certs/etcd/server.crt, key = /var/lib/minikube/certs/etcd/server.key, client-cert=, client-key=, trusted-ca = /var/lib/minikube/certs/etcd/ca.crt, client-cert-auth = true, crl-file = ","cipher-suites":[]}
{"level":"info","ts":"2022-06-30T20:43:46.334Z","caller":"embed/etcd.go:277","msg":"now serving peer/client/metrics","local-member-id":"ce9e8f286885b37e","initial-advertise-peer-urls":["https://192.168.39.215:2380"],"listen-peer-urls":["https://192.168.39.215:2380"],"advertise-client-urls":["https://192.168.39.215:2379"],"listen-client-urls":["https://127.0.0.1:2379","https://192.168.39.215:2379"],"listen-metrics-urls":["http://127.0.0.1:2381"]}
{"level":"info","ts":"2022-06-30T20:43:46.334Z","caller":"embed/etcd.go:763","msg":"serving metrics","address":"http://127.0.0.1:2381"}
{"level":"info","ts":"2022-06-30T20:43:46.338Z","caller":"embed/etcd.go:581","msg":"serving peer traffic","address":"192.168.39.215:2380"}
{"level":"info","ts":"2022-06-30T20:43:46.338Z","caller":"embed/etcd.go:553","msg":"cmux::serve","address":"192.168.39.215:2380"}
{"level":"info","ts":"2022-06-30T20:43:47.366Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e is starting a new election at term 2"}
{"level":"info","ts":"2022-06-30T20:43:47.366Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e became pre-candidate at term 2"}
{"level":"info","ts":"2022-06-30T20:43:47.366Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e received MsgPreVoteResp from ce9e8f286885b37e at term 2"}
{"level":"info","ts":"2022-06-30T20:43:47.366Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e became candidate at term 3"}
{"level":"info","ts":"2022-06-30T20:43:47.366Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e received MsgVoteResp from ce9e8f286885b37e at term 3"}
{"level":"info","ts":"2022-06-30T20:43:47.366Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ce9e8f286885b37e became leader at term 3"}
{"level":"info","ts":"2022-06-30T20:43:47.366Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: ce9e8f286885b37e elected leader ce9e8f286885b37e at term 3"}
{"level":"info","ts":"2022-06-30T20:43:47.368Z","caller":"etcdserver/server.go:2042","msg":"published local member to cluster through raft","local-member-id":"ce9e8f286885b37e","local-member-attributes":"{Name:multinode-20220630203821-10507 ClientURLs:[https://192.168.39.215:2379]}","request-path":"/0/members/ce9e8f286885b37e/attributes","cluster-id":"4cd5d1376c5e8c88","publish-timeout":"7s"}
{"level":"info","ts":"2022-06-30T20:43:47.369Z","caller":"embed/serve.go:98","msg":"ready to serve client requests"}
{"level":"info","ts":"2022-06-30T20:43:47.370Z","caller":"embed/serve.go:98","msg":"ready to serve client requests"}
{"level":"info","ts":"2022-06-30T20:43:47.371Z","caller":"embed/serve.go:188","msg":"serving client traffic securely","address":"192.168.39.215:2379"}
{"level":"info","ts":"2022-06-30T20:43:47.372Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}
{"level":"info","ts":"2022-06-30T20:43:47.372Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}
{"level":"info","ts":"2022-06-30T20:43:47.379Z","caller":"embed/serve.go:188","msg":"serving client traffic securely","address":"127.0.0.1:2379"}
{"level":"info","ts":"2022-06-30T20:53:47.401Z","caller":"mvcc/index.go:214","msg":"compact tree index","revision":1119}
{"level":"info","ts":"2022-06-30T20:53:47.422Z","caller":"mvcc/kvstore_compaction.go:57","msg":"finished scheduled compaction","compact-revision":1119,"took":"19.982201ms"}
*
* ==> kernel <==
* 21:08:22 up 10 min, 0 users, load average: 0.44, 0.34, 0.18
Linux multinode-20220630203821-10507 5.10.57 #1 SMP Tue Jun 28 23:44:16 UTC 2022 x86_64 GNU/Linux
PRETTY_NAME="Buildroot 2021.02.12"
*
* ==> kube-apiserver [3e52c381f83f] <==
* I0630 20:58:33.242162 1 establishing_controller.go:76] Starting EstablishingController
I0630 20:58:33.242175 1 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController
I0630 20:58:33.242186 1 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController
I0630 20:58:33.242198 1 crd_finalizer.go:266] Starting CRDFinalizer
I0630 20:58:33.211992 1 crdregistration_controller.go:111] Starting crd-autoregister controller
I0630 20:58:33.255636 1 shared_informer.go:255] Waiting for caches to sync for crd-autoregister
E0630 20:58:33.329206 1 controller.go:169] Error removing old endpoints from kubernetes service: no master IPs were listed in storage, refusing to erase all endpoints for the kubernetes service
I0630 20:58:33.359319 1 shared_informer.go:262] Caches are synced for crd-autoregister
I0630 20:58:33.376836 1 shared_informer.go:262] Caches are synced for node_authorizer
I0630 20:58:33.394336 1 shared_informer.go:262] Caches are synced for cluster_authentication_trust_controller
I0630 20:58:33.395374 1 cache.go:39] Caches are synced for AvailableConditionController controller
I0630 20:58:33.395791 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller
I0630 20:58:33.414658 1 controller.go:611] quota admission added evaluator for: leases.coordination.k8s.io
I0630 20:58:33.415605 1 cache.go:39] Caches are synced for autoregister controller
I0630 20:58:33.421347 1 apf_controller.go:322] Running API Priority and Fairness config worker
I0630 20:58:33.458400 1 controller.go:611] quota admission added evaluator for: events.events.k8s.io
I0630 20:58:33.873392 1 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
I0630 20:58:34.201268 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist.
I0630 20:58:36.293585 1 controller.go:611] quota admission added evaluator for: daemonsets.apps
I0630 20:58:36.411316 1 controller.go:611] quota admission added evaluator for: serviceaccounts
I0630 20:58:36.423817 1 controller.go:611] quota admission added evaluator for: deployments.apps
I0630 20:58:36.488208 1 controller.go:611] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0630 20:58:36.496213 1 controller.go:611] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
I0630 20:58:46.263852 1 controller.go:611] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0630 20:59:38.294625 1 controller.go:611] quota admission added evaluator for: endpoints
*
* ==> kube-apiserver [8e6a4956b8c4] <==
* I0630 20:43:49.848826 1 controller.go:80] Starting OpenAPI V3 AggregationController
I0630 20:43:49.885361 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/minikube/certs/ca.crt"
I0630 20:43:49.888045 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/minikube/certs/front-proxy-ca.crt"
I0630 20:43:49.908439 1 crdregistration_controller.go:111] Starting crd-autoregister controller
I0630 20:43:49.908605 1 shared_informer.go:255] Waiting for caches to sync for crd-autoregister
I0630 20:43:49.972592 1 shared_informer.go:262] Caches are synced for node_authorizer
I0630 20:43:50.008807 1 shared_informer.go:262] Caches are synced for crd-autoregister
I0630 20:43:50.037552 1 apf_controller.go:322] Running API Priority and Fairness config worker
I0630 20:43:50.037928 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller
I0630 20:43:50.051548 1 shared_informer.go:262] Caches are synced for cluster_authentication_trust_controller
I0630 20:43:50.051904 1 cache.go:39] Caches are synced for autoregister controller
I0630 20:43:50.056664 1 cache.go:39] Caches are synced for AvailableConditionController controller
I0630 20:43:50.058736 1 controller.go:611] quota admission added evaluator for: leases.coordination.k8s.io
I0630 20:43:50.516957 1 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
I0630 20:43:50.840039 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist.
I0630 20:43:52.447791 1 controller.go:611] quota admission added evaluator for: daemonsets.apps
I0630 20:43:52.612609 1 controller.go:611] quota admission added evaluator for: serviceaccounts
I0630 20:43:52.635804 1 controller.go:611] quota admission added evaluator for: deployments.apps
I0630 20:43:52.735283 1 controller.go:611] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0630 20:43:52.749307 1 controller.go:611] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
I0630 20:43:53.744597 1 controller.go:611] quota admission added evaluator for: events.events.k8s.io
I0630 20:44:02.684065 1 controller.go:611] quota admission added evaluator for: endpoints
I0630 20:44:02.912138 1 controller.go:611] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0630 20:57:52.143487 1 controller.go:198] Shutting down kubernetes service endpoint reconciler
I0630 20:57:52.143679 1 object_count_tracker.go:84] "StorageObjectCountTracker pruner is exiting"
*
* ==> kube-controller-manager [2807bd0bc3ef] <==
* I0630 20:44:12.937244 1 event.go:294] "Event occurred" object="kube-system/storage-provisioner" fieldPath="" kind="Pod" apiVersion="" type="Normal" reason="TaintManagerEviction" message="Cancelling deletion of Pod kube-system/storage-provisioner"
I0630 20:44:12.938051 1 event.go:294] "Event occurred" object="default/busybox-d46db594c-dwpcw" fieldPath="" kind="Pod" apiVersion="" type="Normal" reason="TaintManagerEviction" message="Cancelling deletion of Pod default/busybox-d46db594c-dwpcw"
I0630 20:44:12.938400 1 event.go:294] "Event occurred" object="kube-system/coredns-6d4b75cb6d-qjdcj" fieldPath="" kind="Pod" apiVersion="" type="Normal" reason="TaintManagerEviction" message="Cancelling deletion of Pod kube-system/coredns-6d4b75cb6d-qjdcj"
W0630 20:44:42.956322 1 topologycache.go:199] Can't get CPU or zone information for multinode-20220630203821-10507-m03 node
I0630 20:44:42.958038 1 event.go:294] "Event occurred" object="multinode-20220630203821-10507-m02" fieldPath="" kind="Node" apiVersion="v1" type="Normal" reason="NodeNotReady" message="Node multinode-20220630203821-10507-m02 status is now: NodeNotReady"
I0630 20:44:42.968343 1 event.go:294] "Event occurred" object="default/busybox-d46db594c-wnlkb" fieldPath="" kind="Pod" apiVersion="v1" type="Warning" reason="NodeNotReady" message="Node is not ready"
I0630 20:44:42.983905 1 event.go:294] "Event occurred" object="kube-system/kindnet-jjgwh" fieldPath="" kind="Pod" apiVersion="v1" type="Warning" reason="NodeNotReady" message="Node is not ready"
I0630 20:44:42.991380 1 event.go:294] "Event occurred" object="kube-system/kube-proxy-gmc55" fieldPath="" kind="Pod" apiVersion="v1" type="Warning" reason="NodeNotReady" message="Node is not ready"
I0630 20:44:43.003649 1 event.go:294] "Event occurred" object="multinode-20220630203821-10507-m03" fieldPath="" kind="Node" apiVersion="v1" type="Normal" reason="NodeNotReady" message="Node multinode-20220630203821-10507-m03 status is now: NodeNotReady"
I0630 20:44:43.023493 1 event.go:294] "Event occurred" object="kube-system/kindnet-dhpn2" fieldPath="" kind="Pod" apiVersion="v1" type="Warning" reason="NodeNotReady" message="Node is not ready"
I0630 20:44:43.037099 1 event.go:294] "Event occurred" object="kube-system/kube-proxy-7rq6l" fieldPath="" kind="Pod" apiVersion="v1" type="Warning" reason="NodeNotReady" message="Node is not ready"
I0630 20:48:45.890470 1 event.go:294] "Event occurred" object="default/busybox-d46db594c" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: busybox-d46db594c-wzpkt"
I0630 20:48:49.811623 1 event.go:294] "Event occurred" object="default/busybox-d46db594c-wnlkb" fieldPath="" kind="Pod" apiVersion="" type="Normal" reason="TaintManagerEviction" message="Cancelling deletion of Pod default/busybox-d46db594c-wnlkb"
W0630 20:48:49.812380 1 actual_state_of_world.go:541] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true, because nodeName="multinode-20220630203821-10507-m02" does not exist
I0630 20:48:49.819450 1 range_allocator.go:374] Set node multinode-20220630203821-10507-m02 PodCIDR to [10.244.1.0/24]
W0630 20:49:00.206375 1 topologycache.go:199] Can't get CPU or zone information for multinode-20220630203821-10507-m02 node
I0630 20:49:03.098268 1 event.go:294] "Event occurred" object="default/busybox-d46db594c-wnlkb" fieldPath="" kind="Pod" apiVersion="" type="Normal" reason="TaintManagerEviction" message="Cancelling deletion of Pod default/busybox-d46db594c-wnlkb"
W0630 20:53:24.630253 1 topologycache.go:199] Can't get CPU or zone information for multinode-20220630203821-10507-m02 node
W0630 20:53:25.550297 1 topologycache.go:199] Can't get CPU or zone information for multinode-20220630203821-10507-m02 node
W0630 20:53:25.551907 1 actual_state_of_world.go:541] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true, because nodeName="multinode-20220630203821-10507-m03" does not exist
I0630 20:53:25.575486 1 range_allocator.go:374] Set node multinode-20220630203821-10507-m03 PodCIDR to [10.244.2.0/24]
W0630 20:53:46.188500 1 topologycache.go:199] Can't get CPU or zone information for multinode-20220630203821-10507-m02 node
I0630 20:53:48.156999 1 event.go:294] "Event occurred" object="default/busybox-d46db594c-wzpkt" fieldPath="" kind="Pod" apiVersion="" type="Normal" reason="TaintManagerEviction" message="Cancelling deletion of Pod default/busybox-d46db594c-wzpkt"
I0630 20:57:48.778299 1 event.go:294] "Event occurred" object="default/busybox-d46db594c" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: busybox-d46db594c-sqb9f"
W0630 20:57:50.786785 1 topologycache.go:199] Can't get CPU or zone information for multinode-20220630203821-10507-m02 node
*
* ==> kube-controller-manager [65645e95496f] <==
* I0630 20:58:46.236328 1 shared_informer.go:262] Caches are synced for daemon sets
I0630 20:58:46.243217 1 shared_informer.go:262] Caches are synced for resource quota
I0630 20:58:46.250497 1 shared_informer.go:262] Caches are synced for certificate-csrsigning-legacy-unknown
I0630 20:58:46.253724 1 shared_informer.go:262] Caches are synced for endpoint_slice
I0630 20:58:46.264216 1 shared_informer.go:262] Caches are synced for certificate-csrapproving
I0630 20:58:46.264640 1 shared_informer.go:262] Caches are synced for certificate-csrsigning-kube-apiserver-client
I0630 20:58:46.278307 1 shared_informer.go:262] Caches are synced for endpoint_slice_mirroring
I0630 20:58:46.333630 1 shared_informer.go:262] Caches are synced for certificate-csrsigning-kubelet-serving
I0630 20:58:46.341003 1 shared_informer.go:262] Caches are synced for certificate-csrsigning-kubelet-client
I0630 20:58:46.740805 1 shared_informer.go:262] Caches are synced for garbage collector
I0630 20:58:46.761146 1 shared_informer.go:262] Caches are synced for garbage collector
I0630 20:58:46.761168 1 garbagecollector.go:158] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
W0630 20:58:53.975343 1 topologycache.go:199] Can't get CPU or zone information for multinode-20220630203821-10507-m02 node
I0630 20:59:26.100227 1 event.go:294] "Event occurred" object="multinode-20220630203821-10507-m02" fieldPath="" kind="Node" apiVersion="v1" type="Normal" reason="NodeNotReady" message="Node multinode-20220630203821-10507-m02 status is now: NodeNotReady"
I0630 20:59:26.112085 1 event.go:294] "Event occurred" object="kube-system/kindnet-jjgwh" fieldPath="" kind="Pod" apiVersion="v1" type="Warning" reason="NodeNotReady" message="Node is not ready"
I0630 20:59:26.125993 1 event.go:294] "Event occurred" object="kube-system/kube-proxy-gmc55" fieldPath="" kind="Pod" apiVersion="v1" type="Warning" reason="NodeNotReady" message="Node is not ready"
I0630 20:59:26.157258 1 gc_controller.go:81] "PodGC is force deleting Pod" pod="kube-system/kube-proxy-7rq6l"
I0630 20:59:26.172435 1 gc_controller.go:239] "Forced deletion of orphaned Pod succeeded" pod="kube-system/kube-proxy-7rq6l"
I0630 20:59:26.172667 1 gc_controller.go:81] "PodGC is force deleting Pod" pod="kube-system/kindnet-dhpn2"
I0630 20:59:26.185423 1 gc_controller.go:239] "Forced deletion of orphaned Pod succeeded" pod="kube-system/kindnet-dhpn2"
W0630 21:03:29.862970 1 actual_state_of_world.go:541] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true, because nodeName="multinode-20220630203821-10507-m02" does not exist
I0630 21:03:29.863744 1 event.go:294] "Event occurred" object="default/busybox-d46db594c-wnlkb" fieldPath="" kind="Pod" apiVersion="" type="Normal" reason="TaintManagerEviction" message="Cancelling deletion of Pod default/busybox-d46db594c-wnlkb"
I0630 21:03:29.880826 1 range_allocator.go:374] Set node multinode-20220630203821-10507-m02 PodCIDR to [10.244.1.0/24]
W0630 21:03:50.331115 1 topologycache.go:199] Can't get CPU or zone information for multinode-20220630203821-10507-m02 node
I0630 21:03:51.185405 1 event.go:294] "Event occurred" object="default/busybox-d46db594c-wnlkb" fieldPath="" kind="Pod" apiVersion="" type="Normal" reason="TaintManagerEviction" message="Cancelling deletion of Pod default/busybox-d46db594c-wnlkb"
*
* ==> kube-proxy [0fe08b34ff2c] <==
* I0630 20:43:53.603281 1 node.go:163] Successfully retrieved node IP: 192.168.39.215
I0630 20:43:53.603353 1 server_others.go:138] "Detected node IP" address="192.168.39.215"
I0630 20:43:53.603457 1 server_others.go:578] "Unknown proxy mode, assuming iptables proxy" proxyMode=""
I0630 20:43:53.737311 1 server_others.go:199] "kube-proxy running in single-stack mode, this ipFamily is not supported" ipFamily=IPv6
I0630 20:43:53.737330 1 server_others.go:206] "Using iptables Proxier"
I0630 20:43:53.737582 1 proxier.go:259] "Setting route_localnet=1, use nodePortAddresses to filter loopback addresses for NodePorts to skip it https://issues.k8s.io/90259"
I0630 20:43:53.738575 1 server.go:661] "Version info" version="v1.24.2"
I0630 20:43:53.738613 1 server.go:663] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0630 20:43:53.739992 1 config.go:317] "Starting service config controller"
I0630 20:43:53.740193 1 shared_informer.go:255] Waiting for caches to sync for service config
I0630 20:43:53.740244 1 config.go:226] "Starting endpoint slice config controller"
I0630 20:43:53.740250 1 shared_informer.go:255] Waiting for caches to sync for endpoint slice config
I0630 20:43:53.740941 1 config.go:444] "Starting node config controller"
I0630 20:43:53.740974 1 shared_informer.go:255] Waiting for caches to sync for node config
I0630 20:43:53.840780 1 shared_informer.go:262] Caches are synced for endpoint slice config
I0630 20:43:53.840995 1 shared_informer.go:262] Caches are synced for node config
I0630 20:43:53.841009 1 shared_informer.go:262] Caches are synced for service config
*
* ==> kube-proxy [ee5a3e8296ff] <==
* I0630 20:58:38.230399 1 node.go:163] Successfully retrieved node IP: 192.168.39.215
I0630 20:58:38.230640 1 server_others.go:138] "Detected node IP" address="192.168.39.215"
I0630 20:58:38.231059 1 server_others.go:578] "Unknown proxy mode, assuming iptables proxy" proxyMode=""
I0630 20:58:38.311018 1 server_others.go:199] "kube-proxy running in single-stack mode, this ipFamily is not supported" ipFamily=IPv6
I0630 20:58:38.311037 1 server_others.go:206] "Using iptables Proxier"
I0630 20:58:38.312034 1 proxier.go:259] "Setting route_localnet=1, use nodePortAddresses to filter loopback addresses for NodePorts to skip it https://issues.k8s.io/90259"
I0630 20:58:38.313022 1 server.go:661] "Version info" version="v1.24.2"
I0630 20:58:38.313083 1 server.go:663] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0630 20:58:38.315226 1 config.go:317] "Starting service config controller"
I0630 20:58:38.315238 1 shared_informer.go:255] Waiting for caches to sync for service config
I0630 20:58:38.315254 1 config.go:226] "Starting endpoint slice config controller"
I0630 20:58:38.315257 1 shared_informer.go:255] Waiting for caches to sync for endpoint slice config
I0630 20:58:38.316994 1 config.go:444] "Starting node config controller"
I0630 20:58:38.317079 1 shared_informer.go:255] Waiting for caches to sync for node config
I0630 20:58:38.427498 1 shared_informer.go:262] Caches are synced for endpoint slice config
I0630 20:58:38.427591 1 shared_informer.go:262] Caches are synced for service config
I0630 20:58:38.428238 1 shared_informer.go:262] Caches are synced for node config
*
* ==> kube-scheduler [0490c13412f4] <==
* I0630 20:43:47.116727 1 serving.go:348] Generated self-signed cert in-memory
W0630 20:43:49.928714 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0630 20:43:49.928966 1 authentication.go:346] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0630 20:43:49.929167 1 authentication.go:347] Continuing without authentication configuration. This may treat all requests as anonymous.
W0630 20:43:49.929189 1 authentication.go:348] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0630 20:43:49.979364 1 server.go:147] "Starting Kubernetes Scheduler" version="v1.24.2"
I0630 20:43:49.979737 1 server.go:149] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0630 20:43:49.988730 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
I0630 20:43:49.988812 1 shared_informer.go:255] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0630 20:43:49.995764 1 tlsconfig.go:240] "Starting DynamicServingCertificateController"
I0630 20:43:50.000271 1 secure_serving.go:210] Serving securely on 127.0.0.1:10259
I0630 20:43:50.091050 1 shared_informer.go:262] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0630 20:57:52.128665 1 tlsconfig.go:255] "Shutting down DynamicServingCertificateController"
I0630 20:57:52.129022 1 secure_serving.go:255] Stopped listening on 127.0.0.1:10259
I0630 20:57:52.129095 1 configmap_cafile_content.go:223] "Shutting down controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
*
* ==> kube-scheduler [dd8b8a8bcef6] <==
* I0630 20:58:29.935152 1 serving.go:348] Generated self-signed cert in-memory
W0630 20:58:33.292466 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0630 20:58:33.292656 1 authentication.go:346] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0630 20:58:33.292763 1 authentication.go:347] Continuing without authentication configuration. This may treat all requests as anonymous.
W0630 20:58:33.292785 1 authentication.go:348] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0630 20:58:33.339336 1 server.go:147] "Starting Kubernetes Scheduler" version="v1.24.2"
I0630 20:58:33.339434 1 server.go:149] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0630 20:58:33.342723 1 secure_serving.go:210] Serving securely on 127.0.0.1:10259
I0630 20:58:33.346611 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
I0630 20:58:33.346772 1 shared_informer.go:255] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0630 20:58:33.346628 1 tlsconfig.go:240] "Starting DynamicServingCertificateController"
I0630 20:58:33.447887 1 shared_informer.go:262] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
E0630 21:03:50.358284 1 event_broadcaster.go:253] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"busybox-d46db594c-sqb9f.16fd81a17a61047e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"d6cb5904-1492-4e55-b53d-7338f178d419", ResourceVersion:"1911", Generation:0, CreationTimestamp:time.Date(2022, time.June, 30, 20, 59, 28, 0, time.Local), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ZZZ_DeprecatedClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kube-scheduler", Operation:"Update", APIVersion:"events.k8s.io/v1", Time:time.Date(2022, time.June, 30, 20, 59, 28, 0, time.Local), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0009b6798), Subresource:""}}}, EventTime:time.Date(2022, time.June, 30, 20, 59, 28, 485101024, time.Local), Serie
s:(*v1.EventSeries)(0xc00068dec0), ReportingController:"default-scheduler", ReportingInstance:"default-scheduler-multinode-20220630203821-10507", Action:"Scheduling", Reason:"FailedScheduling", Regarding:v1.ObjectReference{Kind:"Pod", Namespace:"default", Name:"busybox-d46db594c-sqb9f", UID:"16c88c3d-5268-4fd8-9b35-24be1ba2c745", APIVersion:"v1", ResourceVersion:"1907", FieldPath:""}, Related:(*v1.ObjectReference)(nil), Note:"0/2 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/unreachable: }, 2 node(s) didn't match pod anti-affinity rules. preemption: 0/2 nodes are available: 1 No preemption victims found for incoming pod, 1 Preemption is not helpful for scheduling.", Type:"Warning", DeprecatedSource:v1.EventSource{Component:"", Host:""}, DeprecatedFirstTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeprecatedLastTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeprecatedCount:0}': 'Event "busybox-d46db594c-sqb9f.16fd81a17a61047e" is invalid: [series.coun
t: Invalid value: "": should be at least 2, eventTime: Invalid value: 2022-06-30 20:59:28.485101 +0000 UTC: field is immutable]' (will not retry!)
*
* ==> kubelet <==
* -- Journal begins at Thu 2022-06-30 20:58:08 UTC, ends at Thu 2022-06-30 21:08:22 UTC. --
Jun 30 21:07:44 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:44.636490 1223 pod_workers.go:951] "Error syncing pod, skipping" err="failed to \"KillPodSandbox\" for \"dde04c59-a5b5-454c-a715-3c4b9b24dfeb\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"busybox-d46db594c-dwpcw_default\\\" network: could not retrieve port mappings: key is not found\"" pod="default/busybox-d46db594c-dwpcw" podUID=dde04c59-a5b5-454c-a715-3c4b9b24dfeb
Jun 30 21:07:47 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:47.637761 1223 remote_runtime.go:248] "StopPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \"coredns-6d4b75cb6d-qjdcj_kube-system\" network: could not retrieve port mappings: key is not found" podSandboxID="3a106f36b825af52c9f08ce59a41444d8ab7f6a4c05265a192eebe7eb98839b1"
Jun 30 21:07:47 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:47.637920 1223 kuberuntime_manager.go:999] "Failed to stop sandbox" podSandboxID={Type:docker ID:3a106f36b825af52c9f08ce59a41444d8ab7f6a4c05265a192eebe7eb98839b1}
Jun 30 21:07:47 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:47.638012 1223 kuberuntime_manager.go:738] "killPodWithSyncResult failed" err="failed to \"KillPodSandbox\" for \"060a0760-fc3d-4823-9720-9e48d846eb51\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"coredns-6d4b75cb6d-qjdcj_kube-system\\\" network: could not retrieve port mappings: key is not found\""
Jun 30 21:07:47 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:47.638039 1223 pod_workers.go:951] "Error syncing pod, skipping" err="failed to \"KillPodSandbox\" for \"060a0760-fc3d-4823-9720-9e48d846eb51\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"coredns-6d4b75cb6d-qjdcj_kube-system\\\" network: could not retrieve port mappings: key is not found\"" pod="kube-system/coredns-6d4b75cb6d-qjdcj" podUID=060a0760-fc3d-4823-9720-9e48d846eb51
Jun 30 21:07:55 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:55.641745 1223 remote_runtime.go:248] "StopPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \"busybox-d46db594c-dwpcw_default\" network: could not retrieve port mappings: key is not found" podSandboxID="ac8b13f76a9b1205d405fa6c0291bc66fd88a28f08393f185569bf5a6beba749"
Jun 30 21:07:55 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:55.642075 1223 kuberuntime_manager.go:999] "Failed to stop sandbox" podSandboxID={Type:docker ID:ac8b13f76a9b1205d405fa6c0291bc66fd88a28f08393f185569bf5a6beba749}
Jun 30 21:07:55 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:55.642283 1223 kuberuntime_manager.go:738] "killPodWithSyncResult failed" err="failed to \"KillPodSandbox\" for \"dde04c59-a5b5-454c-a715-3c4b9b24dfeb\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"busybox-d46db594c-dwpcw_default\\\" network: could not retrieve port mappings: key is not found\""
Jun 30 21:07:55 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:55.642409 1223 pod_workers.go:951] "Error syncing pod, skipping" err="failed to \"KillPodSandbox\" for \"dde04c59-a5b5-454c-a715-3c4b9b24dfeb\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"busybox-d46db594c-dwpcw_default\\\" network: could not retrieve port mappings: key is not found\"" pod="default/busybox-d46db594c-dwpcw" podUID=dde04c59-a5b5-454c-a715-3c4b9b24dfeb
Jun 30 21:07:59 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:59.638699 1223 remote_runtime.go:248] "StopPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \"coredns-6d4b75cb6d-qjdcj_kube-system\" network: could not retrieve port mappings: key is not found" podSandboxID="3a106f36b825af52c9f08ce59a41444d8ab7f6a4c05265a192eebe7eb98839b1"
Jun 30 21:07:59 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:59.639298 1223 kuberuntime_manager.go:999] "Failed to stop sandbox" podSandboxID={Type:docker ID:3a106f36b825af52c9f08ce59a41444d8ab7f6a4c05265a192eebe7eb98839b1}
Jun 30 21:07:59 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:59.639478 1223 kuberuntime_manager.go:738] "killPodWithSyncResult failed" err="failed to \"KillPodSandbox\" for \"060a0760-fc3d-4823-9720-9e48d846eb51\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"coredns-6d4b75cb6d-qjdcj_kube-system\\\" network: could not retrieve port mappings: key is not found\""
Jun 30 21:07:59 multinode-20220630203821-10507 kubelet[1223]: E0630 21:07:59.639619 1223 pod_workers.go:951] "Error syncing pod, skipping" err="failed to \"KillPodSandbox\" for \"060a0760-fc3d-4823-9720-9e48d846eb51\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"coredns-6d4b75cb6d-qjdcj_kube-system\\\" network: could not retrieve port mappings: key is not found\"" pod="kube-system/coredns-6d4b75cb6d-qjdcj" podUID=060a0760-fc3d-4823-9720-9e48d846eb51
Jun 30 21:08:08 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:08.637113 1223 remote_runtime.go:248] "StopPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \"busybox-d46db594c-dwpcw_default\" network: could not retrieve port mappings: key is not found" podSandboxID="ac8b13f76a9b1205d405fa6c0291bc66fd88a28f08393f185569bf5a6beba749"
Jun 30 21:08:08 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:08.637199 1223 kuberuntime_manager.go:999] "Failed to stop sandbox" podSandboxID={Type:docker ID:ac8b13f76a9b1205d405fa6c0291bc66fd88a28f08393f185569bf5a6beba749}
Jun 30 21:08:08 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:08.637241 1223 kuberuntime_manager.go:738] "killPodWithSyncResult failed" err="failed to \"KillPodSandbox\" for \"dde04c59-a5b5-454c-a715-3c4b9b24dfeb\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"busybox-d46db594c-dwpcw_default\\\" network: could not retrieve port mappings: key is not found\""
Jun 30 21:08:08 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:08.637267 1223 pod_workers.go:951] "Error syncing pod, skipping" err="failed to \"KillPodSandbox\" for \"dde04c59-a5b5-454c-a715-3c4b9b24dfeb\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"busybox-d46db594c-dwpcw_default\\\" network: could not retrieve port mappings: key is not found\"" pod="default/busybox-d46db594c-dwpcw" podUID=dde04c59-a5b5-454c-a715-3c4b9b24dfeb
Jun 30 21:08:12 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:12.637990 1223 remote_runtime.go:248] "StopPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \"coredns-6d4b75cb6d-qjdcj_kube-system\" network: could not retrieve port mappings: key is not found" podSandboxID="3a106f36b825af52c9f08ce59a41444d8ab7f6a4c05265a192eebe7eb98839b1"
Jun 30 21:08:12 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:12.638074 1223 kuberuntime_manager.go:999] "Failed to stop sandbox" podSandboxID={Type:docker ID:3a106f36b825af52c9f08ce59a41444d8ab7f6a4c05265a192eebe7eb98839b1}
Jun 30 21:08:12 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:12.638107 1223 kuberuntime_manager.go:738] "killPodWithSyncResult failed" err="failed to \"KillPodSandbox\" for \"060a0760-fc3d-4823-9720-9e48d846eb51\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"coredns-6d4b75cb6d-qjdcj_kube-system\\\" network: could not retrieve port mappings: key is not found\""
Jun 30 21:08:12 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:12.638132 1223 pod_workers.go:951] "Error syncing pod, skipping" err="failed to \"KillPodSandbox\" for \"060a0760-fc3d-4823-9720-9e48d846eb51\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"coredns-6d4b75cb6d-qjdcj_kube-system\\\" network: could not retrieve port mappings: key is not found\"" pod="kube-system/coredns-6d4b75cb6d-qjdcj" podUID=060a0760-fc3d-4823-9720-9e48d846eb51
Jun 30 21:08:22 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:22.639005 1223 remote_runtime.go:248] "StopPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \"busybox-d46db594c-dwpcw_default\" network: could not retrieve port mappings: key is not found" podSandboxID="ac8b13f76a9b1205d405fa6c0291bc66fd88a28f08393f185569bf5a6beba749"
Jun 30 21:08:22 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:22.639100 1223 kuberuntime_manager.go:999] "Failed to stop sandbox" podSandboxID={Type:docker ID:ac8b13f76a9b1205d405fa6c0291bc66fd88a28f08393f185569bf5a6beba749}
Jun 30 21:08:22 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:22.639147 1223 kuberuntime_manager.go:738] "killPodWithSyncResult failed" err="failed to \"KillPodSandbox\" for \"dde04c59-a5b5-454c-a715-3c4b9b24dfeb\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"busybox-d46db594c-dwpcw_default\\\" network: could not retrieve port mappings: key is not found\""
Jun 30 21:08:22 multinode-20220630203821-10507 kubelet[1223]: E0630 21:08:22.639186 1223 pod_workers.go:951] "Error syncing pod, skipping" err="failed to \"KillPodSandbox\" for \"dde04c59-a5b5-454c-a715-3c4b9b24dfeb\" with KillPodSandboxError: \"rpc error: code = Unknown desc = networkPlugin cni failed to teardown pod \\\"busybox-d46db594c-dwpcw_default\\\" network: could not retrieve port mappings: key is not found\"" pod="default/busybox-d46db594c-dwpcw" podUID=dde04c59-a5b5-454c-a715-3c4b9b24dfeb
*
* ==> storage-provisioner [625920e8cb8d] <==
* I0630 20:58:35.831735 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0630 20:59:05.841107 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
*
* ==> storage-provisioner [ba92dd87a7d9] <==
* I0630 20:59:20.827642 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0630 20:59:20.849927 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0630 20:59:20.850678 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0630 20:59:38.297899 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0630 20:59:38.298512 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_multinode-20220630203821-10507_c9cec416-81f0-46f2-ad61-48bceff45541!
I0630 20:59:38.302887 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"ca367dc6-b7af-4116-9a3e-1aa8c67b3df7", APIVersion:"v1", ResourceVersion:"1916", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' multinode-20220630203821-10507_c9cec416-81f0-46f2-ad61-48bceff45541 became leader
I0630 20:59:38.400241 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_multinode-20220630203821-10507_c9cec416-81f0-46f2-ad61-48bceff45541!
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-amd64 status --format={{.APIServer}} -p multinode-20220630203821-10507 -n multinode-20220630203821-10507
helpers_test.go:261: (dbg) Run: kubectl --context multinode-20220630203821-10507 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:270: non-running pods: busybox-d46db594c-sqb9f
helpers_test.go:272: ======> post-mortem[TestMultiNode/serial/ValidateNameConflict]: describe non-running pods <======
helpers_test.go:275: (dbg) Run: kubectl --context multinode-20220630203821-10507 describe pod busybox-d46db594c-sqb9f
helpers_test.go:280: (dbg) kubectl --context multinode-20220630203821-10507 describe pod busybox-d46db594c-sqb9f:
-- stdout --
Name: busybox-d46db594c-sqb9f
Namespace: default
Priority: 0
Node: <none>
Labels: app=busybox
pod-template-hash=d46db594c
Annotations: <none>
Status: Pending
IP:
IPs: <none>
Controlled By: ReplicaSet/busybox-d46db594c
Containers:
busybox:
Image: gcr.io/k8s-minikube/busybox:1.28
Port: <none>
Host Port: <none>
Command:
sleep
3600
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-w89pn (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
kube-api-access-w89pn:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 10m default-scheduler 0/3 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/unschedulable: }, 1 node(s) were unschedulable, 3 node(s) didn't match pod anti-affinity rules. preemption: 0/3 nodes are available: 1 Preemption is not helpful for scheduling, 2 No preemption victims found for incoming pod.
Warning FailedScheduling 10m default-scheduler 0/3 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/unschedulable: }, 1 node(s) were unschedulable, 2 node(s) didn't match pod anti-affinity rules. preemption: 0/3 nodes are available: 1 Preemption is not helpful for scheduling, 2 No preemption victims found for incoming pod.
Warning FailedScheduling 9m49s default-scheduler 0/2 nodes are available: 2 node(s) didn't match pod anti-affinity rules. preemption: 0/2 nodes are available: 2 No preemption victims found for incoming pod.
Warning FailedScheduling 8m57s default-scheduler 0/2 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/unreachable: }, 2 node(s) didn't match pod anti-affinity rules. preemption: 0/2 nodes are available: 1 No preemption victims found for incoming pod, 1 Preemption is not helpful for scheduling.
Warning FailedScheduling 8m54s default-scheduler 0/2 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/unreachable: }, 2 node(s) didn't match pod anti-affinity rules. preemption: 0/2 nodes are available: 1 No preemption victims found for incoming pod, 1 Preemption is not helpful for scheduling.
-- /stdout --
helpers_test.go:283: <<< TestMultiNode/serial/ValidateNameConflict FAILED: end of post-mortem logs <<<
helpers_test.go:284: ---------------------/post-mortem---------------------------------
--- FAIL: TestMultiNode/serial/ValidateNameConflict (30.56s)