=== RUN TestNetworkPlugins/group/calico/Start
net_test.go:98: (dbg) Run: out/minikube-linux-amd64 start -p calico-20210810222820-95538 --memory=2048 --alsologtostderr --wait=true --wait-timeout=5m --cni=calico --driver=docker --container-runtime=docker
=== CONT TestNetworkPlugins/group/calico/Start
net_test.go:98: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p calico-20210810222820-95538 --memory=2048 --alsologtostderr --wait=true --wait-timeout=5m --cni=calico --driver=docker --container-runtime=docker: exit status 80 (52.01661876s)
-- stdout --
* [calico-20210810222820-95538] minikube v1.22.0 on Debian 9.13 (kvm/amd64)
- KUBECONFIG=/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/kubeconfig
- MINIKUBE_BIN=out/minikube-linux-amd64
- MINIKUBE_HOME=/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube
- MINIKUBE_LOCATION=12230
* Using the docker driver based on user configuration
- More information: https://docs.docker.com/engine/install/linux-postinstall/#your-kernel-does-not-support-cgroup-swap-limit-capabilities
* Starting control plane node calico-20210810222820-95538 in cluster calico-20210810222820-95538
* Pulling base image ...
* Creating docker container (CPUs=2, Memory=2048MB) ...
* Preparing Kubernetes v1.21.3 on Docker 20.10.7 ...
- Generating certificates and keys ...
- Booting up control plane ...
- Configuring RBAC rules ...
* Configuring Calico (Container Networking Interface) ...
- Generating certificates and keys ...
- Booting up control plane ...
- Configuring RBAC rules ...
* Configuring Calico (Container Networking Interface) ...
-- /stdout --
** stderr **
I0810 22:32:01.536218 328117 out.go:298] Setting OutFile to fd 1 ...
I0810 22:32:01.536301 328117 out.go:345] TERM=,COLORTERM=, which probably does not support color
I0810 22:32:01.536309 328117 out.go:311] Setting ErrFile to fd 2...
I0810 22:32:01.536312 328117 out.go:345] TERM=,COLORTERM=, which probably does not support color
I0810 22:32:01.536401 328117 root.go:313] Updating PATH: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/bin
I0810 22:32:01.536676 328117 out.go:305] Setting JSON to false
I0810 22:32:01.584785 328117 start.go:111] hostinfo: {"hostname":"debian-jenkins-agent-5","uptime":8083,"bootTime":1628626639,"procs":281,"os":"linux","platform":"debian","platformFamily":"debian","platformVersion":"9.13","kernelVersion":"4.9.0-16-amd64","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"c29e0b88-ef83-6765-d2fa-208fdce1af32"}
I0810 22:32:01.584912 328117 start.go:121] virtualization: kvm guest
I0810 22:32:01.586915 328117 out.go:177] * [calico-20210810222820-95538] minikube v1.22.0 on Debian 9.13 (kvm/amd64)
I0810 22:32:01.588331 328117 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/kubeconfig
I0810 22:32:01.587067 328117 notify.go:169] Checking for updates...
I0810 22:32:01.589729 328117 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0810 22:32:01.591155 328117 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube
I0810 22:32:01.592339 328117 out.go:177] - MINIKUBE_LOCATION=12230
I0810 22:32:01.593005 328117 driver.go:335] Setting default libvirt URI to qemu:///system
I0810 22:32:01.651032 328117 docker.go:132] docker version: linux-19.03.15
I0810 22:32:01.651151 328117 cli_runner.go:115] Run: docker system info --format "{{json .}}"
I0810 22:32:01.737387 328117 info.go:263] docker info: {ID:LQL6:IQEY:TAE3:NG47:ROZQ:WA5O:XM2B:XDCN:3VXZ:7JF3:4DHA:WN5N Containers:3 ContainersRunning:3 ContainersPaused:0 ContainersStopped:0 Images:154 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:false KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:45 OomKillDisable:true NGoroutines:50 SystemTime:2021-08-10 22:32:01.692861042 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:4.9.0-16-amd64 OperatingSystem:Debian GNU/Linux 9 (stretch) OSType:linux Architecture:x86_64 IndexServerAddr
ess:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33742200832 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:debian-jenkins-agent-5 Labels:[] ExperimentalBuild:false ServerVersion:19.03.15 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:269548fa27e0089a8b8278fc4fc781d7f65a939b Expected:269548fa27e0089a8b8278fc4fc781d7f65a939b} RuncCommit:{ID:ff819c7e9184c13b7c2607fe6c30ae19403a7aff Expected:ff819c7e9184c13b7c2607fe6c30ae19403a7aff} InitCommit:{ID:fec3683 Expected:fec3683} SecurityOptions:[name=seccomp,profile=default] ProductLicense: Warnin
gs:[WARNING: No swap limit support] ServerErrors:[] ClientInfo:{Debug:false Plugins:[] Warnings:<nil>}}
I0810 22:32:01.737476 328117 docker.go:244] overlay module found
I0810 22:32:01.739555 328117 out.go:177] * Using the docker driver based on user configuration
I0810 22:32:01.739590 328117 start.go:278] selected driver: docker
I0810 22:32:01.739602 328117 start.go:751] validating driver "docker" against <nil>
I0810 22:32:01.739625 328117 start.go:762] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
W0810 22:32:01.739664 328117 oci.go:119] Your kernel does not support memory limit capabilities or the cgroup is not mounted.
W0810 22:32:01.739682 328117 out.go:242] ! Your cgroup does not allow setting memory.
! Your cgroup does not allow setting memory.
I0810 22:32:01.741217 328117 out.go:177] - More information: https://docs.docker.com/engine/install/linux-postinstall/#your-kernel-does-not-support-cgroup-swap-limit-capabilities
I0810 22:32:01.742148 328117 cli_runner.go:115] Run: docker system info --format "{{json .}}"
I0810 22:32:01.825049 328117 info.go:263] docker info: {ID:LQL6:IQEY:TAE3:NG47:ROZQ:WA5O:XM2B:XDCN:3VXZ:7JF3:4DHA:WN5N Containers:3 ContainersRunning:3 ContainersPaused:0 ContainersStopped:0 Images:154 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:false KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:45 OomKillDisable:true NGoroutines:50 SystemTime:2021-08-10 22:32:01.77864923 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:4.9.0-16-amd64 OperatingSystem:Debian GNU/Linux 9 (stretch) OSType:linux Architecture:x86_64 IndexServerAddre
ss:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33742200832 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:debian-jenkins-agent-5 Labels:[] ExperimentalBuild:false ServerVersion:19.03.15 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:269548fa27e0089a8b8278fc4fc781d7f65a939b Expected:269548fa27e0089a8b8278fc4fc781d7f65a939b} RuncCommit:{ID:ff819c7e9184c13b7c2607fe6c30ae19403a7aff Expected:ff819c7e9184c13b7c2607fe6c30ae19403a7aff} InitCommit:{ID:fec3683 Expected:fec3683} SecurityOptions:[name=seccomp,profile=default] ProductLicense: Warning
s:[WARNING: No swap limit support] ServerErrors:[] ClientInfo:{Debug:false Plugins:[] Warnings:<nil>}}
I0810 22:32:01.825182 328117 start_flags.go:263] no existing cluster config was found, will generate one from the flags
I0810 22:32:01.825332 328117 start_flags.go:697] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0810 22:32:01.825354 328117 cni.go:93] Creating CNI manager for "calico"
I0810 22:32:01.825360 328117 start_flags.go:272] Found "Calico" CNI - setting NetworkPlugin=cni
I0810 22:32:01.825368 328117 start_flags.go:277] config:
{Name:calico-20210810222820-95538 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.21.3 ClusterName:calico-20210810222820-95538 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: Ne
tworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:calico NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:5m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false ExtraDisks:0}
I0810 22:32:01.827163 328117 out.go:177] * Starting control plane node calico-20210810222820-95538 in cluster calico-20210810222820-95538
I0810 22:32:01.827199 328117 cache.go:117] Beginning downloading kic base image for docker with docker
I0810 22:32:01.828563 328117 out.go:177] * Pulling base image ...
I0810 22:32:01.828592 328117 preload.go:131] Checking if preload exists for k8s version v1.21.3 and runtime docker
I0810 22:32:01.828622 328117 preload.go:147] Found local preload: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v11-v1.21.3-docker-overlay2-amd64.tar.lz4
I0810 22:32:01.828659 328117 cache.go:56] Caching tarball of preloaded images
I0810 22:32:01.828665 328117 image.go:75] Checking for gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 in local docker daemon
I0810 22:32:01.828832 328117 preload.go:173] Found /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v11-v1.21.3-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0810 22:32:01.828863 328117 cache.go:59] Finished verifying existence of preloaded tar for v1.21.3 on docker
I0810 22:32:01.828979 328117 profile.go:148] Saving config to /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/config.json ...
I0810 22:32:01.829005 328117 lock.go:36] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/config.json: {Name:mk95c3e484757d34ea06258d82e5bf4f38d38e40 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0810 22:32:01.905695 328117 image.go:79] Found gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 in local docker daemon, skipping pull
I0810 22:32:01.905721 328117 cache.go:139] gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 exists in daemon, skipping load
I0810 22:32:01.905735 328117 cache.go:205] Successfully downloaded all kic artifacts
I0810 22:32:01.905771 328117 start.go:313] acquiring machines lock for calico-20210810222820-95538: {Name:mk1d4dc9d1e00fc4e8cbccbef1e7313b1a46f96c Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0810 22:32:01.905899 328117 start.go:317] acquired machines lock for "calico-20210810222820-95538" in 101.293µs
I0810 22:32:01.905922 328117 start.go:89] Provisioning new machine with config: &{Name:calico-20210810222820-95538 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.21.3 ClusterName:calico-20210810222820-95538 Namespace:default APIServerName:minikubeCA APIServerName
s:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:calico NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.21.3 ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:5m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false ExtraDisks:0} &{Name: IP: Port:8443 KubernetesVersion:v1.21.3 ControlPlane:true Worker:true}
I0810 22:32:01.905997 328117 start.go:126] createHost starting for "" (driver="docker")
I0810 22:32:01.908165 328117 out.go:204] * Creating docker container (CPUs=2, Memory=2048MB) ...
I0810 22:32:01.908368 328117 start.go:160] libmachine.API.Create for "calico-20210810222820-95538" (driver="docker")
I0810 22:32:01.908394 328117 client.go:168] LocalClient.Create starting
I0810 22:32:01.908454 328117 main.go:130] libmachine: Reading certificate data from /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/ca.pem
I0810 22:32:01.908515 328117 main.go:130] libmachine: Decoding PEM data...
I0810 22:32:01.908540 328117 main.go:130] libmachine: Parsing certificate...
I0810 22:32:01.908682 328117 main.go:130] libmachine: Reading certificate data from /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/cert.pem
I0810 22:32:01.908708 328117 main.go:130] libmachine: Decoding PEM data...
I0810 22:32:01.908726 328117 main.go:130] libmachine: Parsing certificate...
I0810 22:32:01.909204 328117 cli_runner.go:115] Run: docker network inspect calico-20210810222820-95538 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0810 22:32:01.950881 328117 cli_runner.go:162] docker network inspect calico-20210810222820-95538 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0810 22:32:01.950948 328117 network_create.go:255] running [docker network inspect calico-20210810222820-95538] to gather additional debugging logs...
I0810 22:32:01.950966 328117 cli_runner.go:115] Run: docker network inspect calico-20210810222820-95538
W0810 22:32:01.991818 328117 cli_runner.go:162] docker network inspect calico-20210810222820-95538 returned with exit code 1
I0810 22:32:01.991847 328117 network_create.go:258] error running [docker network inspect calico-20210810222820-95538]: docker network inspect calico-20210810222820-95538: exit status 1
stdout:
[]
stderr:
Error: No such network: calico-20210810222820-95538
I0810 22:32:01.991880 328117 network_create.go:260] output of [docker network inspect calico-20210810222820-95538]: -- stdout --
[]
-- /stdout --
** stderr **
Error: No such network: calico-20210810222820-95538
** /stderr **
I0810 22:32:01.991971 328117 cli_runner.go:115] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0810 22:32:02.033188 328117 network.go:240] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 Interface:{IfaceName:br-76460e7e925a IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:02:42:6a:60:3a:43}}
I0810 22:32:02.037022 328117 network.go:240] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 Interface:{IfaceName:br-f1326809ab6d IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:02:42:db:3e:63:af}}
I0810 22:32:02.039175 328117 network.go:240] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 Interface:{IfaceName:br-9ebc8479ad43 IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:02:42:f8:8c:4d:77}}
I0810 22:32:02.041346 328117 network.go:288] reserving subnet 192.168.76.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[] amended:true}} dirty:map[192.168.76.0:0xc0006a0018] misses:0}
I0810 22:32:02.041384 328117 network.go:235] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0810 22:32:02.041397 328117 network_create.go:106] attempt to create docker network calico-20210810222820-95538 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0810 22:32:02.041450 328117 cli_runner.go:115] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true calico-20210810222820-95538
I0810 22:32:02.132378 328117 network_create.go:90] docker network calico-20210810222820-95538 192.168.76.0/24 created
I0810 22:32:02.132416 328117 kic.go:106] calculated static IP "192.168.76.2" for the "calico-20210810222820-95538" container
I0810 22:32:02.132481 328117 cli_runner.go:115] Run: docker ps -a --format {{.Names}}
I0810 22:32:02.187141 328117 cli_runner.go:115] Run: docker volume create calico-20210810222820-95538 --label name.minikube.sigs.k8s.io=calico-20210810222820-95538 --label created_by.minikube.sigs.k8s.io=true
I0810 22:32:02.229158 328117 oci.go:102] Successfully created a docker volume calico-20210810222820-95538
I0810 22:32:02.229225 328117 cli_runner.go:115] Run: docker run --rm --name calico-20210810222820-95538-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=calico-20210810222820-95538 --entrypoint /usr/bin/test -v calico-20210810222820-95538:/var gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 -d /var/lib
I0810 22:32:02.998580 328117 oci.go:106] Successfully prepared a docker volume calico-20210810222820-95538
W0810 22:32:02.998632 328117 oci.go:135] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
W0810 22:32:02.998639 328117 oci.go:119] Your kernel does not support memory limit capabilities or the cgroup is not mounted.
I0810 22:32:02.998687 328117 cli_runner.go:115] Run: docker info --format "'{{json .SecurityOptions}}'"
I0810 22:32:02.998725 328117 preload.go:131] Checking if preload exists for k8s version v1.21.3 and runtime docker
I0810 22:32:02.998760 328117 kic.go:179] Starting extracting preloaded images to volume ...
I0810 22:32:02.998860 328117 cli_runner.go:115] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v11-v1.21.3-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v calico-20210810222820-95538:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 -I lz4 -xf /preloaded.tar -C /extractDir
I0810 22:32:03.088694 328117 cli_runner.go:115] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname calico-20210810222820-95538 --name calico-20210810222820-95538 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=calico-20210810222820-95538 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=calico-20210810222820-95538 --network calico-20210810222820-95538 --ip 192.168.76.2 --volume calico-20210810222820-95538:/var --security-opt apparmor=unconfined --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79
I0810 22:32:03.662527 328117 cli_runner.go:115] Run: docker container inspect calico-20210810222820-95538 --format={{.State.Running}}
I0810 22:32:03.711667 328117 cli_runner.go:115] Run: docker container inspect calico-20210810222820-95538 --format={{.State.Status}}
I0810 22:32:03.756591 328117 cli_runner.go:115] Run: docker exec calico-20210810222820-95538 stat /var/lib/dpkg/alternatives/iptables
I0810 22:32:03.875363 328117 oci.go:278] the created container "calico-20210810222820-95538" has a running status.
I0810 22:32:03.875403 328117 kic.go:210] Creating ssh key for kic: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/calico-20210810222820-95538/id_rsa...
I0810 22:32:04.053889 328117 kic_runner.go:188] docker (temp): /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/calico-20210810222820-95538/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0810 22:32:04.483612 328117 cli_runner.go:115] Run: docker container inspect calico-20210810222820-95538 --format={{.State.Status}}
I0810 22:32:04.526978 328117 kic_runner.go:94] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0810 22:32:04.527000 328117 kic_runner.go:115] Args: [docker exec --privileged calico-20210810222820-95538 chown docker:docker /home/docker/.ssh/authorized_keys]
I0810 22:32:06.730333 328117 cli_runner.go:168] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v11-v1.21.3-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v calico-20210810222820-95538:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 -I lz4 -xf /preloaded.tar -C /extractDir: (3.731401432s)
I0810 22:32:06.730361 328117 kic.go:188] duration metric: took 3.731600 seconds to extract preloaded images to volume
I0810 22:32:06.730442 328117 cli_runner.go:115] Run: docker container inspect calico-20210810222820-95538 --format={{.State.Status}}
I0810 22:32:06.772162 328117 machine.go:88] provisioning docker machine ...
I0810 22:32:06.772207 328117 ubuntu.go:169] provisioning hostname "calico-20210810222820-95538"
I0810 22:32:06.772277 328117 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" calico-20210810222820-95538
I0810 22:32:06.813418 328117 main.go:130] libmachine: Using SSH client type: native
I0810 22:32:06.813609 328117 main.go:130] libmachine: &{{{<nil> 0 [] [] []} docker [0x802ea0] 0x802e60 <nil> [] 0s} 127.0.0.1 33049 <nil> <nil>}
I0810 22:32:06.813627 328117 main.go:130] libmachine: About to run SSH command:
sudo hostname calico-20210810222820-95538 && echo "calico-20210810222820-95538" | sudo tee /etc/hostname
I0810 22:32:06.940232 328117 main.go:130] libmachine: SSH cmd err, output: <nil>: calico-20210810222820-95538
I0810 22:32:06.940309 328117 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" calico-20210810222820-95538
I0810 22:32:06.980048 328117 main.go:130] libmachine: Using SSH client type: native
I0810 22:32:06.980192 328117 main.go:130] libmachine: &{{{<nil> 0 [] [] []} docker [0x802ea0] 0x802e60 <nil> [] 0s} 127.0.0.1 33049 <nil> <nil>}
I0810 22:32:06.980211 328117 main.go:130] libmachine: About to run SSH command:
if ! grep -xq '.*\scalico-20210810222820-95538' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 calico-20210810222820-95538/g' /etc/hosts;
else
echo '127.0.1.1 calico-20210810222820-95538' | sudo tee -a /etc/hosts;
fi
fi
I0810 22:32:07.092058 328117 main.go:130] libmachine: SSH cmd err, output: <nil>:
I0810 22:32:07.092085 328117 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube CaCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/key.pem Serve
rCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube}
I0810 22:32:07.092112 328117 ubuntu.go:177] setting up certificates
I0810 22:32:07.092125 328117 provision.go:83] configureAuth start
I0810 22:32:07.092175 328117 cli_runner.go:115] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" calico-20210810222820-95538
I0810 22:32:07.143056 328117 provision.go:137] copyHostCerts
I0810 22:32:07.143140 328117 exec_runner.go:145] found /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/ca.pem, removing ...
I0810 22:32:07.143153 328117 exec_runner.go:190] rm: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/ca.pem
I0810 22:32:07.143210 328117 exec_runner.go:152] cp: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/ca.pem (1082 bytes)
I0810 22:32:07.143339 328117 exec_runner.go:145] found /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/cert.pem, removing ...
I0810 22:32:07.143351 328117 exec_runner.go:190] rm: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/cert.pem
I0810 22:32:07.143369 328117 exec_runner.go:152] cp: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/cert.pem (1123 bytes)
I0810 22:32:07.143429 328117 exec_runner.go:145] found /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/key.pem, removing ...
I0810 22:32:07.143439 328117 exec_runner.go:190] rm: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/key.pem
I0810 22:32:07.143455 328117 exec_runner.go:152] cp: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/key.pem (1679 bytes)
I0810 22:32:07.143496 328117 provision.go:111] generating server cert: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/ca-key.pem org=jenkins.calico-20210810222820-95538 san=[192.168.76.2 127.0.0.1 localhost 127.0.0.1 minikube calico-20210810222820-95538]
I0810 22:32:07.378214 328117 provision.go:171] copyRemoteCerts
I0810 22:32:07.378278 328117 ssh_runner.go:149] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0810 22:32:07.378336 328117 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" calico-20210810222820-95538
I0810 22:32:07.418256 328117 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33049 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/calico-20210810222820-95538/id_rsa Username:docker}
I0810 22:32:07.503499 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0810 22:32:07.519757 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/server.pem --> /etc/docker/server.pem (1257 bytes)
I0810 22:32:07.535869 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0810 22:32:07.551418 328117 provision.go:86] duration metric: configureAuth took 459.27925ms
I0810 22:32:07.551440 328117 ubuntu.go:193] setting minikube options for container-runtime
I0810 22:32:07.551646 328117 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" calico-20210810222820-95538
I0810 22:32:07.592773 328117 main.go:130] libmachine: Using SSH client type: native
I0810 22:32:07.592940 328117 main.go:130] libmachine: &{{{<nil> 0 [] [] []} docker [0x802ea0] 0x802e60 <nil> [] 0s} 127.0.0.1 33049 <nil> <nil>}
I0810 22:32:07.592973 328117 main.go:130] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0810 22:32:07.712880 328117 main.go:130] libmachine: SSH cmd err, output: <nil>: overlay
I0810 22:32:07.712909 328117 ubuntu.go:71] root file system type: overlay
I0810 22:32:07.713085 328117 provision.go:308] Updating docker unit: /lib/systemd/system/docker.service ...
I0810 22:32:07.713156 328117 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" calico-20210810222820-95538
I0810 22:32:07.761828 328117 main.go:130] libmachine: Using SSH client type: native
I0810 22:32:07.762023 328117 main.go:130] libmachine: &{{{<nil> 0 [] [] []} docker [0x802ea0] 0x802e60 <nil> [] 0s} 127.0.0.1 33049 <nil> <nil>}
I0810 22:32:07.762091 328117 main.go:130] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0810 22:32:07.888748 328117 main.go:130] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0810 22:32:07.888829 328117 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" calico-20210810222820-95538
I0810 22:32:07.928356 328117 main.go:130] libmachine: Using SSH client type: native
I0810 22:32:07.928506 328117 main.go:130] libmachine: &{{{<nil> 0 [] [] []} docker [0x802ea0] 0x802e60 <nil> [] 0s} 127.0.0.1 33049 <nil> <nil>}
I0810 22:32:07.928525 328117 main.go:130] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0810 22:32:08.586683 328117 main.go:130] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2021-06-02 11:54:50.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2021-08-10 22:32:07.881870291 +0000
@@ -1,30 +1,32 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
+BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
-Requires=docker.socket containerd.service
+Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutSec=0
-RestartSec=2
-Restart=always
-
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
+Restart=on-failure
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
@@ -32,16 +34,16 @@
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0810 22:32:08.586723 328117 machine.go:91] provisioned docker machine in 1.814539627s
I0810 22:32:08.586735 328117 client.go:171] LocalClient.Create took 6.678332853s
I0810 22:32:08.586747 328117 start.go:168] duration metric: libmachine.API.Create for "calico-20210810222820-95538" took 6.678378953s
I0810 22:32:08.586760 328117 start.go:267] post-start starting for "calico-20210810222820-95538" (driver="docker")
I0810 22:32:08.586768 328117 start.go:277] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0810 22:32:08.586826 328117 ssh_runner.go:149] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0810 22:32:08.586876 328117 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" calico-20210810222820-95538
I0810 22:32:08.630314 328117 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33049 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/calico-20210810222820-95538/id_rsa Username:docker}
I0810 22:32:08.716645 328117 ssh_runner.go:149] Run: cat /etc/os-release
I0810 22:32:08.719817 328117 main.go:130] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0810 22:32:08.719843 328117 main.go:130] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0810 22:32:08.719854 328117 main.go:130] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0810 22:32:08.719865 328117 info.go:137] Remote host: Ubuntu 20.04.2 LTS
I0810 22:32:08.719879 328117 filesync.go:126] Scanning /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/addons for local assets ...
I0810 22:32:08.719926 328117 filesync.go:126] Scanning /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/files for local assets ...
I0810 22:32:08.720029 328117 filesync.go:149] local asset: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/files/etc/ssl/certs/955382.pem -> 955382.pem in /etc/ssl/certs
I0810 22:32:08.720147 328117 ssh_runner.go:149] Run: sudo mkdir -p /etc/ssl/certs
I0810 22:32:08.726531 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/files/etc/ssl/certs/955382.pem --> /etc/ssl/certs/955382.pem (1708 bytes)
I0810 22:32:08.742570 328117 start.go:270] post-start completed in 155.794363ms
I0810 22:32:08.742860 328117 cli_runner.go:115] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" calico-20210810222820-95538
I0810 22:32:08.781837 328117 profile.go:148] Saving config to /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/config.json ...
I0810 22:32:08.782066 328117 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0810 22:32:08.782120 328117 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" calico-20210810222820-95538
I0810 22:32:08.818586 328117 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33049 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/calico-20210810222820-95538/id_rsa Username:docker}
I0810 22:32:08.900598 328117 start.go:129] duration metric: createHost completed in 6.994589781s
I0810 22:32:08.900620 328117 start.go:80] releasing machines lock for "calico-20210810222820-95538", held for 6.994710421s
I0810 22:32:08.900711 328117 cli_runner.go:115] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" calico-20210810222820-95538
I0810 22:32:08.938994 328117 ssh_runner.go:149] Run: systemctl --version
I0810 22:32:08.939056 328117 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" calico-20210810222820-95538
I0810 22:32:08.939056 328117 ssh_runner.go:149] Run: curl -sS -m 2 https://k8s.gcr.io/
I0810 22:32:08.939127 328117 cli_runner.go:115] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" calico-20210810222820-95538
I0810 22:32:08.983265 328117 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33049 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/calico-20210810222820-95538/id_rsa Username:docker}
I0810 22:32:08.987101 328117 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33049 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/machines/calico-20210810222820-95538/id_rsa Username:docker}
I0810 22:32:09.064743 328117 ssh_runner.go:149] Run: sudo systemctl is-active --quiet service containerd
I0810 22:32:09.112944 328117 ssh_runner.go:149] Run: sudo systemctl cat docker.service
I0810 22:32:09.123205 328117 cruntime.go:249] skipping containerd shutdown because we are bound to it
I0810 22:32:09.123261 328117 ssh_runner.go:149] Run: sudo systemctl is-active --quiet service crio
I0810 22:32:09.132658 328117 ssh_runner.go:149] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
image-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0810 22:32:09.148935 328117 ssh_runner.go:149] Run: sudo systemctl unmask docker.service
I0810 22:32:09.230233 328117 ssh_runner.go:149] Run: sudo systemctl enable docker.socket
I0810 22:32:09.297713 328117 ssh_runner.go:149] Run: sudo systemctl cat docker.service
I0810 22:32:09.306874 328117 ssh_runner.go:149] Run: sudo systemctl daemon-reload
I0810 22:32:09.366392 328117 ssh_runner.go:149] Run: sudo systemctl start docker
I0810 22:32:09.376911 328117 ssh_runner.go:149] Run: docker version --format {{.Server.Version}}
I0810 22:32:09.426750 328117 ssh_runner.go:149] Run: docker version --format {{.Server.Version}}
I0810 22:32:09.475649 328117 out.go:204] * Preparing Kubernetes v1.21.3 on Docker 20.10.7 ...
I0810 22:32:09.475726 328117 cli_runner.go:115] Run: docker network inspect calico-20210810222820-95538 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0810 22:32:09.520564 328117 ssh_runner.go:149] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0810 22:32:09.523780 328117 ssh_runner.go:149] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0810 22:32:09.532763 328117 preload.go:131] Checking if preload exists for k8s version v1.21.3 and runtime docker
I0810 22:32:09.532829 328117 ssh_runner.go:149] Run: docker images --format {{.Repository}}:{{.Tag}}
I0810 22:32:09.574105 328117 docker.go:535] Got preloaded images: -- stdout --
k8s.gcr.io/kube-apiserver:v1.21.3
k8s.gcr.io/kube-scheduler:v1.21.3
k8s.gcr.io/kube-proxy:v1.21.3
k8s.gcr.io/kube-controller-manager:v1.21.3
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/pause:3.4.1
kubernetesui/dashboard:v2.1.0
k8s.gcr.io/coredns/coredns:v1.8.0
k8s.gcr.io/etcd:3.4.13-0
kubernetesui/metrics-scraper:v1.0.4
-- /stdout --
I0810 22:32:09.574130 328117 docker.go:466] Images already preloaded, skipping extraction
I0810 22:32:09.574179 328117 ssh_runner.go:149] Run: docker images --format {{.Repository}}:{{.Tag}}
I0810 22:32:09.618793 328117 docker.go:535] Got preloaded images: -- stdout --
k8s.gcr.io/kube-apiserver:v1.21.3
k8s.gcr.io/kube-scheduler:v1.21.3
k8s.gcr.io/kube-proxy:v1.21.3
k8s.gcr.io/kube-controller-manager:v1.21.3
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/pause:3.4.1
kubernetesui/dashboard:v2.1.0
k8s.gcr.io/coredns/coredns:v1.8.0
k8s.gcr.io/etcd:3.4.13-0
kubernetesui/metrics-scraper:v1.0.4
-- /stdout --
I0810 22:32:09.618824 328117 cache_images.go:74] Images are preloaded, skipping loading
I0810 22:32:09.618882 328117 ssh_runner.go:149] Run: docker info --format {{.CgroupDriver}}
I0810 22:32:09.715054 328117 cni.go:93] Creating CNI manager for "calico"
I0810 22:32:09.715080 328117 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0810 22:32:09.715092 328117 kubeadm.go:153] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.21.3 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:calico-20210810222820-95538 NodeName:calico-20210810222820-95538 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NoTaintMaster:true NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/
minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[]}
I0810 22:32:09.715212 328117 kubeadm.go:157] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "calico-20210810222820-95538"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.21.3
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0810 22:32:09.715282 328117 kubeadm.go:909] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.21.3/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=calico-20210810222820-95538 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.21.3 ClusterName:calico-20210810222820-95538 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:calico NodeIP: NodePort:8443 NodeName:}
I0810 22:32:09.715333 328117 ssh_runner.go:149] Run: sudo ls /var/lib/minikube/binaries/v1.21.3
I0810 22:32:09.723656 328117 binaries.go:44] Found k8s binaries, skipping transfer
I0810 22:32:09.723723 328117 ssh_runner.go:149] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0810 22:32:09.732002 328117 ssh_runner.go:316] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (374 bytes)
I0810 22:32:09.743803 328117 ssh_runner.go:316] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0810 22:32:09.755317 328117 ssh_runner.go:316] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2070 bytes)
I0810 22:32:09.766902 328117 ssh_runner.go:149] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0810 22:32:09.769635 328117 ssh_runner.go:149] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0810 22:32:09.777905 328117 certs.go:52] Setting up /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538 for IP: 192.168.76.2
I0810 22:32:09.777954 328117 certs.go:179] skipping minikubeCA CA generation: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/ca.key
I0810 22:32:09.777980 328117 certs.go:179] skipping proxyClientCA CA generation: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/proxy-client-ca.key
I0810 22:32:09.778031 328117 certs.go:294] generating minikube-user signed cert: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/client.key
I0810 22:32:09.778046 328117 crypto.go:69] Generating cert /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/client.crt with IP's: []
I0810 22:32:09.947512 328117 crypto.go:157] Writing cert to /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/client.crt ...
I0810 22:32:09.947542 328117 lock.go:36] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/client.crt: {Name:mk0bf84574059375efab805864a632f72f983f6f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0810 22:32:09.947728 328117 crypto.go:165] Writing key to /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/client.key ...
I0810 22:32:09.947747 328117 lock.go:36] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/client.key: {Name:mk16f91d8c144d3bfe3ef015f460fa4c9a3c7194 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0810 22:32:09.947846 328117 certs.go:294] generating minikube signed cert: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.key.31bdca25
I0810 22:32:09.947857 328117 crypto.go:69] Generating cert /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.crt.31bdca25 with IP's: [192.168.76.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0810 22:32:10.030210 328117 crypto.go:157] Writing cert to /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.crt.31bdca25 ...
I0810 22:32:10.030241 328117 lock.go:36] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.crt.31bdca25: {Name:mk0cfca724bee62a5c10aaeec75aef642d61055b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0810 22:32:10.030945 328117 crypto.go:165] Writing key to /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.key.31bdca25 ...
I0810 22:32:10.030973 328117 lock.go:36] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.key.31bdca25: {Name:mk5141755f6dad14a4306980db76a9cb7471a3ba Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0810 22:32:10.031111 328117 certs.go:305] copying /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.crt.31bdca25 -> /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.crt
I0810 22:32:10.031186 328117 certs.go:309] copying /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.key.31bdca25 -> /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.key
I0810 22:32:10.031249 328117 certs.go:294] generating aggregator signed cert: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/proxy-client.key
I0810 22:32:10.031263 328117 crypto.go:69] Generating cert /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/proxy-client.crt with IP's: []
I0810 22:32:10.178277 328117 crypto.go:157] Writing cert to /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/proxy-client.crt ...
I0810 22:32:10.178310 328117 lock.go:36] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/proxy-client.crt: {Name:mk43e01cf4f9c28038dabbe065b8c92ac1b665f6 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0810 22:32:10.178472 328117 crypto.go:165] Writing key to /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/proxy-client.key ...
I0810 22:32:10.178486 328117 lock.go:36] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/proxy-client.key: {Name:mka5420909174a481d1e7299e80295ef6e57ef18 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0810 22:32:10.178654 328117 certs.go:373] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/95538.pem (1338 bytes)
W0810 22:32:10.178690 328117 certs.go:369] ignoring /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/95538_empty.pem, impossibly tiny 0 bytes
I0810 22:32:10.178700 328117 certs.go:373] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/ca-key.pem (1675 bytes)
I0810 22:32:10.178721 328117 certs.go:373] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/ca.pem (1082 bytes)
I0810 22:32:10.178741 328117 certs.go:373] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/cert.pem (1123 bytes)
I0810 22:32:10.178762 328117 certs.go:373] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/key.pem (1679 bytes)
I0810 22:32:10.178804 328117 certs.go:373] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/files/etc/ssl/certs/home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/files/etc/ssl/certs/955382.pem (1708 bytes)
I0810 22:32:10.179867 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0810 22:32:10.197479 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0810 22:32:10.252921 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0810 22:32:10.271026 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/profiles/calico-20210810222820-95538/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0810 22:32:10.288304 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0810 22:32:10.305292 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0810 22:32:10.321091 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0810 22:32:10.336590 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0810 22:32:10.352078 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/files/etc/ssl/certs/955382.pem --> /usr/share/ca-certificates/955382.pem (1708 bytes)
I0810 22:32:10.368197 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0810 22:32:10.383491 328117 ssh_runner.go:316] scp /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/certs/95538.pem --> /usr/share/ca-certificates/95538.pem (1338 bytes)
I0810 22:32:10.398804 328117 ssh_runner.go:316] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0810 22:32:10.422041 328117 ssh_runner.go:149] Run: openssl version
I0810 22:32:10.428080 328117 ssh_runner.go:149] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0810 22:32:10.436658 328117 ssh_runner.go:149] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0810 22:32:10.440020 328117 certs.go:416] hashing: -rw-r--r-- 1 root root 1111 Aug 10 22:07 /usr/share/ca-certificates/minikubeCA.pem
I0810 22:32:10.440073 328117 ssh_runner.go:149] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0810 22:32:10.444990 328117 ssh_runner.go:149] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0810 22:32:10.452993 328117 ssh_runner.go:149] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/95538.pem && ln -fs /usr/share/ca-certificates/95538.pem /etc/ssl/certs/95538.pem"
I0810 22:32:10.461305 328117 ssh_runner.go:149] Run: ls -la /usr/share/ca-certificates/95538.pem
I0810 22:32:10.464722 328117 certs.go:416] hashing: -rw-r--r-- 1 root root 1338 Aug 10 22:11 /usr/share/ca-certificates/95538.pem
I0810 22:32:10.464777 328117 ssh_runner.go:149] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/95538.pem
I0810 22:32:10.469385 328117 ssh_runner.go:149] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/95538.pem /etc/ssl/certs/51391683.0"
I0810 22:32:10.476661 328117 ssh_runner.go:149] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/955382.pem && ln -fs /usr/share/ca-certificates/955382.pem /etc/ssl/certs/955382.pem"
I0810 22:32:10.484032 328117 ssh_runner.go:149] Run: ls -la /usr/share/ca-certificates/955382.pem
I0810 22:32:10.486813 328117 certs.go:416] hashing: -rw-r--r-- 1 root root 1708 Aug 10 22:11 /usr/share/ca-certificates/955382.pem
I0810 22:32:10.486859 328117 ssh_runner.go:149] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/955382.pem
I0810 22:32:10.491384 328117 ssh_runner.go:149] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/955382.pem /etc/ssl/certs/3ec20f2e.0"
I0810 22:32:10.498421 328117 kubeadm.go:390] StartCluster: {Name:calico-20210810222820-95538 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.25@sha256:6f936e3443b95cd918d77623bf7b595653bb382766e280290a02b4a349e88b79 Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.21.3 ClusterName:calico-20210810222820-95538 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] D
NSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:calico NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.21.3 ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:5m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: MultiNodeRequested:false ExtraDisks:0}
I0810 22:32:10.498542 328117 ssh_runner.go:149] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0810 22:32:10.538132 328117 ssh_runner.go:149] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0810 22:32:10.546325 328117 ssh_runner.go:149] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0810 22:32:10.552923 328117 kubeadm.go:220] ignoring SystemVerification for kubeadm because of docker driver
I0810 22:32:10.552974 328117 ssh_runner.go:149] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0810 22:32:10.559517 328117 kubeadm.go:151] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0810 22:32:10.559573 328117 ssh_runner.go:240] Start: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.3:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0810 22:32:11.422492 328117 out.go:204] - Generating certificates and keys ...
I0810 22:32:13.953690 328117 out.go:204] - Booting up control plane ...
I0810 22:32:23.504607 328117 out.go:204] - Configuring RBAC rules ...
I0810 22:32:23.917253 328117 cni.go:93] Creating CNI manager for "calico"
I0810 22:32:23.922115 328117 out.go:177] * Configuring Calico (Container Networking Interface) ...
I0810 22:32:23.922183 328117 cni.go:187] applying CNI manifest using /var/lib/minikube/binaries/v1.21.3/kubectl ...
I0810 22:32:23.922196 328117 ssh_runner.go:316] scp memory --> /var/tmp/minikube/cni.yaml (22469 bytes)
I0810 22:32:23.937700 328117 ssh_runner.go:149] Run: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml
W0810 22:32:24.379546 328117 out.go:242] ! initialization failed, will try again: apply cni: cni apply: cmd: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml output: -- stdout --
configmap/calico-config created
-- /stdout --
** stderr **
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
** /stderr **: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml: Process exited with status 1
stdout:
configmap/calico-config created
stderr:
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
! initialization failed, will try again: apply cni: cni apply: cmd: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml output: -- stdout --
configmap/calico-config created
-- /stdout --
** stderr **
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
** /stderr **: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml: Process exited with status 1
stdout:
configmap/calico-config created
stderr:
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
I0810 22:32:24.379602 328117 ssh_runner.go:149] Run: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.3:$PATH kubeadm reset --cri-socket /var/run/dockershim.sock --force"
I0810 22:32:37.758903 328117 ssh_runner.go:189] Completed: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.3:$PATH kubeadm reset --cri-socket /var/run/dockershim.sock --force": (13.379280746s)
I0810 22:32:37.758962 328117 ssh_runner.go:149] Run: sudo systemctl stop -f kubelet
I0810 22:32:37.769957 328117 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0810 22:32:37.809587 328117 kubeadm.go:220] ignoring SystemVerification for kubeadm because of docker driver
I0810 22:32:37.809653 328117 ssh_runner.go:149] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0810 22:32:37.839076 328117 kubeadm.go:151] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0810 22:32:37.839122 328117 ssh_runner.go:240] Start: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.21.3:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0810 22:32:39.684798 328117 out.go:204] - Generating certificates and keys ...
I0810 22:32:40.352246 328117 out.go:204] - Booting up control plane ...
I0810 22:32:51.625587 328117 out.go:204] - Configuring RBAC rules ...
I0810 22:32:52.039910 328117 cni.go:93] Creating CNI manager for "calico"
I0810 22:32:52.041867 328117 out.go:177] * Configuring Calico (Container Networking Interface) ...
I0810 22:32:52.041976 328117 cni.go:187] applying CNI manifest using /var/lib/minikube/binaries/v1.21.3/kubectl ...
I0810 22:32:52.041992 328117 ssh_runner.go:316] scp memory --> /var/tmp/minikube/cni.yaml (22469 bytes)
I0810 22:32:52.061071 328117 ssh_runner.go:149] Run: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml
I0810 22:32:52.358561 328117 kubeadm.go:392] StartCluster complete in 41.860138409s
I0810 22:32:52.358659 328117 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0810 22:32:52.440701 328117 logs.go:270] 1 containers: [4b8cf3cc848e]
I0810 22:32:52.440780 328117 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0810 22:32:52.541951 328117 logs.go:270] 1 containers: [c5d7cefb2dda]
I0810 22:32:52.542037 328117 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0810 22:32:52.651602 328117 logs.go:270] 0 containers: []
W0810 22:32:52.651634 328117 logs.go:272] No container was found matching "coredns"
I0810 22:32:52.651694 328117 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0810 22:32:52.742280 328117 logs.go:270] 1 containers: [b93240cbc2b3]
I0810 22:32:52.742363 328117 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0810 22:32:52.802857 328117 logs.go:270] 0 containers: []
W0810 22:32:52.802886 328117 logs.go:272] No container was found matching "kube-proxy"
I0810 22:32:52.802936 328117 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0810 22:32:52.854681 328117 logs.go:270] 0 containers: []
W0810 22:32:52.854704 328117 logs.go:272] No container was found matching "kubernetes-dashboard"
I0810 22:32:52.854754 328117 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0810 22:32:52.905135 328117 logs.go:270] 0 containers: []
W0810 22:32:52.905155 328117 logs.go:272] No container was found matching "storage-provisioner"
I0810 22:32:52.905205 328117 ssh_runner.go:149] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0810 22:32:52.940805 328117 logs.go:270] 1 containers: [f0281aa36c46]
I0810 22:32:52.940839 328117 logs.go:123] Gathering logs for kube-controller-manager [f0281aa36c46] ...
I0810 22:32:52.940851 328117 ssh_runner.go:149] Run: /bin/bash -c "docker logs --tail 400 f0281aa36c46"
I0810 22:32:52.983838 328117 logs.go:123] Gathering logs for Docker ...
I0810 22:32:52.983866 328117 ssh_runner.go:149] Run: /bin/bash -c "sudo journalctl -u docker -n 400"
I0810 22:32:53.001482 328117 logs.go:123] Gathering logs for kubelet ...
I0810 22:32:53.001511 328117 ssh_runner.go:149] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0810 22:32:53.085921 328117 logs.go:123] Gathering logs for dmesg ...
I0810 22:32:53.085960 328117 ssh_runner.go:149] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0810 22:32:53.105004 328117 logs.go:123] Gathering logs for kube-scheduler [b93240cbc2b3] ...
I0810 22:32:53.105041 328117 ssh_runner.go:149] Run: /bin/bash -c "docker logs --tail 400 b93240cbc2b3"
I0810 22:32:53.159750 328117 logs.go:123] Gathering logs for container status ...
I0810 22:32:53.159794 328117 ssh_runner.go:149] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0810 22:32:53.246741 328117 logs.go:123] Gathering logs for describe nodes ...
I0810 22:32:53.246767 328117 ssh_runner.go:149] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.21.3/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0810 22:32:53.380723 328117 logs.go:123] Gathering logs for kube-apiserver [4b8cf3cc848e] ...
I0810 22:32:53.380757 328117 ssh_runner.go:149] Run: /bin/bash -c "docker logs --tail 400 4b8cf3cc848e"
I0810 22:32:53.433824 328117 logs.go:123] Gathering logs for etcd [c5d7cefb2dda] ...
I0810 22:32:53.433852 328117 ssh_runner.go:149] Run: /bin/bash -c "docker logs --tail 400 c5d7cefb2dda"
W0810 22:32:53.490219 328117 out.go:371] Error starting cluster: apply cni: cni apply: cmd: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml output: -- stdout --
configmap/calico-config created
-- /stdout --
** stderr **
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
** /stderr **: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml: Process exited with status 1
stdout:
configmap/calico-config created
stderr:
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
W0810 22:32:53.490269 328117 out.go:242] *
*
W0810 22:32:53.490414 328117 out.go:242] X Error starting cluster: apply cni: cni apply: cmd: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml output: -- stdout --
configmap/calico-config created
-- /stdout --
** stderr **
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
** /stderr **: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml: Process exited with status 1
stdout:
configmap/calico-config created
stderr:
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
X Error starting cluster: apply cni: cni apply: cmd: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml output: -- stdout --
configmap/calico-config created
-- /stdout --
** stderr **
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
** /stderr **: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml: Process exited with status 1
stdout:
configmap/calico-config created
stderr:
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
W0810 22:32:53.490433 328117 out.go:242] *
*
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
W0810 22:32:53.492566 328117 out.go:242] ╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please attach the following file to the GitHub issue: │
│ * - /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/logs/lastStart.txt │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please attach the following file to the GitHub issue: │
│ * - /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/logs/lastStart.txt │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
I0810 22:32:53.494952 328117 out.go:177]
W0810 22:32:53.495078 328117 out.go:242] X Exiting due to GUEST_START: apply cni: cni apply: cmd: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml output: -- stdout --
configmap/calico-config created
-- /stdout --
** stderr **
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
** /stderr **: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml: Process exited with status 1
stdout:
configmap/calico-config created
stderr:
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
X Exiting due to GUEST_START: apply cni: cni apply: cmd: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml output: -- stdout --
configmap/calico-config created
-- /stdout --
** stderr **
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
** /stderr **: sudo /var/lib/minikube/binaries/v1.21.3/kubectl apply --kubeconfig=/var/lib/minikube/kubeconfig -f /var/tmp/minikube/cni.yaml: Process exited with status 1
stdout:
configmap/calico-config created
stderr:
error: error validating "/var/tmp/minikube/cni.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
W0810 22:32:53.495092 328117 out.go:242] *
*
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
[warning]: invalid value provided to Color, using default
W0810 22:32:53.497152 328117 out.go:242] ╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please attach the following file to the GitHub issue: │
│ * - /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/logs/lastStart.txt │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please attach the following file to the GitHub issue: │
│ * - /home/jenkins/minikube-integration/linux-amd64-docker-docker-12230-92576-4e32f41c836e9c021a12ab8ec720ab6aea4bc3f0/.minikube/logs/lastStart.txt │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
I0810 22:32:53.498466 328117 out.go:177]
** /stderr **
net_test.go:100: failed start: exit status 80
--- FAIL: TestNetworkPlugins/group/calico/Start (52.04s)