I0204 00:33:29.497025 12 serving.go:386] Generated self-signed cert in-memory I0204 00:33:29.880734 12 serving.go:386] Generated self-signed cert in-memory I0204 00:33:30.323370 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 00:33:30.323422 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0204 00:33:30.323430 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true I0204 00:33:30.323435 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0204 00:33:30.323441 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false W0204 00:33:32.850935 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true I0204 00:33:35.411926 12 serving.go:386] Generated self-signed cert in-memory I0204 00:33:35.541383 12 serving.go:386] Generated self-signed cert in-memory I0204 00:33:36.171240 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0204 00:33:36.171273 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false I0204 00:33:36.171280 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 00:33:36.171286 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0204 00:33:36.171292 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true W0204 00:33:36.178413 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true I0204 00:33:49.365233 12 serving.go:386] Generated self-signed cert in-memory I0204 00:33:49.654045 12 serving.go:386] Generated self-signed cert in-memory I0204 00:33:49.820239 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0204 00:33:49.820274 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false I0204 00:33:49.820282 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 00:33:49.820289 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0204 00:33:49.820294 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true W0204 00:33:49.826993 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true I0204 00:34:19.328280 12 serving.go:386] Generated self-signed cert in-memory I0204 00:34:19.828131 12 serving.go:386] Generated self-signed cert in-memory I0204 00:34:20.012598 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 00:34:20.012633 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0204 00:34:20.012640 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true I0204 00:34:20.012647 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0204 00:34:20.012653 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false W0204 00:34:20.019569 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true I0204 00:35:06.143766 12 serving.go:386] Generated self-signed cert in-memory I0204 00:35:06.321439 12 serving.go:386] Generated self-signed cert in-memory I0204 00:35:06.494469 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 00:35:06.494505 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0204 00:35:06.494512 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true I0204 00:35:06.494519 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0204 00:35:06.494525 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false W0204 00:35:06.501140 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true I0204 00:36:41.332887 12 serving.go:386] Generated self-signed cert in-memory I0204 00:36:41.548193 12 serving.go:386] Generated self-signed cert in-memory I0204 00:36:41.670616 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true I0204 00:36:41.670652 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0204 00:36:41.670659 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false I0204 00:36:41.670665 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 00:36:41.670671 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false W0204 00:36:41.677969 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true I0204 00:39:24.165905 12 serving.go:386] Generated self-signed cert in-memory I0204 00:39:24.381124 12 serving.go:386] Generated self-signed cert in-memory I0204 00:39:24.649788 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0204 00:39:24.649844 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false I0204 00:39:24.649852 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 00:39:24.649858 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0204 00:39:24.649864 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true W0204 00:39:24.656794 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true I0204 00:44:25.301033 12 serving.go:386] Generated self-signed cert in-memory I0204 00:44:25.469333 12 serving.go:386] Generated self-signed cert in-memory I0204 00:44:25.758240 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false I0204 00:44:25.758286 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 00:44:25.758293 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0204 00:44:25.758300 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true I0204 00:44:25.758306 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false W0204 00:44:25.764837 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true I0204 00:49:46.289350 12 serving.go:386] Generated self-signed cert in-memory I0204 00:49:46.495578 12 serving.go:386] Generated self-signed cert in-memory I0204 00:49:46.823389 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 00:49:46.823424 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0204 00:49:46.823431 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true I0204 00:49:46.823437 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0204 00:49:46.823451 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false W0204 00:49:46.831425 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true I0204 00:55:00.355771 12 serving.go:386] Generated self-signed cert in-memory I0204 00:55:00.500852 12 serving.go:386] Generated self-signed cert in-memory I0204 00:55:00.686617 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 00:55:00.686656 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0204 00:55:00.686664 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true I0204 00:55:00.686671 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0204 00:55:00.686679 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false W0204 00:55:00.694017 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true I0204 01:00:04.609660 12 serving.go:386] Generated self-signed cert in-memory I0204 01:00:04.753997 12 serving.go:386] Generated self-signed cert in-memory I0204 01:00:05.120154 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 01:00:05.120191 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0204 01:00:05.120198 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true I0204 01:00:05.120204 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0204 01:00:05.120210 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false W0204 01:00:05.126981 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true I0204 01:05:08.160203 12 serving.go:386] Generated self-signed cert in-memory I0204 01:05:08.478734 12 serving.go:386] Generated self-signed cert in-memory I0204 01:05:08.596299 12 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false I0204 01:05:08.596333 12 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false I0204 01:05:08.596341 12 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false I0204 01:05:08.596347 12 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false I0204 01:05:08.596353 12 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true W0204 01:05:08.603066 12 requestheader_controller.go:204] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Error: unable to load configmap based request-header-client-ca-file: configmaps "extension-apiserver-authentication" is forbidden: User "system:cloud-controller-manager" cannot get resource "configmaps" in API group "" in the namespace "kube-system" Usage: cloud-controller-manager [flags] Debugging flags: --contention-profiling Enable block profiling, if profiling is enabled --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) Leader-migration flags: --enable-leader-migration Whether to enable controller leader migration. --leader-migration-config string Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1. Generic flags: --allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider. Requires --cluster-cidr. --cidr-allocator-type string Type of CIDR allocator to use (default "RangeAllocator") --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file. --cloud-provider string The provider for cloud services. Empty string for no provider. --cluster-cidr string CIDR Range for Pods in cluster. Only used when --allocate-node-cidrs=true; if false, this option will be ignored. --cluster-name string The instance prefix for the cluster. (default "kubernetes") --configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true) --controller-start-interval duration Interval between starting controller managers. --controllers strings A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. All controllers: cloud-node-controller, cloud-node-lifecycle-controller, gke-service-lb-controller, gkenetworkparamset, node-ipam-controller, node-route-controller, service-lb-controller Disabled-by-default controllers: gke-service-lb-controller, gkenetworkparamset (default [*]) --external-cloud-volume-plugin string The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers. --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are: APIResponseCompression=true|false (BETA - default=true) APIServerIdentity=true|false (BETA - default=true) APIServingWithRoutine=true|false (ALPHA - default=false) AllAlpha=true|false (ALPHA - default=false) AllBeta=true|false (BETA - default=false) AllowParsingUserUIDFromCertAuth=true|false (BETA - default=true) AllowUnsafeMalformedObjectDeletion=true|false (ALPHA - default=false) CBORServingAndStorage=true|false (ALPHA - default=false) CloudControllerManagerWebhook=true|false (ALPHA - default=false) ComponentFlagz=true|false (ALPHA - default=false) ComponentStatusz=true|false (ALPHA - default=false) ConcurrentWatchObjectDecode=true|false (BETA - default=false) ContextualLogging=true|false (BETA - default=true) CoordinatedLeaderElection=true|false (BETA - default=false) DetectCacheInconsistency=true|false (BETA - default=true) ListFromCacheSnapshot=true|false (BETA - default=true) LoggingAlphaOptions=true|false (ALPHA - default=false) LoggingBetaOptions=true|false (BETA - default=true) MutatingAdmissionPolicy=true|false (BETA - default=false) OpenAPIEnums=true|false (BETA - default=true) RemoteRequestHeaderUID=true|false (BETA - default=true) SizeBasedListCostEstimate=true|false (BETA - default=true) StorageVersionAPI=true|false (ALPHA - default=false) StorageVersionHash=true|false (BETA - default=true) StructuredAuthenticationConfigurationEgressSelector=true|false (BETA - default=true) TokenRequestServiceAccountUIDValidation=true|false (BETA - default=true) UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true) WatchCacheInitializationPostStartHook=true|false (BETA - default=false) WatchList=true|false (BETA - default=true) --kube-api-burst int32 Burst to use while talking with kubernetes apiserver. (default 30) --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf") --kube-api-qps float32 QPS to use while talking with kubernetes apiserver. (default 20) --leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true) --leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s) --leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled. (default 10s) --leader-elect-resource-lock string The type of resource object that is used for locking during leader election. Supported options are 'leases'. (default "leases") --leader-elect-resource-name string The name of resource object that is used for locking during leader election. (default "cloud-controller-manager") --leader-elect-resource-namespace string The namespace of resource object that is used for locking during leader election. (default "kube-system") --leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s) --min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod. (default 12h0m0s) --node-monitor-period duration The period for syncing NodeStatus in cloud-node-lifecycle-controller. (default 5s) --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s) --use-service-account-credentials If true, use individual service account credentials for each controller. Cloud-node-controller flags: --concurrent-node-syncs int32 Number of workers concurrently synchronizing nodes. (default 1) Service-lb-controller flags: --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1) Webhook flags: --webhooks strings A list of webhooks to enable. '*' enables all on-by-default webhooks, 'foo' enables the webhook named 'foo', '-foo' disables the webhook named 'foo'. All webhooks: Disabled-by-default webhooks: Webhook serving flags: --webhook-bind-address ip The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address (0.0.0.0 or ::), all interfaces will be used. If unset, defaults to 0.0.0.0. (default 0.0.0.0) --webhook-cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --webhook-secure-port int Secure port to serve cloud provider webhooks. If 0, don't serve webhooks at all. (default 10260) --webhook-tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --webhook-tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. Secure serving flags: --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used. (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. --disable-http2-serving If true, HTTP2 serving will be disabled [default=false] --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --permit-address-sharing If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false] --permit-port-sharing If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false] --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 10258) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_RC4_128_SHA. --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) Authentication flags: --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authentication-tolerate-lookup-failure If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous. --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-uid-headers strings List of request headers to inspect for UIDs. X-Remote-Uid is suggested. Requires the RemoteRequestHeaderUID feature to be enabled. --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) Authorization flags: --authorization-always-allow-paths strings A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server. (default [/healthz,/readyz,/livez]) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) Misc flags: --kubeconfig string Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag). --master string The address of the Kubernetes API server (overrides any value in kubeconfig). --node-status-update-frequency duration Specifies how often the controller updates nodes' status. (default 5m0s) Global flags: -h, --help help for cloud-controller-manager --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log-text-info-buffer-size quantity [Alpha] In text format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. --log-text-split-stream [Alpha] In text format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. --logging-format string Sets the log format. Permitted formats: "text". (default "text") -v, --v Level number for the log level verbosity (default 0) --version version[=true] --version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format) GCE Cloud Provider flags: --enable-l4-deny-firewall Enable creation and updates of Deny VPC Firewall Rules for L4 external load balancers. Requires --enable-pinhole and --enable-l4-deny-firewall-rollback-cleanup to be true. --enable-l4-deny-firewall-rollback-cleanup Enable cleanup codepath of the deny firewalls for rollback. The reason for it not being enabled by default is the additional GCE API calls that are made for checking if the deny firewalls exist/deletion which will eat up the quota unnecessarily. --enable-l4-lb-annotations Enables Annotations for GCE L4 LB Services --enable-multi-project Enables project selection from Node providerID for GCE API calls. CAUTION: Only enable if Node providerID is configured by a trusted source. --enable-rbs-default-l4-netlb Enables RBS defaulting for GCE L4 NetLB Nodeipam controller flags: --enable-multi-networking Enabled multi-networking related logics such as multi-networking IPAM. --enable-multi-subnet-cluster Enabled multi-subnet cluster feature. This enables generating updated nodeTopology custom resource. --node-cidr-mask-size int32 Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6. --node-cidr-mask-size-ipv4 int32 Mask size for IPv4 node cidr in dual-stack cluster. Default is 24. --node-cidr-mask-size-ipv6 int32 Mask size for IPv6 node cidr in dual-stack cluster. Default is 64. --service-cluster-ip-range string CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true