Job

List of experiments that should be used by the job. An example value is ["enable_stackdriver_agent_metrics"].

Enable/disable the use of Streaming Engine for the job. Note that Streaming Engine is enabled by default for pipelines developed against the Beam SDK for Python v2.21.0 or later when using Python 3.

The configuration for VM IPs. Options are "WORKER_IP_PUBLIC" or "WORKER_IP_PRIVATE".

The unique ID of this job.

The name for the Cloud KMS key for the job. Key format is: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY

User labels to be specified for the job. Keys and values should follow the restrictions specified in the labeling restrictions page. NOTE: Google-provided Dataflow templates often provide default labels that begin with goog-dataflow-provided. Unless explicitly set in config, these labels will be ignored to prevent diffs on re-apply.

The machine type to use for the job.

The number of workers permitted to work on the job. More workers may improve processing speed at additional cost.

A unique name for the resource, required by Dataflow.

The network to which VMs will be assigned. If it is not provided, "default" will be used.

One of "drain" or "cancel". Specifies behavior of deletion during terraform destroy.

Key/Value pairs to be passed to the Dataflow job (as used in the template).

The project in which the resource belongs. If it is not provided, the provider project is used.

The region in which the created job should run.

The Service Account email used to create the job.

If true, treat DRAINING and CANCELLING as terminal job states and do not wait for further changes before removing from terraform state and moving on. WARNING: this will lead to job name conflicts if you do not ensure that the job names are different, e.g. by embedding a release ID or by using a random_id.

The current state of the resource, selected from the JobState enum

The subnetwork to which VMs will be assigned. Should be of the form "regions/REGION/subnetworks/SUBNETWORK". If the subnetwork is located in a Shared VPC network, you must use the complete URL. For example "googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/subnetworks/SUBNET_NAME"

A writeable location on GCS for the Dataflow job to dump its temporary data.

The GCS path to the Dataflow job template.

Only applicable when updating a pipeline. Map of transform name prefixes of the job to be replaced with the corresponding name prefixes of the new job. This field is not used outside of update.

The type of this job, selected from the JobType enum

The zone in which the created job should run. If it is not provided, the provider zone is used.