Pipeline

Creates a pipeline. For a batch pipeline, you can pass scheduler information. Data Pipelines uses the scheduler information to create an internal scheduler that runs jobs periodically. If the internal scheduler is not configured, you can use RunPipeline to run jobs.

Properties

Link copied to clipboard
val createTime: Output<String>

Immutable. The timestamp when the pipeline was initially created. Set by the Data Pipelines service.

Link copied to clipboard
val displayName: Output<String>

The display name of the pipeline. It can contain only letters (A-Za-z), numbers (0-9), hyphens (-), and underscores (_).

Link copied to clipboard
val id: Output<String>
Link copied to clipboard
val jobCount: Output<Int>

Number of jobs.

Link copied to clipboard
val lastUpdateTime: Output<String>

Immutable. The timestamp when the pipeline was last modified. Set by the Data Pipelines service.

Link copied to clipboard
val location: Output<String>
Link copied to clipboard
val name: Output<String>

The pipeline name. For example: projects/PROJECT_ID/locations/LOCATION_ID/pipelines/PIPELINE_ID. * PROJECT_ID can contain letters (A-Za-z), numbers (0-9), hyphens (-), colons (:), and periods (.). For more information, see Identifying projects. * LOCATION_ID is the canonical ID for the pipeline's location. The list of available locations can be obtained by calling google&#46;cloud&#46;location&#46;Locations&#46;ListLocations. Note that the Data Pipelines service is not available in all regions. It depends on Cloud Scheduler, an App Engine application, so it's only available in App Engine regions. * PIPELINE_ID is the ID of the pipeline. Must be unique for the selected project and location.

Link copied to clipboard

Immutable. The sources of the pipeline (for example, Dataplex). The keys and values are set by the corresponding sources during pipeline creation.

Link copied to clipboard
val project: Output<String>
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard

Internal scheduling information for a pipeline. If this information is provided, periodic jobs will be created per the schedule. If not, users are responsible for creating jobs externally.

Link copied to clipboard

Optional. A service account email to be used with the Cloud Scheduler job. If not specified, the default compute engine service account will be used.

Link copied to clipboard
val state: Output<String>

The state of the pipeline. When the pipeline is created, the state is set to 'PIPELINE_STATE_ACTIVE' by default. State changes can be requested by setting the state to stopping, paused, or resuming. State cannot be changed through UpdatePipeline requests.

Link copied to clipboard
val type: Output<String>

The type of the pipeline. This field affects the scheduling of the pipeline and the type of metrics to show for the pipeline.

Link copied to clipboard
val urn: Output<String>
Link copied to clipboard

Workload information for creating new jobs.