Creates a transfer job that runs periodically.
get(jobName, projectId=None, x__xgafv=None)
Gets a transfer job.
list(pageSize=None, pageToken=None, x__xgafv=None, filter=None)
Lists transfer jobs.
list_next(previous_request, previous_response)
Retrieves the next page of results.
patch(jobName, body, x__xgafv=None)
Updates a transfer job. Updating a job's transfer spec does not affect
create(body, x__xgafv=None)
Creates a transfer job that runs periodically.
Args:
body: object, The request body. (required)
The object takes the form of:
{ # This resource represents the configuration of a transfer job that runs
# periodically.
"transferSpec": { # Configuration for running a transfer. # Transfer specification.
"objectConditions": { # Conditions that determine which objects will be transferred. Applies only # Only objects that satisfy these object conditions are included in the set
# of data source and data sink objects. Object conditions based on
# objects' `lastModificationTime` do not exclude objects in a data sink.
# to S3 and GCS objects.
"maxTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` on or after
# `NOW` - `maxTimeElapsedSinceLastModification` and objects that don't have
# a `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
"includePrefixes": [ # If `includePrefixes` is specified, objects that satisfy the object
# conditions must have names that start with one of the `includePrefixes`
# and that do not start with any of the `excludePrefixes`. If
# `includePrefixes` is not specified, all objects except those that have
# names starting with one of the `excludePrefixes` must satisfy the object
# conditions.
#
# Requirements:
#
# * Each include-prefix and exclude-prefix can contain any sequence of
# Unicode characters, of max length 1024 bytes when UTF8-encoded, and
# must not contain Carriage Return or Line Feed characters. Wildcard
# matching and regular expression matching are not supported.
#
# * Each include-prefix and exclude-prefix must omit the leading slash.
# For example, to include the `requests.gz` object in a transfer from
# `s3://my-aws-bucket/logs/y=2015/requests.gz`, specify the include
# prefix as `logs/y=2015/requests.gz`.
#
# * None of the include-prefix or the exclude-prefix values can be empty,
# if specified.
#
# * Each include-prefix must include a distinct portion of the object
# namespace, i.e., no include-prefix may be a prefix of another
# include-prefix.
#
# * Each exclude-prefix must exclude a distinct portion of the object
# namespace, i.e., no exclude-prefix may be a prefix of another
# exclude-prefix.
#
# * If `includePrefixes` is specified, then each exclude-prefix must start
# with the value of a path explicitly included by `includePrefixes`.
#
# The max size of `includePrefixes` is 1000.
"A String",
],
"excludePrefixes": [ # `excludePrefixes` must follow the requirements described for
# `includePrefixes`.
#
# The max size of `excludePrefixes` is 1000.
"A String",
],
"minTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` before
# `NOW` - `minTimeElapsedSinceLastModification` and objects that don't have a
# `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
},
"gcsDataSource": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data source.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"httpDataSource": { # An HttpData resource specifies a list of objects on the web to be transferred # An HTTP URL data source.
# over HTTP. The information of the objects to be transferred is contained in
# a file referenced by a URL. The first line in the file must be
# "TsvHttpData-1.0", which specifies the format of the file. Subsequent lines
# specify the information of the list of objects, one object per list entry.
# Each entry has the following tab-delimited fields:
#
# * HTTP URL - The location of the object.
#
# * Length - The size of the object in bytes.
#
# * MD5 - The base64-encoded MD5 hash of the object.
#
# For an example of a valid TSV file, see
# [Transferring data from
# URLs](https://cloud.google.com/storage/transfer/create-url-list).
#
# When transferring data based on a URL list, keep the following in mind:
#
# * When an object located at `http(s)://hostname:port/` is
# transferred to a data sink, the name of the object at the data sink is
# `/`.
#
# * If the specified size of an object does not match the actual size of the
# object fetched, the object will not be transferred.
#
# * If the specified MD5 does not match the MD5 computed from the transferred
# bytes, the object transfer will fail. For more information, see
# [Generating MD5 hashes](https://cloud.google.com/storage/transfer/#md5)
#
# * Ensure that each URL you specify is publicly accessible. For
# example, in Google Cloud Storage you can
# [share an object publicly]
# (https://cloud.google.com/storage/docs/cloud-console#_sharingdata) and get
# a link to it.
#
# * Storage Transfer Service obeys `robots.txt` rules and requires the source
# HTTP server to support `Range` requests and to return a `Content-Length`
# header in each response.
#
# * [ObjectConditions](#ObjectConditions) have no effect when filtering objects
# to transfer.
"listUrl": "A String", # Required. The URL that points to the file that stores the object list
# entries. This file must allow public access. Currently, only URLs with
# HTTP and HTTPS schemes are supported.
},
"transferOptions": { # TransferOptions uses three boolean parameters to define the actions # If the option `deleteObjectsUniqueInSink` is `true`, object conditions
# based on objects' `lastModificationTime` are ignored and do not exclude
# objects in a data source or a data sink.
# to be performed on objects in a transfer.
"overwriteObjectsAlreadyExistingInSink": True or False, # Whether overwriting objects that already exist in the sink is allowed.
"deleteObjectsFromSourceAfterTransfer": True or False, # Whether objects should be deleted from the source after they are
# transferred to the sink. Note that this option and
# `deleteObjectsUniqueInSink` are mutually exclusive.
"deleteObjectsUniqueInSink": True or False, # Whether objects that exist only in the sink should be deleted. Note that
# this option and `deleteObjectsFromSourceAfterTransfer` are mutually
# exclusive.
},
"gcsDataSink": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data sink.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"awsS3DataSource": { # An AwsS3Data resource can be a data source, but not a data sink. # An AWS S3 data source.
# In an AwsS3Data resource, an object's name is the S3 object's key name.
"awsAccessKey": { # AWS access key (see # Required. AWS access key used to sign the API requests to the AWS S3
# bucket. Permissions on the bucket must be granted to the access ID of the
# AWS access key.
# [AWS Security
# Credentials](http://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html)).
"secretAccessKey": "A String", # Required. AWS secret access key. This field is not returned in RPC
# responses.
"accessKeyId": "A String", # Required. AWS access key ID.
},
"bucketName": "A String", # Required. S3 Bucket name (see
# [Creating a
# bucket](http://docs.aws.amazon.com/AmazonS3/latest/dev/create-bucket-get-location-example.html)).
},
},
"status": "A String", # Status of the job. This value MUST be specified for
# `CreateTransferJobRequests`.
#
# NOTE: The effect of the new job status takes place during a subsequent job
# run. For example, if you change the job status from `ENABLED` to
# `DISABLED`, and an operation spawned by the transfer is running, the status
# change would not affect the current operation.
"deletionTime": "A String", # Output only. The time that the transfer job was deleted.
"schedule": { # Transfers can be scheduled to recur or to run just once. # Schedule specification.
"startTimeOfDay": { # Represents a time of day. The date and time zone are either not significant # The time in UTC at which the transfer will be scheduled to start in a day.
# Transfers may start later than this time. If not specified, recurring and
# one-time transfers that are scheduled to run today will run immediately;
# recurring transfers that are scheduled to run on a future date will start
# at approximately midnight UTC on that date. Note that when configuring a
# transfer with the Cloud Platform Console, the transfer's start time in a
# day is specified in your local timezone.
# or are specified elsewhere. An API may choose to allow leap seconds. Related
# types are google.type.Date and `google.protobuf.Timestamp`.
"hours": 42, # Hours of day in 24 hour format. Should be from 0 to 23. An API may choose
# to allow the value "24:00:00" for scenarios like business closing time.
"nanos": 42, # Fractions of seconds in nanoseconds. Must be from 0 to 999,999,999.
"seconds": 42, # Seconds of minutes of the time. Must normally be from 0 to 59. An API may
# allow the value 60 if it allows leap-seconds.
"minutes": 42, # Minutes of hour of day. Must be from 0 to 59.
},
"scheduleStartDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # Required. The first day the recurring transfer is scheduled to run. If
# `scheduleStartDate` is in the past, the transfer will run for the first
# time on the following day.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
"scheduleEndDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # The last day the recurring transfer will be run. If `scheduleEndDate`
# is the same as `scheduleStartDate`, the transfer will be executed only
# once.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
},
"projectId": "A String", # The ID of the Google Cloud Platform Project that owns the job.
"lastModificationTime": "A String", # Output only. The time that the transfer job was last modified.
"creationTime": "A String", # Output only. The time that the transfer job was created.
"name": "A String", # A globally unique name assigned by Storage Transfer Service when the
# job is created. This field should be left empty in requests to create a new
# transfer job; otherwise, the requests result in an `INVALID_ARGUMENT`
# error.
"description": "A String", # A description provided by the user for the job. Its max length is 1024
# bytes when Unicode-encoded.
}
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # This resource represents the configuration of a transfer job that runs
# periodically.
"transferSpec": { # Configuration for running a transfer. # Transfer specification.
"objectConditions": { # Conditions that determine which objects will be transferred. Applies only # Only objects that satisfy these object conditions are included in the set
# of data source and data sink objects. Object conditions based on
# objects' `lastModificationTime` do not exclude objects in a data sink.
# to S3 and GCS objects.
"maxTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` on or after
# `NOW` - `maxTimeElapsedSinceLastModification` and objects that don't have
# a `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
"includePrefixes": [ # If `includePrefixes` is specified, objects that satisfy the object
# conditions must have names that start with one of the `includePrefixes`
# and that do not start with any of the `excludePrefixes`. If
# `includePrefixes` is not specified, all objects except those that have
# names starting with one of the `excludePrefixes` must satisfy the object
# conditions.
#
# Requirements:
#
# * Each include-prefix and exclude-prefix can contain any sequence of
# Unicode characters, of max length 1024 bytes when UTF8-encoded, and
# must not contain Carriage Return or Line Feed characters. Wildcard
# matching and regular expression matching are not supported.
#
# * Each include-prefix and exclude-prefix must omit the leading slash.
# For example, to include the `requests.gz` object in a transfer from
# `s3://my-aws-bucket/logs/y=2015/requests.gz`, specify the include
# prefix as `logs/y=2015/requests.gz`.
#
# * None of the include-prefix or the exclude-prefix values can be empty,
# if specified.
#
# * Each include-prefix must include a distinct portion of the object
# namespace, i.e., no include-prefix may be a prefix of another
# include-prefix.
#
# * Each exclude-prefix must exclude a distinct portion of the object
# namespace, i.e., no exclude-prefix may be a prefix of another
# exclude-prefix.
#
# * If `includePrefixes` is specified, then each exclude-prefix must start
# with the value of a path explicitly included by `includePrefixes`.
#
# The max size of `includePrefixes` is 1000.
"A String",
],
"excludePrefixes": [ # `excludePrefixes` must follow the requirements described for
# `includePrefixes`.
#
# The max size of `excludePrefixes` is 1000.
"A String",
],
"minTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` before
# `NOW` - `minTimeElapsedSinceLastModification` and objects that don't have a
# `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
},
"gcsDataSource": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data source.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"httpDataSource": { # An HttpData resource specifies a list of objects on the web to be transferred # An HTTP URL data source.
# over HTTP. The information of the objects to be transferred is contained in
# a file referenced by a URL. The first line in the file must be
# "TsvHttpData-1.0", which specifies the format of the file. Subsequent lines
# specify the information of the list of objects, one object per list entry.
# Each entry has the following tab-delimited fields:
#
# * HTTP URL - The location of the object.
#
# * Length - The size of the object in bytes.
#
# * MD5 - The base64-encoded MD5 hash of the object.
#
# For an example of a valid TSV file, see
# [Transferring data from
# URLs](https://cloud.google.com/storage/transfer/create-url-list).
#
# When transferring data based on a URL list, keep the following in mind:
#
# * When an object located at `http(s)://hostname:port/` is
# transferred to a data sink, the name of the object at the data sink is
# `/`.
#
# * If the specified size of an object does not match the actual size of the
# object fetched, the object will not be transferred.
#
# * If the specified MD5 does not match the MD5 computed from the transferred
# bytes, the object transfer will fail. For more information, see
# [Generating MD5 hashes](https://cloud.google.com/storage/transfer/#md5)
#
# * Ensure that each URL you specify is publicly accessible. For
# example, in Google Cloud Storage you can
# [share an object publicly]
# (https://cloud.google.com/storage/docs/cloud-console#_sharingdata) and get
# a link to it.
#
# * Storage Transfer Service obeys `robots.txt` rules and requires the source
# HTTP server to support `Range` requests and to return a `Content-Length`
# header in each response.
#
# * [ObjectConditions](#ObjectConditions) have no effect when filtering objects
# to transfer.
"listUrl": "A String", # Required. The URL that points to the file that stores the object list
# entries. This file must allow public access. Currently, only URLs with
# HTTP and HTTPS schemes are supported.
},
"transferOptions": { # TransferOptions uses three boolean parameters to define the actions # If the option `deleteObjectsUniqueInSink` is `true`, object conditions
# based on objects' `lastModificationTime` are ignored and do not exclude
# objects in a data source or a data sink.
# to be performed on objects in a transfer.
"overwriteObjectsAlreadyExistingInSink": True or False, # Whether overwriting objects that already exist in the sink is allowed.
"deleteObjectsFromSourceAfterTransfer": True or False, # Whether objects should be deleted from the source after they are
# transferred to the sink. Note that this option and
# `deleteObjectsUniqueInSink` are mutually exclusive.
"deleteObjectsUniqueInSink": True or False, # Whether objects that exist only in the sink should be deleted. Note that
# this option and `deleteObjectsFromSourceAfterTransfer` are mutually
# exclusive.
},
"gcsDataSink": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data sink.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"awsS3DataSource": { # An AwsS3Data resource can be a data source, but not a data sink. # An AWS S3 data source.
# In an AwsS3Data resource, an object's name is the S3 object's key name.
"awsAccessKey": { # AWS access key (see # Required. AWS access key used to sign the API requests to the AWS S3
# bucket. Permissions on the bucket must be granted to the access ID of the
# AWS access key.
# [AWS Security
# Credentials](http://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html)).
"secretAccessKey": "A String", # Required. AWS secret access key. This field is not returned in RPC
# responses.
"accessKeyId": "A String", # Required. AWS access key ID.
},
"bucketName": "A String", # Required. S3 Bucket name (see
# [Creating a
# bucket](http://docs.aws.amazon.com/AmazonS3/latest/dev/create-bucket-get-location-example.html)).
},
},
"status": "A String", # Status of the job. This value MUST be specified for
# `CreateTransferJobRequests`.
#
# NOTE: The effect of the new job status takes place during a subsequent job
# run. For example, if you change the job status from `ENABLED` to
# `DISABLED`, and an operation spawned by the transfer is running, the status
# change would not affect the current operation.
"deletionTime": "A String", # Output only. The time that the transfer job was deleted.
"schedule": { # Transfers can be scheduled to recur or to run just once. # Schedule specification.
"startTimeOfDay": { # Represents a time of day. The date and time zone are either not significant # The time in UTC at which the transfer will be scheduled to start in a day.
# Transfers may start later than this time. If not specified, recurring and
# one-time transfers that are scheduled to run today will run immediately;
# recurring transfers that are scheduled to run on a future date will start
# at approximately midnight UTC on that date. Note that when configuring a
# transfer with the Cloud Platform Console, the transfer's start time in a
# day is specified in your local timezone.
# or are specified elsewhere. An API may choose to allow leap seconds. Related
# types are google.type.Date and `google.protobuf.Timestamp`.
"hours": 42, # Hours of day in 24 hour format. Should be from 0 to 23. An API may choose
# to allow the value "24:00:00" for scenarios like business closing time.
"nanos": 42, # Fractions of seconds in nanoseconds. Must be from 0 to 999,999,999.
"seconds": 42, # Seconds of minutes of the time. Must normally be from 0 to 59. An API may
# allow the value 60 if it allows leap-seconds.
"minutes": 42, # Minutes of hour of day. Must be from 0 to 59.
},
"scheduleStartDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # Required. The first day the recurring transfer is scheduled to run. If
# `scheduleStartDate` is in the past, the transfer will run for the first
# time on the following day.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
"scheduleEndDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # The last day the recurring transfer will be run. If `scheduleEndDate`
# is the same as `scheduleStartDate`, the transfer will be executed only
# once.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
},
"projectId": "A String", # The ID of the Google Cloud Platform Project that owns the job.
"lastModificationTime": "A String", # Output only. The time that the transfer job was last modified.
"creationTime": "A String", # Output only. The time that the transfer job was created.
"name": "A String", # A globally unique name assigned by Storage Transfer Service when the
# job is created. This field should be left empty in requests to create a new
# transfer job; otherwise, the requests result in an `INVALID_ARGUMENT`
# error.
"description": "A String", # A description provided by the user for the job. Its max length is 1024
# bytes when Unicode-encoded.
}
get(jobName, projectId=None, x__xgafv=None)
Gets a transfer job.
Args:
jobName: string, Required. The job to get. (required)
projectId: string, Required. The ID of the Google Cloud Platform Console project that owns the
job.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # This resource represents the configuration of a transfer job that runs
# periodically.
"transferSpec": { # Configuration for running a transfer. # Transfer specification.
"objectConditions": { # Conditions that determine which objects will be transferred. Applies only # Only objects that satisfy these object conditions are included in the set
# of data source and data sink objects. Object conditions based on
# objects' `lastModificationTime` do not exclude objects in a data sink.
# to S3 and GCS objects.
"maxTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` on or after
# `NOW` - `maxTimeElapsedSinceLastModification` and objects that don't have
# a `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
"includePrefixes": [ # If `includePrefixes` is specified, objects that satisfy the object
# conditions must have names that start with one of the `includePrefixes`
# and that do not start with any of the `excludePrefixes`. If
# `includePrefixes` is not specified, all objects except those that have
# names starting with one of the `excludePrefixes` must satisfy the object
# conditions.
#
# Requirements:
#
# * Each include-prefix and exclude-prefix can contain any sequence of
# Unicode characters, of max length 1024 bytes when UTF8-encoded, and
# must not contain Carriage Return or Line Feed characters. Wildcard
# matching and regular expression matching are not supported.
#
# * Each include-prefix and exclude-prefix must omit the leading slash.
# For example, to include the `requests.gz` object in a transfer from
# `s3://my-aws-bucket/logs/y=2015/requests.gz`, specify the include
# prefix as `logs/y=2015/requests.gz`.
#
# * None of the include-prefix or the exclude-prefix values can be empty,
# if specified.
#
# * Each include-prefix must include a distinct portion of the object
# namespace, i.e., no include-prefix may be a prefix of another
# include-prefix.
#
# * Each exclude-prefix must exclude a distinct portion of the object
# namespace, i.e., no exclude-prefix may be a prefix of another
# exclude-prefix.
#
# * If `includePrefixes` is specified, then each exclude-prefix must start
# with the value of a path explicitly included by `includePrefixes`.
#
# The max size of `includePrefixes` is 1000.
"A String",
],
"excludePrefixes": [ # `excludePrefixes` must follow the requirements described for
# `includePrefixes`.
#
# The max size of `excludePrefixes` is 1000.
"A String",
],
"minTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` before
# `NOW` - `minTimeElapsedSinceLastModification` and objects that don't have a
# `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
},
"gcsDataSource": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data source.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"httpDataSource": { # An HttpData resource specifies a list of objects on the web to be transferred # An HTTP URL data source.
# over HTTP. The information of the objects to be transferred is contained in
# a file referenced by a URL. The first line in the file must be
# "TsvHttpData-1.0", which specifies the format of the file. Subsequent lines
# specify the information of the list of objects, one object per list entry.
# Each entry has the following tab-delimited fields:
#
# * HTTP URL - The location of the object.
#
# * Length - The size of the object in bytes.
#
# * MD5 - The base64-encoded MD5 hash of the object.
#
# For an example of a valid TSV file, see
# [Transferring data from
# URLs](https://cloud.google.com/storage/transfer/create-url-list).
#
# When transferring data based on a URL list, keep the following in mind:
#
# * When an object located at `http(s)://hostname:port/` is
# transferred to a data sink, the name of the object at the data sink is
# `/`.
#
# * If the specified size of an object does not match the actual size of the
# object fetched, the object will not be transferred.
#
# * If the specified MD5 does not match the MD5 computed from the transferred
# bytes, the object transfer will fail. For more information, see
# [Generating MD5 hashes](https://cloud.google.com/storage/transfer/#md5)
#
# * Ensure that each URL you specify is publicly accessible. For
# example, in Google Cloud Storage you can
# [share an object publicly]
# (https://cloud.google.com/storage/docs/cloud-console#_sharingdata) and get
# a link to it.
#
# * Storage Transfer Service obeys `robots.txt` rules and requires the source
# HTTP server to support `Range` requests and to return a `Content-Length`
# header in each response.
#
# * [ObjectConditions](#ObjectConditions) have no effect when filtering objects
# to transfer.
"listUrl": "A String", # Required. The URL that points to the file that stores the object list
# entries. This file must allow public access. Currently, only URLs with
# HTTP and HTTPS schemes are supported.
},
"transferOptions": { # TransferOptions uses three boolean parameters to define the actions # If the option `deleteObjectsUniqueInSink` is `true`, object conditions
# based on objects' `lastModificationTime` are ignored and do not exclude
# objects in a data source or a data sink.
# to be performed on objects in a transfer.
"overwriteObjectsAlreadyExistingInSink": True or False, # Whether overwriting objects that already exist in the sink is allowed.
"deleteObjectsFromSourceAfterTransfer": True or False, # Whether objects should be deleted from the source after they are
# transferred to the sink. Note that this option and
# `deleteObjectsUniqueInSink` are mutually exclusive.
"deleteObjectsUniqueInSink": True or False, # Whether objects that exist only in the sink should be deleted. Note that
# this option and `deleteObjectsFromSourceAfterTransfer` are mutually
# exclusive.
},
"gcsDataSink": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data sink.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"awsS3DataSource": { # An AwsS3Data resource can be a data source, but not a data sink. # An AWS S3 data source.
# In an AwsS3Data resource, an object's name is the S3 object's key name.
"awsAccessKey": { # AWS access key (see # Required. AWS access key used to sign the API requests to the AWS S3
# bucket. Permissions on the bucket must be granted to the access ID of the
# AWS access key.
# [AWS Security
# Credentials](http://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html)).
"secretAccessKey": "A String", # Required. AWS secret access key. This field is not returned in RPC
# responses.
"accessKeyId": "A String", # Required. AWS access key ID.
},
"bucketName": "A String", # Required. S3 Bucket name (see
# [Creating a
# bucket](http://docs.aws.amazon.com/AmazonS3/latest/dev/create-bucket-get-location-example.html)).
},
},
"status": "A String", # Status of the job. This value MUST be specified for
# `CreateTransferJobRequests`.
#
# NOTE: The effect of the new job status takes place during a subsequent job
# run. For example, if you change the job status from `ENABLED` to
# `DISABLED`, and an operation spawned by the transfer is running, the status
# change would not affect the current operation.
"deletionTime": "A String", # Output only. The time that the transfer job was deleted.
"schedule": { # Transfers can be scheduled to recur or to run just once. # Schedule specification.
"startTimeOfDay": { # Represents a time of day. The date and time zone are either not significant # The time in UTC at which the transfer will be scheduled to start in a day.
# Transfers may start later than this time. If not specified, recurring and
# one-time transfers that are scheduled to run today will run immediately;
# recurring transfers that are scheduled to run on a future date will start
# at approximately midnight UTC on that date. Note that when configuring a
# transfer with the Cloud Platform Console, the transfer's start time in a
# day is specified in your local timezone.
# or are specified elsewhere. An API may choose to allow leap seconds. Related
# types are google.type.Date and `google.protobuf.Timestamp`.
"hours": 42, # Hours of day in 24 hour format. Should be from 0 to 23. An API may choose
# to allow the value "24:00:00" for scenarios like business closing time.
"nanos": 42, # Fractions of seconds in nanoseconds. Must be from 0 to 999,999,999.
"seconds": 42, # Seconds of minutes of the time. Must normally be from 0 to 59. An API may
# allow the value 60 if it allows leap-seconds.
"minutes": 42, # Minutes of hour of day. Must be from 0 to 59.
},
"scheduleStartDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # Required. The first day the recurring transfer is scheduled to run. If
# `scheduleStartDate` is in the past, the transfer will run for the first
# time on the following day.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
"scheduleEndDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # The last day the recurring transfer will be run. If `scheduleEndDate`
# is the same as `scheduleStartDate`, the transfer will be executed only
# once.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
},
"projectId": "A String", # The ID of the Google Cloud Platform Project that owns the job.
"lastModificationTime": "A String", # Output only. The time that the transfer job was last modified.
"creationTime": "A String", # Output only. The time that the transfer job was created.
"name": "A String", # A globally unique name assigned by Storage Transfer Service when the
# job is created. This field should be left empty in requests to create a new
# transfer job; otherwise, the requests result in an `INVALID_ARGUMENT`
# error.
"description": "A String", # A description provided by the user for the job. Its max length is 1024
# bytes when Unicode-encoded.
}
list(pageSize=None, pageToken=None, x__xgafv=None, filter=None)
Lists transfer jobs.
Args:
pageSize: integer, The list page size. The max allowed value is 256.
pageToken: string, The list page token.
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
filter: string, Required. A list of query parameters specified as JSON text in the form of:
{"project_id":"my_project_id",
"job_names":["jobid1","jobid2",...],
"job_statuses":["status1","status2",...]}.
Since `job_names` and `job_statuses` support multiple values, their values
must be specified with array notation. `project_id` is required.
`job_names` and `job_statuses` are optional. The valid values for
`job_statuses` are case-insensitive: `ENABLED`, `DISABLED`, and `DELETED`.
Returns:
An object of the form:
{ # Response from ListTransferJobs.
"nextPageToken": "A String", # The list next page token.
"transferJobs": [ # A list of transfer jobs.
{ # This resource represents the configuration of a transfer job that runs
# periodically.
"transferSpec": { # Configuration for running a transfer. # Transfer specification.
"objectConditions": { # Conditions that determine which objects will be transferred. Applies only # Only objects that satisfy these object conditions are included in the set
# of data source and data sink objects. Object conditions based on
# objects' `lastModificationTime` do not exclude objects in a data sink.
# to S3 and GCS objects.
"maxTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` on or after
# `NOW` - `maxTimeElapsedSinceLastModification` and objects that don't have
# a `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
"includePrefixes": [ # If `includePrefixes` is specified, objects that satisfy the object
# conditions must have names that start with one of the `includePrefixes`
# and that do not start with any of the `excludePrefixes`. If
# `includePrefixes` is not specified, all objects except those that have
# names starting with one of the `excludePrefixes` must satisfy the object
# conditions.
#
# Requirements:
#
# * Each include-prefix and exclude-prefix can contain any sequence of
# Unicode characters, of max length 1024 bytes when UTF8-encoded, and
# must not contain Carriage Return or Line Feed characters. Wildcard
# matching and regular expression matching are not supported.
#
# * Each include-prefix and exclude-prefix must omit the leading slash.
# For example, to include the `requests.gz` object in a transfer from
# `s3://my-aws-bucket/logs/y=2015/requests.gz`, specify the include
# prefix as `logs/y=2015/requests.gz`.
#
# * None of the include-prefix or the exclude-prefix values can be empty,
# if specified.
#
# * Each include-prefix must include a distinct portion of the object
# namespace, i.e., no include-prefix may be a prefix of another
# include-prefix.
#
# * Each exclude-prefix must exclude a distinct portion of the object
# namespace, i.e., no exclude-prefix may be a prefix of another
# exclude-prefix.
#
# * If `includePrefixes` is specified, then each exclude-prefix must start
# with the value of a path explicitly included by `includePrefixes`.
#
# The max size of `includePrefixes` is 1000.
"A String",
],
"excludePrefixes": [ # `excludePrefixes` must follow the requirements described for
# `includePrefixes`.
#
# The max size of `excludePrefixes` is 1000.
"A String",
],
"minTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` before
# `NOW` - `minTimeElapsedSinceLastModification` and objects that don't have a
# `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
},
"gcsDataSource": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data source.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"httpDataSource": { # An HttpData resource specifies a list of objects on the web to be transferred # An HTTP URL data source.
# over HTTP. The information of the objects to be transferred is contained in
# a file referenced by a URL. The first line in the file must be
# "TsvHttpData-1.0", which specifies the format of the file. Subsequent lines
# specify the information of the list of objects, one object per list entry.
# Each entry has the following tab-delimited fields:
#
# * HTTP URL - The location of the object.
#
# * Length - The size of the object in bytes.
#
# * MD5 - The base64-encoded MD5 hash of the object.
#
# For an example of a valid TSV file, see
# [Transferring data from
# URLs](https://cloud.google.com/storage/transfer/create-url-list).
#
# When transferring data based on a URL list, keep the following in mind:
#
# * When an object located at `http(s)://hostname:port/` is
# transferred to a data sink, the name of the object at the data sink is
# `/`.
#
# * If the specified size of an object does not match the actual size of the
# object fetched, the object will not be transferred.
#
# * If the specified MD5 does not match the MD5 computed from the transferred
# bytes, the object transfer will fail. For more information, see
# [Generating MD5 hashes](https://cloud.google.com/storage/transfer/#md5)
#
# * Ensure that each URL you specify is publicly accessible. For
# example, in Google Cloud Storage you can
# [share an object publicly]
# (https://cloud.google.com/storage/docs/cloud-console#_sharingdata) and get
# a link to it.
#
# * Storage Transfer Service obeys `robots.txt` rules and requires the source
# HTTP server to support `Range` requests and to return a `Content-Length`
# header in each response.
#
# * [ObjectConditions](#ObjectConditions) have no effect when filtering objects
# to transfer.
"listUrl": "A String", # Required. The URL that points to the file that stores the object list
# entries. This file must allow public access. Currently, only URLs with
# HTTP and HTTPS schemes are supported.
},
"transferOptions": { # TransferOptions uses three boolean parameters to define the actions # If the option `deleteObjectsUniqueInSink` is `true`, object conditions
# based on objects' `lastModificationTime` are ignored and do not exclude
# objects in a data source or a data sink.
# to be performed on objects in a transfer.
"overwriteObjectsAlreadyExistingInSink": True or False, # Whether overwriting objects that already exist in the sink is allowed.
"deleteObjectsFromSourceAfterTransfer": True or False, # Whether objects should be deleted from the source after they are
# transferred to the sink. Note that this option and
# `deleteObjectsUniqueInSink` are mutually exclusive.
"deleteObjectsUniqueInSink": True or False, # Whether objects that exist only in the sink should be deleted. Note that
# this option and `deleteObjectsFromSourceAfterTransfer` are mutually
# exclusive.
},
"gcsDataSink": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data sink.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"awsS3DataSource": { # An AwsS3Data resource can be a data source, but not a data sink. # An AWS S3 data source.
# In an AwsS3Data resource, an object's name is the S3 object's key name.
"awsAccessKey": { # AWS access key (see # Required. AWS access key used to sign the API requests to the AWS S3
# bucket. Permissions on the bucket must be granted to the access ID of the
# AWS access key.
# [AWS Security
# Credentials](http://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html)).
"secretAccessKey": "A String", # Required. AWS secret access key. This field is not returned in RPC
# responses.
"accessKeyId": "A String", # Required. AWS access key ID.
},
"bucketName": "A String", # Required. S3 Bucket name (see
# [Creating a
# bucket](http://docs.aws.amazon.com/AmazonS3/latest/dev/create-bucket-get-location-example.html)).
},
},
"status": "A String", # Status of the job. This value MUST be specified for
# `CreateTransferJobRequests`.
#
# NOTE: The effect of the new job status takes place during a subsequent job
# run. For example, if you change the job status from `ENABLED` to
# `DISABLED`, and an operation spawned by the transfer is running, the status
# change would not affect the current operation.
"deletionTime": "A String", # Output only. The time that the transfer job was deleted.
"schedule": { # Transfers can be scheduled to recur or to run just once. # Schedule specification.
"startTimeOfDay": { # Represents a time of day. The date and time zone are either not significant # The time in UTC at which the transfer will be scheduled to start in a day.
# Transfers may start later than this time. If not specified, recurring and
# one-time transfers that are scheduled to run today will run immediately;
# recurring transfers that are scheduled to run on a future date will start
# at approximately midnight UTC on that date. Note that when configuring a
# transfer with the Cloud Platform Console, the transfer's start time in a
# day is specified in your local timezone.
# or are specified elsewhere. An API may choose to allow leap seconds. Related
# types are google.type.Date and `google.protobuf.Timestamp`.
"hours": 42, # Hours of day in 24 hour format. Should be from 0 to 23. An API may choose
# to allow the value "24:00:00" for scenarios like business closing time.
"nanos": 42, # Fractions of seconds in nanoseconds. Must be from 0 to 999,999,999.
"seconds": 42, # Seconds of minutes of the time. Must normally be from 0 to 59. An API may
# allow the value 60 if it allows leap-seconds.
"minutes": 42, # Minutes of hour of day. Must be from 0 to 59.
},
"scheduleStartDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # Required. The first day the recurring transfer is scheduled to run. If
# `scheduleStartDate` is in the past, the transfer will run for the first
# time on the following day.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
"scheduleEndDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # The last day the recurring transfer will be run. If `scheduleEndDate`
# is the same as `scheduleStartDate`, the transfer will be executed only
# once.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
},
"projectId": "A String", # The ID of the Google Cloud Platform Project that owns the job.
"lastModificationTime": "A String", # Output only. The time that the transfer job was last modified.
"creationTime": "A String", # Output only. The time that the transfer job was created.
"name": "A String", # A globally unique name assigned by Storage Transfer Service when the
# job is created. This field should be left empty in requests to create a new
# transfer job; otherwise, the requests result in an `INVALID_ARGUMENT`
# error.
"description": "A String", # A description provided by the user for the job. Its max length is 1024
# bytes when Unicode-encoded.
},
],
}
list_next(previous_request, previous_response)
Retrieves the next page of results.
Args:
previous_request: The request for the previous page. (required)
previous_response: The response from the request for the previous page. (required)
Returns:
A request object that you can call 'execute()' on to request the next
page. Returns None if there are no more items in the collection.
patch(jobName, body, x__xgafv=None)
Updates a transfer job. Updating a job's transfer spec does not affect
transfer operations that are running already. Updating the scheduling
of a job is not allowed.
Args:
jobName: string, Required. The name of job to update. (required)
body: object, The request body. (required)
The object takes the form of:
{ # Request passed to UpdateTransferJob.
"projectId": "A String", # Required. The ID of the Google Cloud Platform Console project that owns the
# job.
"updateTransferJobFieldMask": "A String", # The field mask of the fields in `transferJob` that are to be updated in
# this request. Fields in `transferJob` that can be updated are:
# `description`, `transferSpec`, and `status`. To update the `transferSpec`
# of the job, a complete transfer specification has to be provided. An
# incomplete specification which misses any required fields will be rejected
# with the error `INVALID_ARGUMENT`.
"transferJob": { # This resource represents the configuration of a transfer job that runs # Required. The job to update. `transferJob` is expected to specify only
# three fields: `description`, `transferSpec`, and `status`. An
# UpdateTransferJobRequest that specifies other fields will be rejected with
# an error `INVALID_ARGUMENT`.
# periodically.
"transferSpec": { # Configuration for running a transfer. # Transfer specification.
"objectConditions": { # Conditions that determine which objects will be transferred. Applies only # Only objects that satisfy these object conditions are included in the set
# of data source and data sink objects. Object conditions based on
# objects' `lastModificationTime` do not exclude objects in a data sink.
# to S3 and GCS objects.
"maxTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` on or after
# `NOW` - `maxTimeElapsedSinceLastModification` and objects that don't have
# a `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
"includePrefixes": [ # If `includePrefixes` is specified, objects that satisfy the object
# conditions must have names that start with one of the `includePrefixes`
# and that do not start with any of the `excludePrefixes`. If
# `includePrefixes` is not specified, all objects except those that have
# names starting with one of the `excludePrefixes` must satisfy the object
# conditions.
#
# Requirements:
#
# * Each include-prefix and exclude-prefix can contain any sequence of
# Unicode characters, of max length 1024 bytes when UTF8-encoded, and
# must not contain Carriage Return or Line Feed characters. Wildcard
# matching and regular expression matching are not supported.
#
# * Each include-prefix and exclude-prefix must omit the leading slash.
# For example, to include the `requests.gz` object in a transfer from
# `s3://my-aws-bucket/logs/y=2015/requests.gz`, specify the include
# prefix as `logs/y=2015/requests.gz`.
#
# * None of the include-prefix or the exclude-prefix values can be empty,
# if specified.
#
# * Each include-prefix must include a distinct portion of the object
# namespace, i.e., no include-prefix may be a prefix of another
# include-prefix.
#
# * Each exclude-prefix must exclude a distinct portion of the object
# namespace, i.e., no exclude-prefix may be a prefix of another
# exclude-prefix.
#
# * If `includePrefixes` is specified, then each exclude-prefix must start
# with the value of a path explicitly included by `includePrefixes`.
#
# The max size of `includePrefixes` is 1000.
"A String",
],
"excludePrefixes": [ # `excludePrefixes` must follow the requirements described for
# `includePrefixes`.
#
# The max size of `excludePrefixes` is 1000.
"A String",
],
"minTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` before
# `NOW` - `minTimeElapsedSinceLastModification` and objects that don't have a
# `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
},
"gcsDataSource": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data source.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"httpDataSource": { # An HttpData resource specifies a list of objects on the web to be transferred # An HTTP URL data source.
# over HTTP. The information of the objects to be transferred is contained in
# a file referenced by a URL. The first line in the file must be
# "TsvHttpData-1.0", which specifies the format of the file. Subsequent lines
# specify the information of the list of objects, one object per list entry.
# Each entry has the following tab-delimited fields:
#
# * HTTP URL - The location of the object.
#
# * Length - The size of the object in bytes.
#
# * MD5 - The base64-encoded MD5 hash of the object.
#
# For an example of a valid TSV file, see
# [Transferring data from
# URLs](https://cloud.google.com/storage/transfer/create-url-list).
#
# When transferring data based on a URL list, keep the following in mind:
#
# * When an object located at `http(s)://hostname:port/` is
# transferred to a data sink, the name of the object at the data sink is
# `/`.
#
# * If the specified size of an object does not match the actual size of the
# object fetched, the object will not be transferred.
#
# * If the specified MD5 does not match the MD5 computed from the transferred
# bytes, the object transfer will fail. For more information, see
# [Generating MD5 hashes](https://cloud.google.com/storage/transfer/#md5)
#
# * Ensure that each URL you specify is publicly accessible. For
# example, in Google Cloud Storage you can
# [share an object publicly]
# (https://cloud.google.com/storage/docs/cloud-console#_sharingdata) and get
# a link to it.
#
# * Storage Transfer Service obeys `robots.txt` rules and requires the source
# HTTP server to support `Range` requests and to return a `Content-Length`
# header in each response.
#
# * [ObjectConditions](#ObjectConditions) have no effect when filtering objects
# to transfer.
"listUrl": "A String", # Required. The URL that points to the file that stores the object list
# entries. This file must allow public access. Currently, only URLs with
# HTTP and HTTPS schemes are supported.
},
"transferOptions": { # TransferOptions uses three boolean parameters to define the actions # If the option `deleteObjectsUniqueInSink` is `true`, object conditions
# based on objects' `lastModificationTime` are ignored and do not exclude
# objects in a data source or a data sink.
# to be performed on objects in a transfer.
"overwriteObjectsAlreadyExistingInSink": True or False, # Whether overwriting objects that already exist in the sink is allowed.
"deleteObjectsFromSourceAfterTransfer": True or False, # Whether objects should be deleted from the source after they are
# transferred to the sink. Note that this option and
# `deleteObjectsUniqueInSink` are mutually exclusive.
"deleteObjectsUniqueInSink": True or False, # Whether objects that exist only in the sink should be deleted. Note that
# this option and `deleteObjectsFromSourceAfterTransfer` are mutually
# exclusive.
},
"gcsDataSink": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data sink.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"awsS3DataSource": { # An AwsS3Data resource can be a data source, but not a data sink. # An AWS S3 data source.
# In an AwsS3Data resource, an object's name is the S3 object's key name.
"awsAccessKey": { # AWS access key (see # Required. AWS access key used to sign the API requests to the AWS S3
# bucket. Permissions on the bucket must be granted to the access ID of the
# AWS access key.
# [AWS Security
# Credentials](http://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html)).
"secretAccessKey": "A String", # Required. AWS secret access key. This field is not returned in RPC
# responses.
"accessKeyId": "A String", # Required. AWS access key ID.
},
"bucketName": "A String", # Required. S3 Bucket name (see
# [Creating a
# bucket](http://docs.aws.amazon.com/AmazonS3/latest/dev/create-bucket-get-location-example.html)).
},
},
"status": "A String", # Status of the job. This value MUST be specified for
# `CreateTransferJobRequests`.
#
# NOTE: The effect of the new job status takes place during a subsequent job
# run. For example, if you change the job status from `ENABLED` to
# `DISABLED`, and an operation spawned by the transfer is running, the status
# change would not affect the current operation.
"deletionTime": "A String", # Output only. The time that the transfer job was deleted.
"schedule": { # Transfers can be scheduled to recur or to run just once. # Schedule specification.
"startTimeOfDay": { # Represents a time of day. The date and time zone are either not significant # The time in UTC at which the transfer will be scheduled to start in a day.
# Transfers may start later than this time. If not specified, recurring and
# one-time transfers that are scheduled to run today will run immediately;
# recurring transfers that are scheduled to run on a future date will start
# at approximately midnight UTC on that date. Note that when configuring a
# transfer with the Cloud Platform Console, the transfer's start time in a
# day is specified in your local timezone.
# or are specified elsewhere. An API may choose to allow leap seconds. Related
# types are google.type.Date and `google.protobuf.Timestamp`.
"hours": 42, # Hours of day in 24 hour format. Should be from 0 to 23. An API may choose
# to allow the value "24:00:00" for scenarios like business closing time.
"nanos": 42, # Fractions of seconds in nanoseconds. Must be from 0 to 999,999,999.
"seconds": 42, # Seconds of minutes of the time. Must normally be from 0 to 59. An API may
# allow the value 60 if it allows leap-seconds.
"minutes": 42, # Minutes of hour of day. Must be from 0 to 59.
},
"scheduleStartDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # Required. The first day the recurring transfer is scheduled to run. If
# `scheduleStartDate` is in the past, the transfer will run for the first
# time on the following day.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
"scheduleEndDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # The last day the recurring transfer will be run. If `scheduleEndDate`
# is the same as `scheduleStartDate`, the transfer will be executed only
# once.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
},
"projectId": "A String", # The ID of the Google Cloud Platform Project that owns the job.
"lastModificationTime": "A String", # Output only. The time that the transfer job was last modified.
"creationTime": "A String", # Output only. The time that the transfer job was created.
"name": "A String", # A globally unique name assigned by Storage Transfer Service when the
# job is created. This field should be left empty in requests to create a new
# transfer job; otherwise, the requests result in an `INVALID_ARGUMENT`
# error.
"description": "A String", # A description provided by the user for the job. Its max length is 1024
# bytes when Unicode-encoded.
},
}
x__xgafv: string, V1 error format.
Allowed values
1 - v1 error format
2 - v2 error format
Returns:
An object of the form:
{ # This resource represents the configuration of a transfer job that runs
# periodically.
"transferSpec": { # Configuration for running a transfer. # Transfer specification.
"objectConditions": { # Conditions that determine which objects will be transferred. Applies only # Only objects that satisfy these object conditions are included in the set
# of data source and data sink objects. Object conditions based on
# objects' `lastModificationTime` do not exclude objects in a data sink.
# to S3 and GCS objects.
"maxTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` on or after
# `NOW` - `maxTimeElapsedSinceLastModification` and objects that don't have
# a `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
"includePrefixes": [ # If `includePrefixes` is specified, objects that satisfy the object
# conditions must have names that start with one of the `includePrefixes`
# and that do not start with any of the `excludePrefixes`. If
# `includePrefixes` is not specified, all objects except those that have
# names starting with one of the `excludePrefixes` must satisfy the object
# conditions.
#
# Requirements:
#
# * Each include-prefix and exclude-prefix can contain any sequence of
# Unicode characters, of max length 1024 bytes when UTF8-encoded, and
# must not contain Carriage Return or Line Feed characters. Wildcard
# matching and regular expression matching are not supported.
#
# * Each include-prefix and exclude-prefix must omit the leading slash.
# For example, to include the `requests.gz` object in a transfer from
# `s3://my-aws-bucket/logs/y=2015/requests.gz`, specify the include
# prefix as `logs/y=2015/requests.gz`.
#
# * None of the include-prefix or the exclude-prefix values can be empty,
# if specified.
#
# * Each include-prefix must include a distinct portion of the object
# namespace, i.e., no include-prefix may be a prefix of another
# include-prefix.
#
# * Each exclude-prefix must exclude a distinct portion of the object
# namespace, i.e., no exclude-prefix may be a prefix of another
# exclude-prefix.
#
# * If `includePrefixes` is specified, then each exclude-prefix must start
# with the value of a path explicitly included by `includePrefixes`.
#
# The max size of `includePrefixes` is 1000.
"A String",
],
"excludePrefixes": [ # `excludePrefixes` must follow the requirements described for
# `includePrefixes`.
#
# The max size of `excludePrefixes` is 1000.
"A String",
],
"minTimeElapsedSinceLastModification": "A String", # If specified, only objects with a `lastModificationTime` before
# `NOW` - `minTimeElapsedSinceLastModification` and objects that don't have a
# `lastModificationTime` are transferred.
#
# Note that, for each `TransferOperation` started by this `TransferJob`,
# `NOW` refers to the `start_time` of the 'TransferOperation`. Also,
# `lastModificationTime` refers to the time of the last change to the
# object's content or metadata - specifically, this would be the `updated`
# property of GCS objects and the `LastModified` field of S3 objects.
},
"gcsDataSource": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data source.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"httpDataSource": { # An HttpData resource specifies a list of objects on the web to be transferred # An HTTP URL data source.
# over HTTP. The information of the objects to be transferred is contained in
# a file referenced by a URL. The first line in the file must be
# "TsvHttpData-1.0", which specifies the format of the file. Subsequent lines
# specify the information of the list of objects, one object per list entry.
# Each entry has the following tab-delimited fields:
#
# * HTTP URL - The location of the object.
#
# * Length - The size of the object in bytes.
#
# * MD5 - The base64-encoded MD5 hash of the object.
#
# For an example of a valid TSV file, see
# [Transferring data from
# URLs](https://cloud.google.com/storage/transfer/create-url-list).
#
# When transferring data based on a URL list, keep the following in mind:
#
# * When an object located at `http(s)://hostname:port/` is
# transferred to a data sink, the name of the object at the data sink is
# `/`.
#
# * If the specified size of an object does not match the actual size of the
# object fetched, the object will not be transferred.
#
# * If the specified MD5 does not match the MD5 computed from the transferred
# bytes, the object transfer will fail. For more information, see
# [Generating MD5 hashes](https://cloud.google.com/storage/transfer/#md5)
#
# * Ensure that each URL you specify is publicly accessible. For
# example, in Google Cloud Storage you can
# [share an object publicly]
# (https://cloud.google.com/storage/docs/cloud-console#_sharingdata) and get
# a link to it.
#
# * Storage Transfer Service obeys `robots.txt` rules and requires the source
# HTTP server to support `Range` requests and to return a `Content-Length`
# header in each response.
#
# * [ObjectConditions](#ObjectConditions) have no effect when filtering objects
# to transfer.
"listUrl": "A String", # Required. The URL that points to the file that stores the object list
# entries. This file must allow public access. Currently, only URLs with
# HTTP and HTTPS schemes are supported.
},
"transferOptions": { # TransferOptions uses three boolean parameters to define the actions # If the option `deleteObjectsUniqueInSink` is `true`, object conditions
# based on objects' `lastModificationTime` are ignored and do not exclude
# objects in a data source or a data sink.
# to be performed on objects in a transfer.
"overwriteObjectsAlreadyExistingInSink": True or False, # Whether overwriting objects that already exist in the sink is allowed.
"deleteObjectsFromSourceAfterTransfer": True or False, # Whether objects should be deleted from the source after they are
# transferred to the sink. Note that this option and
# `deleteObjectsUniqueInSink` are mutually exclusive.
"deleteObjectsUniqueInSink": True or False, # Whether objects that exist only in the sink should be deleted. Note that
# this option and `deleteObjectsFromSourceAfterTransfer` are mutually
# exclusive.
},
"gcsDataSink": { # In a GcsData resource, an object's name is the Google Cloud Storage object's # A Google Cloud Storage data sink.
# name and its `lastModificationTime` refers to the object's updated time,
# which changes when the content or the metadata of the object is updated.
"bucketName": "A String", # Required. Google Cloud Storage bucket name (see
# [Bucket Name
# Requirements](https://cloud.google.com/storage/docs/naming#requirements)).
},
"awsS3DataSource": { # An AwsS3Data resource can be a data source, but not a data sink. # An AWS S3 data source.
# In an AwsS3Data resource, an object's name is the S3 object's key name.
"awsAccessKey": { # AWS access key (see # Required. AWS access key used to sign the API requests to the AWS S3
# bucket. Permissions on the bucket must be granted to the access ID of the
# AWS access key.
# [AWS Security
# Credentials](http://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html)).
"secretAccessKey": "A String", # Required. AWS secret access key. This field is not returned in RPC
# responses.
"accessKeyId": "A String", # Required. AWS access key ID.
},
"bucketName": "A String", # Required. S3 Bucket name (see
# [Creating a
# bucket](http://docs.aws.amazon.com/AmazonS3/latest/dev/create-bucket-get-location-example.html)).
},
},
"status": "A String", # Status of the job. This value MUST be specified for
# `CreateTransferJobRequests`.
#
# NOTE: The effect of the new job status takes place during a subsequent job
# run. For example, if you change the job status from `ENABLED` to
# `DISABLED`, and an operation spawned by the transfer is running, the status
# change would not affect the current operation.
"deletionTime": "A String", # Output only. The time that the transfer job was deleted.
"schedule": { # Transfers can be scheduled to recur or to run just once. # Schedule specification.
"startTimeOfDay": { # Represents a time of day. The date and time zone are either not significant # The time in UTC at which the transfer will be scheduled to start in a day.
# Transfers may start later than this time. If not specified, recurring and
# one-time transfers that are scheduled to run today will run immediately;
# recurring transfers that are scheduled to run on a future date will start
# at approximately midnight UTC on that date. Note that when configuring a
# transfer with the Cloud Platform Console, the transfer's start time in a
# day is specified in your local timezone.
# or are specified elsewhere. An API may choose to allow leap seconds. Related
# types are google.type.Date and `google.protobuf.Timestamp`.
"hours": 42, # Hours of day in 24 hour format. Should be from 0 to 23. An API may choose
# to allow the value "24:00:00" for scenarios like business closing time.
"nanos": 42, # Fractions of seconds in nanoseconds. Must be from 0 to 999,999,999.
"seconds": 42, # Seconds of minutes of the time. Must normally be from 0 to 59. An API may
# allow the value 60 if it allows leap-seconds.
"minutes": 42, # Minutes of hour of day. Must be from 0 to 59.
},
"scheduleStartDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # Required. The first day the recurring transfer is scheduled to run. If
# `scheduleStartDate` is in the past, the transfer will run for the first
# time on the following day.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
"scheduleEndDate": { # Represents a whole or partial calendar date, e.g. a birthday. The time of day # The last day the recurring transfer will be run. If `scheduleEndDate`
# is the same as `scheduleStartDate`, the transfer will be executed only
# once.
# and time zone are either specified elsewhere or are not significant. The date
# is relative to the Proleptic Gregorian Calendar. This can represent:
#
# * A full date, with non-zero year, month and day values
# * A month and day value, with a zero year, e.g. an anniversary
# * A year on its own, with zero month and day values
# * A year and month value, with a zero day, e.g. a credit card expiration date
#
# Related types are google.type.TimeOfDay and `google.protobuf.Timestamp`.
"month": 42, # Month of year. Must be from 1 to 12, or 0 if specifying a year without a
# month and day.
"day": 42, # Day of month. Must be from 1 to 31 and valid for the year and month, or 0
# if specifying a year by itself or a year and month where the day is not
# significant.
"year": 42, # Year of date. Must be from 1 to 9999, or 0 if specifying a date without
# a year.
},
},
"projectId": "A String", # The ID of the Google Cloud Platform Project that owns the job.
"lastModificationTime": "A String", # Output only. The time that the transfer job was last modified.
"creationTime": "A String", # Output only. The time that the transfer job was created.
"name": "A String", # A globally unique name assigned by Storage Transfer Service when the
# job is created. This field should be left empty in requests to create a new
# transfer job; otherwise, the requests result in an `INVALID_ARGUMENT`
# error.
"description": "A String", # A description provided by the user for the job. Its max length is 1024
# bytes when Unicode-encoded.
}