Skip to main content

Options

NameTypeDescription
delete_sourcesboolean
Delete the task's source files upon successful completion of the task.
delete_source_on_completionbooleanWill delete source on successful copy.
throttle_byte_per_secinteger
string
Throttle the copy operation. Useful, for example, for operation with SwiftStack storage on shared network lines.
restore_from_glacierbooleanIf a file is on s3 with the storage class 'GLACIER', issue a restore and wait for arrival before copying.
multi_file_concurrencyintegerIf multiple files are to be copied, limit the concurrency of the copy operation.
multi_source_modeenum                          
use_first
use_all
concatenate
If multiple tasks feed into a single copy task: "use_all": Copy from all incoming tasks to the output location (default). "concatenate": Concatenate the outputs from all incoming tasks into a single output file. "use_first": Only copy output from first task, but wait for all incoming tasks to complete before proceeding in the workflow.

Example options

{
"uid": "copy_task",
"kind": "copy",
"payload": {
"target": {
"location": {
"storage_provider": "s3",
"path": "s3://my_destination_bucket/my_destination_folder"
},
"existing_files": "delete_and_replace"
},
"options": {
"delete_source_on_completion": true
}
}
}