Options
Name | Type | Description |
---|---|---|
delete_sources | boolean | Delete the task's source files upon successful completion of the task. |
delete_source_on_completion | boolean | Will delete source on successful copy. |
throttle_byte_per_sec | integer string | Throttle the copy operation. Useful, for example, for operation with SwiftStack storage on shared network lines. |
restore_from_glacier | boolean | If a file is on s3 with the storage class 'GLACIER', issue a restore and wait for arrival before copying. |
multi_file_concurrency | integer | If multiple files are to be copied, limit the concurrency of the copy operation. |
multi_source_mode | enum use_first use_all concatenate | If multiple tasks feed into a single copy task: "use_all": Copy from all incoming tasks to the output location (default). "concatenate": Concatenate the outputs from all incoming tasks into a single output file. "use_first": Only copy output from first task, but wait for all incoming tasks to complete before proceeding in the workflow. |
Example options
{
"uid": "copy_task",
"kind": "copy",
"payload": {
"target": {
"location": {
"storage_provider": "s3",
"path": "s3://my_destination_bucket/my_destination_folder"
},
"existing_files": "delete_and_replace"
},
"options": {
"delete_source_on_completion": true
}
}
}