Settings

Name	Type	Description
key	string	A unique key to identify this watchfolder for tracking processed source files.
watch_items_persistence	enum                           tracked untracked	When a watchfolder is stopped and later re-started, this parameter indicates whether Hybrik should track all files previously processed by the watchfolder.
interval_sec	integer	Polling interval in seconds for the watch folder. default: 300
recursive	boolean	Recursively watch the folder and its sub-folders.
process_existing_files	boolean	When the watchfolder process initiates, there may be contents already existing in the target location. Setting this value to true will indicate that pre-existing files should trigger new jobs.
pattern_matching	enum                           wildcard regex	Whether to use a simple file-system style wildcard to filter incoming files, or to use a regular expression. default: wildcard
wildcard	string	Which expression to use for pattern matching. default: *
regex	string	A regular expression may be used to match only certain file names for processing.

---

Example settings

{
  "uid": "watchfolder_source",
  "kind": "watchfolder",
  "task": {
    "tags": [
      "WATCH_FOLDER"
    ]
  },
  "payload": {
    "source": {
      "storage_provider": "s3",
      "path": "s3://my_bucket/my_watchfolder"
    },
    "settings": {
      "key": "watch_folder",
      "interval_sec": 5,
      "pattern_matching": "wildcard",
      "wildcard": "*",
      "recursive": true,
      "process_existing_files": true
    }
  }
}