file

The uw mode for handling filesystem files.

uw file --help

usage: uw file [-h] [--version] ACTION ...

Handle files

Optional arguments:
  -h, --help
      Show help and exit
  --version
      Show version info and exit

Positional arguments:
  ACTION
    copy
      Copy files
    link
      Link files

copy

The copy action stages files in a target directory by copying files. Any KEY positional arguments are used to navigate, in the order given, from the top of the config to the file block.

uw file copy --help

usage: uw file copy --target-dir PATH [-h] [--version] [--config-file PATH]
                    [--cycle CYCLE] [--leadtime LEADTIME] [--dry-run]
                    [--quiet] [--verbose]
                    [KEY ...]

Copy files

Required arguments:
  --target-dir PATH
      Path to target directory

Optional arguments:
  -h, --help
      Show help and exit
  --version
      Show version info and exit
  --config-file PATH, -c PATH
      Path to UW YAML config file (default: read from stdin)
  --cycle CYCLE
      The cycle in ISO8601 format (e.g. 2024-05-29T12)
  --leadtime LEADTIME
      The leadtime as HH[:MM[:SS]]
  --dry-run
      Only log info, making no changes
  --quiet, -q
      Print no logging messages
  --verbose, -v
      Print all logging messages
  KEY
      YAML key leading to file dst/src block

Examples

Given copy-config.yaml containing

config:
  files:
    foo: src/foo
    subdir/bar: src/bar

rm -rf copy-dst
uw file copy --target-dir copy-dst --config-file copy-config.yaml config files
echo
tree copy-dst

[2024-05-29T16:50:14]     INFO Validating config against internal schema files-to-stage
[2024-05-29T16:50:14]     INFO 0 UW schema-validation errors found
[2024-05-29T16:50:14]     INFO File copies: Initial state: Pending
[2024-05-29T16:50:14]     INFO File copies: Checking requirements
[2024-05-29T16:50:14]     INFO Copy src/foo -> copy-dst/foo: Initial state: Pending
[2024-05-29T16:50:14]     INFO Copy src/foo -> copy-dst/foo: Checking requirements
[2024-05-29T16:50:14]     INFO Copy src/foo -> copy-dst/foo: Requirement(s) ready
[2024-05-29T16:50:14]     INFO Copy src/foo -> copy-dst/foo: Executing
[2024-05-29T16:50:14]     INFO Copy src/foo -> copy-dst/foo: Final state: Ready
[2024-05-29T16:50:14]     INFO Copy src/bar -> copy-dst/subdir/bar: Initial state: Pending
[2024-05-29T16:50:14]     INFO Copy src/bar -> copy-dst/subdir/bar: Checking requirements
[2024-05-29T16:50:14]     INFO Copy src/bar -> copy-dst/subdir/bar: Requirement(s) ready
[2024-05-29T16:50:14]     INFO Copy src/bar -> copy-dst/subdir/bar: Executing
[2024-05-29T16:50:14]     INFO Copy src/bar -> copy-dst/subdir/bar: Final state: Ready
[2024-05-29T16:50:14]     INFO File copies: Final state: Ready

copy-dst
├── foo
└── subdir
    └── bar

2 directories, 2 files

Here, foo and bar are copies of their respective source files.

The --cycle and --leadtime options can be used to make Python datetime and timedelta objects, respectively, available for use in Jinja2 expression in the config. For example:

config:
  files:
    baz-{{ validtime }}: src/{{ yyyymmdd }}/{{ hh }}/{{ nnn }}/baz
yyyymmdd: "{{ cycle.strftime('%Y%m%d') }}"
hh: "{{ cycle.strftime('%H') }}"
nnn: "{{ '%03d' % (leadtime.total_seconds() // 3600) }}"
validtime: "{{ (cycle + leadtime).strftime('%Y-%m-%dT%H') }}"

rm -rf copy-dst-timedep
uw file copy --target-dir copy-dst-timedep --config-file copy-config-timedep.yaml --cycle 2024-05-29T12 --leadtime 6 config files
echo
tree copy-dst-timedep

[2024-05-29T16:50:14]     INFO Validating config against internal schema files-to-stage
[2024-05-29T16:50:14]     INFO 0 UW schema-validation errors found
[2024-05-29T16:50:14]     INFO File copies: Initial state: Pending
[2024-05-29T16:50:14]     INFO File copies: Checking requirements
[2024-05-29T16:50:14]     INFO Copy src/20240529/12/006/baz -> copy-dst-timedep/baz-2024-05-29T18: Initial state: Pending
[2024-05-29T16:50:14]     INFO Copy src/20240529/12/006/baz -> copy-dst-timedep/baz-2024-05-29T18: Checking requirements
[2024-05-29T16:50:14]     INFO Copy src/20240529/12/006/baz -> copy-dst-timedep/baz-2024-05-29T18: Requirement(s) ready
[2024-05-29T16:50:14]     INFO Copy src/20240529/12/006/baz -> copy-dst-timedep/baz-2024-05-29T18: Executing
[2024-05-29T16:50:14]     INFO Copy src/20240529/12/006/baz -> copy-dst-timedep/baz-2024-05-29T18: Final state: Ready
[2024-05-29T16:50:14]     INFO File copies: Final state: Ready

copy-dst-timedep
└── baz-2024-05-29T18

1 directory, 1 file