file

The uw mode for handling filesystem files.

uw file --help

usage: uw file [-h] [--version] ACTION ...

Handle files

Optional arguments:
  -h, --help
      Show help and exit
  --version
      Show version info and exit

Positional arguments:
  ACTION
    copy
      Copy files
    link
      Link files

copy

The copy action stages files in a target directory by copying files. Any KEY positional arguments are used to navigate, in the order given, from the top of the config to the file block.

uw file copy --help

usage: uw file copy [-h] [--version] [--config-file PATH] [--target-dir PATH]
                    [--cycle CYCLE] [--leadtime LEADTIME] [--dry-run]
                    [--quiet] [--verbose]
                    [KEY ...]

Copy files

Optional arguments:
  -h, --help
      Show help and exit
  --version
      Show version info and exit
  --config-file PATH, -c PATH
      Path to UW YAML config file (default: read from stdin)
  --target-dir PATH
      Root directory for relative destination paths
  --cycle CYCLE
      The cycle in ISO8601 format (e.g. 2024-05-29T12)
  --leadtime LEADTIME
      The leadtime as hours[:minutes[:seconds]]
  --dry-run
      Only log info, making no changes
  --quiet, -q
      Print no logging messages
  --verbose, -v
      Print all logging messages
  KEY
      YAML key leading to file dst/src block

Examples

Given copy-config.yaml containing

config:
  files:
    foo: src/foo
    subdir/bar: src/bar

rm -rf copy-dst
uw file copy --target-dir copy-dst --config-file copy-config.yaml config files
echo
tree copy-dst

[2024-08-02T00:43:08]     INFO Validating config against internal schema: files-to-stage
[2024-08-02T00:43:08]     INFO 0 UW schema-validation errors found
[2024-08-02T00:43:08]     INFO File copies: Initial state: Not Ready
[2024-08-02T00:43:08]     INFO File copies: Checking requirements
[2024-08-02T00:43:08]     INFO Copy src/foo -> copy-dst/foo: Initial state: Not Ready
[2024-08-02T00:43:08]     INFO Copy src/foo -> copy-dst/foo: Checking requirements
[2024-08-02T00:43:08]     INFO Copy src/foo -> copy-dst/foo: Requirement(s) ready
[2024-08-02T00:43:08]     INFO Copy src/foo -> copy-dst/foo: Executing
[2024-08-02T00:43:08]     INFO Copy src/foo -> copy-dst/foo: Final state: Ready
[2024-08-02T00:43:08]     INFO Copy src/bar -> copy-dst/subdir/bar: Initial state: Not Ready
[2024-08-02T00:43:08]     INFO Copy src/bar -> copy-dst/subdir/bar: Checking requirements
[2024-08-02T00:43:08]     INFO Copy src/bar -> copy-dst/subdir/bar: Requirement(s) ready
[2024-08-02T00:43:08]     INFO Copy src/bar -> copy-dst/subdir/bar: Executing
[2024-08-02T00:43:08]     INFO Copy src/bar -> copy-dst/subdir/bar: Final state: Ready
[2024-08-02T00:43:08]     INFO File copies: Final state: Ready

copy-dst
├── foo
└── subdir
    └── bar

2 directories, 2 files

Here, foo and bar are copies of their respective source files.

The --cycle and --leadtime options can be used to make Python datetime and timedelta objects, respectively, available for use in Jinja2 expression in the config. For example:

config:
  files:
    baz-{{ validtime }}: src/{{ yyyymmdd }}/{{ hh }}/{{ nnn }}/baz
yyyymmdd: "{{ cycle.strftime('%Y%m%d') }}"
hh: "{{ cycle.strftime('%H') }}"
nnn: "{{ '%03d' % (leadtime.total_seconds() // 3600) }}"
validtime: "{{ (cycle + leadtime).strftime('%Y-%m-%dT%H') }}"

rm -rf copy-dst-timedep
uw file copy --target-dir copy-dst-timedep --config-file copy-config-timedep.yaml --cycle 2024-05-29T12 --leadtime 6 config files
echo
tree copy-dst-timedep

[2024-08-02T00:43:08]     INFO Validating config against internal schema: files-to-stage
[2024-08-02T00:43:08]     INFO 0 UW schema-validation errors found
[2024-08-02T00:43:08]     INFO File copies: Initial state: Not Ready
[2024-08-02T00:43:08]     INFO File copies: Checking requirements
[2024-08-02T00:43:08]     INFO Copy src/20240529/12/006/baz -> copy-dst-timedep/baz-2024-05-29T18: Initial state: Not Ready
[2024-08-02T00:43:08]     INFO Copy src/20240529/12/006/baz -> copy-dst-timedep/baz-2024-05-29T18: Checking requirements
[2024-08-02T00:43:08]     INFO Copy src/20240529/12/006/baz -> copy-dst-timedep/baz-2024-05-29T18: Requirement(s) ready
[2024-08-02T00:43:08]     INFO Copy src/20240529/12/006/baz -> copy-dst-timedep/baz-2024-05-29T18: Executing
[2024-08-02T00:43:08]     INFO Copy src/20240529/12/006/baz -> copy-dst-timedep/baz-2024-05-29T18: Final state: Ready
[2024-08-02T00:43:08]     INFO File copies: Final state: Ready

copy-dst-timedep
└── baz-2024-05-29T18

1 directory, 1 file

The --target-dir option is optional when all destination paths are absolute, and will never be applied to absolute destination paths. If any destination paths are relative, however, it is an error not to provide a target directory:

config:
  files:
    foo: src/foo
    subdir/bar: src/bar

uw file copy --config-file copy-config.yaml config files

[2024-07-26T21:51:23]    ERROR Relative path 'foo' requires the target directory to be specified