rocoto
The uw
mode for realizing and validating Rocoto XML documents.
uw rocoto --help
usage: uw rocoto [-h] [--version] ACTION ...
Realize and validate Rocoto XML documents
Optional arguments:
-h, --help
Show help and exit
--version
Show version info and exit
Positional arguments:
ACTION
realize
Realize a Rocoto XML workflow document
validate
Validate Rocoto XML
realize
In uw
terminology, to realize
a configuration file is to transform it from its raw form into its final, usable state. In the case of uw rocoto
, that means transforming a structured UW YAML file into a Rocoto XML file. The structured YAML language required by UW closely follows the XML language defined by Rocoto.
More information about the structured UW YAML file for Rocoto can be found here.
uw rocoto realize --help
usage: uw rocoto realize [-h] [--version] [--config-file PATH]
[--output-file PATH] [--quiet] [--verbose]
Realize a Rocoto XML workflow document
Optional arguments:
-h, --help
Show help and exit
--version
Show version info and exit
--config-file PATH, -c PATH
Path to UW YAML config file (default: read from stdin)
--output-file PATH, -o PATH
Path to output file (defaults to stdout)
--quiet, -q
Print no logging messages
--verbose, -v
Print all logging messages
Examples
The examples in this section use a UW YAML file rocoto.yaml
with contents:
workflow:
attrs:
realtime: false
scheduler: slurm
cycledef:
- attrs:
group: howdy
spec: 202209290000 202209300000 06:00:00
entities:
ACCOUNT: myaccount
FOO: test.log
log: /some/path/to/&FOO;
tasks:
task_hello:
attrs:
cycledefs: howdy
account: "&ACCOUNT;"
command: "echo hello $person"
jobname: hello
native: --reservation my_reservation
nodes: 1:ppn=1
walltime: 00:01:00
envars:
person: siri
To realize a UW YAML input file to
stdout
in Rocoto XML format:uw rocoto realize --config-file rocoto.yaml
[2024-08-26T23:11:42] INFO 0 schema-validation errors found in Rocoto config [2024-08-26T23:11:42] INFO 0 Rocoto XML validation errors found <?xml version='1.0' encoding='utf-8'?> <!DOCTYPE workflow [ <!ENTITY ACCOUNT "myaccount"> <!ENTITY FOO "test.log"> ]> <workflow realtime="False" scheduler="slurm"> <cycledef group="howdy">202209290000 202209300000 06:00:00</cycledef> <log>/some/path/to/&FOO;</log> <task name="hello" cycledefs="howdy"> <account>&ACCOUNT;</account> <nodes>1:ppn=1</nodes> <walltime>00:01:00</walltime> <command>echo hello $person</command> <jobname>hello</jobname> <native>--reservation my_reservation</native> <envar> <name>person</name> <value>siri</value> </envar> </task> </workflow>
To realize a UW YAML input file to a file named
rocoto.xml
:rm -f rocoto.xml uw rocoto realize --config-file rocoto.yaml --output-file rocoto.xml echo cat rocoto.xml
[2024-08-26T23:11:41] INFO 0 schema-validation errors found in Rocoto config [2024-08-26T23:11:41] INFO 0 Rocoto XML validation errors found <?xml version='1.0' encoding='utf-8'?> <!DOCTYPE workflow [ <!ENTITY ACCOUNT "myaccount"> <!ENTITY FOO "test.log"> ]> <workflow realtime="False" scheduler="slurm"> <cycledef group="howdy">202209290000 202209300000 06:00:00</cycledef> <log>/some/path/to/&FOO;</log> <task name="hello" cycledefs="howdy"> <account>&ACCOUNT;</account> <nodes>1:ppn=1</nodes> <walltime>00:01:00</walltime> <command>echo hello $person</command> <jobname>hello</jobname> <native>--reservation my_reservation</native> <envar> <name>person</name> <value>siri</value> </envar> </task> </workflow>
To read the UW YAML from
stdin
and write the XML tostdout
:cat rocoto.yaml | uw rocoto realize
[2024-08-26T23:11:42] INFO 0 schema-validation errors found in Rocoto config [2024-08-26T23:11:42] INFO 0 Rocoto XML validation errors found <?xml version='1.0' encoding='utf-8'?> <!DOCTYPE workflow [ <!ENTITY ACCOUNT "myaccount"> <!ENTITY FOO "test.log"> ]> <workflow realtime="False" scheduler="slurm"> <cycledef group="howdy">202209290000 202209300000 06:00:00</cycledef> <log>/some/path/to/&FOO;</log> <task name="hello" cycledefs="howdy"> <account>&ACCOUNT;</account> <nodes>1:ppn=1</nodes> <walltime>00:01:00</walltime> <command>echo hello $person</command> <jobname>hello</jobname> <native>--reservation my_reservation</native> <envar> <name>person</name> <value>siri</value> </envar> </task> </workflow>
To see verbose log output (Rocoto XML and some output elided for brevity):
rm -f rocoto.log uw rocoto realize --config-file rocoto.yaml --verbose >/dev/null 2>rocoto.log head -n10 rocoto.log echo ... tail -n10 rocoto.log
[2025-01-05T21:15:07] DEBUG Command: uw rocoto realize --config-file rocoto.yaml --verbose [2025-01-05T21:15:07] DEBUG [dereference] Dereferencing, current value: [2025-01-05T21:15:07] DEBUG [dereference] workflow: [2025-01-05T21:15:07] DEBUG [dereference] attrs: [2025-01-05T21:15:07] DEBUG [dereference] realtime: false [2025-01-05T21:15:07] DEBUG [dereference] scheduler: slurm [2025-01-05T21:15:07] DEBUG [dereference] cycledef: [2025-01-05T21:15:07] DEBUG [dereference] - attrs: [2025-01-05T21:15:07] DEBUG [dereference] group: howdy [2025-01-05T21:15:07] DEBUG [dereference] spec: 202209290000 202209300000 06:00:00 ... [2025-01-05T21:15:07] DEBUG [dereference] cycledefs: howdy [2025-01-05T21:15:07] DEBUG [dereference] account: '&ACCOUNT;' [2025-01-05T21:15:07] DEBUG [dereference] command: echo hello $person [2025-01-05T21:15:07] DEBUG [dereference] jobname: hello [2025-01-05T21:15:07] DEBUG [dereference] native: --reservation my_reservation [2025-01-05T21:15:07] DEBUG [dereference] nodes: 1:ppn=1 [2025-01-05T21:15:07] DEBUG [dereference] walltime: 00:01:00 [2025-01-05T21:15:07] DEBUG [dereference] envars: [2025-01-05T21:15:07] DEBUG [dereference] person: siri [2025-01-05T21:15:07] INFO 0 Rocoto XML validation errors found
validate
uw rocoto validate --help
usage: uw rocoto validate [-h] [--version] [--input-file PATH] [--quiet]
[--verbose]
Validate Rocoto XML
Optional arguments:
-h, --help
Show help and exit
--version
Show version info and exit
--input-file PATH, -i PATH
Path to input file (defaults to stdin)
--quiet, -q
Print no logging messages
--verbose, -v
Print all logging messages
Examples
The examples in this section use a Rocoto XML file rocoto-good.xml
with contents:
<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE workflow [
<!ENTITY ACCOUNT "myaccount">
<!ENTITY FOO "test.log">
]>
<workflow realtime="False" scheduler="slurm">
<cycledef group="howdy">202209290000 202209300000 06:00:00</cycledef>
<log>/some/path/to/&FOO;</log>
<task name="hello" cycledefs="howdy">
<account>&ACCOUNT;</account>
<nodes>1:ppn=1</nodes>
<walltime>00:01:00</walltime>
<command>echo hello $person</command>
<jobname>hello</jobname>
<envar>
<name>person</name>
<value>siri</value>
</envar>
</task>
</workflow>
To validate XML from
stdin
:cat rocoto-good.xml | uw rocoto validate
[2024-08-26T23:11:42] INFO 0 Rocoto XML validation errors found
To validate XML from file
rocoto-good.xml
:uw rocoto validate --input-file rocoto-good.xml
[2024-08-26T23:11:42] INFO 0 Rocoto XML validation errors found
When the XML is invalid:
In this example, the
<command>
line was removed fromrocoto-good.xml
to createrocoto-bad.xml
.uw rocoto validate --input-file rocoto-bad.xml
[2024-08-26T23:11:41] ERROR 3 Rocoto XML validation errors found [2024-08-26T23:11:41] ERROR <string>:9:0:ERROR:RELAXNGV:RELAXNG_ERR_NOELEM: Expecting an element command, got nothing [2024-08-26T23:11:41] ERROR <string>:9:0:ERROR:RELAXNGV:RELAXNG_ERR_INTERSEQ: Invalid sequence in interleave [2024-08-26T23:11:41] ERROR <string>:9:0:ERROR:RELAXNGV:RELAXNG_ERR_CONTENTVALID: Element task failed to validate content [2024-08-26T23:11:41] ERROR Invalid Rocoto XML: [2024-08-26T23:11:41] ERROR 1 <?xml version='1.0' encoding='utf-8'?> [2024-08-26T23:11:41] ERROR 2 <!DOCTYPE workflow [ [2024-08-26T23:11:41] ERROR 3 <!ENTITY ACCOUNT "myaccount"> [2024-08-26T23:11:41] ERROR 4 <!ENTITY FOO "test.log"> [2024-08-26T23:11:41] ERROR 5 ]> [2024-08-26T23:11:41] ERROR 6 <workflow realtime="False" scheduler="slurm"> [2024-08-26T23:11:41] ERROR 7 <cycledef group="howdy">202209290000 202209300000 06:00:00</cycledef> [2024-08-26T23:11:41] ERROR 8 <log>/some/path/to/&FOO;</log> [2024-08-26T23:11:41] ERROR 9 <task name="hello" cycledefs="howdy"> [2024-08-26T23:11:41] ERROR 10 <account>&ACCOUNT;</account> [2024-08-26T23:11:41] ERROR 11 <nodes>1:ppn=1</nodes> [2024-08-26T23:11:41] ERROR 12 <walltime>00:01:00</walltime> [2024-08-26T23:11:41] ERROR 13 <jobname>hello</jobname> [2024-08-26T23:11:41] ERROR 14 <envar> [2024-08-26T23:11:41] ERROR 15 <name>person</name> [2024-08-26T23:11:41] ERROR 16 <value>siri</value> [2024-08-26T23:11:41] ERROR 17 </envar> [2024-08-26T23:11:41] ERROR 18 </task> [2024-08-26T23:11:41] ERROR 19 </workflow>
To decode the three
ERROR:RELAXNGV
messages, it is easiest to read from the bottom up. They say:At line 9 column 0 (i.e.
<string>:9:0
), the element (i.e.<task>
) failed to validate.The sequence of interleaved elements under
<task>
was invalid.A
<command>
element was expected, but it wasn’t found.