uwtools + ecFlow¶

The notebook demonstrates the use of uwtools to configure and run an ecFlow server, to generate an ecFlow workflow (a suite definition and ecf scripts), and to load the suite into the server and execute it. As a demo application, we perform verification of 2-meter temperature at the 6-hour leadtime of a GFS forecast against a GFS leadtime-zero forecast as a source of truth.

A recipe for creating a software environment in which this notebook can be interacted with live can be found by following the ecFlow Tool link in the Jupyter Notebooks section in the uwtools documentation.

Note that, unlike many Jupyter notebooks, the code cells in this notebook are interpreted by bash rather than Python. So, if you make changes, be sure to enter only valid bash code.

First, get to the correct working directory and activate the virtual environment for the demo:

In [1]:
cd $BASEDIR
source conda/etc/profile.d/conda.sh
conda activate demo

Here's the uwtools-enabled configuration file for the demo. It's not necessary to understand all of it now. The top-level app block defines various application-level items that are useful throughout the workflow. The ecflow block defines server and suite-definition blocks specific to ecFlow. And the wxvx block defines parameters for verification to be performed by wxvx.

Some observations:

  • Note the use of Jinja2 expressions to correctly tie together various config blocks.
  • The key ecflow.suitedef.suite_vx.vars.ECF_JOB_CMD defines the command used to start local jobs, overriding the server's default value to force the use of bash as the shell.
  • Likewise, the ecflow.suitedef.suite_vx.task_verification.vars block defines ECF_JOB_CMD, ECF_KILL_CMD, and ECF_STATUS_CMD to submit a Slurm batch job, kill it, and query its status, respectively. The value of ECF_JOB_CMD not only submits the job script to Slurm with sbatch, but also captures the Slurm job ID and sets the ECF_RID value for the task to this ID, so that the ECF_KILL_CMD and ECF_STATUS_CMD commands can make use of it to perform their functions.
In [2]:
cat config.yaml

app:
  basedir: '{{ "BASEDIR" | env }}'
  cycle: 2026-07-04T12:00:00
  fcst:
    fn: gfs.t{{ app.hh }}z.pgrb2.1p00.f{{ "%03d" % app.leadtime }}
    url: '{{ app.prefix }}/{{ app.fcst.fn }}'
  hh: '{{ app.cycle.strftime("%H") }}'
  leadtime: 6
  prefix: https://nomads.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.{{ app.yyyymmdd }}/{{ app.hh }}/atmos
  yyyymmdd: '{{ app.cycle.strftime("%Y%m%d") }}'
ecflow:
  server:
    ECF_HOME: '{{ app.basedir }}'
  suitedef:
    scheduler: slurm
    suite_vx:
      task_extract_grids:
        script:
          body: |
            init $$
            uw config realize -i {{ app.basedir }}/config.yaml --key-path wxvx >{{ app.basedir }}/vx.yaml
            wxvx -c {{ app.basedir }}/vx.yaml -t grids
          includes:
            entry: [common.h, server.h]
          manual: Extract forecast and truth grids
        trigger: /vx/fetch_truth == complete
      task_fetch_truth:
        script:
          body: |
            init $$
            test -e {{ app.fcst.fn }} || wget --quiet {{ app.fcst.url }}
          includes:
            entry: [common.h, server.h]
          manual: Fetch GRIB truth data
      task_verification:
        script:
          batchargs:
            cores: 1
            export: false
            rundir: '{{ app.basedir }}/vx'
            stdout: verification.out
            walltime: 00:05:00
          body: |
            init $SLURM_JOB_ID
            wxvx -c {{ app.basedir }}/vx.yaml -t stats
          includes:
            entry: [common.h, server.h]
          manual: Perform verification
        trigger: /vx/extract_grids == complete
        vars:
          ECF_JOB_CMD: ecflow_client --alter=add variable ECF_RID $(sbatch --parsable %ECF_JOB%) %ECF_NAME%
          ECF_KILL_CMD: scancel %ECF_RID%
          ECF_STATUS_CMD: sacct -lj %ECF_RID%
      vars:
        ECF_JOB_CMD: /usr/bin/env -S bash %ECF_JOB% >%ECF_JOBOUT% 2>&1
wxvx:
  forecast:
    name: GFSFCST
    path: '{{ app.basedir }}/{{ app.fcst.fn }}'
  meta:
    hh: '{{ app.hh }}'
    prefix: '{{ app.prefix }}'
  paths:
    grids:
      forecast: "{{ app.basedir }}/vx/forecast"
      truth: "{{ app.basedir }}/vx/truth"
    run: "{{ app.basedir }}/vx/run"
  timepairs: [[!datetime '{{ app.cycle }}', !int '{{ app.leadtime}} ']]
  truth:
    name: GFS
    type: grid
    url: '{{ meta.prefix }}/gfs.t{{ meta.hh }}z.pgrb2.1p00.f000'
  variables:
    T2M:
      level_type: heightAboveGround
      levels: [2]
      name: 2t

The ecflow.server block provides the bare minimum for running the ecFlow server: A definition of the ECF_HOME environment variable, which specifies where the server should run and where, by default, it will look for .h include files when creating job scripts from ecf scripts:

In [3]:
uw config realize -i config.yaml --key-path ecflow.server

ECF_HOME: /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip

In this case, ecflow.server refers to app.basedir for its value, and app.basedir refers to the environment variable BASEDIR set by the start script.

Start the ecFlow server using the config specified in config.yaml. Request a JSON-formatted report of important server environment variables, redirect that to server.json, and redirect the rest of the output to server.log:

In [4]:
uw ecflow server -c config.yaml --report >server.json 2>server.log &

[1] 286156

Wait until the server has started up and written server.json:

In [5]:
while [[ ! -s server.json ]]; do sleep 1; done

View server.log and note that there were no errors, and the the server is using a random available TCP port.

Also note that, since ~/.ecflowrc/ssl did not already contain the SSL certificate files required for secure communication with the ecFlow server, these were generated.

In [6]:
cat server.log

[2026-07-09T04:37:43]     INFO Validating config against internal schema: ecflow
[2026-07-09T04:37:43]     INFO Schema validation succeeded for ecFlow config
[2026-07-09T04:37:43]     INFO Generating SSL private key: /home/Paul.Madden/.ecflowrc/ssl/server.key
[2026-07-09T04:37:43]     INFO Generating SSL certificate: /home/Paul.Madden/.ecflowrc/ssl/server.crt
[2026-07-09T04:37:43]     INFO Generating DH parameters: /home/Paul.Madden/.ecflowrc/ssl/dh2048.pem
[2026-07-09T04:37:52]     INFO SSL certificate files written to /home/Paul.Madden/.ecflowrc/ssl
[2026-07-09T04:37:52]     INFO Server started on port 2858

At server-startup time, uw ecflow server also creates a .h include file called server.h, defining environment variables that will be needed by ecf scripts:

In [7]:
cat server.h

export ECF_HOST=%ECF_HOST%
export ECF_NAME=%ECF_NAME%
export ECF_PASS=%ECF_PASS%
export ECF_PORT=%ECF_PORT%
export ECF_TRYNO=%ECF_TRYNO%
export PATH=/scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/conda/envs/demo/bin/:$PATH

The server.json file, containing current values for some useful server-related environment variables, was also created:

In [8]:
jq . server.json

{
  "ECF_HOME": "/scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip",
  "ECF_HOST": "uecflow01",
  "ECF_PORT": "2858",
  "ECF_SSL": "1"
}

Use jq to set environment variables based on these values. Extract values from this file to use to set some crucial environment variables that will allow the ecFlow client tool, ecflow_client, to successfully communicate with the server:

In [9]:
for key in $(jq -r keys[] server.json); do
  export $key="$(jq -r .$key server.json)"
  echo $key: ${!key}
done

ECF_HOME: /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip
ECF_HOST: uecflow01
ECF_PORT: 2858
ECF_SSL: 1

With these environment variables in place, use ecflow_client to ping the server and verify that it is listening:

In [10]:
ecflow_client --ping

ping server(uecflow01:2858) succeeded in 00:00:00.010941  ~10 milliseconds

By default, the server starts halted, so place it into running state with the restart command:

In [11]:
ecflow_client --stats | grep Status
ecflow_client --restart
ecflow_client --stats | grep Status

  Status                          HALTED
  Status                          RUNNING

Now the server is ready to be loaded with a suite definition, provided by the ecflow.suitedef block in config.yaml. The uw ecflow realize command creates a suite.def file in the directory specified by --output-dir and, under the directory specified by --scripts-dir, any ecf scripts defined by script blocks in the suite definition:

In [12]:
uw ecflow realize -c config.yaml --output-dir=$BASEDIR --scripts-dir=$BASEDIR

[2026-07-09T04:37:55]     INFO Validating config against internal schema: ecflow
[2026-07-09T04:37:55]     INFO Schema validation succeeded for ecFlow config

Next let's see the generated suite.def file, in ecFlow's custom configuration language. Note that the fetch_truth task can proceed immediately, but that extract_grids requires fetch_truth to be complete, as it uses the fetched truth data as an input; and that verification requires extract_grids to be complete, as it uses the extracted grids in verification:

In [13]:
cat suite.def

#5.16.0
suite vx
  edit ECF_JOB_CMD '/usr/bin/env -S bash %ECF_JOB% >%ECF_JOBOUT% 2>&1'
  task extract_grids
    trigger /vx/fetch_truth == complete
  task fetch_truth
  task verification
    trigger /vx/extract_grids == complete
    edit ECF_JOB_CMD 'ecflow_client --alter=add variable ECF_RID $(sbatch --parsable %ECF_JOB%) %ECF_NAME%'
    edit ECF_KILL_CMD 'scancel %ECF_RID%'
    edit ECF_STATUS_CMD 'sacct -lj %ECF_RID%'
endsuite
# enddef

The ecf scripts, which are templates used to create executable job scripts that run ecFlow tasks, are created in a directory hierarchy based on the suite name, family names (this demo does not use families), and task names:

In [14]:
tree vx

vx
├── extract_grids.ecf
├── fetch_truth.ecf
└── verification.ecf

1 directory, 3 files

The fetch_truth.ecf file defines a task that will download truth data to verify a forecast against. When rendered by the server to a job script, the contents of the common.h and server.h include files will be inlined where the %include statements appear. This is followed by the main script logic, taken from the task's body element in the config. Finally, it defines some help text between the %manual and %end tags that can be viewed, for example, in the ecFlow GUI application, ecflow_ui

In [15]:
cat vx/fetch_truth.ecf

#!/usr/bin/env bash

%include common.h

%include server.h

init $$
test -e gfs.t12z.pgrb2.1p00.f006 || wget --quiet https://nomads.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.20260704/12/atmos/gfs.t12z.pgrb2.1p00.f006

%manual
Fetch GRIB truth data
%end

We saw the contents of the server.h file generated by uw ecflow server earlier. Here's common.h, which belongs to this demo application and defines some functions and traps that allow task jobs to communicate their status to the server:

In [16]:
cat common.h

abort() {
  trap EXIT # unset previous trap
  ecflow_client --abort=$ECF_RID
  exit 1
}

trap abort ERR SIGTERM

complete() {
  ecflow_client --ssl --complete
}

trap complete EXIT

init() {
  export ECF_RID=$1
  ecflow_client --ssl --init=$ECF_RID
}

set -euxo pipefail

Here's extract_grids.ecf, the ecf script that runs wxvx to extract grids from the forecast and truth datasets to be used in verification:

In [17]:
cat vx/extract_grids.ecf

#!/usr/bin/env bash

%include common.h

%include server.h

init $$
uw config realize -i /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/config.yaml --key-path wxvx >/scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx.yaml
wxvx -c /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx.yaml -t grids

%manual
Extract forecast and truth grids
%end

Finally, we have verification.ecf, representing the final task in the workflow, which defines the task that performs verification with MET.

In [18]:
cat vx/verification.ecf

#!/usr/bin/env bash

#SBATCH --chdir=/scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx
#SBATCH --export=NONE
#SBATCH --ntasks=1
#SBATCH --output=verification.out
#SBATCH --time=00:05:00

%include common.h

%include server.h

init $SLURM_JOB_ID
wxvx -c /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx.yaml -t stats

%manual
Perform verification
%end

Load the suite into the server:

In [19]:
ecflow_client --load=suite.def

Verify that the suite was loaded:

In [20]:
ecflow_client --get

#5.16.0
suite vx
  edit ECF_JOB_CMD '/usr/bin/env -S bash %ECF_JOB% >%ECF_JOBOUT% 2>&1'
  task extract_grids
    trigger /vx/fetch_truth == complete
  task fetch_truth
  task verification
    trigger /vx/extract_grids == complete
    edit ECF_JOB_CMD 'ecflow_client --alter=add variable ECF_RID $(sbatch --parsable %ECF_JOB%) %ECF_NAME%'
    edit ECF_KILL_CMD 'scancel %ECF_RID%'
    edit ECF_STATUS_CMD 'sacct -lj %ECF_RID%'
endsuite
# enddef

Begin suite execution:

In [21]:
ecflow_client --begin=vx

Wait for the tasks to complete:

In [22]:
while true; do
  state=$(ecflow_client --query state /vx/fetch_truth)
  echo "  fetch_truth: $state"
  state=$(ecflow_client --query state /vx/extract_grids)
  echo "extract_grids: $state"
  state=$(ecflow_client --query state /vx/verification)
  echo "verification:  $state"
  test $state == complete && break
  sleep 10
done

  fetch_truth: active
extract_grids: queued
verification:  queued
  fetch_truth: complete
extract_grids: active
verification:  queued
  fetch_truth: complete
extract_grids: active
verification:  queued
  fetch_truth: complete
extract_grids: active
verification:  queued
  fetch_truth: complete
extract_grids: active
verification:  queued
  fetch_truth: complete
extract_grids: active
verification:  queued
  fetch_truth: complete
extract_grids: complete
verification:  active
  fetch_truth: complete
extract_grids: complete
verification:  complete

Now that the workflow tasks are complete, examine some of the files created both by the server and by the workflow tasks.

First, note that the server has created, in the ECF_HOME directory, a log file based on the hostname of the system it is running on. These can be useful for debugging, but its contents are beyond the scope of this tutorial. Depending on when you look, you may also see a .check file, containing a checkpoint of the workflow state, which can be used to resume a workflow if the server is restarted.

In [23]:
ls -l $ECF_HOST.$ECF_PORT.*

-rw-r--r-- 1 Paul.Madden gsd-hpcs 4637 Jul  9 04:39 uecflow01.2858.ecf.log

Here are the updated contents of the vx/ directory:

In [24]:
tree vx

vx
├── extract_grids.1
├── extract_grids.ecf
├── extract_grids.job1
├── fetch_truth.1
├── fetch_truth.ecf
├── fetch_truth.job1
├── forecast
│   └── 20260704
│       └── 12
│           └── 006
│               ├── 2t-heightAboveGround-0002.grib2
│               └── gfs.t12z.pgrb2.1p00.f006.ecidx
├── run
│   └── stats
│       └── 20260704
│           └── 12
│               └── 006
│                   ├── grid_stat_gfsfcst_2t_heightAboveGround_0002_060000L_20260704_180000V_cnt.txt
│                   ├── grid_stat_gfsfcst_2t_heightAboveGround_0002_060000L_20260704_180000V.config
│                   ├── grid_stat_gfsfcst_2t_heightAboveGround_0002_060000L_20260704_180000V.log
│                   ├── grid_stat_gfsfcst_2t_heightAboveGround_0002_060000L_20260704_180000V.sh
│                   └── grid_stat_gfsfcst_2t_heightAboveGround_0002_060000L_20260704_180000V.stat
├── truth
│   └── 20260704
│       └── 18
│           └── 000
│               ├── 2t-heightAboveGround-0002.grib2
│               └── gfs.t12z.pgrb2.1p00.f000.idx
├── verification.ecf
├── verification.job1
└── verification.out

14 directories, 18 files

We previously saw the extract_grids.ecf, fetch_truth.ecf, and verification.ecf template files. The server rendered these to ready-to-run scripts extract_grids.job1, fetch_truth.job1, and verification.job1, respectively. (The 1 in the .job1 filename extensions indicates that these files belong to the first of potentially multiple attempts to run this task. If it had been necessary to retry any of these jobs, corresponding .job2 files would have been created.)

Let's examine the files created for the extract_grids task as an example.

The file extract_grids.job1 is the concatenation of the header file common.h, the header file server.h, and the value of ecflow.suitedef.suite_vs.task_extract_grids.script.body from config.yaml. Here it is:

In [25]:
cat vx/extract_grids.job1

#!/usr/bin/env bash

abort() {
  trap EXIT # unset previous trap
  ecflow_client --abort=$ECF_RID
  exit 1
}

trap abort ERR SIGTERM

complete() {
  ecflow_client --ssl --complete
}

trap complete EXIT

init() {
  export ECF_RID=$1
  ecflow_client --ssl --init=$ECF_RID
}

set -euxo pipefail

export ECF_HOST=uecflow01
export ECF_NAME=/vx/extract_grids
export ECF_PASS=LqnraUf4
export ECF_PORT=2858
export ECF_TRYNO=1
export PATH=/scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/conda/envs/demo/bin/:$PATH

init $$
uw config realize -i /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/config.yaml --key-path wxvx >/scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx.yaml
wxvx -c /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx.yaml -t grids

The file extract_grids.1 contains a log of the output of this job. Since our common.h performs a set -x, we see a trace of the execution of shell commands, as well as the log output of the wxvx utility as it runs.

In [26]:
cat vx/extract_grids.1

+ export ECF_HOST=uecflow01
+ ECF_HOST=uecflow01
+ export ECF_NAME=/vx/extract_grids
+ ECF_NAME=/vx/extract_grids
+ export ECF_PASS=LqnraUf4
+ ECF_PASS=LqnraUf4
+ export ECF_PORT=2858
+ ECF_PORT=2858
+ export ECF_TRYNO=1
+ ECF_TRYNO=1
+ export PATH=/scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/conda/envs/demo/bin/:/scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/conda/envs/demo/bin:/home/Paul.Madden/.local/bin:/usr/local/slurm/default/bin:/usr/local/slurm/default/sbin:/home/Paul.Madden/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/apps/hpss/bin:/apps/bin:/apps/local/bin
+ PATH=/scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/conda/envs/demo/bin/:/scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/conda/envs/demo/bin:/home/Paul.Madden/.local/bin:/usr/local/slurm/default/bin:/usr/local/slurm/default/sbin:/home/Paul.Madden/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/apps/hpss/bin:/apps/bin:/apps/local/bin
+ init 286258
+ export ECF_RID=286258
+ ECF_RID=286258
+ ecflow_client --ssl --init=286258
+ uw config realize -i /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/config.yaml --key-path wxvx
+ wxvx -c /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx.yaml -t grids
[2026-07-09T04:38:47]     INFO Schema validation succeeded for config
[2026-07-09T04:38:47]     INFO Preparing task graph for 'grids'
[2026-07-09T04:38:47]     INFO GRIB index file /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/forecast/20260704/12/006/gfs.t12z.pgrb2.1p00.f006.ecidx at 20260704 12Z 006: Executing
[2026-07-09T04:38:50]     INFO GRIB index file /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/forecast/20260704/12/006/gfs.t12z.pgrb2.1p00.f006.ecidx at 20260704 12Z 006: Wrote /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/forecast/20260704/12/006/gfs.t12z.pgrb2.1p00.f006.ecidx
[2026-07-09T04:38:50]     INFO GRIB index file /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/forecast/20260704/12/006/gfs.t12z.pgrb2.1p00.f006.ecidx at 20260704 12Z 006: Ready
[2026-07-09T04:38:50]     INFO GRIB index file /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/truth/20260704/18/000/gfs.t12z.pgrb2.1p00.f000.idx: Executing
[2026-07-09T04:38:50]     INFO GRIB index file /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/truth/20260704/18/000/gfs.t12z.pgrb2.1p00.f000.idx: Fetching https://nomads.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.20260704/12/atmos/gfs.t12z.pgrb2.1p00.f000.idx
[2026-07-09T04:38:50]     INFO GRIB index file /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/truth/20260704/18/000/gfs.t12z.pgrb2.1p00.f000.idx: Wrote /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/truth/20260704/18/000/gfs.t12z.pgrb2.1p00.f000.idx
[2026-07-09T04:38:50]     INFO GRIB index file /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/truth/20260704/18/000/gfs.t12z.pgrb2.1p00.f000.idx: Ready
[2026-07-09T04:38:50]     INFO Forecast grid /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/forecast/20260704/12/006/2t-heightAboveGround-0002.grib2: Executing
[2026-07-09T04:38:50]     INFO Forecast grid /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/forecast/20260704/12/006/2t-heightAboveGround-0002.grib2: Ready
[2026-07-09T04:38:50]     INFO GRIB index data at 20260704 18Z 000: Executing
[2026-07-09T04:38:50]     INFO GRIB index data at 20260704 18Z 000: Ready
[2026-07-09T04:38:50]     INFO Forecast grids for GFSFCST: Ready
[2026-07-09T04:38:50]     INFO Truth grid /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/truth/20260704/18/000/2t-heightAboveGround-0002.grib2: Executing
[2026-07-09T04:38:50]     INFO Truth grid /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/truth/20260704/18/000/2t-heightAboveGround-0002.grib2: Fetching https://nomads.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.20260704/12/atmos/gfs.t12z.pgrb2.1p00.f000 bytes=33774443-33822702
[2026-07-09T04:38:50]     INFO Truth grid /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/truth/20260704/18/000/2t-heightAboveGround-0002.grib2: Wrote /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/truth/20260704/18/000/2t-heightAboveGround-0002.grib2
[2026-07-09T04:38:50]     INFO Truth grid /scratch4/BMC/gsd-hpcs/Paul.Madden/uwtools/wip/vx/truth/20260704/18/000/2t-heightAboveGround-0002.grib2: Ready
[2026-07-09T04:38:50]     INFO Truth grids for GFS: Ready
[2026-07-09T04:38:50]     INFO Grids: Ready
+ complete
+ ecflow_client --ssl --complete

Finally, we can consider the files created by wxvx.

The forecast/ directory contains the 2-meter temperature grid extracted from the full forecast dataset, along with an index file used to efficiently find specific GRIB messages in the full dataset, for the purposes of verification:

In [27]:
tree vx/forecast/

vx/forecast/
└── 20260704
    └── 12
        └── 006
            ├── 2t-heightAboveGround-0002.grib2
            └── gfs.t12z.pgrb2.1p00.f006.ecidx

4 directories, 2 files

The truth/ directory contains a "truth" grid, extracted from a remote GRIB dataset, to verify the forecast grid against:

In [28]:
tree vx/truth/

vx/truth/
└── 20260704
    └── 18
        └── 000
            ├── 2t-heightAboveGround-0002.grib2
            └── gfs.t12z.pgrb2.1p00.f000.idx

4 directories, 2 files

The run/ directory contains a MET configuration file, a script to invoke MET, .stat and .txt files containing the verification results, and the MET log.

In [29]:
tree vx/run/

vx/run/
└── stats
    └── 20260704
        └── 12
            └── 006
                ├── grid_stat_gfsfcst_2t_heightAboveGround_0002_060000L_20260704_180000V_cnt.txt
                ├── grid_stat_gfsfcst_2t_heightAboveGround_0002_060000L_20260704_180000V.config
                ├── grid_stat_gfsfcst_2t_heightAboveGround_0002_060000L_20260704_180000V.log
                ├── grid_stat_gfsfcst_2t_heightAboveGround_0002_060000L_20260704_180000V.sh
                └── grid_stat_gfsfcst_2t_heightAboveGround_0002_060000L_20260704_180000V.stat

5 directories, 5 files

With the workflow now complete, we can shut down the server by simulating a CTRL-C that a command-line user would press:

In [30]:
kill -INT %1
wait

[1]+  Done                    uw ecflow server -c config.yaml --report > server.json 2> server.log