Unified Workflow Tools
uwtools is a modern, open-source Python package that helps automate common tasks needed for many standard numerical weather prediction (NWP) workflows. It also provides drivers to automate the configuration and execution of Unified Forecast System (UFS) components, providing flexibility, interoperability, and usability to various UFS Applications.
If you’re interested in contributing, check out the Contributor Guide.
For users who want to start using the toolbox and framework, take a peek at our User Guide.
On This Page:
The Tools
The tools are accessible from both a command-line interface (CLI) and a Python API. The CLI automates many operations commonly needed in NWP workflows. The API supports all CLI operations and additionally provides access to in-memory objects to facilitate more novel use cases. We hope these options will let you integrate the package into your pre-existing bash and Python scripts, and give you some handy tools to use in your day-to-day work with running NWP systems.
Configuration Management
uw config -himport uwtools.api.configThe config tool suite helps you compare, transform, modify, and even validate your configuration. The package supports YAML, shell, Fortran namelist, and INI file formats. Configuration in any of these formats may use Jinja2 syntax to express values. These values can reference others, or compute new values by evaluating mathematical expressions, building paths, manipulating strings, etc.
Compare Mode
When the Linux diff tool just doesn’t work for comparing unordered namelists with mixed-case keys, this is your go-to! The Fortran namelists are the real catalyst behind this gem, but it also works on the other configuration formats.
Realize Mode
This mode renders values created by Jinja2 templates, and lets you override values in one file or object with those from others, not necessarily with the same configuration format. With uwtools, you can even reference the content of other files to build up a configuration from its pieces.
Validate Mode
In this mode, you can provide a JSON Schema file alongside your configuration to validate that it meets the requirements set by the schema. We’ve enabled robust logging to make it easier to repair your configs when problems arise.
Templating
uw template -himport uwtools.api.templateRender Mode
The render mode that gives you the full power of rendering a Jinja2 template in the same easy-to-use interface as your other workflow tools.
Translate Mode
This tool helps transform legacy configuration files templated with the atparse tool (common at NOAA) into Jinja2 templates for use with the uw config realize and uw template render tools, or their API equivalents.
Rocoto Configurability
uw rocoto -himport uwtools.api.rocotoThis tool is all about creating a configurable interface to the Rocoto workflow manager tool that produces the Rocoto XML for a totally arbitrary set of tasks. The uwtools package defines a structured YAML interface that relies on tasks you define to run. Paired with the uw config tool suite, this interface becomes highly configurable and requires no XML syntax!
Realize Mode
This is where you put in your structured YAML that defines your workflow of choice, and it pops out a verified Rocoto XML.
Validate Mode
Do you already have a Rocoto XML but don’t want to run Rocoto to make sure it works? Use the validate mode to check to see if Rocoto will be happy.
The Drivers
Drivers for NWP components are available as top-level CLI modes and API modules.
Provided with a valid UW YAML configuration file, and CLI arguments when required, uw can prepare a fully provisioned run directory and execute a component either directly on the current system, or via a batch job submitted to an HPC scheduler.
Each driver produces a list of available TASK arguments from its CLI --help, -h flag. The provisioned_run_directory will do everything except run the executable, but any of the tasks may be requested and only the steps required to produce that task will be performed.
Over time, we’ll add many other drivers to support a variety of UFS components from pre-processing to post-processing, along with many data assimilation components.
Drivers for UFS
To prepare a complete forecast, drivers would typically be run in the order shown here (along with additional drivers still in development).
esg_grid
uw esg_grid -himport uwtools.api.drivers.esg_gridglobal_equiv_resol
uw global_equiv_resol -himport uwtools.api.drivers.global_equiv_resolmake_hgrid
uw make_hgrid -himport uwtools.api.drivers.make_hgridsfc_climo_gen
uw sfc_climo_gen -himport uwtools.api.drivers.sfc_climo_genshave
uw shave -himport uwtools.api.shavechgres_cube
uw chgres_cube -himport uwtools.api.drivers.chgres_cubeFV3
uw fv3 -himport uwtools.api.drivers.fv3JEDI
uw jedi -himport uwtools.api.drivers.jediDrivers for MPAS
Drivers for working with standalone Model for Prediction Across Scales (MPAS), typically run in the order shown here:
ungrib
uw ungrib -himport uwtools.api.drivers.ungribmpas_init
uw mpas_init -himport uwtools.api.drivers.mpas_initmpas
uw mpas -himport uwtools.api.drivers.mpasDisclaimer
The United States Department of Commerce (DOC) GitHub project code is provided on an “as is” basis and the user assumes responsibility for its use. DOC has relinquished control of the information and no longer has responsibility to protect the integrity, confidentiality, or availability of the information. Any claims against the Department of Commerce stemming from the use of its GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.