Skip to content

Tasks

Tasks in PIPES represent units of work performed on datasets that occur often in integrated modeling projects, especially when exchanging data. Tasks are typically planned at Model Run initialization, but can also be added after Model Run initialization if needed. When a task is completed, users can indicate this to PIPES via a Task Creation. Currently, tasks are not automated processes within PIPES, though we are exploring integrating with workflow management tools to allow for task automation.

In PIPES, there are currently three types of tasks:

  1. Transformation
  2. QAQC
  3. Visualization

Note

As defined in the readmap, we plan to add support for a fourth type of task: Knowledge Exchanges

We describe each type of the task in detail below.

Transformation

Transformation tasks describe how a dataset is altered to meet metadata requirements. PIPES expects users to indicate which piece of metadata is being changed and what the change is (i.e., the to and from values). For example, if a dataset has temporal_info.resolution = hourly but a down stream model expects temporal_info.resolution = 1 minute then a transformation that achieves this change in resolution must be performed. PIPES can automatically determine transformations that need to occur based on differences in requirements across models, and this is discussed further in Validation. In general, PIPES assumes that transformations will need to be performed on handoff datasets, but it is not required.

QAQC

QAQC tasks capture dataset validation and acceptance criteria which are crucial for ensuring successful integration of two models. For example, QAQC tasks could ensure that data is within certain bounds or adheres to various metrics. The exact nature of the criteria is totally customizable for users, see Task Planning for more details.

Visualization

Visualization tasks capture which datasets should be visualized and in what manner. When creating a visualization task, it may be useful to indicate the outputs of the task (such as images or videos) in the outputs field in the Task Creation.