Workflows

Workflows are a unique feature of Parallel Works, making it faster and easier to complete work on our platform.

What Is a Workflow?

“Workflow” may mean different things to different people. At PW, a workflow is a series of automated steps run on one or more compute resources.

Some workflows launch only one instance of one application. For example, the Jupyter workflow on the PW Marketplace sets up an interactive session for you to run a Jupyter notebook. The workflow takes care of several setup and connectivity steps in the background, with the ultimate goal of running a single application, JupyterHub.

Parameter sweeps are an example of a more complicated workflow type that launches multiple, independent, parallel-running instances of the same application. Parameter sweeps are great for automating sensitivity studies with the same application and visualizing the results. The MDLite workflow is an example of a parameter sweep with a simple molecular dynamics application whose results are visualized in DesignExplorer. Because the applications are fully independent, this type of workflow is sometimes classified as embarrassingly parallel.

Workflow complexity grows substantially when different applications are chained together. For example, a workflow can automate the process of running pre-processing, main simulation, and post-processing applications.

How Do Workflows Run?

Regardless of workflow size, type, or topology, all workflows on the PW platform need a minimum of a workflow form as a workflow.xml file and a Bash launch script as a workflow.sh file. The workflow.sh file serves as a starting point for the execution defined in the workflow form. The launch script can be the whole workflow, or it can be a wrapper that launches other workflow fabrics in other languages (such as Parsl in Python).

In the example workflow.xml file below, the second line launches workflow.sh.

<tool id='placeholder_id' name='placeholder_name'>
  <command interpreter='bash'>workflow.sh</command>
  <inputs>
    <section name='commands' type='section' conditional='[object Object],[object Object]' title='Commands' expanded='true'>
      <param name='wuser'
	     label='User name'
	     type='text'
	     value='__USER__'
	     help='Username for account on the Workflow Host. Your PW account username will automatically substitute __USER__; other values will be treated as typed.'
	     width='50%_none'>
      </param>
[...continues...]