Configuring workflow jobs¶

Workflow jobs are configured with a small configuration file much like the configuration file used to install forward model jobs. The keywords used in the configuration file are in two classes - those related to how the job should be located/run and the arguments which should be passed from the workflow to the job.

Configure an internal job¶

If you wish to implement your job as a Python class, derived from ErtScript you should use the SCRIPT keyword to point to an existing Python script:

INTERNAL  TRUE                     -- The job will call an internal function or script of the currently running ERT instance.
SCRIPT scripts/my_script.py        -- An existing Python script

Observe that the important thing here is the fact that we are writing an internal Python script; if you are writing an external script to loop through all your realization folders that will typically be an external script, and in that case the implementation language - i.e. Python, Perl, C++, F77 … has no relevance.

NB: note that relative paths are resolved from the location of the job configuration file, not the configuration file provided to ert.

Configure an external job¶

An external job is a workflow job which is implemented in an external executable, i.e. typically a script written in for instance Python. When configuring an external job the most important keyword is EXECUTABLE which is used to give the path to the external executable:

INTERNAL   FALSE                    -- Optional - Set to FALSE by default.
EXECUTABLE path/to/program          -- Path to a program/script which will be invoked by the job.

NB: note that relative paths are resolved from the location of the job configuration file, not the configuration file provided to ert.

Stop Ert execution upon job failure¶

By default, failing jobs (both internal and external) will not stop the entire ert simulation. In some cases it is best to cancel the entire simulation if a job fails. This behavior can be achieved by adding the below line to the job file:

STOP_ON_FAIL TRUE

For example, if a job is defined as follows:

EXECUTABLE script.sh
STOP_ON_FAIL TRUE                   -- Tell the job to stop ert on failure

Shell scripts (Bash) must, in addition to having STOP_ON_FAIL TRUE in ert config, set the “exit immediately” flag (set -e) for the error to propagate. Otherwise, even if errors occur within the script, it will still be “successful”.

::: set -e

For example, this bash external job script will stop on failure

::: #!/usr/bin/env bash set -e ekho ‘Hello World’ # misspelled command fails

For example, this internal job script will stop on failure

from ert import ErtScript
class AScript(ErtScript):
    stop_on_fail = True

    def run(self):
        assert False, "failure"
"""

Configuring the arguments¶

In addition to the INTERNAL and EXECUTABLE keys which are used to configure what the job should do, there are some keys which can be used to configure the number of arguments and their type. These arguments apply to both internal and external jobs:

MIN_ARG    2                 -- The job should have at least 2 arguments.
MAX_ARG    3                 -- The job should have maximum 3 arguments.
ARG_TYPE   0    INT          -- The first argument should be an integer.
ARG_TYPE   1    FLOAT        -- The second argument should be a float value.
ARG_TYPE   2    STRING       -- The third argument should be a string - the default.
ARGLIST    <ARG0> <ARG1>     -- A list of arguments to pass on to the executable.

The MIN_ARG, MAX_ARG and ARG_TYPE arguments are used to validate workflows.

Note¶

When configuring ARGLIST for workflow jobs, named options such as --some-option cannot be used since they are treated as comments by the configuration compiler. Single letter options, i.e. -s, are needed.

Example : Run external script

-- FILE: ECL_HIST --
EXECUTABLE  Script/ecl_hist.py
MIN_ARG     3

This job will invoke the external script Script/ecl_host.py which is expected to have at least three command line arguments. The path to the script, Script/ecl_hist.py is interpreted relative to the location of the configuration file.

Loading workflow jobs into ERT¶

Before the jobs can be used in workflows they must be “loaded” into ERT. This can be done either by specifying jobs by name, or by specifying a directory containing jobs.

Use the keyword LOAD_WORKFLOW_JOB to specify jobs by name:

LOAD_WORKFLOW_JOB     jobConfigFile     JobName

The LOAD_WORKFLOW_JOB keyword will load one workflow job. The name of the job is optional, and will be fetched from the configuration file if not provided. Alternatively, you can use the command WORKFLOW_JOB_DIRECTORY which will load all the jobs in a directory.

Use the keyword WORKFLOW_JOB_DIRECTORY to specify a directory containing jobs:

WORKFLOW_JOB_DIRECTORY /path/to/jobs

The WORKFLOW_JOB_DIRECTORY loads all workflow jobs found in the /path/to/jobs directory. Observe that all the files in the /path/to/jobs directory should be job configuration files. The jobs loaded in this way will all get the name of the file as the name of the job. The WORKFLOW_JOB_DIRECTORY keyword will not load configuration files recursively.