Forward model#

In the context of uncertainty estimation and data assimilation, a forward model refers to a predictive model that simulates how a system evolves over time given certain inputs or initial conditions. The model is called “forward” because it predicts the future state of the system based on the current state and a set of input parameters. The predictive model may include pre-processing and post-processing steps in addition to the physics simulator itself. In ERT, we think of a forward model as a sequence of jobs such as making directories, copying files, executing simulators etc.

Consider a scenario in reservoir management. Here, a forward model might encompass reservoir modeling software like RMS, a fluid simulator like Eclipse or Flow, and custom jobs like relative permeability interpolation and water saturation calculation.

To add a job to the forward model, use the FORWARD_MODEL keyword. Each FORWARD_MODEL keyword instructs ERT to run a specific executable. You can build a series of jobs by listing multiple FORWARD_MODEL keywords.

An alternative to FORWARD_MODEL is the SIMULATION_JOB keyword, which can also configure the forward model. The difference lies in how these keywords pass command-line arguments to the final executable.

You can find all pre-configured jobs to define your forward models here. These jobs form the building blocks for your custom forward models in ERT.

Configuring your own jobs#

ERT imposes no restrictions on the programming language used to write a job. The only requirement is that it should be an executable that can be run. Consequently, it is possible to create a program or script performing any desired function, and then have ERT run it as one of the jobs in the FORWARD_MODEL.

However, for ERT to recognize a job, it must be installed. All predefined jobs are already installed and may be invoked by using the FORWARD_MODEL keyword in the configuration file. If you need to include a custom job, it must first be installed using INSTALL_JOB, as follows:

INSTALL_JOB JOB_NAME JOB_CONFIG

In this command, JOB_NAME is a name of your choice that you can later use in the ERT configuration file to call upon the job. JOB_CONFIG is a file that specifies the location of the executable and provides rules for the behavior of any arguments.

By installing your own jobs in this way, you can extend the capabilities of ERT to meet your specific needs and scenarios.

EXECUTABLE  path/to/program

STDERR      prog.stderr      -- Name of stderr file (defaults to
                             -- name_of_file.stderr.<job_nr>)
STDOUT      prog.stdout      -- Name of stdout file (defaults to
                             -- name_of_file.stdout.<job_nr>)
ARGLIST     <ARG0> <ARG1>    -- A list of arguments to pass on to the
                             --  executable
REQUIRED    <ARG0> <ARG1>    -- A list of arguments required to be passed
                             -- on to the executable

Note#

When configuring ARGLIST for FORWARD_MODEL jobs it is not suitable to use --some-option for named options as it treated as a comment by the configuration compiler. Single letter options, i.e. -s are needed.

Invoking the job is then done by including it in the ert config:

FORWARD_MODEL JOB_NAME(<ARG0>=3, <ARG1>="something")

Note that the following behaviour provides identical results:

DEFINE <ARG0> 3
FORWARD_MODEL JOB_NAME(<ARG1>="something")

see example Creating a Script to Define the Forward Model

Pre-configured forward models#

Modelling#
Reservoir#
RMS#

Forward model for running a given workflow in an existing RMS-project.

The forward model requires explicit knowledge of the version used to produce the RMS project loaded. The Python environment is adapted to work with Python inside RMS.

Category
modelling.reservoir
Source package
ert
Examples#
Running the forward model#

RMS is usually incorporated in ERT configurations using statements like

DEFINE  <RMS_PROJECT>         reek.rms11.0.1
DEFINE  <RMS_VERSION>         11.0.1
DEFINE  <RMS_WORKFLOW_NAME>   MAIN_WORKFLOW
FORWARD_MODEL RMS(<IENS>=<IENS>, <RMS_VERSION>=<RMS_VERSION>, <RMS_PROJECT>=<CONFIG_PATH>/../../rms/model/<RMS_NAME>)
RMS script documentation#

The script must be invoked with minimum three positional arguments:

Positional arguments:
IENS

Realization number

RMS_PROJECT

The RMS project we are running

RMS_WORKFLOW

The rms workflow we intend to run

Optional arguments:
-r, --run-path RUN_PATH

The directory which will be used as cwd when running rms

-t, --target-file TARGET_FILE

name of file which should be created/updated by rms

-i, --import-path IMPORT_PATH

the prefix of all relative paths when rms is importing

-e, --export-path EXPORT_PATH

the prefix of all relative paths when rms is exporting

-v, --version VERSION

The version of rms to use

-a, --allow-no-env

Boolean flag to allow RMS to run without a site configured environment. WARNING: This is typically a sign of using an unsupported version of RMS and the configuration of the environment is now entirely on the shoulders of the user. Consider contacting your system administrators.

Job configuration used by ERT#
EXECUTABLE ../res/script/rms.py

DEFAULT       <RMS_IMPORT_PATH> ./
DEFAULT       <RMS_EXPORT_PATH> ./
DEFAULT       <RMS_RUNPATH>     rms/model
DEFAULT       <RMS_OPTS> ""
TARGET_FILE   <RMS_TARGET_FILE>
ARGLIST <IENS>  <RMS_PROJECT> <RMS_WORKFLOW> -r <RMS_RUNPATH> -t <RMS_TARGET_FILE> -i <RMS_IMPORT_PATH> -v <RMS_VERSION> -e <RMS_EXPORT_PATH> <RMS_OPTS>

EXEC_ENV PYTHONPATH <RMS_PYTHONPATH>
EXEC_ENV PATH_PREFIX <RMS_PATH_PREFIX>
Simulators#
Reservoir#
ECLIPSE100#
Category
simulators.reservoir
Source package
ert
Examples#

The version, number of cpu, and whether or not to ignore errors and whether or not to produce YOUR_CASE_NAME.h5 output files can be configured in the configuration file when adding the job, as such:

FORWARD_MODEL ECLIPSE100(<ECLBASE>, <VERSION>=xxxx, <OPTS>={"--ignore-errors", "--summary-conversion"})

The OPTS argument is optional and can be removed, fully or partially. In absence of --ignore-errors eclipse will fail on errors. Adding the flag --ignore-errors will result in eclipse ignoring errors.

And in absence of --summary-conversions eclipse will run without producing YOUR_CASE_NAME.h5 output files. Add flag --summary-conversions to produce YOUR_CASE_NAME.h5 output files.

Job configuration used by ERT#
EXECUTABLE ../res/script/ecl100.py
REQUIRED <VERSION>
DEFAULT <NUM_CPU> 1
DEFAULT <OPTS> ""
ARGLIST <ECLBASE> -v <VERSION> -n <NUM_CPU> <OPTS>
FLOW#
Category
simulators.reservoir
Source package
ert
Examples#

The version, number of cpu and whether or not to ignore errors can be configured in the configuration file when adding the job, as such:

FORWARD_MODEL FLOW(<ECLBASE>, <VERSION>=xxxx, <OPTS>="--ignore-errors")

The OPTS argument is optional and can be removed, thus running flow without ignoring errors.

Job configuration used by ERT#
EXECUTABLE ../res/script/flow.py
DEFAULT <VERSION> default
DEFAULT <NUM_CPU> 1
DEFAULT <OPTS> ""
ARGLIST <ECLBASE> -v <VERSION> -n <NUM_CPU> <OPTS>
ECLIPSE300#
Category
simulators.reservoir
Source package
ert
Examples#

The version, number of cpu and whether or not to ignore errors can be configured in the configuration file when adding the job, as such:

FORWARD_MODEL ECLIPSE300(<ECLBASE>, <VERSION>=xxxx, <OPTS>="--ignore-errors")

The OPTS argument is optional and can be removed, thus running eclipse without ignoring errors

Job configuration used by ERT#
EXECUTABLE ../res/script/ecl300.py
DEFAULT <VERSION> version
DEFAULT <NUM_CPU> 1
DEFAULT <OPTS> ""
ARGLIST <ECLBASE> -v <VERSION> -n <NUM_CPU> <OPTS>
Utility#
File_system#
COPY_FILE#

The COPY_FILE job will copy a file. If the <TO> argument has a directory component, that directory will be created. This is an extension of the normal cp command which will not create directories in this way.

Category
utility.file_system
Source package
ert
Examples#
FORWARD_MODEL COPY_FILE(<FROM>=file1, <TO>=path/to/directory/file1)
Job configuration used by ERT#
EXECUTABLE  ../../shell_scripts/copy_file.py
ARGLIST     <FROM> <TO>

MIN_ARG    2
MAX_ARG    2
ARG_TYPE 0 STRING
ARG_TYPE 1 STRING
DELETE_FILE#

The DELETE_FILE job will only remove files which are owned by the current user, even if file system permissions would have allowed the delete operation to proceed. The DELETE_FILE will not delete a directory, and if presented with a symbolic link it will only delete the link, and not the target.

Category
utility.file_system
Source package
ert
Job configuration used by ERT#
EXECUTABLE  ../../shell_scripts/delete_file.py
ARGLIST     <FILES>

MIN_ARG    1
MAX_ARG    1
ARG_TYPE 0 STRING
DELETE_DIRECTORY#

The DELETE_DIRECTORY job will recursively remove a directory and all the files in the directory. Like the DELETE_FILE job it will only delete files and directories which are owned by the current user. If one delete operation fails the job will continue, but unless all delete calls succeed (parts of) the directory structure will remain.

Category
utility.file_system
Source package
ert
Job configuration used by ERT#
EXECUTABLE  ../../shell_scripts/delete_directory.py
ARGLIST     <DIRECTORY>

MIN_ARG    1
MAX_ARG    1
ARG_TYPE 0 STRING
MOVE_FILE#

The MOVE_FILE job will move file to target directory. If file already exists, this job will move file to the target directory and then replace the existing file.

Category
utility.file_system
Source package
ert
Job configuration used by ERT#
EXECUTABLE    ../../shell_scripts/move_file.py
ARGLIST       <FROM>  <TO>

MIN_ARG    2
MAX_ARG    2
ARG_TYPE   0 STRING
ARG_TYPE   1 STRING
MAKE_DIRECTORY#

Will create the directory <DIRECTORY>, with all sub directories.

Category
utility.file_system
Source package
ert
Job configuration used by ERT#
ARGLIST   <DIRECTORY>
EXECUTABLE    ../../shell_scripts/make_directory.py

MIN_ARG    1
MAX_ARG    1
ARG_TYPE 0 STRING
CAREFUL_COPY_FILE#

The CAREFUL_COPY_FILE job will copy a file. If the <TO> argument has a directory component, that directory will be created. This is an extension of the normal cp command which will not create directories in this way. This job superseded an older version called CAREFULL_COPY and should be used instead.

Category
utility.file_system
Source package
ert
Examples#
FORWARD_MODEL CAREFUL_COPY_FILE(<FROM>=file1, <TO>=path/to/directory/file1)
Job configuration used by ERT#
EXECUTABLE  ../../shell_scripts/careful_copy_file.py
ARGLIST     <FROM> <TO>

MIN_ARG    2
MAX_ARG    2
ARG_TYPE 0 STRING
ARG_TYPE 1 STRING
COPY_DIRECTORY#

The job copies the directory <FROM> to the target <TO>. If <TO> points to a non-existing directory structure, it will be created first. If the <TO> folder already exist it creates a new directory within the existing one. E.g. COPY_DIRECTORY (<FROM>=foo, <TO>=bar) creates bar/foo if the directory bar already exists. If bar does not exist it becomes a copy of foo.

Category
utility.file_system
Source package
ert
Job configuration used by ERT#
EXECUTABLE ../../shell_scripts/copy_directory.py
ARGLIST    <FROM> <TO>

MIN_ARG    2
MAX_ARG    2
ARG_TYPE 0 STRING
ARG_TYPE 1 STRING
Templating#
TEMPLATE_RENDER#

Loads the data from each file (some/path/filename.xxx) in INPUT_FILES and exposes it as the variable filename. It then loads the Jinja2 template TEMPLATE_FILE and dumps the rendered result to OUTPUT.

Category
utility.templating
Source package
ert
Examples#

Given an input file my_input.json:

{
    "my_variable": "my_value"
}

And a template file tmpl.jinja:

This is written in my file together with {{my_input.my_variable}}

This job will produce an output file:

This is written in my file together with my_value

By invoking the FORWARD_MODEL as such:

FORWARD_MODEL TEMPLATE_RENDER(<INPUT_FILES>=my_input.json, <TEMPLATE_FILE>=tmpl.jinja, <OUTPUT_FILE>=output_file)
Job configuration used by ERT#
EXECUTABLE  ../templating/script/template_render.py
ARGLIST -i <INPUT_FILES> -o <OUTPUT_FILE> -t <TEMPLATE_FILE>