Control File

Data Management Interface (DMI) : Control File-executable DMI : Control File

Control File

The control file is an ASCII file which the user provides. The purpose of the control file is as follows:

• Map between object.slots in the Workspace and data files. In the case of an import DMI, data is imported from the data files to the object.slots; in the case of an export DMI, data is exported from the object.slots to the data files.

• Provide information about the data files.

Each control file entry contains three pieces of information:

• An object.slot specification.

• One or more file names listed as file = name

• Optional keyword = value pairs.

The control file format is:

object.slot: file=name keyword=value keyword=value ...

If a file name contains spaces then it must be enclosed in double quotations marks (“), for example:

object.slot: file=”file name” keyword=value keyword=value ...

The DMI must also be configured to allow spaces in file paths; see Allow Spaces in File Paths for details.

Object.Slot Specifications

The object.slot specifications indicate which slots will import from or export to data files. There are four ways of specifying the object:

• <object name> - A specific object name, e.g., “FtLoudoun,” which matches the object with that name. Aggregate object elements are specified as <aggregate name>:<element name>.

• <object type> - An object type, e.g., “LevelPowerReservoir,” which matches all objects of that type.

• <sub-basin name> - A sub-basin name, which matches all objects in the sub-basin.

• * - A wild card which matches all objects of all types.

There are three ways of specifying the slot:

• <slot name> - A specific slot name, e.g., “Inflow,” which matches the slot with that name.

• <slot type> - A slot type, e.g., “SeriesSlot,” which matches all slots of that type.

• dispatch - Any dispatching slots.

There are two ways of specifying a column within multi-column slots:

• <column name> - A specific column name, which matches one of the columns in the selected slot.

• no name - If no name is given, the DMI defaults to the first column.

An account or supply can be specified like so:

• <object name>^<account name>

• <supply name>.Supply - the “.Supply” portion of the string is necessary to tell RiverWare that this is a supply in the accounting system and not a slot in the physical system. If a DMI is used to bring in values, it should be setting supplies in the accounting system rather than accounting slots. See Setting Slots Versus Setting Supplies in Accounting for details.

Also, the slots in a Slot Set can be specified using the SlotSet keyword, as follows. See Slot Sets in User Interface) for details.

SlotSet.<slot set name>

For example, SlotSet.Reservoir Inflows would expand to all of the slots in the Reservoir Inflows set.

The above slot specifications lead to many ways of specifying the object.slot pairs, as follows:

<sub-basin name>.<slot name>.<column name>

<sub-basin name>.<slot name>

<sub-basin name>.<slot type>.<column name>

<sub-basin name>.<slot type>

<sub-basin name>.dispatch.<column name>

<sub-basin name>.dispatch

*.<slot name>.<column name>

*.<slot name>

*.<slot type>.<column name>

*.<slot type>

*.dispatch.<column name>

*.dispatch

<supply name>.Supply

SlotSet.<Slot Set Name>

These pairs are listed from most exact to least exact. It is possible that more than one control file entry will select a specific object.slot pair in the Workspace. In this case, the more exact entry will always take precedence. For example, in the control file:

# All Confluence objects, Inflow1 slot.

Confluence.Inflow1: file=~/%o.inflow1.dat

# Confluence01 object, Inflow1 slot.

Confluence01.Inflow1: file=~/confluence01.inflow1.dat

Both entries will select Confluence01's Inflow1 slot (the first through an object type, the second through an object name). The more exact entry, the second entry, will take precedence.

There are two other concepts this example introduces: the control file may contain comments (lines beginning with #) and the file name may contain special characters (~ and %), described below.

Finally, note that object and slot names and types must be specified exactly as they appear in the user interface, including case and whitespace.

File Name and Directory Specification

The file name indicates from which data files the object.slots will be imported or to which data files the objects.slots will be exported. The file name may be either relative (beginning with ~/) or absolute (beginning with /). If the file name is relative, then it is relative to the DMI's working directory. The DMI's working directory is where it creates its temporary files. If the RIVERWARE_DMI_DIR environment variable is set, the DMI's working directory is $RIVERWARE_DMI_DIR/dmi_name; otherwise, the DMI's working directory is /tmp/dmi_name. In both cases, dmi_name is the name that appears in the DMI Selector list.

The file name may contain %o and %s directives (currently, it may contain at most one of each). The %o is replaced with the object name, and the %s is replaced with the slot name. These are useful when using object types or slot types in the object.slot specifications.

Note: Because dmi names, object names and slot names are mapped to directory and file names, characters which are not valid in directory and file names (currently all characters other than A-Za-z0-9/_.) are replaced with _ when the directory or file name is created.

Consider the following example:

RIVERWARE_DMI_DIR is /usr/local/riverware

The DMI name is “test dmi”

The control file contains the following entry:

Canal.Flow 1: file=~/%o.%s.dat

The file which the “Tellico Canal.Flow 1" object.slot would import from or export to would be:

/usr/local/riverware/test_dmi/Tellico_Canal.Flow_1.dat

The ~ has been replaced with the value of the working directory, RIVERWARE_DMI_DIR/<dmi name>, %o has been replaced with the object name, and %s has been replaced with the slot name (with the spaces having been replaced with _).

Multiple files can be specified on a single control file line entry for input dmi’s. This enables the user to import data into a slot from two or more files. Use of the data_date file keyword allows the user to specify where in the series slot each file should be placed. The syntax for multiple files is

Object.Slot: file=name file=name

The data files are imported from left to right. If the import = resize is specified, the slot will grow to accommodate the data. Overlapping data is allowed but values will be overwritten by the later data files. If the incoming data does not overlap and there are gaps, any untouched values that existed on the slot will remain.

Another control file available is %tempdir, which is replaced by a guaranteed unique temporary directory. When the DMI is invoked RiverWare generates the directory name, makes the directory and passes the directory to the DMI executable via the RW_DMI_TEMPDIR environment variable. The executable should be able to recognize RW_DMI_TEMPDIR. After the DMI finishes RiverWare removes the directory. The directory name will be similar to the following:

b05741ad-2b57-4f82-8044-bda67606e4c9

Because RiverWare removes the directory you most likely won't see these directories, but if you kill RiverWare or RiverWare crashes a directory might be left behind, in which case you can safely remove it.

Optional keyword=value Pairs

The optional keyword=value pairs provide additional information about the data files the slots will import. (The one exception to this is the file keyword, which identifies the data file itself.) Although the keywords are not case sensitive, the values can be.

The currently supported keywords are:

• file=<data file>

The data file the slot will import from or export to, as described above.

• units=<units>

The units of the data the slot will import or export. <units> must match exactly one of the units defined in the Units List dialog (see Display Units in User Interface for details on the Units List).

• scale=<scale> (float)

The scale of the data the slot will import or export.

Note: If either units or scale is not specified in the control file, then it defaults to the current setting in the user interface as specified by the active Unit Scheme; (see Unit Schemes in User Interface for details). Since the unit scheme can change from invocation to invocation, it is strongly recommended that the control file always contain units and scale (even if the scale is 1.0).

• precision=<display>|<model>

The precision of the data the slot will import or export. The <display> precision is that specified in the Unit Scheme for the slot. The <model> precision is 17, the same precision as all internal calculations. The default is <display>.

• import=<fixed>|<resize>

This keyword controls how the DMI sets the time series range of the series slot. The default is fixed.

For either import option, the Begin Timestep of the slot is set in the same manner. If the data file contains a start_date header, the slot Begin Timestep is set to the start_date from the data file. If the data file does not contain a start_date header, the slot maintains its existing Begin Timestep.The data import always starts on the new Begin Timestep of the slot with the first value in the data file.

For import=resize, if the data file contains an end_date header, the End Timestep of the time series range of the series slot will be set to the later of the end_date header in the data file or the number of timesteps necessary hold the number of data values in the data file given the updated Begin Timestep of the slot from above. If the data file does not contain an end_date header, the End Timestep of the time series range of the slot will be extended, if necessary, to hold the number of data values in the data file given the updated Begin Timestep of the slot from above. The End Timestep of the slot will never be moved earlier if there is no end_date header in the data file.

For import=fixed, if the data file contains an end_date header, the End Timestep of the series slot is set to the end_date from the data file. Data will not be imported past the new End Timestep of the slot. If the data file does not contain an end_date header and the existing End Timestep on the slot is later than the updated Begin Timestep, the slot maintains its existing End Timestep. Data will not be imported past the End Timestep of the slot. If the existing End Timestep on the slot is equal to or earlier than the updated Begin Timestep, then the slot will be expanded to hold all of the data values in the data file starting with the updated Begin Timestep.

• flags=<true>|<false>

The flags keyword specifies whether or not Series Slot flag values (Input, Target, Best Efficiency, Max Capacity, Drift, and Unit Values) will be exported or imported with the data. The syntax is flags=true; the default is <false>. If an input DMI, the keyword pairs in the data file should be specified as “value flag” with the flag being an integer value, e.g. Input = 1, Output = 2, etc.

• aggregate=<true>|<false>

The aggregate keyword is used to specify whether or not the DMI should import/export all time series slots on an Aggregate Series slot or just the first time series column. If present, with the value “true”, and if the slot indicated is an Aggregate Series slot, the data file is expected to (on import) or will (on export) contain data for all subslots (columns) in the Aggregate Series slot. If the keyword=value pair is absent, or if its value is “false”, only the data for the first subslot (column) of the Aggregate Series slot are present.

When Aggregate Series slot data are imported or exported in total, the data are in row-major format, one timestep per line. The subslots of the aggregation must be homogeneous in start-date, end-date and step size, and all subslots (columns) must be represented on each line.

If used with the ‘flags=true’ keyword-value pair, the data will appear in the form:

value flag value flag value flag...

The flag is an integer number representing the flag. If the named slot is in fact the first subslot (column) of an Aggregate Series slot, the behavior is the same as if the subslot (column) name were absent.

The following three keywords are used to include text string metadata. They are mutually exclusive; you can only specify one at a time.

• slot_anno = <true>|<false>

The slot_anno keyword indicates whether to import or export series slot notes; see Exporting Notes Using DMIs in User Interfacefor details. The default value for slot_anno is false, meaning no Notes will be written.

• slot_set_info = <true>|<false>

The slot_set_info keyword indicates whether to export series slot value information. The default value for slot_set_info is false, meaning no information will be written.When true, the information corresponds to the tool tips seen on slot values on slot and SCT dialogs () will be written to the file; see Series Slots in User Interface for details. The string appears after the value. Examples of the information are “Set by DMI: Input DMI", “Set by Rule: (7) Flood Control”.

• slot_opt_sol_info = <true>|<false>

The slot_opt_sol_info keyword indicates whether to export series slot optimization solution information . The default value for slot_opt_sol_info is false, meaning no information will be written. When true, the available per slot/value optimization solution information will be written to the file. The string appears after each value. Examples of the information are “Frozen at Lower Bound”, or “Frozen by (3) Minimum Load 47.6% between limits set by 3.1.1.1 and 2.1.1.1.”. See Tooltips on Variables in Optimization for details.

User keyword=value Pairs

The user keyword=value pairs provide an additional means by which the user can pass parameters to the DMI executable. The user keyword=value pairs are specified in the control file as:

!keyword=value

The keyword=value pairs are written to the metaControl file, with the ‘!’ removed, on all lines they apply to. The DMI executable can then use the keyword=value pairs to influence its behavior.

Example 2.1

# All LevelPowerReservoir Outflows

LevelPowerReservoir.Outflow: file=~/%o.Outflow units=cfs scale=1.0

# All LevelPowerReservoir Pool Elevations,

# but for Hiwassee use a different scale.

LevelPowerReservoir.Pool Elevation: file=~/%o.%s units=ft scale=1.0

Hiwassee.Pool Elevation: file=~/%o.%s units=ft scale=10.0

# All LevelPowerReservoir Storages.

LevelPowerReservoir.Storage: file=~/%o.%s units=cfs-day scale=1.0

# All LevelPowerReservoir Hydrologic Inflows.

LevelPowerReservoir.Hydrologic Inflow: file=~/%o.%s units=cfs scale=1.0

# Finally, all SeriesSlots which are not selected by one of

# the preceding lines. (Remember that *.<slot type> is the

# least exact specification.) The file ~/start_end is a data

# file which contains only the start_date: and end_date:

# attributes. This will serve to change the start date and

# end date for all series slots which import this file.

*.SeriesSlot: file=~/start_end

Revised: 08/04/2020