RiverWare
  RiverSMART Software Suite Help
    Contents
   
1)
  RiverSMART Help Information
   
2)
  Hydrology Simulator Plugin Help Information
   
3)
  Spatial Disaggregation Plugin Help Information
   
4)
  Temporal Disaggregation Plugin Help Information
   
5)
  RiverWare DMI Plugin Help Information
   
6)
  RiverWare MRM Plugin Help Information
   
7)
  RiverWare Model Plugin Help Information
   
8)
  RiverWare Policy Plugin Help Information
   
9)
  RiverWare Run Range Help Information
   
10)
  RiverWare Plugin Help Information
   
11)
  RdfAnnualizer Plugin Help Information
   
12)
  RdfToExcel Plugin Help Information
   
13)
  GPAT Graphs Plugin Help Information
   
14)
  R Plugin Help Information
   
15)
  CSV Combiner
   
16)
  Download R for Windows
 

RiverSMART Help Information

Table of Contents

Background

The RiverWare Study Manager and Research Tool (RiverSMART) is a software tool to facilitate the creation, execution and archiving of planning studies that compare the results of several scenarios with differing supplies (hydrologic scenarios), demands, and alternative options and strategies such as operating policies and infrastructure alternatives. It manages the execution of the simulations to multiple processors, and keeps track of the output files for analysis. It can serve as an archiving structure to save the results of an entire study. The software was designed and developed to address the needs of complex modeling studies such as the Basin Studies conducted by the U.S. Bureau of Reclamation to explore options for meeting projected imbalances of future supplies and demands due to changing climate and water uses.

RiverSMART development was funded through Reclamation’s WaterSMART Grants to Develop Climate Analysis Tools. It was developed at CU-CADSWES during the period that Reclamation was working on the Colorado River Basin Study and several of the features were directly influenced by needs identified by that study.

In addition to RiverWare, some of the other tools included in RiverSMART such as the Graphical Policy Analysis Tool  and the Demand Input Tool were developed in part under other Reclamation contracts and grants and further enhanced and adapted to fit into this software package. Other components such as the Hydrology Simulator were developed fully under the WaterSMART grant. RiverSMART can be further enhanced to include other plugins; we expect future development based on needs of users.

Overview

RiverSMART has RiverWare at its core – it organizes input and output files for RiverWare, manages execution of RiverWare with multiple run management, and post-processes rdf (RiverWare Data File) formatted output files. The main components or “events” of RiverSMART are:

Hydrology Simulator - generates an ensemble of hydrologic traces at an annual timestep based on reference data and a selection of one of several available algorithms. Along with this are the Spatial Disaggregation and Temporal Disaggregation Plugins that disaggregate annual single-site data to monthly, multi-site data.

DMI (Data Management Interface) Plugin - allows any arbitrary input of data into the model either as a single series to be imported before the multiple runs are executed or trace-based, i.e., run as an ensemble. For example, the outputs of the Hydrology Simulator or Demand Input Tool are input to RiverWare via the DMI Plugin.

MRM (Multiple Run Management) Plugin - The RiverWare model can support a number of Multiple Run Management configurations – these are named in the Model file. Each MRM configuration event names a configuration that will be used in the study. The combination of DMIs, models, policies and MRM configurations are selected according to the user’s needs in the Scenario Organizer dialog.

RiverWare – specifies the RiverWare executable to use for the study.

RDF Annualizer – annualizes monthly rdf output data.

RDF to Excel – converts the rdf output data to an excel workbook.

GPAT – (Graphical Policy Analysis Tool) – generates graphs that have been pre-configured in an Excel output file.

R Plugin – allows RiverSMART to call a user-created script in the R programming language for post-processing of study results.

CSV Combiner – combines CSV output files into a single CSV file.

Starting a Study

When starting a new study, the study must be given a name and the directory (Study Folder) must be chosen where the study and all of its sub-directories will be written.

Study Name and Folder
Environment variables can be used in the Study Folder path. If environment variables are used, they must be preceded by a $.

The File menu has items for saving the study, opening an existing study, and exiting RiverSMART.

File Menu

The menu bar has a Help menu with a RiverSMART Help item that will open this help file from RiverSMART.

Help Menu

The icon in the toolbar located below the menu bar will also open the RiverSMART help file.

 Note that individual help is available for each event from its configuration dialog as discussed in the Events section below.

RiverSMART Workspace

The RiverSMART workspace is populated with events, files, and links to create a study. These building blocks of a study are discussed below.

Events

Events are added to the workspace by right-clicking the workspace where you want the event, clicking Add Event, and then choosing the desired event to add to the study.

Adding an event

When an event is added to the workspace, it is shown as an icon representing the event type. The event types can be categorized based on their function in the study. Following are the currently available events by category along with their icons and a brief description.

Hydrology Events

         Hydrology Simulator - takes observed streamflows and synthesizes an ensemble of flow data for a specified number of traces

        Spatial Disaggregation - takes an ensemble of flow data for a single site and disaggregates it to a number of sites

         Temporal Disaggregation - takes an ensemble of annual flow data for a site and temporally disaggregates it to an ensemble of monthly flow data

RiverWare Input Events

         RiverWare DMI - specifies data to import into a RiverWare model via the Data Management Interface

         RiverWare MRM - specifies a Multiple Run Management configuration from a RiverWare model

         RiverWare Model - specifies a RiverWare model file

         RiverWare Policy - specifies a RiverWare policy set to use with a RiverWare model

         RiverWare Run Range - configure scenarios to have different run ranges


RiverWare Event

         RiverWare - specifies a RiverWare executable to use for the study

Post-Processing Events

         RDF Annualizer - takes a RiverWare Data File (RDF) of timestep less than a year and aggregates the data to an annual RDF file

         RDF To Excel - takes a RiverWare Data File and converts it into an Excel workbook

         or    GPAT Graphs (input by Scenario or Scenario Set) - takes Excel files with series data from RiverWare and generates graphs in an                                                                                                                                 Excel workbook

         or   R Plugin (input by Scenario or Scenario Set) - calls a user created script in the R programming language to post-process study results

         CSV Combiner - takes CSV files from a set of scenarios and combines them to a single CSV file

Double-clicking an event's icon will open a configuration dialog for the event. Note that configuring events and files is mutually exclusive of generating, simulating, and post-processing scenarios; if dialogs for processing scenarios in these ways are open, configuration dialogs for event and files cannot be opened. Following is an example configuration dialog for a RiverWare DMI event:

Event Configuration Example

The Name field at the top is common to all of the event configuration dialogs. This name must be unique among all events in the study, and a unique name is automatically generated when an event is created. The user can replace the automatic name with a more descriptive one. The event name is displayed on the workspace as a label directly below the icon for the event, so the name should be descriptive as a label, but not overly long. As discussed elsewhere, this name will be used as a directory name by RiverSMART when managing files related to the event.

Also common to all event configuration dialogs is the Help item in the menu bar. This allows the user to access a help file that explains the configuration options for this event. Each event type has its own help information to assist the user in filling out the configuration dialog.

One field worth noting that is specific to the DMI configuration dialog is the Category field. This user-created category name allows the user to group together DMIs based on the type of data being imported (i.e. Supply, Demand, Evaporation, etc.). The category is used as an input type when generating alternative RiverWare scenarios as discussed in a later section.

Sometimes in a study the choice is not between inputs of a given type (e.g. between Demands: High and Low) but rather between having an input or not (e.g. between Supplemental Flow A or No Supplemental Flow). For these cases, RiverSMART allows the use of a Placeholder event for any input event type (DMI, MRM, Policy or Model). An event can be designated as a placeholder event by selecting Placeholder from the Edit menu in the event configuration dialog.

Placeholder Event

If an event is designated as a Placeholder event, all of its configuration options are disabled with the exceptions of the Name field and the Category field for DMI events. The Placeholder event will be treated as any other input event in its category when generating scenario combinations. When the scenarios are executed, however, the event will do nothing. For example, a Placeholder DMI event will not invoke any DMI for its category when a scenario containing the Placeholder DMI event is simulated.

An event can be deleted from the workspace by right-clicking on the event and selecting the Delete Event option.

Delete an Event

Files

File items are added to the study to specify file names for outputs from events, and when linked to a following event will specify an input file name for that event. Files are added to the workspace by right-clicking the workspace where you want the file, clicking Add File, and then choosing the type of file to add to the study.

Add File

When a file is added to the workspace, it is shown as an icon representing the file type. There are currently four file types available in RiverSMART:

         RiverWare Data File - ASCII file representing slot data from RiverWare, typically output to capture data from all traces of a multiple run

        Excel Workbook File -  Excel  workbook, which could be an output from the RDF To Excel event, or input or output to a GPAT Graphs event

         CSV File -  Comma Separated Values file representing slot data from RiverWare, typically output to capture data from all traces of a multiple run

         netCDF File - Network Common Data Form file representing slot data from RiverWare, typically output to capture data from all traces of a multiple run


Double-clicking the file icon will open a configuration dialog for the file.

File Configuration Example

Files do not have to have unique names and are given a default label of the file type (i.e. RDF or Excel Workbook) beneath the icon on the workspace. They can be given a descriptive name in the Name field of the configuration dialog, which will then be used as the label on the workspace. The actual file name is either typed into the text box or chosen from the pull down menu. The menu lists all of the files of that type that are found in the model's MRM output configurations. Note, the actual path to a particular instance of this file is determined by RiverSMART when it is processing a scenario, you do not enter the full path, just the name.

A file is deleted from the study by right-clicking its icon and selecting the Delete File item.

Delete File

Linking

Links are created on the workspace to represent how information will flow between events and files. In some cases, an event is linked directly to another event, but in cases where the event output is preserved as a file, the event is linked to a file item, which then can be linked to a following event. Hydrology events link directly to each other, RiverWare input events link directly to RiverWare, and Post-Processing events link through file items. Links have an arrow to show the direction the link represents.

A link is created by right-clicking an icon for the event or file and clicking the Link item. This shows a cascading menu containing a Begin Link item and also a list of possible destinations for the link.

Linking

If one of the possible destinations is clicked, the link is automatically created between the starting item and the clicked destination item. Beneath the covers, each event or file has a type and knows what types it is allowed to link to, both for input and output. It also knows the number of inputs and outputs allowed for each type. This information permits the list of available destinations for a link to be determined for populating the link dialog list.

An alternative linking approach is to click the Begin Link item in the cascading menu. This creates a black rubber-banded line that is attached to the starting item. Moving the mouse over other events or files in the study makes the line turn green if that is a legitimate destination or red if it is not. Right-clicking over a legitimate destination brings up a menu where Create Link can be clicked to complete the link.

End Link

A link is deleted by right-clicking on its line and choosing Delete Link.

Delete Link

Adding Text

Text can be added to the workspace to identify like items that may be grouped together or to add other additional information. Text is added by right-clicking the desired location on the workspace and selecting the Add Text item. This creates a text edit box for typing in the desired information. When finished typing, clicking outside the edit box will close it and leave just the text on the workspace. Single clicking text will select its text box and allow it to be dragged to a different location. Double clicking text will allow editing of the text in the text box. Right-clicking text will bring up a Delete Text context menu for removing the text.

The size and color of text can be modified with controls in the toolbar on the workspace.

Text Attributes

One or more existing text items can be selected, and their color and font size can then be modified by the toolbar controls. Any new text added to the workspace will have the font size and color currently selected in the controls.

Multiple Selection

Multiple selections (multiple events, files and text items) can be selected by dragging out a rectangle:




Multiple selections can also be made by left mouse clicking on an event, file or text item to begin the selection and then control-left mouse clicking on events, files and text items to add them to or remove them from the selection. Right-click menu items apply to the selection.

Copy and Paste

The selected events, files and text items can be copied to the system clipboard, and the events, files and text items in the system clipboard can be pasted into the current study. The spatial relationships between the selected items are preserved as are the links between selected events and files. The selected items are copied to the system clipboard by right-clicking on a selected item and choosing Copy. (The wording of the Copy option will vary depending on the selected items.)




The items in the system clipboard are pasted into the current study by right-clicking on the workspace and choosing one of the Paste options:



The paste options behave as follows:

Here The items are pasted at  the current mouse position
(the current mouse position is the upper-left corner of the pasted items)
Left of the Study The items are pasted centered on the left border of the study
Right of the Study The items are pasted centered on the right border of the study
Above the Study The items are pasted centered on the top border of the study
Below the Study The items are pasted centered on the bottom border of  the study

The pasted events, files and text items remain as the current selection, making it easy to "fine tune" their placement.

The selected items can also be copied to, and pasted from, applications which support copying and pasting text (for example, text editors and email clients).

File Menu

The File menu item in the RiverSMART menu bar contains items to open and save study files. Of particular note is the Reopen Study menu, which shows the 6 most recently accessed study files and directories.



 The files and directories can be selected with the mouse or by entering their number. Selecting a file opens the study file, selecting a directory opens the file chooser in the directory.

Workspace Menu

The Workspace menu item in the RiverSMART menu bar contains the controls for enabling the collection of diagnostic messages during execution of the study and for actually showing the diagnostic output dialog that displays any collected diagnostic messages. The Validate Study menu item will validate the configurations of the events and files shown on the workspace. Validation varies with the event or file, but generally includes checking that required configuration information is provided and verifying that referenced files that should already exist are available and readable. The Workspace menu also contains an item to clear the workspace. This will remove all events, files, links and text from the workspace to present a clean slate.

Workspace Menu


Zooming  the Workspace

Controls on the toolbar allow the workspace to be zoomed in or out by increments of 10% within the range of 10% to 400 %. The current zoomed percent of the workspace is displayed adjacent to the controls.

Zoom Controls

Status Bar

Finally, near the bottom of the workspace, there is a status bar. As shown, the status bar has four elements:
1. The study status (blue background), for example “Executing event RiverWare”.
2. The event status (green background), for example “Simulating scenario C Inflow 2, E Inflow2, R Inflow1, T Vs, CRSS Policy 2”.
3. The event progress bar: If an event is able to provide a percent complete then the progress bar shows the percent complete; otherwise is functions as a busy indicator.
4. The diagnostic window button: The diagnostic window button is a tristate button. In its normal state it displays a “log file” icon. If warnings have been posted to the diagnostic window which haven’t been viewed, it changes to the warning color and blinks (warning state). Similarly, if errors have been posted to the diagnostic window which haven’t been viewed, it changes to the error color and blinks (error state). Thus it provides a graphical indication of whether there are warning or error messages which users haven’t viewed. It should be noted that “haven’t been viewed” means that users haven’t opened the diagnostic window from the RiverSMART user interface since the messages were posted. Users might have to scroll
the diagnostic window to actually view the messages. Thus, it is a graphical indication that there are messages users may want to view.

Example Configured Workspace

The following shows an example configured workspace with events, files and links.

Configured Workspace

The example study shows two sets of hydrology events that are linked in the upper right portion of the workspace. RiverWare inputs are all linked to RiverWare in the left part of the dialog (note that DMIs are grouped into Demands, Supplies, and Evap). Only one RiverWare event is allowed in a study. RDF files coming out of RiverWare are post-processed through a series of events and files in the right side of the workspace to end up with GPAT graphs in Excel workbooks.

Approaches for Setting Up RiverWare Inputs

When configuring a study, users must consider what to configure in the RiverWare model versus what to configure as inputs in RiverSMART. This is particularly true for MRM and DMI configurations.

MRM configurations in RiverWare can specify initialization and per-trace DMIs as well as policy. A user could create a RiverSMART event for a RiverWare MRM configuration that has these specified and and not show the DMIs and policy as events on the RiverSMART workspace. However, this hides the scenario components so that they cannot be seen in RiverSMART. The RiverWare model has to be opened to to see the DMIs and the policy that are used. This reduces the usefulness of RiverSMART for visualizing how inputs are combined into scenarios. A more transparent approach would be to have DMI and policy events in RiverSMART that will get paired with an MRM configuration event. The DMIs and policy configured in the RiverSMART events will get substituted into that MRM configuration n RiverWare to create the scenario. In this setup, the RiverSMART workspace provides a visual representation of how the scenario components are combined into scenarios.

As an example of DMI configuration trade offs, take the four demand DMIs in the above example. These may be Excel Direct Connect DMIs where the data  for each demand is contained on a different worksheet in an Excel workbook. One approach is to set up four fully configured DMIs in RiverWare and have the four DMI events in RiverSMART configured to refer to each RiverWare DMI by name. To set up the four fully configured DMIs in RiverWare, much of the DMI configuration needs to be repeated four times. The actual Excel dataset configurations that specify the same Excel workbook, but each with a different worksheet, would be largely repeated, but more importantly, the slots into which the demand data will be imported need to be repeated in the setup of each  DMI. This may be hundreds of slots and keeping the slot selections with their begin dates, end dates, and units consistent across all of the DMIs would be tedious and error prone. Here is an example of the RiverWare configuration of an Excel direct connect DMI with slot selections:



A better approach would be to create one dataset and one DMI in RiverWare that functions as a template for RiverSMART to fill in for each scenario. The Excel dataset configuration would refer to the Excel workbook, but would leave the Single Run Name (the worksheet name) blank:



The one DMI in RiverWare would use this "template" dataset, and would contain the slot selections as shown above. The four RiverSMART DMI events would have configurations that refer to that one RiverWare DMI, but would provide the worksheet name that will be substituted into the "template" dataset for that particular demand:



In this way, only one dataset and DMI need to be maintained in RiverWare. RiverSMART can use these as a template and substitute in the correct worksheet for each demand as configured in its DMI events. A disadvantage of  using a template DMI is that the demand DMIs cannot be run interactively from within RiverWare itself without manually inserting the missing information into the template or using a script to do this.

Hydrology Processing

The hydrology-related plugins link to each other, but not to other types of events. Their purpose is to create traces of hydrology data that can be moved into RiverWare via a separately configured RiverWare DMI event. Generating hydrologies can be very time-consuming and, once configured properly, these events would not typically be rerun unless some input data has changed.

Execution of hydrology events are initiated via the context menu. Right-clicking a hydrology event will bring up the following menu.

Hydrology Event Execution

The execution options allow the user to execute the selected event, execute all previous linked events up to and including the selected event, or to execute all events after the selected event. These options give a lot of flexibility for executing all or parts of a linked hydrology sequence so that only the pieces that need to can be rerun.

The location of generated hydrology results is controlled by RiverSMART. Under the specified Study Folder for the study, a Hydrology directory is created. Each event that generates hydrology data will have a directory created under the Hydrology directory named with the event's name. Trace folders are then created under the event's folder with the generated data files for each trace.

There is also a Working directory created under the Study Folder. A subdirectory is created here for each event that will contain information like log files or intermediate results generated during the event's execution. These can be useful for debugging problems in event execution.

The R Project for Statistical Computing along with some of its component packages must be installed on the user's computer for the hydrology events to execute successfully. See section 16 for instuctions on downloading and installing R and its packages.

Scenario Processing

A scenario refers to a particular combination of RiverWare inputs, which create a RiverWare run. The generation, simulation and post-processing of RiverWare scenarios are controlled from the Scenarios menu in RiverSMART. Note that these menus are disabled if configuration dialogs are open for files or events on the workspace; the configuration and the running of scenarios are mutually exclusive.

Scenarios Menu

Generate Scenarios

The Scenario List Organizer dialog that is accessed from the Generate Scenarios item in the Scenarios menu in RiverSMART is where all of the potential combinations of the inputs linked into RiverWare are made into scenarios. An example follows:

Generate Scenarios Dialog

A column is created for each of the RiverWare input categories. Note that the RiverWare DMIs have been assigned one of three different category names in their configuration dialogs, so the three categories show up as separate columns in this dialog (Supply, Demand, and Evap). By default the columns show up in alphabetical order but can be selected and dragged to move them into an order which makes the most sense to the user for identifying the scenarios. This is important because the order of the columns here determines the scenario name, which is a concatenation of the column names from left to right. There is an additional dialog available from the Edit menu that can further modify scenario names.

Scenario Name Menu Item

The Scenario Name Configuration dialog presents controls to select which columns should become components of the name and what the separator between the components should be.

Scenario Name Configuration Dialog

In this example, the same MRM Configuration and Model are used for all scenarios, so including these in the name adds no additional information for differentiating among scenarios. These are unchecked to leave them out of the name. The separator for the components of the name is selected via the combo box. An example of a scenario name as configured is given at the bottom of the dialog.

Once set in the Scenario List Organizer dialog, the scenario name cannot be changed without coming back to this dialog to regenerate the scenarios, which will cause the loss of any simulation and post-processing state information previously associated with the scenarios. Scenario names are fixed in this way because directories for files generated by RiverSMART are automatically created using the scenario names as directory names.

You can see at the bottom of the Scenario List Organizer that 48 scenarios have been created from the combination of inputs to RiverWare in the example. Not all of these make sense because, for example, VIC Evap should only be used with VIC supplies. In reality there are only twelve scenarios among the 48 that should be run for this model. To eliminate the unwanted combinations from cluttering up this and subsequent dialogs, the unwanted scenarios can be checked and deactivated. Checking is accomplished by using the Check drop down menu at the top of the dialog.

Check Dropdown Menu

All will check every row (a shortcut for this is to check the box to the left of the menu, which will also check all rows). None will uncheck any checked rows. Selected allows you to check any rows that you have selected in the dialog. Since there may be many rows in this dialog with the need to check many individual rows, this approach allows you to incrementally select and check things as you work through the list.

Once unwanted scenarios are checked, they can be deactivated from the edit menu.

Deactivate Scenarios

When a scenario is deactivated, that state is persistent across all dialogs. The scenario will not show up in any of the other dialogs in RiverSMART. If scenarios are mistakenly deactivated, the Activate Inactive Scenarios menu item will bring them all back and the correct ones can then be checked and deactivated.

There is also functionality to hide and show scenarios. Hiding simply hides scenarios from view in this dialog only and does not deactivate them. This is accomplished by checking scenarios and using the Hide Checked Scenarios item under the View menu.

Hide Scenarios

The Show Hidden Scenarios menu item will bring back all hidden scenarios into the dialog view. Note that the message line at the bottom of the dialog says how many active scenarios there are, how many are hidden and how many are currently checked.

Simulate Scenarios

The Scenario List Simulator dialog is accessed from the Simulate Scenarios item in the Scenarios menu in RiverSMART.

This is a two level tree-view that lists the activated scenarios with sub-items for activities that have been completed during simulation for each scenario. Checking a scenario and clicking the Simulate Checked Scenarios menu item will initiate a RiverWare MRM run utilizing that scenario's inputs. Note that the MRM run are typically set up as distributed runs. If they are not set up to be distributed, then are automatically converted into distributed runs using the default number of processors on the machine. If more than one scenario is checked in the Scenario List Simulator, each will run sequentially until all are completed. If scenarios are being run, the Stop Simulations menu item will be active. Clicking this allows the current scenario to finish, but will stop execution before the following scenario starts.

Simulate Scenarios

The location of scenario simulation results is controlled by RiverSMART. Under the specified Study Folder, a directory named Scenario is created. Under here a directory is created for each scenario using the scenario's name. The files generated by RiverWare (i.e RDF, CSV, and netCDF) from simulating a scenario go into this folder. There is also a Working directory created by RiverSMART under the Study Folder. A subdirectory is created here with the RiverWare event's name that contains a directory for each scenario. This is where log files or intermediate results generated during simulation of the scenario are placed. These can be useful for debugging problems.

When the RiverWare MRM configuration is set up to distribute individual runs of a multiple run across cores on the computer, the distributed MRM interface will be displayed during a scenario's simulation. This shows the progress of the traces distributed to each core and the status of the current individual trace running on each core. The Stop All button on the interface can be used to interrupt a currently running scenario.

Distributed MRM Interface

If a scenario's simulation was successful, the icon in the run column of the Scenario List Simulator dialog will turn from gray to green. An unsuccessful simulation will make the icon red. When a scenario is selected in the dialog, the status line near the bottom will show the status information for that scenario. This state information of success or failure of simulations is saved with the study and persists if the study is closed and reopened. The Mark Checked Scenarios As Simulated and Mark Checked Scenarios As Not Simulated menu items allow manual control over whether the run icon is gray or green. For instance if a scenario is successfully simulated, but the study was not saved after that, the icon could appear gray when the study is reopened even though the scenario has been simulated. The icon could then be made green manually to reflect the true state of the scenario. The yellow file folder button at the end of a scenario line will generate a Windows Explorer dialog open to the Scenario directory where output files for that scenario reside.

If the scenario's tree is opened, the activities for a simulated scenario are shown. An activity represents a step in completing the scenario simulation, such as initializing the model, executing the distributed MRM controller, or simulating traces. The activity status is shown as a green check for success and a red check for failure. Note that an activity may fail, but the icon for the scenario could still show green. It is advisable to open the activity list for a simulated scenario to check the activities for problems. Highlighting an activity line will give a status message for the activity at the bottom of the dialog. If RiverWare created a log file associated with the activity, the file icon in the status message will be enabled and will open the log file for viewing. This link can aid in identifying a problem if the RiverWare activity was unsuccessful.

When a scenario is simulated, keyword/value descriptors are created to characterize the scenario and its input events. These descriptors are written to the MRM configuration in the RiverWare distributed model as MRM descriptors and from there are optionally available as outputs to CSV and netCDF files from the multiple run. Descriptor keyword/value pairs are created as follows:
The Scenario List Simulator dialog also contains menu items to deactivate/activate scenarios under the Edit menu and hide/show scenarios under the View menu. These work as described above for the Scenario List Organizer dialog. Deactivated scenarios apply across all dialogs in RiverSMART. Hidden scenarios apply to this dialog only (if a large number of scenarios have been successfully run, for example, you may want to hide them to reduce dialog clutter).

Post-Process Scenarios

The Scenario List Post-Processor dialog is accessed from the Simulate Scenarios item in the Scenarios menu in RiverSMART.

This is a three level tree-view that lists the activated scenarios followed by the events performed via post-processing followed by any completed activities for an event. Checking one or more scenarios and clicking the Post-Process Checked Scenarios menu item will initiate their post-processing. When a scenario is post-processed, its output files (i.e. RDF or CSV) are run through the linked processes that follow each output file icon on the RiverSMART workspace. Post-processing typically might involve taking an RDF file and running it through events like the RDF Annualizer and/or RDF to Excel to generate Excel workbooks that can then be used for data analysis. Files resulting from post-processing are managed by RiverSMART and are placed where the scenario's output files reside, under the Scenario directory of the study under a subdirectory with the scenario name. Log files or intermediate results from post-processing go in the Working directory of the study under a subdirectory with the event name. If scenarios are post-processing, the Stop Post-Processes menu item will be active. Clicking this allows the current scenario to finish processing, but will stop before the following scenario starts.

Post-Processor Dialog

For convenience, the simulation status of scenarios is shown with icons in the run column of the Scenario List Post-Processor (green for successful, red for unsuccessful, gray for not run). The next column of icons is the post-processing status. For a scenario, the post-processing status can be green for all successfully run, red for all failed, part red and part green for some steps successful and some failed, and gray for not processed. The Mark Checked Scenarios As Post-Processed and Mark Checked Scenarios As Not Post-Processed menu items allow for some manual control over the post-processing icon for a scenario (in case the study was not saved  to properly record the status after some post-processing). The yellow file folder button at the end of a scenario line will generate a Windows Explorer dialog open to the Scenario directory where output files for that scenario reside.

If the scenario's tree is opened, the post-processing status is marked for each event in the post-processing of the scenario, green for success and red for failure. If an event line is highlighted, a status message for the post-processing of that event is displayed at the bottom of the dialog. If unsuccessful, the reported error message is given in a second line of the status message. If an event reports individual activities in its post-processing, these appear as another sublevel in the tree view under the event, with a green check meaning success and a red check meaning failure of the activity. Highlighting an activity line will give a status message for the activity at the bottom of the dialog. An activity can create a log file with information about what transpired in the activity. If a log file is available, the file icon in the status message will be enabled and will open the log file for viewing. Following is a sample log file generated by GPAT for the successful Stateline Flow Graphs activity highlighted in the above dialog:



This Scenario List Post-Processor dialog also contains menu items to deactivate/activate scenarios under the Edit menu and hide/show scenarios under the View menu. These work as described above for the Scenario List Organizer dialog. Deactivated scenarios apply across all dialogs in RiverSMART. Hidden scenarios apply to this dialog only (if a large number of scenarios have been successfully post-processed, for example, you may want to hide them to reduce dialog clutter).

Display Orders

When generating, simulating or post-processing scenarios the order of the scenarios is determined by the order of the RiverWare inputs, with the leftmost input changing least frequently and the rightmost input changing most frequently. By default the order is ascending:


The View menu:


allows for the scenarios to be displayed in descending order:


and custom order (controlled by the up and down arrows, which move the selected scenarios up and down one row at a time):


Updating Scenario Inputs

When modifications are made to input events (DMI, MRM, Model or Policy), RiverSMART provides visual indicators to alert you to portions of the study that are no longer up-to-date. If any item, other than the name, in the configuration dialog for an input event is modified, the label of the event will turn red to indicate that scenarios that include the event need to be regenerated.

It is also possible to manually specify that an event has been modified. For example, if one or more rules in a ruleset gets modified, this could change the outputs for scenarios that include the ruleset; however RiverSMART is not aware of any changes made to external files, such as a ruleset or data files. In order to transmit the information that the ruleset has been modified, and thus its associated scenarios are out-of-date, the Externally Modified option can be manually selected from the Edit menu of the event configuration dialog.



Selecting Externally Modified will have the same effect as modifying the event configuration. The event label will turn red, and RiverSMART will be aware that scenarios that include the event need to be regenerated. If a new input event is added to the study, its label will also be red indicating that scenarios should be regenerated to incorporate the new event.

As a further indicator that scenarios might be out-of-date, in the Scenario List Simulator and Scenario List Post-Processor dialogs, any scenario that contains an event that has been modified will be colored red. This again indicates that scenarios should be regenerated, and these scenarios should be re-run to bring study results up-to-date. Scenarios that were unaffected by changes to input events will remain black.

Scenarios with modified inputs colored red

Note that the icon in the run status column is still green, indicating that the scenario has been successfully simulated. This is because scenarios have not yet been regenerated, and thus the scenario has simulated successfully based on the current scenario inputs.

The next step is to regenerate scenarios in order to update all scenarios that have had input events modified. Regenerate Scenarios is selected from the Scenario menu in the RiverSMART menu bar.

Regenerate Scenarios

Selecting Regenerate Scenarios will bring up the Scenario Confirmation dialog.

Scenario Confirmation dialog

The Scenario Confirmation dialog indicates scenarios that will be added, will be removed, will have their results invalidated or will be unaffected based on any modifications made to input events since the last time scenarios were generated. Clicking on the arrow next to the row for a given category will expand the row to show all of the individual scenarios that apply. Clicking OK will regenerate scenarios. At this point, all options described in the Generate Scenarios section can be re-applied.  Once scenarios are regenerated, all event labels will return to black, and the scenario names in the Simulator List Simulator and Scenario List Post-Processor dialogs will return to black. Any scenarios that had their results invalidated when scenarios were regenerated (or any new scenarios added) will now display a gray box in the run status column indicating that the scenario has not yet been run (needs to be resimulated). Scenarios that were unaffected will continue to display the run status they displayed before regenerating scenarios.

Scenarios not yet run after regenerating

At this point, the scenarios with a gray box can be resimulated and post-processed. Scenarios with a green box do not need to be resimulated because their results were not affected by the changes to the input events.

Scenario Set Definition and Processing

Scenario Sets are precisely that, a number of scenarios that have been grouped together into a set. The reason for creating scenario sets is to facilitate analyzing and comparing results across scenarios. With many scenarios in the study, it may be impractical, for example, to graph desired output data for all scenarios on a single plot because it gets too cluttered. Scenario sets can be created to group scenarios that you do want to compare together. For example a set could be created for all scenarios that have the same hydrology but different demands and policies. Or scenarios with a certain policy, but differing demand and hydrologies could be grouped into a set. The following sections discuss defining and processing scenario sets.

Definition

Scenario sets are defined in the Scenario Set Manager, available from the Scenario Set Management menu item under the Scenarios menu of RiverSMART.

Scenario Set Manager

The Scenario Set Manager dialog shows scenario sets that have been defined in the left-hand window. If a set is selected on the left, the scenarios that are members of the selected set are shown in the right-hand window. A selected set on the left can be deleted with the Delete button. Clicking the Add button or clicking the Edit button for a selected set brings up the Scenario Set Editor dialog.

Scenario Set Editor Dialog

The Scenario Set Editor dialog is where a set is assigned a name and where activated scenarios can be selected for inclusion in the set. In this example, we are defining a set of the scenarios having the DNF hydrology. If a scenario was included in a set and then was later deactivated, it will show up in the Scenario Set Editor (and in the Scenario Set Manager if that scenario set is selected ) as grayed out. It can be unchecked in the Scenario Set Editor to be removed from the set and will then disappear from display in these dialogs.

Processing

All of the post-processing as described in an earlier section are by individual scenarios. Processing involving sets of scenarios is initiated differently. Where scenario sets are utilized, they are specified as inputs in the configuration of a particular event. GPAT Graphs, the R Plugin, and the CSV Combiner are currently the events where scenario sets can be configured as input. Events using scenario sets as input are grouped by event type and given a Process menu item in the Scenarios menu of RiverSMART.

Processing with Scenario Sets

The cascading menu under Process GPAT Graphs allows individual GPAT Graphs events that use scenario sets to be processed, similarly for Process R Plugin. When a scenario set is processed, the input file(s) linked to the event is retrieved for each scenario in the set, and all are passed together to the event for processing. In the case of GPAT Graphs, the input Excel files for all scenarios in the set are passed to GPAT where graphs are generated that contain lines for each scenario to present a visual comparison of results from the scenarios. The graphs generated for the scenario set go to an output Excel workbook. The directory for this result workbook is managed by RiverSMART and is located in the ScenarioSet directory under the study folder. Here a directory is created for each scenario set name, with a subdirectory for each event that processed the scenario set, which will contain the result file. In the case of CSV Combiner, the linked CSV files for all scenarios in the set are passed to the event where they are combined to a single CSV output file. The use of linked input files by an R Plugin is dependent on the user-created R script called by the plugin. Any log files generated by a GPAT Graphs event, a CSV Combiner event, or R Plugin event go under the Working directory of the study folder in a directory with the event's name.

Directory Structure

An important aspect of RiverSMART is its ability to manage all of the files associated with a complex study. The idea is that all of the files for the study are organized together in a directory structure so that the study can be easily archived or moved. The user specifies a Study Folder directory in the interface that is the root for all of the directories and files. RiverSMART automatically creates the following directories under the study folder:


Hydrology Simulator Plugin Help Information

Table of Contents

Overview

The Hydrology Simulator will take observed streamflow values and synthesize an ensemble of flow data for a user specified number of traces. These trace flow files can then be imported via a DMI to a RiverWare multiple run. The scripts to generate hydrology ensembles use the R Project for Statistical Computing program, so R must be installed on the user's computer (R Installation Instructions).

Available Functions

The Hydrology Simulator has three functions to choose from for generating ensembles:

  1. Resampled KNN -  This function resamples historic streamflow values using the k-Nearest Neighbor algorithm.
  2. Paleo Conditioned Homogeneous Markov Chain - This function resamples historic streamflow values using paleo-reconstructed streamflow sequences.  First it creates a transition-probability matrix from the entire paleo period.  Then the historic streamflow data is divided into 2 or 3 transitional states (based on user preference).  Using these transition-probabilities as weights, a state is selected for each time step corresponding to the Trace Length.  An observed streamflow value from the selected state is then chosen using the k-Nearest Neighbor algorithm.  This process is repeated for each time step until the Trace Length is reached.
  3. Paleo Conditioned Nonhomogeneous Markov Chain -  This function resamples historic streamflow values using paleo-reconstructed streamflow sequences.  First it creates a transition-probability matrix from a randomly-selected window from the paleo period, of length equal to Trace Length.  This is what differentiates this method from the Homogeneous Markov Chain method.  Then the historic streamflow data is divided into the same transitional states.  Using these transition-probabilities as weights, a state is selected for each time step.  An observed streamflow value from the selected state is then chosen using the k-Nearest Neighbor algorithm.  This process is repeated for each time step until the Trace Length is reached.

User Interface

When the Hydrology Simulator plugin is configured, a user interface is presented where the user gives the plugin a unique name. Other general controls are the Help menu item at the top that will bring up this help file, and the OK button at the bottom that will save the configuration and dismiss the dialog. The user can select the desired function from a drop-down list. The following sections discuss the user interface presented for each function.

Resampled KNN Interface

Resampled Knn Interface

Input File

In the Input File frame, the user can type in a file name or select a file via the file chooser button. This file is a single time series of observed historic flows for the desired site. The R scripts expect that the file will start with data values, so if there are non-data header lines in the file, the number of lines to skip needs to be specified by the user with the Number of Header Lines spin box. The eye icon will bring up a viewer showing the file (if the file exists) allowing the user to see what header lines are present.

Generated Ensemble

The Generated Ensemble frame contains controls to configure ensemble information. The number of traces to generate in the ensemble and the number of timesteps to put into each trace are specified here. The optional Climate Change Trend control allows a positive or negative percent change to be applied, so that over the course of all the timesteps in the trace, values will trend up or down by this percent.

Output Files

The Output Files frame contains a number of controls for specifying output generated by the plugin.  The output directory is determined by RiverSMART and is displayed as read-only here for information purposes (this directory is named with the name of the plugin and is located under the Hydrology subdirectory of the study's directory). A directory for each trace, named trace1, trace2, etc., will be created under the output directory. The File Name is the name that will be given to the resulting output file generated in each trace directory. This name (typically a site name) can be typed in by the user, or optionally selected from a RiverWare DMI control file. If this option is chosen, the DMI Control File name can be typed in or chosen with a file selector button. The eye icon will allow the user to view the entire control file. The Choose... button brings up a list of the slot names present in the control file and the user can select one that will be populated to the File Name line. If the slot has units specified in the control file, the Scale/Units line will also be populated (this is for informational purposes only and has no impact in changing the ensemble calculations). The Clear button will clear out entries in the File Name and Scale/Units lines.

The remaining controls in the Output Files frame are optional checkboxes. The Copy Header Lines from Input File checkbox allows the user to copy the header lines identified as present in the Input File over as header lines to the output files. Add Header Lines allows the user to type in additional header lines to be put into the output files. The Edit.. buttons brings up a text box to type in the additional lines, and when saved, the number of additional lines is displayed in the adjacent box. The Create R Data File option allows the R session for generating the ensembles to be saved so that it can be reopened later in R. The path for the saved file can be typed in or selected with the file chooser button.

Paleo Conditioned Homogeneous Markov Chain Interface

Paleo Conditioned Homogeneous Markov Chain Interface

Input Files

In the Input Files frame, the user can type in file names for Historic Streamflows and Paleo Streamflows or can select files via file chooser buttons. Historic Streamflows are a single time series of observed historic flows for the desired site. Paleo Streamflows are a single time series of paleo-reconstructed streamflows for the site. The R scripts expect that the files will start with data values. If there are non-data header lines in the Historic Streamflows file, the number of lines to skip can be specified by the user with the Number of Header Lines spin box. The eye icon will bring up a viewer showing this file (if the file exists) allowing the user to see what header lines are present.

Generated Ensemble

The Generated Ensemble frame contains controls to configure ensemble information. The number of traces to generate in the ensemble and the number of timesteps to put into each trace are specified here. The optional Climate Change Trend control allows a positive or negative percent change to be applied, so that over the course of all the timesteps in the trace, values will trend up or down by this percent.

The Hydrologic Classification States controls allow definition of the hydrologic states used in the calculations. The State Count can be 2 or 3 (i.e. corresponding to "dry/wet," or "very dry/normal/very wet" for classifying the hydrology for each year). Radio buttons allow the given number of states to be equally sized or have their sizes specified by entering the separating threshold values in a line widget (comma separated if there are more than 2 states). Radio buttons allow the entered threshold values to represent either flow values or quartiles (from 0.0 to 1.0).

Output Files

The Output Files frame contains a number of controls for specifying output generated by the plugin.  The output directory is determined by RiverSMART and is displayed as read-only here for information purposes (this directory is named with the name of the plugin and is located under the Hydrology subdirectory of the study's directory). A directory for each trace, named trace1, trace2, etc., will be created under the output directory. The File Name is the name that will be given to the resulting output file generated in each trace directory. This name (typically a site name) can be typed in by the user, or optionally selected from a RiverWare DMI control file. If this option is chosen, the DMI Control File name can be typed in or chosen with a file selector button. The eye icon will allow the user to view the entire control file. The Choose... button brings up a list of the slot names present in the control file and the user can select one that will be populated to the File Name line. If the slot has units specified in the control file, the Scale/Units line will also be populated (this is for informational purposes only and has no impact in changing the ensemble calculations). The Clear button will clear out entries in the File Name and Scale/Units lines.

The remaining controls in the Output Files frame are optional checkboxes. The Copy Header Lines from Input File checkbox allows the user to copy the header lines identified as present in the Input File over as header lines to the output files. Add Header Lines allows the user to type in additional header lines to be put into the output files. The Edit.. buttons brings up a text box to type in the additional lines, and when saved, the number of additional lines is displayed in the adjacent box. The Create R Data File option allows the R session for generating the ensembles to be saved so that it can be reopened later in R. The path for the saved file can be typed in or selected with the file chooser button.

Paleo Conditioned Nonhomogeneous Markov Chain Interface

Interface controls for the Paleo Conditioned Nonhomogeneous Markov Chain function are the same as discussed above for the homogeneous function, except there is an additonal optional checkbox in the Climate Change Trend controls to Enable Paleo Weighting for the climate change calculation.

Enable Paleo Weighting Control

R Installation Instructions

The R Project for Statistical Computing along with some of its component packages must be installed on the user's computer for the Hydrology Simulator to successfully generate ensemble flow data. See section 16 on instructions for downloading and installing R and its packages.


Spatial Disaggregation Plugin Help Information

Table of Contents

Overview

The Spatial Disaggregation plugin will take an existing hydrology ensemble consisting of trace directories each containing a file of flow data for a single site, and spatially disaggregating that site's flow data to result in trace directories containing files of flow data for a number of sites . These trace flow files can then be imported via a DMI to a RiverWare multiple run. The scripts to spatially disaggregate flow data use the R Project for Statistical Computing program, so R must be installed on the user's computer (see R Installation Instructions).

User Interface

When the Spatial Disaggregation plugin is configured, a user interface is presented where the user gives the plugin a unique name. Other general controls in the interface are the Help menu item at the top that will bring up this help file, and the OK button at the bottom that will save the configuration and dismiss the dialog. The following section discusses the spatial disaggregation controls in the interface.

Spatial Disaggregation Interface

Timestep Size

This radio button allows the user to specify if the input ensemble data discussed below is monthly or annual data.

Input Traces

In the Input Traces frame, the user specifies information related to the ensemble of trace data for a site that is being input to the spatial disaggregation process. If the plugin is linked on the input side, the directory, number of traces, and flows file name fields are disabled. In this case, these fields are automatically filled in when output from the upstream plugin is generated. If the spatial disaggregation plugin is not linked on the input side, these fields are enabled and must be filled in by the user. The Directory of Traces is the directory that contains all of the trace folders for the input ensemble, and can be typed in or selected via the directory chooser button. The Number of Traces in the ensemble must be entered, and the name of the site data file that exists in each trace directory must be entered in the Flows File Name field. A site data file should be a single time series flows for the desired site. The R scripts expect that the file will start with data values, so if there are non-data header lines in the site data file, the number of lines to skip needs to be specified by the user with the Number of Header Lines spin box. The eye icon will bring up a viewer showing a sample site data file for trace1 (if the file exists) allowing the user to see what header lines are present.

Pattern Data

The Pattern Data frame contains the widgets to specify data to provide a pattern for how to spatially disaggregate the input site. The Aggregated Flow File is a file containing a single time series of historical observed values for the input site. The Disaggregated Flows File is a file containing a column for each site that will be disaggregated to (values space separated in the row). Each column will be a time series of historical observed values for that disaggregation site. These file paths can be typed into the interface or chosen with the file chooser buttons. The eye icons will bring up a viewer showing the specifed files. The Location Count box is used to specify the number of sites to disaggregate to (the number of columns in the Disaggregated Flows File). The year of the first observation in the Disaggregated Flows File is entered into the First Year box.

Output Files

The Output Files frame contains a number of controls for specifying the output generated by the plugin. The output directory is determined by RiverSMART and is displayed as read-only here for information purposes (this directory is named with the name of the plugin and is located under the Hydrology subdirectory of the study's directory). A directory for each trace, named trace1, trace2, etc., will be created under the output directory.

The File Name Mapping controls are used to specify the names of the output files that will be generated (one name for each site that is disaggregated to). An entry can be put into the list by clicking the Add button and then that entry can be edited by double clicking on it to type in the file name (typically the name of the site). Alternatively, an entry can be chosen from a DMI Control File. The DMI Control File name can be typed in or chosen with a file selector button. The eye icon will allow the user to view the entire control file. The Add from Control File... button will then bring up a list of the slot names present in the control file and the user can select one that will be added to the file name mapping list. If units are specifed in the control file for the site, the units column in the list will be populated for informational purposes only. The order of the file names in the list must match the order of the columns of site data in the Disaggregated Flows File. The name entrys in the list can be reordered by selecting and using the up or down arrow buttons under the list. Selected entries can be removed from the list with the Remove button. If the number of entries in the list does not match the Location Count specification, a red warning note above the name mapping list is displayed.

The remaining controls in the Output Files frame are optional checkboxes. The Copy Header Lines from Input File checkbox allows the user to copy the header lines identified as present in the Input Traces over as header lines to the output files. Add Header Lines allows the user to type in additional header lines to be put into the output files. The Edit.. buttons brings up a text box to type in the additional lines, and when saved, the number of additional lines is displayed in the adjacent box.

R Installation Instructions

The R Project for Statistical Computing along with some of its component packages must be installed on the user's computer for Spatial Disaggregation to successfully generate ensemble flow data. See section 16 for instructions on downloading and installing R and its packages.


Temporal Disaggregation Plugin Help Information

Table of Contents

Overview

The Temporal Disaggregation plugin will take an existing hydrology ensemble consisting of trace directories each containing files of annual flow data for a number of sites, and temporally disaggrege the flow data to result in trace directories containing files of monthly flow data for the sites. These trace monthly flow files can then be imported via a DMI to a RiverWare multiple run. The scripts to temporally disaggregate flow data use the R Project for Statistical Computing program, so R must be installed on the user's computer (see R Installation Instructions).

User Interface

When the Temporal Disaggregation plugin is configured, a user interface is presented where the user gives the plugin a unique name. Other general controls in the interface are the Help menu item at the top that will bring up this help file, and the OK button at the bottom that will save the configuration and dismiss the dialog. The following section discusses the temporal disaggregation controls in the interface.

Temporal Disaggregation Interface


Input Traces

In the Input Traces frame, the user specifies information related to the ensemble of trace data for sites that are being input to the temporal disaggregation process. If the plugin is linked on the input side, the directory, number of traces, and flows file names list fields are disabled. In this case, these fields are automatically filled in when output from the upstream plugin is generated. If the temporal disaggregation plugin is not linked on the input side, these fields are enabled and must be filled in by the user. The Directory of Traces  is the directory that contains all of the trace folders for the input ensemble, and can be typed in or selected via the directory chooser button. The Number of Traces in the ensemble must be entered, and the names of the site data files that exist in each trace directory must be entered in the Flows File Names list. An entry can be added to the list with the Add button and then edited by double clicking on it to type in the name of the file. A selected entry can be removed from the list with the Remove button. The order of site names in the list does not matter. A site data file should be a single time series of flows for a desired site. The R scripts expect that a file will start with data values, so if there are non-data header lines in the site data files, the number of lines to skip needs to be specified by the user with the Number of Header Lines spin box. The eye icon will bring up a viewer showing a sample site data file for trace1 of the first site (if the file exists) allowing the user to see what header lines are present. 

Pattern Data

The Pattern Data frame contains the widgets to specify data to provide a pattern for how to temporally disaggregate the input sites. The Monthly Flow Directory is a directory containing a file of monthly flow data for each site in the Flows File Names list. Each file has to be named with the same site name as in the Flows File Names list. A file should contain a single time series of monthly historical observed values starting with January for that input site, and can contain no header rows. The first year of the monthly flow data needs to be specified in the First Year box. The optional Annual Flows Directory is a directory containing a file of annual flow data for each site in the Flows File Directory. Files in this directory must also be named with the same site names as in the Flows File Names list. These files contain a single time series of annual historic observed data with no header information. If the Annual Flows Directory option is not selected, the required monthly flow files are aggregated to get annual values for the calculations. The names of these directories can either be typed into the interface or chosen with the adjacent directory chooser buttons.

Output Files

The Output Files frame contains controls related to the output generated by the plugin. The output directory is determined by RiverSMART and is displayed as read-only here for information purposes (this directory is named with the name of the plugin and is located under the Hydrology subdirectory of the study's directory). A directory for each trace, named trace1, trace2, etc., will be created under the output directory. A monthly flows file for each site is then generated in each trace directory. The other controls in the Output Files frame are optional checkboxes. The Copy Header Lines from Input File checkbox allows the user to copy the header lines identified as present in the Input Traces over as header lines to the output files. Add Header Lines allows the user to type in additional header lines to be put into the output files. The Edit.. buttons brings up a text box to type in the additional lines, and when saved, the number of additional lines is displayed in the adjacent box.

R Installation Instructions

The R Project for Statistical Computing along with some of its component packages must be installed on the user's computer for Temporal Disaggregation to successfully generate monthly ensemble flow data. See section 16 for instructions on downloading and installing R and its packages.


RiverWare DMI Plugin Help Information

Table of Contents

Overview

The RiverWare DMI plugin allows the user to specify configuration information that is applied to a DMI in a RiverWare model. DMI is an acronym for Data Management Interface and represents a mechanism for importing data to a RiverWare model. Linking this plugin to the RiverWare plugin in the  RiverSMART workspace will result in this DMI configuration being included as a variable in the development of RiverWare scenarios. The DMI configurations made in the plugin will be passed to RiverWare before RiverWare execution and are used to modify the DMI in the model for that RiverWare invocation. 

User Interface

When the RiverWare DMI plugin is configured, a user interface is presented where the user gives the plugin a unique name. Other general controls are the Help menu item at the top that will bring up this help file, and the OK button at the bottom that will save the configuration and dismiss the dialog. A screenshot of the DMI interface follows:

 

Under DMI Configuration, there is a Name and a Category field. The Name is the name of the DMI in the corresponding RiverWare model. Select the name of a DMI in the RiverWare model. The pull-down menu lists the DMIs defined in the model that have that DMI Type. The Category is a user-defined category name. This serves as a way to group together DMI plugin instances on the RiverSMART workspace that import similar data (for instance Supply, Demand, etc.). This Category name is used when developing RiverWare scenarios, such that all Supply DMIs would be considered as alternative choices for a dimension named Supply when inputs to the RiverWare plugin are combined into alternative scenarios.

There are three types of DMIs that can be configured in the interface, selectable via the DMI Type radio button. The interfaces differ for these types and are discussed separately below.

Direct Connect DMI Interface

The interface presented when the user has clicked the Direct Connect radio button under DMI Type is as follows:

Direct Connect DMI Interface

The Execution Time radio buttons specify when the DMI should be executed within the invocation of RiverWare. Pre-run (Initialization) DMIs are executed once before the multiple run is started. Per Trace DMIs are run at the beginning of each trace (run) of the multiple run. If Per Trace is selected, the Trace Count box must be populated with the number of traces (runs) that the user wants in the multiple run. This number will be incorporated into the multiple run configuration to control the number of runs when RiverWare is invoked.

The Direct Connect Datasets list can be populated by the user to change some configuration items for datasets specified for this existing DMI in the RiverWare model. Note that only Excel datasets are currently supported by the plugin, and that the configurations that can be changed for a dataset are the configured workbook, or in the case of Pre-Run execution, the configured worksheet. The plus button will add an entry to the list and the minus button will remove a selected entry from the list. When an entry is added, a Dataset Name must be entered by double clicking on this column. The dataset name must match exactly the name of a dataset configured for the DMI in the RiverWare model. The Type column will be automatically populated with "Excel". The Workbook column can then optionally be double clicked and populated with the path and name of an Excel workbook that should be used as inputwhen the DMI is invoked instead of the one configured in the dataset. Similarly, if applicable, a worksheet name can be entered in the Sheet column to tell the dataset what sheet to use when invoked.

Control File DMI Interface

The interface presented when the user has clicked the Control File / Executable  radio button under DMI Type is as follows:

Control File Dmi Interface

The Execution Time radio buttons specify when the DMI should be executed within the invocation of RiverWare. Pre-run (Initialization) DMIs are executed once before the multiple run is started. Per Trace DMIs are run at the beginning of each trace (run) of the multiple run. If Per Trace is selected, the Trace Count box must be populated with the number of traces (runs) that the user wants in the multiple run. This number will be incorporated into the multiple run configuration to control the number of runs when RiverWare is invoked.

A different executable than the one configured for the DMI in the model can be specified by entering  its path and name in theExecutable field or by selecting it via the file chooser button. This is the executable that will then be invoked when the existing DMI in the model is run. Executables are typically developed by users to do certain data-related tasks. In the example, the executable is a Perl script that copies trace data to a certain location.

The Argument List is a text box where arguments can be entered (one per line) that will be passed to the DMI's executable when the DMI is run. In the example, a trace directory is being passed to tell the DMI executable named CopyTraces.pl where to find the traces it will be copying.

Trace Directory DMI Interface

The interface presented when the user has clicked the Trace Directory radio button under DMI Type is as follows:

Trace Based Input Dmi Interface

The Execution Time radio buttons specify when the DMI should be executed within the invocation of RiverWare. Pre-run (Initialization) DMIs are executed once before the multiple run is started. Per Trace DMIs are run at the beginning of each trace (run) of the multiple run. If Per Trace is selected, the Trace Count box must be populated with the number of traces (runs) that the user wants in the multiple run. This number will be incorporated into the multiple run configuration to control the number of runs when RiverWare is invoked. Per Trace is the configuration that makes sense for Trace Directory DMIs (if Pre-Run  is selected only the information for trace one will be imported prior to the multiple run being started).

The Top Directory Containing Traces must be specified by typing the name in the field or selecting it via the drectory chooser button. This is the top-level directory that contains the trace directories that will be imported. It is assumed that trace directories are labeled trace1, trace2, etc. Specification of this top-level directory allows RiverWare to find the trace data without an executable being specified as is required for the Control File / Executable DMI.

The Control File is specified by typing the file path and name in the field or selecting the control file via the file chooser button. This control file will be used as a basis for RiverWare to create temporary control files for each trace of the multiple run. The slots to be imported would be listed along with any keyword pairs, like units and scale, that can be used with control files in Control File/Executable DMIs. The file keyword and value can be specified or not for the slots. If specified, the path up to the file name will be substituted with the path to the appropriate trace directory when the DMI is run in RiverWare. If no file keyword is used, RiverWare will insert the trace path with the %o%s file name, meaning the name of the file is expected to match the object and slot name.


RiverWare MRM Plugin Help Information

Table of Contents

Overview

The RiverWare MRM plugin allows the user to specify the name of a Multiple Run Management (MRM) configuration that has been set up in a RiverWare model. Linking this event to the RiverWare event will result in this MRM configuration being included as a variable in the development of RiverWare scenarios. Note that the MRM configuration must specify that runs be distributed to work correctly with RiverSMART.

User Interface

When the RiverWare MRM event is configured, a user interface is presented where the user gives the event a unique name. Other general controls in the interface are the Help menu item at the top that will bring up this help file, and the OK button at the bottom that will save the configuration and dismiss the dialog. The interface appears as follows:


The MRM configuration name can be selected from the pull-down menu, which lists all defined MRM configurations in the specified RiverWare model.

User-defined MRM deescriptors can be added with the plus button or a selected one can be deleted with the minus button. The Keyword and Value fields are editable by double clicking on their fields. These are provided because an MRM configuration could actually include policies and DMIs that go together (such as VIC demand, VIC supply, and VIC operations) instead of having these broken out as separate input events in RiverSMART. In this case, it may be useful to associate one or more keyword/value pairs with the configuration to describe the inputs it represents. In the above example, a DMI in the MRM configuration brings in a hydrology based on climate change projections, so it is given a descriptor that the Supply is Climate Change. When a scenario is simulated, these descriptors are written to the MRM configuration in the RiverWare model as MRM descriptors and will appear in the Description tab of the MRM configuration dialog of the model that is running the scenario. MRM descriptors can optionally be configured to write to CSV and netCDF files that may be outputs from the multiple run for the scenario.


RiverWare Model Plugin Help Information

Table of Contents

Overview

The RiverWare Model plugin allows the user to specify a RiverWare model file. Linking this plugin to the RiverWare plugin will result in this model being included in the development of RiverWare scenarios.

User Interface

When the  RiverWare Model plugin is configured, a user interface is presented where the user gives the plugin a unique name. Other general controls in the interface are the Help menu item at the top that will bring up this help file, and the OK button at the bottom that will save the configuration and dismiss the dialog. The interface appears as follows:.

RiverWare Model Plugin Interface

The model file and its path can be typed into the user interface or can be selected with the adjacent file chooser button.


RiverWare Policy Plugin Help Information

Table of Contents

Overview

The RiverWare Policy plugin allows the user to specify a RiverWare policy set (RPL Set). Linking this plugin to the RiverWare plugin will result in this policy being included in the development of RiverWare scenarios.

User Interface

When the  RiverWare Policy plugin is configured, a user interface is presented where the user gives the plugin a unique name. Other general controls in the interface are the Help menu item at the top that will bring up this help file, and the OK button at the bottom that will save the configuration and dismiss the dialog. The interface appears as follows:

RiverWare Policy Plugin Interface

The RPL Set file path and name can be typed into the interface or can be selected with the adjacent file chooser button. Any separate global function sets that are needed for the RPL set should be specified by using the Add... button to choose the global set and add it to the Global Function Set list. The text for an entry in the list can be edited by double clicking on the entry, and selected entries can be removed with the Remove button.


RiverWare Run Range Event Help Information

Table of Contents

Overview

The RiverWare Run Range Event provides a mechanism to configure a study in which scenarios have different run ranges (e.g., different start or end dates). The event configure a series of run ranges which are used to generate scenarios. In describing the event it is helpful to use an example, as follows:

For the sample study there are 12 run ranges with a monthly timestep, beginning with each month from 1980-01 through 1980-12. The run duration is 24 months, with the exception that the policy requires that the run ranges extend through at least May of the end year. In tabular form, there are 12 run ranges:

Start Date End Date # Timesteps
1980-01 1982-05 28
1980-02 1982-05 27
1980-03 1982-05 26
1980-04 1982-05 25
1980-05 1982-05 24
1980-06 1982-06 24
1980-07 1982-07 24
1980-08 1982-08 24
1980-09 1982-09 24
1980-10 1982-10 24
1980-11 1982-11 24
1980-12 1982-12 24

Each of the 12 run ranges is a unique input to the study and is used to generate scenarios. Continuing with the example, if the study has 2 RiverWare Policy events, 4 RiverWare DMI events with a Supply category and 2 RiverWare DMI events with the Demand category, then the study has  12 x 2 x 4 x 2 = 192 scenarios.

User Interface

The Run Range Event configuration dialog is shown below with values consistent with the example:

The dialog has a help menu to access this help document. It also provides the event with a unique name (here "Run Range"). It also has the controls to configure the series of run ranges, as follows:

Label An optional label which is prepended to the run range's name, either LABEL-YYYY-MM-DD or LABELYYYYMMDD (as determined by the name format). If a study contains multiple run range events whose dates overlap, the label can be used to distinguish the events.
Name Format The name format specifies how a run range's start date is formatted to form the run range's name (which in turn is used to form scenario names). Although the dialog shows full YYYY-MM-DD precision, only the precision necessary for the simulation timestep is used (YYYY, YYYY-MM or YYYY-MM-DD).
Timestep The simulation timestep (yearly, monthly or daily), which must match the model's simulation timestep.
Start Date The start date of the first run range in the series. Only the precision necessary for the simulation timestep is used (YYYY, YYYY-MM or YYYY-MM-DD). If the timestep is yearly or monthly the date is edited by selecting the year or month and either typing the new value or clicking the up/down arrows: . If the timestep is daily, the date is edited either by selecting the year, month or day and typing the new value or by clicking the pull-down menu to open a calendar dialog to select the date: .
Duration The duration of each run range in the series. For a yearly timestep the duration is specified in years, for a monthly timestep it's specified in years or months and for a daily timestep it's specified in years, months or days.
Start Date Increment The offset added to the start date of each run range in the series to determine the start date of the next run range in the series. For a yearly timestep, the increment is specified in years. For a monthly timestep, it is specified in years or months. For a daily timestep, it is specified in years, months or days.
Number of Runs The number of  run ranges in the series.
R Language Script An optional R Language script which can adjust the run ranges for policy specific requirements (described in detail below).

For daily timestep durations or start date increments specified in years or months, the results might have unintended offsets. For example, if the start date is 1980-01-31 and the start date increment is 1 month, the start dates of the first few run ranges in the series will be (assuming a non-leap year) 1980-01-31, 1980-02-28 and 1980-03-28, not the last day of each month (as might have been intended).

R Language Script

In the example, the policy requires that run ranges extend through at least May of the end year, which means that the run ranges beginning in January through April have durations greater than 24 months. This type of policy specific behavior ca not be captured in this configuration; instead the Run Range Event enables you to provide an R language script file which can adjust the run ranges as necessary. The R language script file must define the function:

adjustDates <- function(inputFile, outputFile)
{
}

The inputFile and outputFile parameters are the full paths to the function's input and output files, described below. Other aspects of the function are also described below. The workflow is:

  1. The event generates the run ranges from its configuration. The run ranges adhere to the duration and start date increment.
  2. The event writes the run ranges to a CSV file (whose format is described below).
  3. The event invokes Rscript.exe to execute the adjustDates() function. The function's inputFile parameter is the full path to the file written in step 2, its outputFile parameter is the full path to a file which the function is expected to write.
  4. The adjustDates() function parses its input file, adjusts the run ranges as necessary, and writes its output file - a CSV file whose format is identical to that of its input file.
  5. If the adjustDates() function finished successfully, and if it wrote its output file, the event parses the file and validates the adjusted run ranges contained in the file. If the adjusted run ranges are valid, the event replaces the run ranges it generated with the adjusted run ranges.

adjustDates() Function

The function can print informational messages and error messages which will be displayed in RiverSMART's diagnostic window. (The information messages will only be displayed if RiverSMART diagnostics are enabled.) There are undoubtedly many ways to do this in the R language; one way to do this is to define the functions:

info <- function(...) cat(sprintf(...), sep='', file=stdout());
error <- function(...) cat(sprintf(...), sep='', file=stderr())

which can be called as one would call the R language sprintf() function.
The function's exit status indicates whether it finished successfully or not:

# Successful:
quit(, 0, )

# Error:
quit(, 1, )

An error exit status should be accompanied by error message(s).

CSV File Format

The adjustDates() function's input and output CSV files contain 10 fields:
  1. Run number (1 .. N)
  2. Run name
  3. Timestep (year, month or day)
  4. Start year
  5. Start month
  6. Start day
  7. End year
  8. End month
  9. End day
  10. Number of timesteps
The CSV files contain full precision (yyyy,mm,dd) even if the timestep doesn't require it. For the example, the first few lines of the adjustDates() function's input file are:

run_number,run_name,timestep,start_year,start_month,start_day,end_year,end_month,end_day,num_timesteps
1,1980-01,month,1980,01,31,1982,01,31,24
2,1980-02,month,1980,02,29,1982,02,28,24
3,1980-03,month,1980,03,31,1982,03,31,24

Preview

Understanding the run ranges the Run Range Event generates and how the R language script adjusts them would be difficult. Users would need to generate the scenarios and then compare 2 CSV files in the event's working directory. To address this difficulty the event provides a preview capability, which executes the workflow described above and presents the results in the preview dialog. Previewing the example shows the run ranges extended through May of the end year:

Preview Dialog

The initially generated run ranges are shown in blue, the adjusted run ranges are shown in green and the differences are highlighted in violet. The preview capability makes it easy to understand the run ranges and how the R language script adjusts them.

DMI Integration

In some uses of the Run Range Event, run ranges are associated with DMIs. For example, the run range 1980-01 might have different initial conditions than the run range 1980-02. Users would organize their data to include the run range names, for example, in trace directories:

C:/Study/InitCond/1980-01
C:/Study/InitCond/1980-02
...

Or, alternatively, in an Excel workbook with sheets named "1980-01", "1980-02", ...

To facilitate configuring the DMI events, DMI event configurations can include the variable %RunRange%, which is replace with the run range's name. This shows a Trace Directory DMI using the variable:

DMI Configuration

With this configuration, the scenarios using run range 1980-01 will import the initial conditions in C:/Study/InitCond/1980-01, the scenarios using run range 1980-02 will import the initial conditions in C:/Study/InitCond/1980-02 and so on. The DMI event configuration fields which support the %RunRange% variable are:

 


RiverWare Plugin Help Information

Table of Contents

Overview

The RiverWare plugin allows the user to specify the RiverWare executable that will be used when RiverWare is invoked by RiverSMART.

User Interface

When the RiverWare plugin is configured, a user interface is presented where the user gives the plugin a unique name. Other general controls in the interface are the Help menu item at the top that will bring up this help file, and the OK button at the bottom that will save the configuration and dismiss the dialog. The interface appears as follows:

RiverWare Process Interface

The RiverWare Executable path and name can be typed into its field or selected via the file chooser button.



RdfAnnualizer Plugin Help Information


Table of Contents


Overview

The RdfAnnualizer plugin will use a RiverWare Data Format (RDF) file with a timestep of less than a year and will aggregate its data to generate an RDF file with an annual timestep. Methods for aggregating data for the slots in the RDF file are specified via a method control file. The user also specifies the end month for the aggregated year, so calendar years or water years can be generated, for example.

Source RDF File

RDF files are generated from RiverWare and contain data for slots specified by the user. Non-series slots can be written to RDF files, but only data for series slots will be processed by the RdfAnnualizer tool. The exact format of an RDF file is described in the Output and Plotting section of RiverWare Online Help. An RDF file for results from a single run in RiverWare can be created from the Output Manager via a RiverWare Data File device. Slot data for a multiple run in RiverWare can be output to an RDF file by configurations in the Output tab of the MRM Configuration dialog.

The source RDF file should contain series data with a timestep of less than a year.Currently in RiverWare this would include timesteps of  1 hour, 6 hour, 12 hour, daily, weekly, and monthly. If a yearly timestep file is specified as input to the RdfAnnualizer, an error message will be returned that the run is already in an annual timestep.

Method Control File

The method control file specifies the slots to be processed and the annualization method associated with the slots. The file is a text file and the syntax is for each line to contain an entry of the form:
Object.Slot: Method
where Object can be:
where Slot can be:
and where Method can be any of the following:
The behavior of the non-Nan methods is to report NaN for the summary value if any of the input values for the year are NaN. The Nan suffix on a method means that NaN input values are ignored in calculating the summary value.

If the same slot is identified more than once in the method control file through specific declarations or wildcards, the method is set at the first identification and not changed thereafter.

Example Method Control File

An example method control file might contain the following entries:

*.Outflow: Average
*.Pool Elevation: End
Big Reservoir.Evaporation: SumNan
DataObj.*: EndNan

Outflow for all objects in the file will be averaged over the year and if any outflows are NaN, the result for the year will be NaN. Pool Elevation for all abjects in the file will be the last value for the year and if any values over the year are NaN, the result will be NaN. For the object named Big Reservoir, its evaporation slot will be summed over the year, and if any values are NaN, they will be skipped and the remaining values will be used in the sum. For all objects in the file that are data objects, all of their slots will be assigned the last value of the year whether there are any NaNs or not in the data over the year.

End Year Month

The RdfAnnualizer plugin will take a month specification that becomes the end of year for the annualization calculations that generate the result RDF file. The default is December for annualizing to calendar years, but any month can be specified, such as September for water years. Another example would be if you want an RDF file with storage for reservoirs at the end of March, you could annualize with March as the end month and use the End method to get the last value for the year.

Note that if the timesteps in the source RDF file generate partial years at the beginning or end of the time range when annualized into the defined year, the partial years are dropped when the result RDF file is written. The result annual RDF file only contains data for full annualized years.

Result Annual RDF File

The result RDF file contains all of the same descriptive fields as the original source RDF file, such as name, owner, description, etc., and will contain the same number of runs. Note that only slots from the original file that have a method defined in the method control file will be annualized, so the output RDF file could have  fewer slots than the input file (also any non-series slots in the source RDF file will not be annualized). The time_step_unit field is written as year, and the end times for the specific annualized years are listed as the new timesteps for the file.

The same slot header information is listed in the result file for annualized  slots, except a method field is added to show the specific method for how that slot's data was annualized. Also, if the method was Sum or SumNan, the units field for the slot has the phrase "summed over the year" added to it (such as "acre-ft/month summed over the year"). Otherwise, the result annual RDF file looks exactly like an RDF file that was output from a RiverWare model with a yearly timestep.

User Interface

When the RdfAnnualizer plugin is activated, the following user interface is presented (note that the source RDF file and the output annual RDF file are configured in the program calling the plugin, so are not part of this interface):

RdfAnnualizer Plugin Interface

The Method Control File can be indicated by either using the associated "Select..." button to open a file chooser dialog, or by typing in the path and file name into the file's text field.  An environment variable of the form $NAME, for example, can be typed in and the NAME environment variable defined on the system would then be substituted into the Method Control File path when the program is run.

The End of Year Month is a combo box where the month defining the end of the annualized year can be selected by the user.

RdfToExcel Plugin Help Information

Table Of Contents


Overview

The RdfToExcel plugin will use a RiverWare Data Format (RDF) file and convert its contents into an Excel workbook using user-specified configuration options. This is accomplished, in part, by automating Excel on the user's computer, so the tool is Windows only, and the user must have Excel. installed. Since the tool automates whatever version of Excel the user has installed, the tool is not tied any particular version of Excel and its associated workbook format or limitations.

RDF Files

RDF files are generated from RiverWare and contain data for slots specified by the user. Non-series slots can be written to RDF files, but only data for series slots will be processed by the RdfToExcel tool for writing to Excel. The exact format of an RDF file is described in the Output and Plotting section of RiverWare Online Help. An RDF file for results from a single run in RiverWare can be created from the Output Manager via a RiverWare Data File device. Slot data for a multiple run in RiverWare can be output to an RDF file by configurations in the Output tab of the MRM Configuration dialog.

Excel Workbook

When an RDF file is written to an Excel workbook, a Header worksheet is created followed by sheets containing the slot data. These are discussed below along with the available slot name options.

Header Sheet

The header sheet contains a summary of the information in the RDF file that is not slot data. This includes:
File name
Owner
Description
Creation Date
Number of Runs
Number of Slots
Number of Timesteps
Information for Each Run
Information for Each Slot
Listing of the Timestep Dates

Data Sheets

The appearance of the data sheets depends on the workbook orientation option selected by the user. A workbook has three "dimensions", rows, columns, and worksheets, that can be mapped to the data dimensions of timesteps, slots, and runs. Typical orientations would be to put timesteps as rows, slots as columns, and runs as sheets, or timesteps as rows, runs as columns, and slots as sheets. In this last orientation, for example, the first column would contain a timestep label  in each row, the first row would contain a run label in each column, each sheet would be labeled as a slot, and cells in the sheets would contain the corresponding timestep data as indicated by the header labels and sheet name. Some orientations are more common, but any of the six possible orientations are available.

Slot Names

Because slot names from RiverWare can be very long, fitting them into the workbook can be problematic, most notably in the case where they are sheet tab labels, which are limited to 31 characters. For this reason, the tool provides three options for writing slot names to the workbook:
Slot Index Labels
Full Slot Name (truncated if necessary)
Automatically Shortened Slot Names

Slot index labels are Slot0, Slot1, Slot2, etc. The index names are mapped to the actual slot names on the Header sheet, but the index labels are used in the data sheets.

The full slot names are the object and slot name concatenated. If these are used on sheet tabs, colons are removed because these are not legal characters on tabs. Since the number of characters on tabs is limited to 31, the name is truncated to 30 characters to fit, if necessary, and a '~' is appended to the end to show it is truncated.. If this truncating does not make the name unique with respect to other slot names, a sequential number is prepended to the name to make it unique. Note that the full object slot name is placed into the first cell (A1) of the sheet so the actual slot can be easily determined..

Automatically shortened slot names are shortened according to the following criteria:
"And" in the name is replace with "&"
Colons and spaces are removed
All lower case vowels are removed

As an example, PowellStorage becomes PwllStrg. If the shortened name is used on sheet tabs and it still exceeds 31 characters, it is truncated to 30 characters and a '~' is added to the end to indicate the name is shortened. If the truncated name is not unique with respect to other slot names, a sequential number is prepended to the name to make it unique. The full object slot name is placed into the first cell (A1) of the sheet so the actual slot can be easily determined.

User Interface

When the RdfToExcel plugin is activated, the following user interface is presented (note that the source RDF file and the result workbook are configured in the program calling the plugin, so are not part of this interface):

RdfToExcel Plugin dialog

The Workbook Orientation frame of the dialog provides a combo box with choices of how the workbook's rows, columns, and worksheets will map to the RDF file's timesteps, runs, and slots. The following choices are available:

Orientation Choices

If slots are selected to go onto worksheets, then the Slot Labels for Sheet Tabs frame of the dialog is visible as shown above. With radio buttons, the user selects which slot name option to use for the worksheet's tab labels.

If slots are not selected to go onto worksheets in the Workbook Orientation combo box, then the Slot Labels for Sheet Tabs frame is not visible, and instead the user gets a checkbox option for naming as shown below:

Slot Name Checkbox

If the checkbox is checked, automatically shortened slot names are used in the header columns or rows, depending on the orientation. This may be desired even though the number of characters are not limited in this situation as in sheet tab labels in order to make the slot names discernible without having to widen the cells so much. If the checkbox is unchecked, the full slot name (object and slot name concatenated) is used in header cells.

GPAT Graphs Plugin Help Information


Table of Contents


Overview

The GPAT Graphs plugin will take Excel files with series data from RiverWare or other data sources and generate graphs in an Excel output file according to graph configurations specified in a GPAT graphs configuration file. The Graphical Policy Analysis Tool (GPAT) is an add-in to Excel, so Excel must be available on the system and the GPAT add-in must be installed in Excel for the plugin to function. A GPAT graphs configuration file is created in GPAT and is specified in the plugin interface to indicate what graphs are to be produced.

User Interface

When the GPAT Graphs plugin is activated, the following user interface is presented (note that the source Excel data files and the output Excel file with graphs are configured in RiverSMART, so are not part of this interface):


GPAT Graphs Interface

The GPAT Graphs configuration file indicating what graphs should be generated needs to be specified by typing in the path or choosing the file with the Select... button. This is an XML-based file that is created in GPAT and records graph configurations created in the GPAT interface. If GPAT is installed in Excel on the system, the Open GPAT... button will open the GPAT interface where configuration files can be created or modified. If a configuration file has been specified, GPAT will be opened with that configuration file already loaded.

The next field allows for optionally typing in or selecting an Excel workbook for the copy result option in GPAT. More information on this GPAT option is available in the GPAT help file. If graphs have been saved into the configuration file that utilize the copy result option, then the workbook that results will be copied into needs to be selected here. Otherwise, selection of the copy result workbook is not applicable.

The optional "Process input workbooks by Scenario Set" control allows input workbooks from across scenarios to be processed together into graphs in an output workbook. The Scenario Set list will show all scenario sets that have been defined in the study (these are defined in the Scenario Set Manager within RiverSMART, which allows grouping of scenarios into defined sets). When this option is selected, each scenario set that is checked in the list will be processed individually by the GPAT plugin. When a set is processed, the input workbook is found for each scenario in the set and these are passed as inputs to GPAT. GPAT will then create graphs specified in the selected configuration file for the input data from the scenarios in the set, and write the graphs and associated data to the scenario set directory in RiverSMART with the linked output workbook name. If more than one scenario set is checked, processing then moves to the next set and its scenarios are processed into graphs written to an output workbook in its scenario set directory.

Source Excel Data Workbooks

In RiverSMART, one or more Excel workbooks can be linked as source data for the GPAT graphs plugin. As explained in the GPAT Help, source data workbook requirements are that rows represent timesteps, the first column of a worksheet contains the timestep names, and the first row of a worksheet contains the column names. In RiverWare, generating workbooks of this type directly can be accomplished by specifying an Excel output device through the Output Manager or by specifying the creation of Excel files in the Output panel of the Multiple Run Management Configuration dialog. RiverWare output files (.rdf) can also be converted to Microsoft Excel files using the RdfToExcel plugin or RdfToExcel stand-alone tool distributed by CADSWES. Note that column names and worksheet names that are specified in the saved graph configurations in the GPAT Graphs configuration file must be present in the source data workbooks for the graphs to be successfully generated by the plugin.

Output Excel Workbook 

The output Excel workbook is specified by an output link from the GPAT Graphs plugin in RiverSMART. The output for an invocation of the GPAT Graphs code is a single workbook that contains all of the graphs generated by the execution of the plugin based on the specified graphs configuration file. Each graph has a data worksheet that contains the data for the graph, and an adjacent graph worksheet that contains the Excel plot. If an existing workbook is specified as the output workbook, the graph worksheets are added to the workbook. Existing worksheets in the workbook are not deleted before new graphs are created.

A log file is generated in the plugin's Working directory under a subdirectory having the name of the processed scenario or scenario set. This file will contain any informational messages, warnings, or error messages generated during creation of the graphs by plugin execution.

R Plugin Help Information

Table of Contents

Overview

The R Plugin allows scripts in the R programming language to be executed as a post-processing event in a study. R must be installed on the user's computer  in order to utilize the R Plugin (R Installation Instructions).

The user specifies an R script file as well as a function that is defined within the specified R script file. The user can also specify arguments to pass to the function. When the plugin is executed, it will call the function and pass the arguments that have been defined..

There are no formal restrictions on the user-defined R function that can be called. The R Plugin does, however, provide utilities to facilitate the integration of the post-processing outputs from R into the general RiverSMART file management framework. This is done by making predefined arguments available that pass information such as scenario names, linked input files and output file locations. (Note that in this R Plugin Help document, "input files" refer to files that are inputs to the R Plugin. These will be outputs from previous events in the study, such as a RiverWare event or an RDFtoExcel event.)

Using the R Plugin

An R Plugin event can be added to the post-processing of scenario data in RiverSMART. When the R Plugin is executed, it will run a user-defined R script in the background (i.e. no R interface will be visible).

The file types defined and used in post-processing, RDFs and Excel Workbooks, are linkable to the R Plugin as input files. It is up to the user's R script to process the data provided within any input files. The R Plugin is not linkable in the interface to any files on its output side (i.e. the R Plugin will be the terminus of its post-processing stream), but there is no limit to the number of R Plugin events in a study (each as its own terminus).

RiverSMART will provide an output directory as an optional predefined argument to the plugin. The user's R script can use this specified output directory as a location to save any files generated by the R code, but RiverSMART will not link these files into subsequent processes. Though RiverSMART does not force files generated by the R code to be saved in the predefined output directory, it is highly recommended that R scripts called from RiverSMART be written to save all output files to this location for two reasons. First, the predefined output directory will guarantee that R ouput files will be saved in a file structure that is consistent with the general RiverSMART file management framework. Second, writing files to another location within the RiverSMART directory structure could potentially result in file naming conflicts, or it could result in the unintentional deletion of output files if RiverSMART clears a directory of its contents as part of the re-execution of a portion of a study.

Steps to Use an R Plugin Event

Outside of RiverSMART

  1. Install R
  2. Create R script: The R script must contain, at a minimum, a single user-defined function that matches the function named in the plugin interface. The script may contain additional commands before or after the function definition. These commands will get executed when the script is sourced from the RiverSMART-generated R script, but RiverSMART will only pass arguments for the single function call. The user-defined R script is responsible for all processing of the data, including reading the data from the input files. The user's script can make use of any R functionality including sourcing other script files, calling other functions or reading data from other files outside of the RiverSMART directory structure.

Within RiverSMART

  1. Add R Plugin Event to the Workspace
  2. Link Input Files: one or more RDF files or Excel Workbooks on the RiverSMART workspace that are outputs from another RiverSMART event
  3. Specify R Script
  4. Specify R Function
  5. Specify Arguments and their Values
  6. Specify How to Process Scenarios
  7. Generate and Simulate Scenarios (if not done previously)
  8. Post-Process Scenarios

Linking Files

RDF and Excel Workbooks can be linked to the R Plugin as input files. Multiple input files can be linked to a single R Plugin event, and a single input file can be linked to multiple R Plugin events. The R Plugin is not linkable in the interface to any files on its output side (i.e. the R Plugin will be the terminus of its post-processing stream).

The linked input files will be used by RiverSMART to define the values of the <linked input file paths> and <linked input file names> predefined arguments.

User Interface

Once an R Plugin event has been added to a study, double clicking on the R Plugin Event icon R Plugin Iconwill open the R Plugin Configuration dialog. The R Plugin Event can be given a name at the top of the dialog. This name will be displayed below the event icon on the RiverSMART workspace.

The R Plugin Configuration dialog contains three panels. In the first, the R script file and function are specified. In the second, arguments and their values are defined. In the third, the method for processing scenarios is specified. Each of these is described in greater detail in the following sections.

R Plugin User Interface

Specifying an R Script and Function

In the top panel, the user must specify an R script file in the File field in the R Function panel of the configuration dialog. The plugin will create its own R script file that will source this user-specified script file when the plugin is executed. The file can be selected either by clicking on the Select ... button, which will bring up a file chooser, or by typing in the file path. Environment variables can be used in the file path, preceeded by a "$" as shown in the image below. Once a valid file path has been specified, the preview button Preview Button will become active. Clicking on the preview button will open a separate window with a view of the specified file.
Specify an R script file and an R function within the file
In addition to an R script file, a function that is within that script file must be specified. This function allows the plugin to pass user-specified arguments to the R script. The Plugin will call the function after sourcing the specified R script file. (Click here to see the form of the RiverSMART-generated R script.)

It is the user's responsibility to write an R script and function that contain appropriate R syntax. There are no limits, however, to what can be included within the script other than it must contain the specified function, and the function must accept the arguements specified in the R Plugin Configuration dialog. The user-created R script can source other files, call other functions or require any R packages that the user has installed.

Arguments

Argurments can be passed in one of two manners: By Name or By Position. The selection is made by clicking on the appropriate radio button in the Arguments panel of the Configuration dialog.

Specify Arguments By Name or Position

If "By Name" is selected, the user must specify a name for each argument in the plugin that exactly matches the name in the function definition in their R script. In this case the order of the arguments in the R Plugin Configuration does not matter.

If "By Position" is selected, the argument is not given a name in the R Plugin Configuration. Instead, it is given an index number, its position in the list of arguments. In this case, the order of the arguments in the Plugin must match the order of the arguments in the function definition. In general, the By Name option is recommended.
A new argument can be added to the list by clicking on the green plus button Add an argument. A selected argument can be removed by clicking on the red minus button Remove an argument. The order of arguments in the Plugin can be adjusted by clicking on an argument to highlight it then clicking on the up/down arrows Up and Down Arrows

Adding an argument to the list will activate the Edit panel. In the Edit panel, the name (if using By Name), type, description and value for the argument can be specified. The name for the argument can be any valid name in the R programming language. All alphanumeric symbols are allowed plus ‘.’ and ‘_’, with the restriction that a name must start with ‘.’ or a letter, and if it starts with ‘.’ the second character must not be a digit.

There are two general classes of arguments: User-defined and Predefined, which are described below. The options for editing the type, description and value depend on the selection of User-defined or Predefined. The class of argument is set by clicking on the appropriate radio button.
User Defined vs Predefined

User-defined Arguments

When a User-defined argument is selected, the user can then select the type by clicking on the Type dropdown menu:
Type Menu
The type of argument must match the type that is expected by the user's function that is being called by the R Plugin. The available argument types are described below.

An optional description of the argument can be typed into the Description field. This is for the user's reference only.

The value for the argument can be entered in the Value field.  Values for numeric, integer and character arguments can either be a single value or a vector with the individual elements separated by commas. Note that it is not necessary to type in the R concatenate command, c(...), for a vector of values. Only the commas separating the values are required. The plugin requires a value to be specified for all arguments included in the list.
Numeric :
A numeric argument passes a double precision vector of values (the vector could have a length of 1). Possible valid entries for numeric argument values are shown below.
Numeric Scalar Argument   Numeric Vector Argument
Integer:
An integer argument is the same as a numeric, but any digits after a decimal point will be truncated (i.e. all values will be rounded down to the nearest whole number). So a value entered as 12.75 would get passed to the R function as 12.
Character:
A character argument allows a character string or a vector of character strings to be passed. Possible valid entries for character argument values are shown below.
Single Character Argument    Character Vector Argument
The elements of the character argument can be written with or without quotation marks. They will be treated the same in both cases. For example, the following would produce the same result as the characterVector argument definition shown above:
Character Vector argument without quotation marks
Logical:
A logical argument passes either TRUE or FALSE. When the logical type is selected, the appropriate TRUE/FALSE radio button can be selected to set the value for the argument. A logical argument cannot contain multiple values.
Logical Argument
R Expression:
The R expression argument is generic and can contain any valid R expression. Technically, any of the previous arguments could be defined as an R expression argument, but this type also allows commands that are part of the R programming language to be passed as an argument value. For example, a vector of a single value repeated 10 times could be passed through an R expression argument using the R rep() function.
R Expresssion Argument
It is up to the user to make sure that the R expression argument is specified using appropriate R syntax. Note that it is possible to use other arguments defined previously in the list within an R expression. For example, say that a previous argument was of type integer with a name N and a value of 10. The expression for the repeatYears argument shown above could be written as
rep(2015, N)
Function Name:
The final User-defined argument type is a function name. With this argument, the name of a function is passed as an argument to the function being called by the plugin. For example, in one season, values may be processed in a specific way, but another method may be used in other seasons.
Function Name Argument
The function name specified could be either a user-defined function or a predefined function in the R programming language. In the case of a user-defined function, it is up to the user to make sure that the function is defined within their script. For a pre-defined R function, the user must make sure that any required packages are loaded by their script. No arguments are passed within the function name argument. Only the function name is passed.

Predefined Arguments

When a pre-defined argument is selected, the user chooses which predefined argument they would like and gives the argument a name (if using By Name). The argument's type, description and value are fixed (filled in by RiverSMART). 
Predefined Argument Menu
The actual value of the argument that will be passed must be determined at runtime, so within the argument list in the Configuration dialog, the generic argument value will be shown (e.g. <linked input directory>). Examples of actual values that will be passed by the predefined arguments are shown in the table below.

Assume for this example that the study has two supply secenarios (Wet and Dry) and two demand scenarios (HighDemand and LowDemand), for a total of four Supply + Demand scenarios. There are two linked rdf files: UpperBasinOutput.rdf and LowerBasinOutput.rdf. The scenarios are processed by a scenario set called "AllScenarios" that includes all four  scenarios. The R Plugin event is named "BasinResultsR."
Argument Value Passed to R
<linked input directory> “C:/users/UserName/RiverSMARTFiles/MyStudy/Scenario/”
<scenario names> c(“Dry,HighDemand”, “Dry,LowDemand”, “Wet,HighDemand”, “Wet,LowDemand”)
<number of scenarios> 4
<linked input file names> c("UpperBasinOutput.rdf", "LowerBasinOutput.rdf")
<linked input file paths> c("C:/users/UserName/RiverSMARTFiles/MyStudy/Scenario/Dry,HighDemand/UpperBasinOutput.rdf",
"C:/users/UserName/RiverSMARTFiles/MyStudy/Scenario/Dry,HighDemand/
LowerBasinOutput.rdf",
"C:/users/UserName/RiverSMARTFiles/MyStudy/Scenario/Dry,LowDemand/
UpperBasinOutput.rdf",
...)

<output directory> “C:/users/UserName/RiverSMARTFiles/MyStudy/ScenarioSet/AllScenarios/BasinResultsR/”

All predefined arguments are of the character type, with the exception of <number of scenarions>, which is an integer.

The <linked input directory> will always be the Study Folder specified in the main RiverSMART workspace with the /Scenario/ sub-directory appended. The use of the term "linked input" indicates that this directory contains files that will be inputs to the R script. The linked input directory will contain the "raw" RiverWare outputs (typically rdf files) or outputs from another RiverSMART event.

The <scenario names> will match the scenarios defined by the selected scenario set, if Scenario Processing By Scenario Sets is selected. If Scenario Processing By Scnenarios is selected, the <scenario names> argument will be a single scenario name each time it is passed.

The <number of scenarios> argument will be an integer equal to the length of the <scenario names> argument.

The <linked input file names> argument contains just the general file names (as seen in the RiverSMART workspace). It does not contain a separate element for each individual scenario.

The <linked input file paths> argument allows the actual files to be fully specified for each individual scenario. The length of this character vector will be <number of scenarions> * length<linked input file names>.

The <output directory> argument specifies a file path where outputs from the R script can be saved. RiverSMART will create the director but otherwise does nothing with this argument directly after passing it to R. It is up to the user to incorporate this argument into their R script commands that write output files. RiverSMART does nothing to enforce the use of this output location; however it is highly recommended that R scripts called from RiverSMART be written to save all output files to this location for two reasons. First, the predefined output directory will guarantee that R ouput files will be saved in a file structure that is consistent with the general RiverSMART file management framework. Second, writing files to another location within the RiverSMART directory structure could potentially result in file naming conflicts, or it could result in the unintentional deletion of output files if RiverSMART clears a directory of its contents as part of the re-execution of a portion of a study.

For R Plugin events with processing by Scenario Sets the <output directory> will be the Study Folder specified in the main RiverSMART workspace with the /ScenarioSet/<Scenario Set Name>/<R Plugin Name>/ sub-directory appended. If processing by Scenarios has been selected, the <output directory> will be the Study Folder with the /Scenario/<Sceanrio Name>/<R Plugin Name>/ sub-directory appended.

Scenario Processing

At the bottom of the R Plugin Configuration, the user can choose whether processing should be "By Scenarios" or "By Scenario Sets."

Scenario Processing

If "By Scenarios" is selected, the Plugin will be executed once for each scenario that is selected in the Scenario List Post-Processor dialog (accessed by selecting Scenarios -> Post-Process Scenarios ... in the main RiverSMART workspace). The Plugin can be executed by selecting Scenarios -> Post-Process Checked Scenarios in the Scenario List Post-Processor dialog, which will execute all post-processing events for the selected scenarios. If the Plugin is using the <scenario names> Predefined argument, its value will be a single scenario name each time the Plugin is invoked. The <number of scenarios> argument will have a value of 1.

If "By Scenario Sets" is selected, a list of all scenario sets defined in the study will appear. (Scenario sets are defined in the Scenario Set Manager by selecting Scenarios -> Scenario Set Management ... in the main RiverSMART workspace.) The user can then select which scenario sets should be applied. The R Plugin will be invoked once for each scenario set. The <scenario names> Predefined argument will be a vector with the list of scenarios defined by the scenario set, and the <number of scenarios> argument will be equal to the length of that vector. When "By Scenario Sets" is selected, the Plugin can be executed by selecting Scenarios -> R Plugin in the main RiverSMART workspace, then selecting the appropriate R Plugin event from the dropdown menu.

Error Handling and Log File

RiverSMART has no knowledge of the user-created R script and thus will not report success or failure of the script execution. It is the responsibility of the user to verify that their R script executed successfully. Note that RiverSMART will not automatically delete old output files generated by the user's R script, so it is possible for old output files to exist if the R script fails on a current invocation. It is therefore important to check the time stamps on output files to verify that they are from the current invocation.

To assist in verifying successful execution or identifying errors, RiverSMART will create a log file. After the Plugin has executed, the log can be viewed by navigating to
<Study Folder>\Working\<Plugin Name>\<Scenario Set Name>\RPlugin.log
if using Scenario Sets or
<Study Folder>\Working\<Plugin Name>\<Scenario Name>\RPlugin.log
if post-processing by Scenarios. The RPlugin.log file can be opened in a text editor for viewing.

The log file will contain all messages sent to R Standard Out and Standard Error. For an execution with no errors and no output messages, the only contents of the log file may be
STANDARD OUT ...

STANDARD ERROR ...


If an error occurs during execution messages will be printed under STANDARD ERROR ...

Any print commands in the user's script will print to the STANDARD OUT ... in the log file.

RiverSMART-Generated R Script

The R Plugin creates an R script file that is executed by R in the background each time the plugin is invoked. After the Plugin has executed, this file can be viewed by navigating to
<Study Folder>\Working\<Plugin Name>\<Scenario Set Name>\RPlugin.R
if using Scenario Sets or
<Study Folder>\Working\<Plugin Name>\<Scenario Name>\RPlugin.R

if post-processing by Scenarios. The RPlugin.R file can be opened in a text editor for viewing. This can be useful for debugging purposes when the user's R script does not execute as expected.

When the user has selected to pass arguments "By Position," the RiverSMART-Generated R Script has the following general form:

source(<User-specified R file>)

arg1 <- <value>

arg2 <- <value>
...
argN <- <value>

<UserSpecifiedFunction>(arg1, arg2, arg3, ..., argN)

If the user has selected to pass arguments "By Name,"  the commands assigning values to the arguments will use the argument names specified in the R Plugin Configuration dialog. The final function call would then have the form:

<UserSpecifiedFunction>(arg1 = arg1, arg2 = arg2, arg3 = arg3, ..., argN = argN)

This form allows the ordering of the arguments in the plugin to differ from the ordering in the function definition.

Note that the user's R script gets sourced as the first command in the script. Then arguments are assigned values. Then the function gets called. This ordering allows the user to include additional commands in their R script before the specified function is called. These could even include additional function definitions that are then used as R expression arguments in the plugin. This ordering also allows an argument to be used in the definition of another argument value farther down in the list. For example, the following list of arguments would be valid:
List of arguments

CSV Combiner Plugin Help Information


Table of Contents


Overview

The CSV Combiner plugin takes separate CSV files and combines them into a single CSV file. Input is specified as scenario sets, so CSV files from separate scenarios can be combined into a single file that will include data from all the scenarios in a specified set. The scenario name is added as the first field in each line to distinguish lines for the different scenarios. The input and output files for the event are specified by linking CSV file icons on the workspace as input to and output from the CSV Combiner icon. The input CSV files can have the same columns, as named in the file header line, or can have different columns. If a line of data does not have a value for a column in the combined file, it shows as a blank value(i.e. ,2,,,4,).

User Interface

When the CSV Combiner event is opened, the following user interface is presented:


CSV Combiner Dialog

Specify the event name. All scenario sets, as defiined in the Scenario Set Manager available through the Scenarios/Scenario Set Management menu items, are presented in a checkable list. Checking a scenario set means an output file will be generated for the scenarios in that set, and the file will appear in the ScenarioSet directory named with the scenario set name. More than one set can be checked and an output CSV file will be generated for each set.


Downloading and Configuring the R Project for Statistical Computing on Windows

  1. Download R for Windows
    1. Navigate to http://cran.r-project.org/
    2. Select “Download R for Windows”
    3. Select “base”
    4. This takes you to the download page for the current R version. The Study Manager plugins were developed and tested with version 2.14.2, but they should work with later versions. If necessary, versions older than the current one can be downloaded by clicking the “Previous Releases” link and selecting an older version.
    5. Save the download file to your computer.
    6. As administrator, run the download file to install R on your system. The default installation directory and other default options are fine.
  2. Add the R executable location to your computer’s “Path” environment variable
    1. As administrator, go to Computer, Properties, Advanced System Settings, and click Environment Variables.
    2. Under the System Variables section, highlight “Path” and click Edit.
    3. Add a semicolon to the end of the variable’s value and then append the path to the directory containing your desired R executable, for example “C:\Program Files\R\R-2.14.2\bin\x64”.
    4. Click Ok to save the edit
  3. Install packages needed by R scripts in the study manager
    1. Start R as an administrator by right-clicking its desktop icon and using the “Run as administrator” menu item.
    2. In the R menu bar click on Packages, then Install Package(s)…
    3. You may be asked to select a mirror site from which to download packages.
    4. You will get a list of available packages. Select the following packages from the list and then click OK:
      1. e1071
      2. ggplot2
      3. gplots
      4. msm
      5. Reshape
      6. Zoo
    5. The packages should then be downloaded and installed for you.
    6. Exit the R interface.