Once a run has been made, there is typically a large volume of data to analyze. This section provides an overview (and links to additional documentation) of some of the features available to view, analyze, an debug model results. This includes plotting, statistical slots, SCTs, expression slots, and the model run analysis tool. The model run analysis tool includes a special results section which includes information specific to USACE methods.

Saved plots configurations can be moved amongst models by exporting the configuration from one model and then importing into another. See “Exporting and Importing Output Devices” in Output Utilities and Data Visualization for details on this utility.

Statistical Table slots compute statistics on a series of data. The user creates the slots and then selects the series slot for which the analysis is to be performed. Then, the user selects the statistical function to use and applies data filters as necessary. (Make sure to configure the statistical slots to only analyze data from the run range, not data before the start timestep or after the finish timestep. To do this, in the filter, select the Run Range option) The result is a table of data that can be viewed, plotted, or exported. Statistical slots that result in data that is decimal or percent (i.e. an exceedence curve) can be plotted with a probability scale. This is similar to plotting the data on probability paper.

See “Statistical Table Slots” in User Interface for details on statistical slots.

Another useful tool for viewing data is the System Control Table (SCT). The SCT is a customizable, editable spreadsheet view of slots in a RiverWare model. The SCT presents series data in a scrollable grid of numeric values and lists other types of slots for easy access. The SCT is a view into the model and as such, the user can have more than one SCT for a given model. For example, the user may create (and save) an SCT that shows all reservoir Outflows or one for all reservoir Storages. Or, the user may configure an SCT to show all of the desired information about a single reservoir.

See “System Control Table (SCT)” in User Interface for details.

Expression slots are computational expressions that can contain slot names and other variables. The user can either create a Series Slot with Expression or a Scalar Slot with Expression. They are used to calculate any quantity the user wishes to see. For example, the user may create slots that compute “combined Storage of all Reservoirs,” “weekly average Outflow of Reservoir 1,” “the ratio of Hydrologic Inflow to Control Point Inflow.” The logic for an expression slot is developed in the RiverWare Policy Language (RPL). Expression slots can be evaluated automatically at the beginning of run, beginning of timestep, end of timestep or end of run. They can also be evaluated outside of a run.

Therefore, there is a lot of flexibility with expression slots and are very useful for debugging, post processing data, and other analysis. See “Series Slots With Expression” in User Interface for details on expression slots.

The Model Run Analysis tool is a grid showing how each object solved on each timestep; see “Rulebased Simulation” in Debugging and Analysis for details on this utility. The Special Results Details dialog provides additional information for each object/timestep including information specific to USACE methods, as shown in Figure 6.1. This section provides more information on accessing and using the tool.

Access the Special Results Details dialog using the following methods.

As with other model run details dialogs, Special Results Details dialogs can be undocked to display the information in a separate window. Because this dialog is naturally large due to the amount of information it displays, viewing it undocked is most likely preferred. Select the Detach button to undock and use it as a separate dialog. Use the File, then Dock in Model Run Analysis to redock it. When docked in the Model Run Analysis dialog, a splitter allows the dialog to be resized as necessary.

The Special Results Details dialog contains a tab labeled USACE Methods which presents information related to certain USACE methods described in this document. Only Reservoirs, Control Points, and Computational Subbasins have information that is presented by this dialog. Figure 6.3 shows the tool.

The USACE Methods tab displays results in two tables. The upper table displays the values of several series slots in the time range from the given timestep through the Forecast Period (the forecast period is taken from the computational subbasin which has a non-null Flood Control method selected, of which there should only be one).

The slots displayed are ordered into functionality groups: Flood Control, Surcharge, Inflow Forecasting, Low-flow Releases, and Hydropower. The displayed slots are those likely to be helpful in understanding the behavior of the various related USACE methods. This list of slots is based on the method selections and includes both intermediate results that are not normally visible to the user as well as regular slots that are closely related to the flood control algorithm. The slots are shown in a tree-view so you can control the display by selecting the + and - operations.

Any slot listed in this dialog (including the invisible ones) can be opened or plotted using right-click context menus.

Table 6.1 provides diagnostic information on why the Flood Control release was limited. This table is only displayed on Reservoirs and is only populated when Operating Level Balancing is selected.

It is recommended that the user test the model at each stage of development. For example, the surcharge release results should be verified before moving on to flood control. The following testing strategies explain how to compare RiverWare results with results from the SUPER model. However, a similar testing strategy may be devised to compare RiverWare with results from another model or historical data.

For testing purposes, it may be useful to create rules that set the storage slot on each reservoir to the final, known storage value taken from another model or from historical data. These rules can be used to incrementally test portions of the model. Since these rules need to overwrite the values computed by the surcharge release and flood control rules, they must be higher priority. Add a policy group above the flood control policy group and add a rule for each reservoir in the model. The rules should be ordered such that the upstream reservoirs execute first and the downstream reservoirs execute last (same order as the surcharge release rules). Each rule should set the reservoir storage slot equal to the final, known storage value (either taken from historical data or another model run). These values should be stored in a custom slot in RiverWare. Figure 6.4 shows a sample rule. These rules should be used for testing purposes only and would not be a part of the final flood control model.

When these storage rules are active, they will overwrite the results of the policy. Each rule will reset the reservoir storage to the known values (stored in a custom slot). This will trigger each reservoir to redispatch with the solveMB_givenInflowStorage method. Each reservoir will compute a new outflow which will be routed downstream.

The set storage rules, discussed above, are required to accomplish this incremental testing. These rules allow the RiverWare model to be reset, at the end of each timestep, to a state equivalent to the SUPER model at that timestep. This ensures that the RiverWare model begins the next timestep in the same state as the SUPER model. As discussed above, the set storage rules should be the highest priority rules and will reset the reservoir storage to the SUPER storage value for that timestep. The SUPER storage values need to be obtained from a SUPER run that only has the desired calculations enabled. Store these values in RiverWare on custom slots.

In order to test the surcharge release results, the surcharge release rules and set storage rules need to be active. The regulation discharge and flood control policy groups should be turned off. The goal of this testing is to compare the surcharge releases computed in RiverWare with the surcharge releases computed in SUPER. Both models should produce the same results, within some accepted tolerance. The set storage rules are necessary because, without them, the reservoirs in RiverWare will fill up to the top of the flood pool and stay there (because only the surcharge release rule would be active). The user may be wondering why the flood control rules couldn’t be used instead of the set storage rules. The reason is that it would defeat the purpose of incremental testing. If the RiverWare flood control results deviate from SUPER, then the models have deviated and it would be meaningless to compare surcharge release results.

When the RiverWare model is run with the surcharge release and set storage rules, the model will compute the surcharge releases at each timestep. Then the set storage rules will reset the reservoir storages to the known SUPER storage values. The final surcharge release values can then be compared to the final SUPER surcharge release values. If the values do not compare, debug the problem. When these values match up, proceed to the next level of testing.

To test the regulation discharge results, the surcharge release, regulation discharge, and set storage rules should be active. The regulation discharge and empty space results computed by RiverWare should compare with the SUPER results.

Once the surcharge release and regulation discharge calculations have been verified, the flood control calculations can be tested. To test these, the surcharge release, regulation discharge, and flood control rules should be active. The set storage rules should not be active because they would overwrite the flood control results (although some types of specific testing may need to make use of the set storage rules).

Follow a similar approach as described above to test these three policies. The user should make a determination on a case-by-case basis as to whether the set storage rules should be enabled.

The Data Management Interface (DMI) can be used to import or export data from a model to an external data repository or database. There are two types of DMIs:

The USACE‑SWD uses DSS as it data repository, so this section will focus on the use of the Database DMI connection to DSS. In USACE‑SWD models, DMIs are used to import the following:

After a run is made, DMIs are used to export data to the DSS file to store the results of the run. The following section provides links to the DMI interface, the utility to record invocations of each DMI and then clear the values set by the DMI, and documentation on the interaction of RiverWare and CWMS.

The Database DMI interface allows the user to configure Name Maps and Datasets, and then define a Database DMI that uses them. See “Overview” in Data Management Interface (DMI) for details on the database DMI interface.

The USACE‑SWD uses DMIs to bring large volumes of table and series data into the model. Frequently, a previously developed model will be used for a study, but all of the data will be replaced with new data. To prevent using any of the previous data, a utility was developed to allow the user to clear out values set by a DMI. This utility has two parts, recording an invocation of the DMI and then clearing values set during that invocation. Following is an overview of the process a user would take. See “DMI Invocation Manager Dialog” in Data Management Interface (DMI) for details on this utility.

To summarize, to clear out all imported data in the model, the user should make sure that all DMIs have the Record Invocations toggle enabled. Then, when it is time to clear out the data the Invocation Manager can be used.

Run time of the USACE‑SWD models is always an important feature to consider. See “Improving Model Run Performance” in Debugging and Analysis for general tips and approaches.

Following are specific tips for USACE‑SWD models.

When the Diversions from Reservoirs conservation operation was initially implemented, the reservoir and the diversion object required the following links:

This was required because the diversion object had Available For Diversion as one of the required knowns in its dispatch conditions.

It was later found that the link between the Available For Diversion slots was leading to issues when the reservoir was low. To fix this, we removed Available For Diversion as a required known in the Diversion object's dispatch method. This was implemented in RiverWare 4.9. With this change, the link between the slots is no longer necessary.

The current analysis shows that there is unnecessary dispatching of the diversion objects because of this link. Any time the reservoir sets a new storage, the Available For Diversion at the next timestep is set. With the link in place, the value propagates to the Diversion object and the Diversion object redispatches, which then can cause the reservoir to redispatch. Thus, the Diversion objects are dispatching over the forecast period unnecessarily. Remove this link if they exist.

The following sections describe changes to the code to specifically improve performance in USACE‑SWD models.

The default behavior in RiverWare is for objects to dispatch whenever they have adequate information to do so. It was noted in the analyses that objects were dispatching well into the future beyond the end of the forecast period at the current timestep due to the step response routing method on reaches that distributes a current inflow into outflows at future timesteps. No information beyond the forecast period is used for calculations at the current timestep, and information from dispatching beyond the forecast period is overwritten anyway by subsequent calculations as the model steps forward. Dispatching beyond the end of the forecast period is, therefore, not necessary.

To limit dispatching to the forecast period only, changes were made to the controller. At the beginning of the run, the flood basin computational object tells the controller what the forecast period is. If there are multiple flood basin computational objects, the controller uses the largest forecast period. At the beginning of each timestep, the controller calculates the end time of the forecast period by adding the forecast period to the current controller time. A condition was added to the checkDispatching method so that if the date of the dispatch is beyond the end time of the forecast period, the dispatch entry is not put on the dispatch queue for solving.

The step response routing method in reach calculates and sets outflow values into future timesteps. If the timesteps are beyond the end of the forecast period, this work is unnecessary because this information is not needed in the model for solving at the current timestep.

The step response routing method was modified so that if forecasting is being used, the reach asks the controller for the end time of the forecast period. The step response calculations are then limited so that outflow values are not calculated or set beyond the end of the forecast period.

On the computational subbasin, there was significant time spent getting the appropriate control point and looking up the routing vector on its routing coefficients slot for the upstream reservoir.

Instead of doing this lookup each time, the code was changed to cache the routing vector information on a data structure in the computational object the first time coefficients are requested for a particular upstream and downstream object pair. Subsequent requests can then access the routing information from the data structure instead of redoing the lookup. If variable routing coefficients that can change with each timestep are being used, the data structure is cleared at the beginning of each timestep. Otherwise, the data structure can be used throughout the whole run.

In analyzing dispatching, it was noted that excess dispatching occurs where values have been set on a number of different objects across the model. Dispatching order in RiverWare is based on a queue where objects dispatch in the order in which they come onto the queue. For example, in flood control where a value is set on each reservoir in the system, the reservoirs all go onto the queue and each reservoir dispatches in order before any objects dispatch that have come onto the queue as a result of the reservoir dispatches. This means the effect of an upstream reservoir's dispatching is not propagated to the downstream reservoir before it dispatches the first time, so it must go back on the queue to dispatch again (and maybe a third or fourth time due to additional upstream reservoirs).

To address this problem, a custom downstream dispatch order was implemented. The first time custom dispatching is called, the rule controller creates an ordered upstream to downstream list of objects in the network based on the topology information of upstream and downstream links between objects. Custom dispatching then iterates the dispatch queue successively for each object in the ordered list and dispatches all of that object's entries so that a downstream object does not dispatch before upstream effects have been propagated to it.

There is overhead associated with iterating the queue by object, so custom dispatching is only called in places where a number of objects across the basin are being put on the queue at the same time. For the COE model this has been implemented for computing incremental local inflows, the flood control rule, and the hydropower rule.