skip to main content
Miscellaneous Topics
USACE‑SWD Modeling Techniques
Miscellaneous Topics
Analysis and Output Tools
Once a run has been made, there is typically a large volume of data to analyze. This section provides an overview (and links to additional documentation) of some of the features available to view, analyze, an debug model results. This includes plotting, statistical slots, SCTs, expression slots, and the model run analysis tool. The model run analysis tool includes a special results section which includes information specific to USACE methods.
Plotting
The RiverWare plotting utility allows the user to plot series, tables, contours, parametric, and periodic curves; see “Plotting” in Output Utilities and Data Visualization for details on this utility. In the plot utility, you can quickly choose to plot either all the data in a slot , only data in the run range , or only data in a specified range .
Saved plots configurations can be moved amongst models by exporting the configuration from one model and then importing into another. See “Exporting and Importing Output Devices” in Output Utilities and Data Visualization for details on this utility.
Statistical Slots and Probability Plots
Statistical Table slots compute statistics on a series of data. The user creates the slots and then selects the series slot for which the analysis is to be performed. Then, the user selects the statistical function to use and applies data filters as necessary. (Make sure to configure the statistical slots to only analyze data from the run range, not data before the start timestep or after the finish timestep. To do this, in the filter, select the Run Range option) The result is a table of data that can be viewed, plotted, or exported. Statistical slots that result in data that is decimal or percent (i.e. an exceedence curve) can be plotted with a probability scale. This is similar to plotting the data on probability paper.
See “Statistical Table Slots” in User Interface for details on statistical slots.
SCT
Another useful tool for viewing data is the System Control Table (SCT). The SCT is a customizable, editable spreadsheet view of slots in a RiverWare model. The SCT presents series data in a scrollable grid of numeric values and lists other types of slots for easy access. The SCT is a view into the model and as such, the user can have more than one SCT for a given model. For example, the user may create (and save) an SCT that shows all reservoir Outflows or one for all reservoir Storages. Or, the user may configure an SCT to show all of the desired information about a single reservoir.
Expression Slots
Expression slots are computational expressions that can contain slot names and other variables. The user can either create a Series Slot with Expression or a Scalar Slot with Expression. They are used to calculate any quantity the user wishes to see. For example, the user may create slots that compute “combined Storage of all Reservoirs,” “weekly average Outflow of Reservoir 1,” “the ratio of Hydrologic Inflow to Control Point Inflow.” The logic for an expression slot is developed in the RiverWare Policy Language (RPL). Expression slots can be evaluated automatically at the beginning of run, beginning of timestep, end of timestep or end of run. They can also be evaluated outside of a run.
Therefore, there is a lot of flexibility with expression slots and are very useful for debugging, post processing data, and other analysis. See “Series Slots With Expression” in User Interface for details on expression slots.
Model Run Analysis—Special Results Details Dialog
The Model Run Analysis tool is a grid showing how each object solved on each timestep; see “Rulebased Simulation” in Debugging and Analysis for details on this utility. The Special Results Details dialog provides additional information for each object/timestep including information specific to USACE methods, as shown in Figure 6.1. This section provides more information on accessing and using the tool.
Figure 6.1   
Accessing and Navigation
Access the Special Results Details dialog using the following methods.
• From the model run analysis tool
– Select the Details button and select Special Results. The details will be shown docked in the main Model Run Analysis dialog. Select the Detach button to use it as a separate dialog.
– On any cell in the Model Run Analysis dialog, right-click and select Show Special Results Details to show the details in a new window.
• From the Open Object dialog
– Select View, then Special Results Details menu to show the details in a new window.
– Shift-click the Dispatch button to show the details in a new window. The button's tooltip summarizes the dispatch behavior but also lists the behavior when the button is selected (open the Dispatch Behavior Details dialog) and when it is shift-clicked (open the Special Results Details dialog).
Figure 6.2   
As with other model run details dialogs, Special Results Details dialogs can be undocked to display the information in a separate window. Because this dialog is naturally large due to the amount of information it displays, viewing it undocked is most likely preferred. Select the Detach button to undock and use it as a separate dialog. Use the File, then Dock in Model Run Analysis to redock it. When docked in the Model Run Analysis dialog, a splitter allows the dialog to be resized as necessary.
Layout and Overview
The Special Results Details dialog contains a tab labeled USACE Methods which presents information related to certain USACE methods described in this document. Only Reservoirs, Control Points, and Computational Subbasins have information that is presented by this dialog. Figure 6.3 shows the tool.
Figure 6.3   
Related Slots
The USACE Methods tab displays results in two tables. The upper table displays the values of several series slots in the time range from the given timestep through the Forecast Period (the forecast period is taken from the computational subbasin which has a non-null Flood Control method selected, of which there should only be one).
The slots displayed are ordered into functionality groups: Flood Control, Surcharge, Inflow Forecasting, Low-flow Releases, and Hydropower. The displayed slots are those likely to be helpful in understanding the behavior of the various related USACE methods. This list of slots is based on the method selections and includes both intermediate results that are not normally visible to the user as well as regular slots that are closely related to the flood control algorithm. The slots are shown in a tree-view so you can control the display by selecting the + and - operations.
Note:  Invisible slots are shown with an asterisk. This indicate they are not visible or accessible in other dialogs but are saved in the model file. These are slots that were temporary prior to RiverWare 6.1. This dialog is the only place to access these invisible slots. Display units are configured on the Unit Scheme. Also, you can create an exception to the scheme for this slot by opening the slot, then modify using the View, then Configure menu.
Any slot listed in this dialog (including the invisible ones) can be opened or plotted using right-click context menus.
Reason For Limiting Release Table
Table 6.1 provides diagnostic information on why the Flood Control release was limited. This table is only displayed on Reservoirs and is only populated when Operating Level Balancing is selected.
Note:  Table 6.1 displays information that, prior to RiverWare 6.1, was encoded on the “Temp Reason” and “Temp Limiting Control Point” slots. Each row represents a pass of the flood control algorithm. Each time the flood control release is reduced by a constraint, the reason for the last reduction is written. On dates that have few or no passes, some rows will not have values. While diagnostics might refer to other forecast timestep, only the first forecast timestep’s reason (i.e. current timestep) is saved here. The Downstream Object and Date columns will have values if the reason is “Channel Space”. Otherwise they will be blank.
 
Table 6.1    
Diagnostic String
Discussion
Spillway constraint
The spillway cannot accommodate any more
outflow. This constraint applies to all outflows, including surcharge releases, through releases, and this object’s flood control release.
Stepped hydrograph smoothing
The release was limited by the first release in a stepped-down hydrograph that releases the entire proposed flood pool volume over the rest of the forecast period with an Allowable Falling Release Change reduction at each timestep.
This constraint is applied once, to the first timestep only.
Channel space
Channel space at a downstream control point limited the release. The value in the Downstream Object column indicates which downstream control point is the cause and at which forecast Date. This information indicates which downstream control point is lacking in adequate channel space and limited a flood control release on each pass of the flood control algorithm.
No increase after timestep 0
On its last pass, the algorithm does not allow an increase in release over the prior pass’s proposed release for any timestep after the first timestep in the release hydrograph. This string will show up in diagnostics, but not in the slot, since the slot contains only those reasons that apply to the first timestep of the forecast period.
Key Control Point maximum flood control release
The Maximum Flood Control Release slot value (computed by a key control point) limited the release.
Volume above key Control Point balance level
The volume intended to be released is that found to be above the balance level assigned by a key control point at the end of the balance period. This volume is reduced by each scheduled release in the forecast period, and the remaining amount limits subsequent scheduled releases.
Volume in flood pool
The flood control release was limited by the total volume forecast to be in the flood pool at the time this reservoir is trying to make a release (this takes into account tandem storages and forecast flood control releases).
Allowable falling release change
The release was limited by the first release in a stepped-down hydrograph that releases the remaining proposed flood pool volume over the rest of the forecast period with an Allowable Falling Release Change reduction at each timestep. This constraint is applied to each timestep in the forecast period.
Allowable rising release change
The release was limited by the Allowable Rising Release Change added to the prior timestep’s release.
Comparing to SUPER
It is recommended that the user test the model at each stage of development. For example, the surcharge release results should be verified before moving on to flood control. The following testing strategies explain how to compare RiverWare results with results from the SUPER model. However, a similar testing strategy may be devised to compare RiverWare with results from another model or historical data.
Set Storage Rules
For testing purposes, it may be useful to create rules that set the storage slot on each reservoir to the final, known storage value taken from another model or from historical data. These rules can be used to incrementally test portions of the model. Since these rules need to overwrite the values computed by the surcharge release and flood control rules, they must be higher priority. Add a policy group above the flood control policy group and add a rule for each reservoir in the model. The rules should be ordered such that the upstream reservoirs execute first and the downstream reservoirs execute last (same order as the surcharge release rules). Each rule should set the reservoir storage slot equal to the final, known storage value (either taken from historical data or another model run). These values should be stored in a custom slot in RiverWare. Figure 6.4 shows a sample rule. These rules should be used for testing purposes only and would not be a part of the final flood control model.
When these storage rules are active, they will overwrite the results of the policy. Each rule will reset the reservoir storage to the known values (stored in a custom slot). This will trigger each reservoir to redispatch with the solveMB_givenInflowStorage method. Each reservoir will compute a new outflow which will be routed downstream.
Figure 6.4   
Testing Strategies
The set storage rules, discussed above, are required to accomplish this incremental testing. These rules allow the RiverWare model to be reset, at the end of each timestep, to a state equivalent to the SUPER model at that timestep. This ensures that the RiverWare model begins the next timestep in the same state as the SUPER model. As discussed above, the set storage rules should be the highest priority rules and will reset the reservoir storage to the SUPER storage value for that timestep. The SUPER storage values need to be obtained from a SUPER run that only has the desired calculations enabled. Store these values in RiverWare on custom slots.
Testing Surcharge Release
In order to test the surcharge release results, the surcharge release rules and set storage rules need to be active. The regulation discharge and flood control policy groups should be turned off. The goal of this testing is to compare the surcharge releases computed in RiverWare with the surcharge releases computed in SUPER. Both models should produce the same results, within some accepted tolerance. The set storage rules are necessary because, without them, the reservoirs in RiverWare will fill up to the top of the flood pool and stay there (because only the surcharge release rule would be active). The user may be wondering why the flood control rules couldn’t be used instead of the set storage rules. The reason is that it would defeat the purpose of incremental testing. If the RiverWare flood control results deviate from SUPER, then the models have deviated and it would be meaningless to compare surcharge release results.
When the RiverWare model is run with the surcharge release and set storage rules, the model will compute the surcharge releases at each timestep. Then the set storage rules will reset the reservoir storages to the known SUPER storage values. The final surcharge release values can then be compared to the final SUPER surcharge release values. If the values do not compare, debug the problem. When these values match up, proceed to the next level of testing.
Testing Regulation Discharge
To test the regulation discharge results, the surcharge release, regulation discharge, and set storage rules should be active. The regulation discharge and empty space results computed by RiverWare should compare with the SUPER results.
Testing Flood Control Results
Once the surcharge release and regulation discharge calculations have been verified, the flood control calculations can be tested. To test these, the surcharge release, regulation discharge, and flood control rules should be active. The set storage rules should not be active because they would overwrite the flood control results (although some types of specific testing may need to make use of the set storage rules).
Testing of Low-flow Releases, Reservoir Diversions, and Hydropower Releases
Follow a similar approach as described above to test these three policies. The user should make a determination on a case-by-case basis as to whether the set storage rules should be enabled.
DMIs to Import and Export Data
The Data Management Interface (DMI) can be used to import or export data from a model to an external data repository or database. There are two types of DMIs:
• Control File/Executable. Provide a link to any external database or repository
• Database DMIs. Provide an automated connection to HEC‑DSS and HDB.
The USACE‑SWD uses DSS as it data repository, so this section will focus on the use of the Database DMI connection to DSS. In USACE‑SWD models, DMIs are used to import the following:
• Table and scalar data used to define the model
• Initial values
• Time series data representing uncontrolled area flows, evaporation, power demands, etc.
• Observed or existing data used for comparison
After a run is made, DMIs are used to export data to the DSS file to store the results of the run. The following section provides links to the DMI interface, the utility to record invocations of each DMI and then clear the values set by the DMI, and documentation on the interaction of RiverWare and CWMS.
Database DMI-DSS Interface
The Database DMI interface allows the user to configure Name Maps and Datasets, and then define a Database DMI that uses them. See “Overview” in Data Management Interface (DMI) for details on the database DMI interface.
DMI Invocations and Clearing Input DMI Values
The USACE‑SWD uses DMIs to bring large volumes of table and series data into the model. Frequently, a previously developed model will be used for a study, but all of the data will be replaced with new data. To prevent using any of the previous data, a utility was developed to allow the user to clear out values set by a DMI. This utility has two parts, recording an invocation of the DMI and then clearing values set during that invocation. Following is an overview of the process a user would take. See “DMI Invocation Manager Dialog” in Data Management Interface (DMI) for details on this utility.
1. Within the DMI manager, the user configures that an input DMI should record invocations. With this toggle enabled, the DMI will record every time the DMI is invoked and the slots set.
2. The user executes the input DMI and the invocation is recorded. Also, with invocations enabled, all of the data that is set is given the “Z” flag instead of the “I” flag.
Note:  The Z flag behaves identically to the I flag.The user then performs the run as usual.
3. When the user decides that they want to start a new study with new data, the user opens the Invocation Manager and selects the appropriate invocation. The user then selects the Edit, then Clear Selected Values menu and any data that was set by that DMI is cleared out (i.e. set to NaN).
4. In this way, the user can go through all of the input DMIs and clear out any value that was set by a DMI.
To summarize, to clear out all imported data in the model, the user should make sure that all DMIs have the Record Invocations toggle enabled. Then, when it is time to clear out the data the Invocation Manager can be used.
Performance Tips and Information
Run time of the USACE‑SWD models is always an important feature to consider. See “Improving Model Run Performance” in Debugging and Analysis for general tips and approaches.
User Tips
Following are specific tips for USACE‑SWD models.
• Make sure each rule is not executing more often than necessary. Typically, each rule only needs to execute once per timestep. See “Execution Constraints to Execute Rule Once per Timestep” for details on the approach to control this.
• When diverting from reservoirs using the ComputeReservoirDiversions function, make sure that Reservoir.Available For Diversion is not linked to Diversion.Available For Diversion. This link is unnecessary and can lead to excessive dispatching of the diversion object. See “Linking Setup” for details on the linking structure.
• If not debugging, disable the following, as they are computationally intensive and are typically not necessary:
– Diagnostics
– RPL Set Analysis
• Close any unused open windows, particularly the Model Run Analysis tool. This and other dialogs attempt to redraw after every timestep and can slow down the model.
• Close other applications that are competing for computer resources.
When the Diversions from Reservoirs conservation operation was initially implemented, the reservoir and the diversion object required the following links:
Reservoir.Diversion <---> Diversion.Diversion
Reservoir.Available For Diversion <---> Diversion.Available For Diversion
This was required because the diversion object had Available For Diversion as one of the required knowns in its dispatch conditions.
It was later found that the link between the Available For Diversion slots was leading to issues when the reservoir was low. To fix this, we removed Available For Diversion as a required known in the Diversion object's dispatch method. This was implemented in RiverWare 4.9. With this change, the link between the slots is no longer necessary.
The current analysis shows that there is unnecessary dispatching of the diversion objects because of this link. Any time the reservoir sets a new storage, the Available For Diversion at the next timestep is set. With the link in place, the value propagates to the Diversion object and the Diversion object redispatches, which then can cause the reservoir to redispatch. Thus, the Diversion objects are dispatching over the forecast period unnecessarily. Remove this link if they exist.
Internal RiverWare Changes
The following sections describe changes to the code to specifically improve performance in USACE‑SWD models.
No Dispatching Beyond Forecast Period
The default behavior in RiverWare is for objects to dispatch whenever they have adequate information to do so. It was noted in the analyses that objects were dispatching well into the future beyond the end of the forecast period at the current timestep due to the step response routing method on reaches that distributes a current inflow into outflows at future timesteps. No information beyond the forecast period is used for calculations at the current timestep, and information from dispatching beyond the forecast period is overwritten anyway by subsequent calculations as the model steps forward. Dispatching beyond the end of the forecast period is, therefore, not necessary.
To limit dispatching to the forecast period only, changes were made to the controller. At the beginning of the run, the flood basin computational object tells the controller what the forecast period is. If there are multiple flood basin computational objects, the controller uses the largest forecast period. At the beginning of each timestep, the controller calculates the end time of the forecast period by adding the forecast period to the current controller time. A condition was added to the checkDispatching method so that if the date of the dispatch is beyond the end time of the forecast period, the dispatch entry is not put on the dispatch queue for solving.
No Setting of Values Beyond Forecast Period in Step Response Routing
The step response routing method in reach calculates and sets outflow values into future timesteps. If the timesteps are beyond the end of the forecast period, this work is unnecessary because this information is not needed in the model for solving at the current timestep.
The step response routing method was modified so that if forecasting is being used, the reach asks the controller for the end time of the forecast period. The step response calculations are then limited so that outflow values are not calculated or set beyond the end of the forecast period.
Cache Routing Vectors
On the computational subbasin, there was significant time spent getting the appropriate control point and looking up the routing vector on its routing coefficients slot for the upstream reservoir.
Instead of doing this lookup each time, the code was changed to cache the routing vector information on a data structure in the computational object the first time coefficients are requested for a particular upstream and downstream object pair. Subsequent requests can then access the routing information from the data structure instead of redoing the lookup. If variable routing coefficients that can change with each timestep are being used, the data structure is cleared at the beginning of each timestep. Otherwise, the data structure can be used throughout the whole run.
Custom Downstream Dispatch Order
In analyzing dispatching, it was noted that excess dispatching occurs where values have been set on a number of different objects across the model. Dispatching order in RiverWare is based on a queue where objects dispatch in the order in which they come onto the queue. For example, in flood control where a value is set on each reservoir in the system, the reservoirs all go onto the queue and each reservoir dispatches in order before any objects dispatch that have come onto the queue as a result of the reservoir dispatches. This means the effect of an upstream reservoir's dispatching is not propagated to the downstream reservoir before it dispatches the first time, so it must go back on the queue to dispatch again (and maybe a third or fourth time due to additional upstream reservoirs).
To address this problem, a custom downstream dispatch order was implemented. The first time custom dispatching is called, the rule controller creates an ordered upstream to downstream list of objects in the network based on the topology information of upstream and downstream links between objects. Custom dispatching then iterates the dispatch queue successively for each object in the ordered list and dispatches all of that object's entries so that a downstream object does not dispatch before upstream effects have been propagated to it.
There is overhead associated with iterating the queue by object, so custom dispatching is only called in places where a number of objects across the basin are being put on the queue at the same time. For the COE model this has been implemented for computing incremental local inflows, the flood control rule, and the hydropower rule.
Note:  The flood control algorithm is sensitive to small changes in input values, even changes that may be within convergence of slots. Changing the dispatch order of objects can affect how very small changes get propagated downstream or not and can change results by kicking off an operation at an earlier or later timestep. The custom dispatch order actually ensures that even small changes are propagated upstream to downstream, and due to this, gives different results at particular timesteps compared to the baseline executable. However, even though results at individual timesteps may differ, the overall trends and form of the solution is the same.
 
Revised: 06/03/2019