skip to main content
Data Management Interface (DMI) : Database DMI : Web Service Datasets
Web Service Datasets
Note:  Web service datasets are new to RiverWare 8.2 and their features, diagnostic tools, and error messages may not be as robust as other datasets. Keep this in mind when using them.
A web service dataset provides a connection to a website that accesses the data in a database. The dataset creates the query URL and sends it to the website, which then accesses the data and returns it in the appropriate format. RiverWare then reads the data and imports it into the slots. Web service datasets support Input DMIs only.
A web service URL uses the form:
base_URL?query_string
The query_string is a series of keyword=value pairs separated by “&”.
Your organization may use multiple web services, each with a different base URL. The following web services are supported:
You can access the Web Service Dataset dialog from either of the following:
• Database DMI
• Dataset Manager
In the Web Service Dataset dialog, you can rename and configure the dataset. The dialog has the following tabs:
General Tab
The General tab includes options general to nearly all datasets. This section describes how to use these options for a web service dataset.
Name Map
Select the desired name map from the Name Map menu. The selections are populated based on the name maps configured in the Name Map Manager. See Name Mapping for details on creating name maps.
Name maps have special relevance for web service DMIs because they provide specific information on how a RiverWare slot maps to a location number or parameter code in the web database. See the following sections for details on specifying the names maps for each service:
Missing Values
Specify how missing values are handled when data is imported. The options are as follows:
• NaN—on import, missing values in the database are set to NaNs in RiverWare.
• Unchanged—on import, RiverWare data is left unchanged for a missing value in the database.
• Replaced With—on import, missing values are replaced by a specified input value.
Units
A menu in the Dataset dialog allows you to specify how units should be handled for a dataset. The options are as follows:
• Use Database Units
• Prefer Database Units
• Use Dataset Units
Units are currently not read from any web service; therefore, Use Dataset Units is the most applicable selection. If the unit type of the data does not match the unit type of the slot in RiverWare and the units cannot be converted, the slot is skipped and a warning message is given.
Web Service Tab
The Web Service tab includes the web service to use, a test tool URL, the base URL, and any other necessary information. Select the desired Web Service from the menu.
The dialog also includes the following common functionality.
• Test Tool URL (where supported)—URL that leads to the web service test query tool, which you can use to find locations and the desired parameters. Details for each web service are provided in the sections below. Use the Web button to open a browser to the URL.
• Base URL—URL on which all queries are formed.
You can edit the URLs to change them, if necessary. Use the green arrow Reset button to reset to the default.
The following sections describe each web service.
HDB Web Service
This web service connects to the HDB web service.
Note:  This dataset requires OpenSSL to be installed on your computer. For details, see the OpenSSL Installation document at: www.riverware.org/guides/index.html. If you get the following message, you likely need to install OpenSSL: Network error 6: the SSL/TLS handshake failed and the encrypted channel could not be established.
Note:  HDB datasets allow direct connections to and from an HDB database. See HDB Datasets for details.
The default URLs are as follows:
Configuration
When setting up an HDB web service dataset, specify the HDB to use and the type of data.
First, select the Desired HDB.
Then, select either Observed or Model Run ID. For the latter, also specify the Model Run ID.
HDB Web Service Use of Name Maps
RiverWare identifies data by Object.Slot, while HDB uses Site / Data type IDs (SDIs) or other identifiers.
Note:  In the HDB web service dataset, you must create a name map to map the Object.Slot to the SDI or other identifier.
In the name map, create a slot selection for each slot, then specify the SDI in the Name column. In Figure 4.26, “Coachella Valley.Diversion” is mapped to SDI 10066.
Figure 4.26  HDB web service name map example
CWMS RADAR Web Service
This web service connects to the CWMS RESTful API for Data Retrieval (CWMS RADAR) web service. This is a publicly available web service that pulls data from the public national CWMS database.
Note:  This web service does not require OpenSSL.
The default URL is as follows:
There is no test query tool at this time, but one will be added to the interface when it is implemented. Contact the developers of the CWMS RADAR web service for more information on the query formats.
For the CWMS RADAR web service, there is no configuration in the dataset other than the base URL. The main configuration takes place in the Name Map; see CWMS RADAR Use of Name Maps for details.
CWMS RADAR Use of Name Maps
RiverWare identifies data by Object.Slot, while CWMS uses a query string.
Note:  In the CWMS RADAR web service dataset, you must create a name map to map the Object.Slot to the correct location and parameter.
In the name map, create a slot selection for each slot. Then, in the Name column, specify the text that represents location, parameter, timestep, and other information. In Figure 4.27, “Keystone.Pool Elevation” is mapped to KEYS.Elev.Inst.1hour.0.Ccp-Rev. Contact the developers of the CWMS RADAR web service for more information on the query formats.
Figure 4.27  CWMS RADAR name map example
USGS Daily Values
This web service connects to the USGS Daily Values web service.
Note:  This dataset requires OpenSSL to be installed on your computer. For details on installing OpenSSL, see the OpenSSL Installation document at: www.riverware.org/guides/index.html. If you get the following message, you likely need to install OpenSSL: Network error 6: the SSL/TLS handshake failed and the encrypted channel could not be established.
The default URLs are as follows:
• Base URL: https://waterservices.usgs.gov/nwis/dv This is editable if the Base URL changes.
Note:  Since this is a daily values web service, data can only be imported to slots that have a 1 Day timestep size.
RiverWare identifies data by Object.Slot, while USGS uses a site and parameter ID. You must create a site map and/or a name map to specify how the Object.Slot are connected to the gage number and parameter.
See Site Maps for details.
See USGS Daily Values Use of Name Maps for details.
Note:  Date ranges are handled automatically by the specification of start and finish timesteps in the Database DMI.
Site Maps
The web service site map allows you to map object names to site IDs by querying the web service for potential site matches.
From the Web Service tab, select the Site Map button. The dialog lists all objects, by object type in the model. Navigate the tree view to find the desired object. When you double click in the Site Name column, the site map queries the web service, and an hourglass is shown as this could take a few moments:
f
Tip:  If you get the following message, you likely need to install OpenSSL: Network error 6: the SSL/TLS handshake failed and the encrypted channel could not be established. This Site Map queries the web service so it requires OpenSSL to be installed on your computer. For details on installing OpenSSL, see the OpenSSL Installation document at: www.riverware.org/guides/index.html.
Once it is finished, it shows a list of the sites whose name contains the object name as a substring (in this example Heron):
Select the desired name from the list and the Site ID is populated:
Tip:  The Site Name and Site ID field are editable, enabling you to enter the site name and ID if querying the web service didn't return the appropriate site. This would occur in the object name wasn't a substring of the site name, for example if the name in RiverWare is "Blue Water" but the USGS lists is as "Bluewater". You could enter Bluewater as the Site Name and then enter the correct Site ID.
USGS Daily Values Use of Name Maps
Site Maps, (Site Maps) map Object names to sites. Name Maps must be used to link slot names to parameter IDs.
Most likely similar slots (e.g., gage inflows, reservoir inflows) have the same parameter ID. As a result, use the name map's wildcarding capability to configure all similar slots with a single mapping.
This slot selection was created using the slot selector as follows; notice the wildcard and filters highlighted:
If similar slots don't all have the same parameter ID the name map's prioritized selections make it easy to configure the general case with exceptions, see Name Map Ordering and Priorities.
Also, in the Name Map, specify the parameter ID in the Name column. Some common USGS parameter IDs include:
– Mean discharge in cubic feet per second = 00060
– Elevation of reservoir water surface above datum, feet = 00062
– Stage or gage height, feet = 00065
More information can be found in the service documentation at:
Although the Site Map utility and wild carding in Name Maps are the preferred approach, you can still use the original approach where the name map specifies the full query string. In the name map, create a slot selection for each slot, then specify the query strings in the Name column. These are typically in the following form:
&Argument=value&Argument2=value
Some common arguments are as follows:
• Site or location—eight-digit site location
• parameterCd or variable—data to return from the gage. Common variables values are as follows:
See Figure 4.28 for an example, Otowi.Inflow is mapped to gage 08313000 and parameter 00060 (mean daily flow).
Figure 4.28  USGS Daily Values name map example using query syntax
Note:  Date ranges are handled automatically by the specification of start and finish timesteps in the Database DMI; they do not need to be specified in the name map. Similarly, format is also automatic.
Revised: 08/02/2021