skip to main content
Data Management Interface (DMI) : Database DMI : Web Service Datasets
Web Service Datasets
A web service dataset provides a connection to a website that accesses the data in a database. The dataset creates the query URL and sends it to the website, which then accesses the data and returns it in the appropriate format. RiverWare then reads the data and imports it into the slots. Web service datasets support Input DMIs only.
A web service URL uses the form:
base_URL?query_string
The query_string is a series of keyword=value pairs separated by “&”.
Your organization may use multiple web services, each with a different base URL. The following web services are supported:
You can access the Web Service Dataset dialog from either of the following:
• Database DMI
• Dataset Manager
In the Web Service Dataset dialog, you can rename and configure the dataset. The dialog has the following tabs:
General Tab
The General tab includes options general to nearly all datasets. This section describes how to use these options for a web service dataset.
Name Map
Select the desired name map from the Name Map menu. The selections are populated based on the name maps configured in the Name Map Manager. See Name Mapping for details on creating name maps.
Name maps have special relevance for web service DMIs because they provide specific information on how a RiverWare slot maps to a location number or parameter code in the web database. See the following sections for details on specifying the names maps for each service:
Missing Values
Specify how missing values are handled when data is imported. The options are as follows:
• NaN—on import, missing values in the database are set to NaNs in RiverWare.
• Unchanged—on import, RiverWare data is left unchanged for a missing value in the database.
• Replaced With—on import, missing values are replaced by a specified input value.
Units
A menu in the Dataset dialog allows you to specify how units should be handled for a dataset. The options are as follows:
• Use Database Units
• Prefer Database Units
• Use Dataset Units
Units are currently not read from any web service; therefore, Use Dataset Units is the most applicable selection. If the unit type of the data does not match the unit type of the slot in RiverWare and the units cannot be converted, the slot is skipped and a warning message is given.
Web Service Tab
The Web Service tab includes the web service to use, a test tool URL, the base URL, and any other necessary information. Select the desired Web Service from the menu.
The dialog also includes the following common functionality.
• Test Tool URL (where supported)—URL that leads to the web service test query tool, which you can use to find locations and the desired parameters. Details for each web service are provided in the sections below. Use the Web button to open a browser to the URL.
• Base URL—URL on which all queries are formed.
You can edit the URLs to change them, if necessary. Use the green arrow Reset button to set it to the default.
The following sections describe each web service.
CDSS Surface Water Time Series - Day
This web service connects to Colorado’s Decision Support Systems (CDSS). The web service is called CDSS Surface Water Time Series - Day.
The default URLs are as follows:
• Test query tool URL: https://dwr.state.co.us/Rest/GET/Help/SurfaceWaterTSDayGenerator
• Base URL: https://dwr.state.co.us/Rest/GET/api/v2/surfacewater/surfacewatertsday/ This is editable if the Base URL changes.
Note:  Since this is a daily values web service, data can only be imported to slots that have a 1 Day timestep size.
RiverWare identifies data by Object.Slot, while CDSS uses a site name and site ID. You must create a site map and/or a name map to specify how the Object.Slot are connected to the site ID.
See CDSS Site Maps for details.
See CDSS Daily Values Use of Name Maps for details.
Note:  Date ranges are handled automatically by the specification of start and finish timesteps in the Database DMI.
CDSS Site Maps
The web service site map allows you to map object names to site IDs by querying the web service for potential site matches.
From the Web Service tab, select the Site Map button. The dialog lists all objects, by object type in the model. Navigate the tree view to find the desired object.
When you double click in the Site Name column, the site map queries the web service, in order, for the
1. Alt Search Name - For example, if your object is named, “South Platte at Denver”, it will not find it as the actual name is “South Platte River at Denver”, so you can enter an Alt Search Name, “South Platte” for example and it will look for that string.
2. Object Name - If your objects names closely match the Station Names, no Alt Search Name is necessary. It will search for the Object Name directly.
Also specify at the top of the dialog if you want to search in the Station Name or Abbreviated Name, as the example shows.
f
Once it is finished, it shows a drop down list of the sites whose name contains the Alt Search Name or Object Name as a substring (in this example Alamosa):
Select the desired name from the list and the Site ID is populated, for example 670:
Tip:  In the screenshot above, the gage Alamosa queried form the object name directly. For the other gage, Otowi, the user typed in Otowi Bridge as the Alt Search Name. This narrowed down the query to be more detailed.
Tip:  The Site Name and Site ID field are editable, enabling you to enter the site name and ID if querying the web service didn't return the appropriate site. This would occur in the object name wasn't a substring of the site name, for example if the name in RiverWare is "Frying Pan" but the CDSS lists is as “Fryingpan”. You could enter Fryingpan as the Site Name and then enter the correct Site ID or just enter Fryingpan as the Alt Search Name.
CDSS Daily Values Use of Name Maps
Site Maps, (CDSS Site Maps) map Object names to sites and are the recommended approach. If those don’t work for some database reason, you can map Object names using normal Name Maps. Below is a screenshot of a name map that maps Platte River.Gage Inflow to Site ID 637.
Note:  Unlike USGS gages that have multiple parameters, the CDSS sites are for a single parameter. For example Site ID 637 returns SOUTH PLATTE RIVER AT DENVER gage flows in cfs. For that reason, Name Maps are not required to map the slots to the parameters.
More information can be found in the service documentation at:
https://dwr.state.co.us/Rest/GET/Help/SurfaceWaterTSDayGenerator
 
CWMS Data API Web Service
This web service connects to the CWMS Data API (CDA) web service. This is a publicly available web service that pulls data from the public national CWMS database.
The default URL is as follows:
• Base URL: https://water.usace.army.mil/cwms-data/timeseries
There is no test query tool at this time, but one will be added to the interface when it is implemented. Contact the developers of the CWMS Data API web service for more information on the query formats.
For the CWMS Data API web service, there is no configuration in the dataset other than the base URL. The main configuration takes place in the Name Map; see CWMS Data API Use of Name Maps for details.
CWMS Data API Use of Name Maps
RiverWare identifies data by Object.Slot, while CWMS uses a query string.
Note:  In the CWMS Data API web service dataset, you must create a name map to map the Object.Slot to the correct location and parameter.
In the name map, create a slot selection for each slot. Then, in the Name column, specify the text that represents location, parameter, timestep, and other information. In Figure 4.28, “Keystone.Pool Elevation” is mapped to KEYS.Elev.Inst.1hour.0.Ccp-Rev. Contact the developers of the CWMS Data API web service for more information on the query formats.
Figure 4.28  CWMS Data API name map example
HDB Web Service
This web service connects to the HDB web service.
Note:  HDB datasets allow direct connections to and from an HDB database. See HDB Datasets for details.
The default URLs are as follows:
Configuration
When setting up an HDB web service dataset, specify the HDB to use and the type of data.
First, select the Desired HDB.
 
Then, select either Observed or Model Run ID. For the latter, also specify the Model Run ID.
HDB Web Service Use of Name Maps
RiverWare identifies data by Object.Slot, while HDB uses Site / Data type IDs (SDIs) or other identifiers.
Note:  In the HDB web service dataset, you must create a name map to map the Object.Slot to the SDI or other identifier.
In the name map, create a slot selection for each slot, then specify the SDI in the Name column. In Figure 4.29, “Coachella Valley.Diversion” is mapped to SDI 10066.
Figure 4.29  HDB web service name map example
HDB Web Service Timesteps
The HDB Web Service returns values that include dates at the beginning of the timestep. The values are imported at the appropriate timestep in the slot. For example, for daily values using RiverWare dates 9/1/2019 24:00 through 10/1/2019 24:00, the query is:
https://www.usbr.gov/pn-bin/hdb/hdb.pl?svr=lchdb&sdi=10066&tstp=DY&t1=2019-09-01T00:00&t2=2019-10-01T00:00&table=R&mrid=0&format=json
Here the bold value indicates 9/1/2019 00:00 which is the beginning of the timestep 9/1 00:00 - 24:00. In RiverWare, this daily timestep is9/1/2019 24:00. The HDB web service uses beginning of timestep notation, RiverWare uses end of timestep.
This can look misleading for longer timesteps. For example, the query for data for RiverWare monthly values December 2019 to January 2020 is:
https://www.usbr.gov/pn-bin/hdb/hdb.pl?svr=lchdb&sdi=1930&tstp=MN&t1=2019-12-1&t2=2020-1-1&table=R&mrid=0&format=json
It returns the two data point:
{"t":"12/1/2019 12:00:00 AM","v":"1090.49"},
{"t":"1/1/2020 12:00:00 AM","v":"1094.68"}
The DMI then imports these beginning of timestep values into the rows for 12-2019 and 1-2020, which of course are represented internally in RiverWare as 12/31/2019 24:00 and 1/31/2020 24:00, respectively.
Figure 4.30  Sample HDB web service imported cells
Reclamation Hydromet
This web service connects to Reclamation’s Hydromet web service hosted by the Missouri Basin and Arkansas Rio Grande-Texas Gulf region. (Previously, the Great Plains region)
The default URLs are as follows:
• Test query tool URL: https://www.usbr.gov/gp/hydromet/archive_form.html
• Base URL: https://www.usbr.gov/gp-bin/arcread.pl This is editable if the Base URL changes or other Reclamation offices have similar webservices.
Note:  Since this is a daily values web service, data can only be imported to slots that have a 1 Day timestep size.
Note:  Date ranges are handled automatically by the specification of start and finish timesteps in the Database DMI.
Reclamation Hydromet use of Name Maps
RiverWare identifies data by Object.Slot, while Hydromet uses site and parameters.
Note:  In the HDB web service dataset, you must create a name map to map the Object.Slot to the site/parameter.
In the name map, create a slot selection for each slot, then specify the Site and Parameter in the Name column. In Figure 4.31, “Buffalo Bill.Outflow” is mapped to BBR QD.
Figure 4.31  Hydromet web service name map example
More information can be found in the web service documentation at
https://www.usbr.gov/gp/hydromet/automated_retrieval.pdf
USGS Daily Values
This web service connects to the USGS Daily Values web service.
The default URLs are as follows:
• Base URL: https://waterservices.usgs.gov/nwis/dv This is editable if the Base URL changes.
Note:  Since this is a daily values web service, data can only be imported to slots that have a 1 Day timestep size.
RiverWare identifies data by Object.Slot, while USGS uses a site and parameter ID. You must create a site map and/or a name map to specify how the Object.Slot are connected to the gage number and parameter.
See USGS Site Maps for details.
See USGS Daily Values Use of Name Maps for details.
Note:  Date ranges are handled automatically by the specification of start and finish timesteps in the Database DMI.
USGS Site Maps
The web service site map allows you to map object names to site IDs by querying the web service for potential site matches.
From the Web Service tab, select the Site Map button. The dialog lists all objects, by object type in the model. Navigate the tree view to find the desired object.
When you double click in the Site Name column, the site map queries the web service, in order, for the
1. Alt Search Name - For example, if your objects are named something different than the station name, it will not find it, so you can enter an Alt Search Name, “Heron Reservoir” for example and it will look for that string.
2. Object Name - If your objects names match the Station Names, no Alt Search Name is necessary. It will search for the Object Name directly.
Again, double click the Site Name field to initiate the query.
f
Once it is finished querying, it shows a list of the sites whose name contains the object name as a substring (in this example Otowi, a location on the Rio Grande):
Select the desired name from the list and the Site ID is populated:
Tip:  The Site Name and Site ID field are editable, enabling you to enter the site name and ID if querying the web service didn't return the appropriate site. This would occur in the object name wasn't a substring of the site name, for example if the name in RiverWare is "Blue Water" but the USGS lists is as "Bluewater". You could enter Bluewater as the Site Name and then enter the correct Site ID, or just enter Bluewater as the Alt Search Name.
USGS Daily Values Use of Name Maps
Use Site Maps (USGS Site Maps) to map Object names to sites. Name Maps must be used to connect slot names to parameter IDs.
Most likely, similar slots (e.g., gage inflows, reservoir inflows) have the same parameter ID. As a result, use the name map's wildcarding capability to configure all similar slots with a single mapping.
This slot selection was created using the slot selector as follows; notice the wildcard and filters highlighted:
If similar slots don't all have the same parameter ID the name map's prioritized selections make it easy to configure the general case with exceptions, see Name Map Ordering and Priorities.
Also, in the Name Map, specify the parameter ID in the Name column. Some common USGS parameter IDs include:
– Mean discharge in cubic feet per second = 00060
– Elevation of reservoir water surface above datum, feet = 00062
– Stage or gage height, feet = 00065
More information can be found in the service documentation at:
Although the Site Map utility and wild carding in Name Maps are the preferred approach, you can still use the original approach where the name map specifies the full query string. In the name map, create a slot selection for each slot, then specify the query strings in the Name column. These are typically in the following form:
&Argument=value&Argument2=value
Some common arguments are as follows:
• Site or location—eight-digit site location
• parameterCd or variable—data to return from the gage. Common variables values are as follows:
See Figure 4.32 for an example, Otowi.Inflow is mapped to gage 08313000 and parameter 00060 (mean daily flow).
Figure 4.32  USGS Daily Values name map example using query syntax
Note:  Date ranges are handled automatically by the specification of start and finish timesteps in the Database DMI; they do not need to be specified in the name map. Similarly, format is also automatic.
Revised: 07/03/2024