Data Extractor Tool
The Data Extractor Tool (DET) extracts slot data from a collection of model files that were saved as part of an MRM run. Slot data can only be extracted to an RDF file from MRM-created model files. In the future, this may be extended to other collections of model files and output formats.
Why use this tool? Sometime you don’t know what data you want or need to analyze before you stat the multiple runs. Using the functionality to save a per-trace model file, you can make the time-consuming runs, and preserve the results. If you realized you forgot or need to get additional data out of the model files, you don’t need to re-run the MRM, but can instead use the Data Extractor Tool to extract the necessary data.
Requirements for Use
There are two main requirements to use the DET:
1. Multiple Run Management (MRM) enables you to save per-trace model files. Before making the MRM runs, you must enable this feature to save the per-trace model files. See Save Model File for more information on how to enable this tool.
2. The DET initially only extract slot data to an RDF file. Thus, you must have an output control file that specifies the slots that you would like to extract from each model file. See Control File for more information on the format for the control file.
Accessing the DET
The Data Extractor Tool is available from two locations:
• On the main RiverWare workspace, use Utilities and then Data Extractor Tool menu.
• On the MRM Run control, use the View and then Data Extractor Tool menu.
Using the DET
The DET has two tabs - one for configuring the extraction and the other for displaying diagnostics during the extraction. This section first describes the configuration tab, how the extraction works, and then discusses the diagnostics tab.
Configuration Tab
Figure 3.23 shows the DET for a sample model that has been run in MRM and has saved per-trace models.
Figure 3.23 Annotated Sample DET configuration tab
The numbers in the above Figure 3.23 correspond to the following:
1. Select the MRM configuration.
2. The folder location of the saved model files is shown. This came directly from the MRM configuration.
3. The per-trace model files are shown with their modification times. Missing per-trace model files are displayed as disabled and unchecked. Check any existing per-trace model files to use in the extraction process. Unchecked per-trace model files are not included in the extraction.
4. Text provides the number of traces, existing per-trace model files and selected per-trace model files.
5. If desired, select to back up existing RDF data files. When this is selected, the RDF files will be backed up by appending “(N)”, where N is the next highest value for an RDF file. For example, KeySlots.rdf would be backed up to KeySlots (1).rdf, KeySlots (2).rdf, and so on.
6. The same RDF Options are available as in the MRM configuration as described in RDF Options. A typical use case is that you would either specify a new or different RDF control file or an existing (but edited) RDF file. Change any other desired options for the RDF output.
7. Once all configuration is complete, there is a button to start the extraction process. There are also buttons to stop the extraction and to close the DET.
Note: The configuration changes made to the DET don't persist, either during the RiverWare session or in the model file. If you close and reopen the DET, the configuration must start over.
Performing the Extraction
Select the Start button to perform the extraction.
Tip: There can be a lag between pressing Start and the first extraction as the MRM must be initialized.
The DET performs the extraction by creating a batch script and invoking RiverWare in batch mode. The batch script loads each model file in turn and executes one or more batch commands for each model file. Conceptually it works as follows:
For each model specified in the configuration
Open the model
Update the MRM configuration with the new RDF options.
Generate the RDF file
Close the model
END FOR
After all models are visited, the first model is re-opened and it then starts the process to create the Excel files from the RDF files, if specified.
Note: None of the per-trace models are saved as a part of the batch runs, so they will not have any of the modified RDF options if subsequently opened interactively.
More specifically, the automatically-created batch script looks like this:
OpenWorkspace {first per-trace model file}
ExtractMrmData {MRM configuration name} keyword=value …
CloseWorkspace
…
OpenWorkspace {last per-trace model file}
ExtractMrmData {MRM configuration name} keyword=value …
CloseWorkspace
OpenWorkspace {first per-trace model file}
GenerateMrmExcel {MRM configuration name} keyword=value …
CloseWorkspace
ExtractMrmData {MRM configuration name} keyword=value …
CloseWorkspace
…
OpenWorkspace {last per-trace model file}
ExtractMrmData {MRM configuration name} keyword=value …
CloseWorkspace
OpenWorkspace {first per-trace model file}
GenerateMrmExcel {MRM configuration name} keyword=value …
CloseWorkspace
See ExtractMrmData in Automation Tools and GenerateMrmExcel in Automation Tools for more information on the RCL batch commands.
When the extraction is finished, the status dialog opens.
Figure 3.24 DET Status Dialog
If the extraction failed an error message is displayed; clicking the triangle will show the diagnostic messages. An extraction will fail if any of the batch commands fails.
Diagnostics Tab
The diagnostic messages from the batch mode invocation of RiverWare are captured and displayed in the diagnostic tab as shown in Figure 3.25.
Note: If validating the MRM configuration generates warnings, or if informational diagnostic messages are enabled, there can be many diagnostic messages.
Figure 3.25 Data Extractor Tool Diagnostics Tab
Revised: 07/03/2024