Package {MVLs}

Title: Model Validation Levels
Version: 2.1
Description: Calculates Model Validation Levels from user provided model data, referent data, and scope files as described by Stafford, Provost, & Jones (2025) <doi:10.1080/17477778.2025.2536096>. Data files must be structured according to one of the accepted Excel templates as shown by Jones, Provost, Stafford, & Young (2026) https://www.afit.edu/docs/MVL%20User%20Guide%20v4.pdf.
License: MIT + file LICENSE
Imports: dplyr, DT, geometry, htmlwidgets, lhs, magrittr, openxlsx, purrr, RANN, readxl, shiny, shinyWidgets, stats, tibble, tidyselect, markdown, utils
Encoding: UTF-8
RoxygenNote: 7.3.3
Suggests: spelling
Language: en-US
NeedsCompilation: no
Packaged: 2026-07-17 17:13:51 UTC; cweeks
Author: Kyle Provost [aut], Corinne Stafford [aut, cre], Ronald Prescott [ctb], Chloe Young [ctb], Nicholas Jones [ctb]
Maintainer: Corinne Stafford <corinne.stafford.ctr@us.af.mil>
Repository: CRAN
Date/Publication: 2026-07-27 14:40:24 UTC

Model Validation Level (MVL)

Description

Quantifies validity of model outputs compared to referent data according to a defined scope. It generates a validation report containing MVL metrics.

Usage

MVL(data_files, scope_files, is.MVLa = FALSE)

Arguments

data_files

A character vector of file paths to the data files. This should include paths to both the model output files and the referent data files. All files are expected to be in Excel format (.xlsx or .xls).

scope_files

A character vector of file paths to the scope file(s). These Excel files define the model outputs (responses), factors, and factor levels to be included in the validation analysis.

is.MVLa

A logical value. If TRUE, performs an accuracy only MVLa analysis. Defaults to FALSE.

Details

This function calculates MVLs for defined model outputs. It reads model and referent data from Excel files, understands the scope of the validation from scope file(s), and computes a suite of metrics to assess model validity.

Value

The function returns a validation report as a list object that contains the computed MVL metrics. The exact structure of the report depends on the data and/or scope(s) provided.

References

Jones, N., Provost, K., Stafford, C., & Young, C. (2026). Model Validation Level (MVL) R Tool User Guide. Scientific Test and Analysis Techniques Center of Excellence. STAT COE

Stafford, C., Jones, N., & Provost, K. (2024). Case Studies on Model Validation Levels. Scientific Test and Analysis Techniques Center of Excellence. STAT COE

Stafford, C., Provost, K., & Jones, N. (2025). Model Validation Levels: Methods and Implementation. Scientific Test and Analysis Techniques Center of Excellence. STAT COE

Stafford, C., Provost, K., & Jones, N. (2025). Model validation levels: an automatable framework for model validation. Journal of Simulation. doi:10.1080/17477778.2025.2536096

Examples

#Find file paths for example model, referent, and scope files
example_model <- system.file("extdata", "example_stochastic_model.xlsx", package = "MVLs")
example_referent <- system.file("extdata", "example_referent.xlsx", package = "MVLs")
example_scope <- system.file("extdata", "example_scope.xlsx", package = "MVLs")

#Define data files (model outputs and referent data)
data_files <- c(
  example_model,
  example_referent
)

#Define scope file detailing scope of intended use
scope_files <- c(
  example_scope
)

#Calculate validation metrics
result <- MVL(data_files, scope_files)

#Display results
result

Run the MVL app user interface

Description

Launches an interactive user interface for the Model Validation Level (MVL) tool.

Usage

MVL_app()

Details

This function starts a local Shiny web application, providing a graphical user interface (GUI) built around the MVL function. The R console will be occupied while the app is running.

The application's key features include:

Value

This function does not return a value to the R session. Its primary effect is to launch the Shiny application.

References

Jones, N., Provost, K., Stafford, C., & Young, C. (2026). Model Validation Level (MVL) R Tool User Guide. Scientific Test and Analysis Techniques Center of Excellence. STAT COE

Stafford, C., Jones, N., & Provost, K. (2024). Case Studies on Model Validation Levels. Scientific Test and Analysis Techniques Center of Excellence. STAT COE

Stafford, C., Provost, K., & Jones, N. (2025). Model Validation Levels: Methods and Implementation. Scientific Test and Analysis Techniques Center of Excellence. STAT COE

Stafford, C., Provost, K., & Jones, N. (2025). Model validation levels: an automatable framework for model validation. Journal of Simulation. doi:10.1080/17477778.2025.2536096

Examples

# Open the MVL tool User Interface
 if (interactive()) {
  MVL_app()
}

Model Validation Level accuracy-only method (MVLa)

Description

Quantifies validity of model outputs compared to referent data according to a defined scope of intended use. It generates a validation report containing MVL metrics. MVLa() does not consider the variability of model outputs and is suitable for deterministic models.

Usage

MVLa(data_files, scope_files)

Arguments

data_files

A character vector of file paths to the data files. This should include paths to both the model output files and the referent data files. All files are expected to be in Excel format (.xlsx or .xls).

scope_files

A character vector of file paths to the scope file(s). These Excel file(s) define the model outputs (responses), factors, and factor levels to be included in the validation analysis.

Details

This function calculates the accuracy-only version of the MVL. It reads model and referent data from Excel files, understands the scope of the validation from scope file(s), and computes a suite of metrics to assess model validity.

Value

The function returns a validation report as a list object that contains the computed MVL metrics. The exact structure of the report depends on the data and/or scope(s) provided.

References

Jones, N., Provost, K., Stafford, C., & Young, C. (2026). Model Validation Level (MVL) R Tool User Guide. Scientific Test and Analysis Techniques Center of Excellence. STAT COE

Stafford, C., Jones, N., & Provost, K. (2024). Case Studies on Model Validation Levels. Scientific Test and Analysis Techniques Center of Excellence. STAT COE

Stafford, C., Provost, K., & Jones, N. (2025). Model Validation Levels: Methods and Implementation. Scientific Test and Analysis Techniques Center of Excellence. STAT COE

Stafford, C., Provost, K., & Jones, N. (2025). Model validation levels: an automatable framework for model validation. Journal of Simulation. doi:10.1080/17477778.2025.2536096

Examples

#Find file paths for example files
example_model <- system.file("extdata", "example_deterministic_model.xlsx", package = "MVLs")
example_referent <- system.file("extdata", "example_referent.xlsx", package = "MVLs")
example_scope <- system.file("extdata", "example_scope.xlsx", package = "MVLs")

#Define data files (model outputs and referent data)
data_files <- c(
  example_model,
  example_referent
)

#Define scope file detailing scope of intended use
scope_files <- c(
  example_scope
)

#Calculate validation metrics
result <- MVLa(data_files, scope_files)

#Display results
result