A brief {clairvoyant} example • clairvoyant

The {clairvoyant} package was created to provide a rudimentary, flexible way to specify evidence and run models, where {clairvoyant} ties everything together and handles the sensitivity analyses.

You specify a {clairvoyant} project with four files, all in YAML format:

The project file;
One or more model files;
One or more scenario files;
Probably many assertion files.

The project file organizes the information about this project, such as its identifier, metadata such as a title, and project-level specifications such as the parameters (the “input” for the model) and the entities (the units in which evidence is extracted).

A model file specifies a model, consisting of the equations that form the model, organized into one or more modeling steps that will typically use the products of earlier steps.

A scenario file specifies a scenario, consisting of the lower bound, upper bound, point estimate, and step values for each parameter.

Finally, assertion files each contain an assertion as to a parameter value. These assertions are formed by evidence in the form of one or more entities from a source; often extracted entities from articles or other scientific reports (also see https://metabefor.opens.science and https://sysrevving.com). Each entity has an identifier, specifying what is being extracted/specified, and linking it to validation rules. In addition, each assertion has provenance, specifying metadata about the entities’ origin. These contextualize and so constraint generalization of the assertion.

This aim of this {clairvoyant} data model is threefold: first, to separate 1) the act of finding and specifying evidence from 2) the act of deciding exactly which model(s) to run and which scenario(s) to process; second, to allow maximal flexibility as to sensitivity analyses and changes in models at a later stage; and third, to, as much as possible, store the evidence in a format that makes it easily available to other purposes in the future.

### Potentially install from Codeberg
# remotes::install_git("https://codeberg.org/R-packages/clairvoyant");

exampleProjectPath <-
  system.file(
    "example1",
    package = "clairvoyant"
  );

### Import project
importedProject <-
  clairvoyant::read_project(
    exampleProjectPath,
    silent = FALSE
  );

## 
## Retrieved file list in project directory '/tmp/RtmphTfzbm/temp_libpath3fa6a44fed9a5f/clairvoyant/example1': 12 YAML files found.
## Starting to read files from disk.
##   - Importing file 'assertion---prevalence_source1.yml'...
##   - Importing file 'assertion---prevalence_source2.yml'...
##   - Importing file 'assertion---prevalence_source3.yml'...
##   - Importing file 'assertion---prevalence_source4.yml'...
##   - Importing file 'assertion---productionCosts---source5.yml'...
##   - Importing file 'assertion---productionCosts---source6.yml'...
##   - Importing file 'assertion---productionCosts---source7.yml'...
##   - Importing file 'assertion---productionCosts---source8.yml'...
##   - Importing file 'model---basic_model.yml'...
##   - Importing file 'project---dealing_with_drugs.yml'...
##   - Importing file 'scenario---policy_model_1.yml'...
##   - Importing file 'scenario---policy_model_2.yml'...
## 
## Read files from disk. Starting to parse contents as YAML.
##   - Parsing file 'assertion---prevalence_source1.yml'...
##   - Parsing file 'assertion---prevalence_source2.yml'...
##   - Parsing file 'assertion---prevalence_source3.yml'...
##   - Parsing file 'assertion---prevalence_source4.yml'...
##   - Parsing file 'assertion---productionCosts---source5.yml'...
##   - Parsing file 'assertion---productionCosts---source6.yml'...
##   - Parsing file 'assertion---productionCosts---source7.yml'...
##   - Parsing file 'assertion---productionCosts---source8.yml'...
##   - Parsing file 'model---basic_model.yml'...
##   - Parsing file 'project---dealing_with_drugs.yml'...
##   - Parsing file 'scenario---policy_model_1.yml'...
##   - Parsing file 'scenario---policy_model_2.yml'...
## 
## Parsed files. Looking for project specification.
## 
## Found project information. The project identifier is 'dealing_with_drugs' and the title is ''.
## 
## Starting to process models.
##   - Found model with identifier 'basic_model'.
## Imported 1 models.
## 
## Starting to process scenarios.
##   - Found scenario with identifier 'policy_model_1'.
##   - Found scenario with identifier 'policy_model_2'.
## Imported 2 scenarios.
## 
## Starting to process assertions.
##   - Found assertion with identifier 'prevalence_source1'.
##   - Found assertion with identifier 'prevalence_source2'.
##   - Found assertion with identifier 'prevalence_source3'.
##   - Found assertion with identifier 'prevalence_source4'.
##   - Found assertion with identifier 'productionCosts_source5'.
##   - Found assertion with identifier 'productionCosts_source6'.
##   - Found assertion with identifier 'productionCosts_source7'.
##   - Found assertion with identifier 'productionCosts_source8'.
## Imported 8 assertions.
## Reorganizing assertions by parameter.
## Reorganized assertions.
## Reorganizing scenarios.
## Reorganized scenarios.
## Reorganizing project-level parameter information.
## Reorganized project-level parameter information.

Once a project is imported, it can use used to run modeling steps. As an illustration, this runs the first step of the model, with two manually specified values.

clairvoyant::run_model_step(
  project = importedProject,
  modelId = "basic_model",
  modelStep = 1,
  parameterValues = 
    list(
      salePrice = 10,
      productionCosts = 5
    ),
  silent = FALSE
);

## Running model basic_model, step 1...

## [1] 5

This runs the entire model, computing intermediate values, and returning the input, the outcome, and the intermediate values:

clairvoyant::run_model(
  project = importedProject,
  modelId = "basic_model",
  parameterValues = 
    list(
      salePrice = 10,
      productionCosts = 5,
      prevalence = 3.6,
      population_size = 17000000
    ),
  silent = FALSE
);

## $parameterValues
## $parameterValues$salePrice
## [1] 10
## 
## $parameterValues$productionCosts
## [1] 5
## 
## $parameterValues$prevalence
## [1] 3.6
## 
## $parameterValues$population_size
## [1] 1.7e+07
## 
## 
## $intermediateValues
## $intermediateValues$profit_per_pill
## [1] 5
## 
## $intermediateValues$number_of_customers
## [1] 612000
## 
## 
## $outcomeValue
## [1] 3060000
## 
## $df
##   salePrice productionCosts prevalence population_size profit_per_pill
## 1        10               5        3.6         1.7e+07               5
##   number_of_customers total_profit
## 1              612000      3060000
## 
## attr(,"class")
## [1] "clairvoyant"             "clairvoyant_modelResult"

That function is the called recursively on a list of vectors of parameter values. For example:

model1 <-
  clairvoyant::papply(
    parameterList = list(
      salePrice = seq(from = 5, to = 10, by = .1),
      productionCosts = seq(from = 1, to = 8, by = .5),
      prevalence = c(2, 3.6, 7),
      population_size = 17000000
    ),
    func = clairvoyant::run_model,
    args = list(project = importedProject,
                modelId = "basic_model"),
    progressbar = TRUE
  );

## Starting to process 4 parameters ('salePrice', 'productionCosts', 'prevalence' & 'population_size') with a total of 2295 parameter value combinations.

We can then combine this functionality to run the model for a scenario:

policy_model_1_results <-
  clairvoyant::run_scenario(
    project = importedProject,
    scenarioId = "policy_model_1",
    modelId = "basic_model",
    progressbar = TRUE
  );

## Starting to process 4 parameters ('prevalence', 'population_size', 'productionCosts' & 'salePrice') with a total of 575 parameter value combinations.

policy_model_1_results;

## 🔮 {clairvoyant} scenario results
## 
## Ran model 'basic_model' for scenario 'policy_model_1', which specified 4 parameters ('prevalence', 'population_size', 'productionCosts' & 'salePrice'). Computed 2 intermediate values ('profit_per_pill' & 'number_of_customers') to arrive at final outcome 'total_profit'.
## 
## For the point estimate values, I yielded the following results:
## 
##     prevalence population_size productionCosts salePrice profit_per_pill
## 288       5.15        17000000               6         7               1
##     number_of_customers total_profit
## 288              875500       875500

And then we can show how these results vary by the value of a parameter:

clairvoyant::show_parameter_range(
  policy_model_1_results
);