Last updated on 2025-10-31 00:51:03 CET.
| Flavor | Version | Tinstall | Tcheck | Ttotal | Status | Flags |
|---|---|---|---|---|---|---|
| r-devel-linux-x86_64-debian-clang | 1.2.1 | 8.42 | 202.89 | 211.31 | ERROR | |
| r-devel-linux-x86_64-debian-gcc | 1.2.1 | 6.24 | 141.12 | 147.36 | ERROR | |
| r-devel-linux-x86_64-fedora-clang | 1.2.1 | 16.00 | 302.50 | 318.50 | ERROR | |
| r-devel-linux-x86_64-fedora-gcc | 1.2.1 | 17.00 | 302.67 | 319.67 | ERROR | |
| r-devel-windows-x86_64 | 1.2.1 | 12.00 | 187.00 | 199.00 | ERROR | |
| r-patched-linux-x86_64 | 1.2.1 | 8.45 | 193.43 | 201.88 | ERROR | |
| r-release-linux-x86_64 | 1.2.1 | 7.21 | 192.74 | 199.95 | ERROR | |
| r-release-macos-arm64 | 1.2.1 | 4.00 | 107.00 | 111.00 | NOTE | |
| r-release-macos-x86_64 | 1.2.1 | 7.00 | 206.00 | 213.00 | NOTE | |
| r-release-windows-x86_64 | 1.2.1 | 12.00 | 188.00 | 200.00 | ERROR | |
| r-oldrel-macos-arm64 | 1.2.1 | 4.00 | 95.00 | 99.00 | NOTE | |
| r-oldrel-macos-x86_64 | 1.2.1 | 5.00 | 187.00 | 192.00 | NOTE | |
| r-oldrel-windows-x86_64 | 1.2.1 | 14.00 | 249.00 | 263.00 | ERROR |
Version: 1.2.1
Check: CRAN incoming feasibility
Result: NOTE
Maintainer: ‘Jakub Wiśniewski <jakwisn@gmail.com>’
The Description field contains
<arXiv:2104.00507>.
Please refer to arXiv e-prints via their arXiv DOI <doi:10.48550/arXiv.YYMM.NNNNN>.
Flavors: r-devel-linux-x86_64-debian-clang, r-devel-linux-x86_64-debian-gcc
Version: 1.2.1
Check: Rd files
Result: NOTE
checkRd: (-1) choose_metric.Rd:35: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) choose_metric.Rd:36: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) choose_metric.Rd:37: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) confusion_matrx.Rd:23: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) disparate_impact_remover.Rd:28: Lost braces
28 | pigeonholing. The number of pigeonholes is fixed and equal to min{101, unique(a)}, where a is vector with values for subgroup. So if some subgroup is not numerous and
| ^
checkRd: (-1) fairness_check.Rd:47: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:48: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:49: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:50: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:51: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:52: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:53: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:54: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:55: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:56: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:57: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:58: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:61: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:62: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:63: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:64: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:65: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_check.Rd:66: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:12: Lost braces
12 | \item{scale}{logical, if code{TRUE} metrics will be scaled to mean 0 and sd 1. Default \code{FALSE}}
| ^
checkRd: (-1) fairness_heatmap.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_heatmap.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:18: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_pca.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_radar.Rd:18: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) fairness_radar.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:25: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:26: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:27: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_matrices.Rd:28: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:30: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:31: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:32: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) group_metric.Rd:33: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) metric_scores.Rd:18: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) metric_scores.Rd:19: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:20: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:21: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:22: Lost braces in \itemize; \value handles \item{}{} directly
checkRd: (-1) performance_and_fairness.Rd:23: Lost braces in \itemize; \value handles \item{}{} directly
Flavors: r-devel-linux-x86_64-debian-clang, r-devel-linux-x86_64-debian-gcc, r-devel-linux-x86_64-fedora-clang, r-devel-linux-x86_64-fedora-gcc, r-devel-windows-x86_64, r-patched-linux-x86_64, r-release-linux-x86_64, r-release-macos-arm64, r-release-macos-x86_64, r-release-windows-x86_64, r-oldrel-macos-arm64, r-oldrel-macos-x86_64, r-oldrel-windows-x86_64
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in ‘fairmodels-Ex.R’ failed
The error most likely occurred in:
> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 4.352454e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavors: r-devel-linux-x86_64-debian-clang, r-devel-linux-x86_64-debian-gcc
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [36s/45s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.3).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1607198 , mean = 0.5447872 , max = 0.8647956
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8523345 , mean = 9.287262e-05 , max = 0.7862265
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -252.3956 , mean = 744.946 , max = 1560.699
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -303.3584 , mean = -2.221964e-12 , max = 284.3314
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -252.3956 , mean = 744.946 , max = 1560.699
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -303.3584 , mean = -2.221964e-12 , max = 284.3314
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 6/13 metrics calculated for all models ( <1b>[33m7 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6349206
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-linux-x86_64-debian-clang
Version: 1.2.1
Check: re-building of vignette outputs
Result: ERROR
Error(s) in re-building vignettes:
...
--- re-building ‘Advanced_tutorial.Rmd’ using rmarkdown
--- finished re-building ‘Advanced_tutorial.Rmd’
--- re-building ‘Basic_tutorial.Rmd’ using rmarkdown
Quitting from Basic_tutorial.Rmd:254-257 [unnamed-chunk-19]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<error/rlang_error>
Error in `rep()`:
! attempt to replicate an object of type 'object'
---
Backtrace:
▆
1. ├─base::plot(fheatmap, text_size = 3)
2. └─fairmodels:::plot.fairness_heatmap(fheatmap, text_size = 3)
3. └─base::ifelse(...)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Error: processing vignette 'Basic_tutorial.Rmd' failed with diagnostics:
attempt to replicate an object of type 'object'
--- failed re-building ‘Basic_tutorial.Rmd’
SUMMARY: processing the following file failed:
‘Basic_tutorial.Rmd’
Error: Vignette re-building failed.
Execution halted
Flavors: r-devel-linux-x86_64-debian-clang, r-devel-linux-x86_64-debian-gcc, r-patched-linux-x86_64, r-release-linux-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [23s/28s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.3).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1650944 , mean = 0.5451945 , max = 0.8664059
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8489308 , mean = -0.0003144042 , max = 0.7765741
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -86.93445 , mean = 755.0572 , max = 1816.04
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -368.6442 , mean = 1.705729e-13 , max = 317.2715
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -86.93445 , mean = 755.0572 , max = 1816.04
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -368.6442 , mean = 1.705729e-13 , max = 317.2715
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 6/13 metrics calculated for all models ( <1b>[33m7 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6348175
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-linux-x86_64-debian-gcc
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in ‘fairmodels-Ex.R’ failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 1.280546e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.06718651 , mean = 0.6975836 , max = 0.9967857
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7225752 , mean = 0.002416363 , max = 0.634748
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-devel-linux-x86_64-fedora-clang
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [59s/66s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.3).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1559462 , mean = 0.5452587 , max = 0.8694043
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8527447 , mean = -0.000378572 , max = 0.7864532
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -151.5396 , mean = 753.8965 , max = 1779.219
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -362.4764 , mean = 5.860634e-13 , max = 310.5362
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -151.5396 , mean = 753.8965 , max = 1779.219
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -362.4764 , mean = 5.860634e-13 , max = 310.5362
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 6/13 metrics calculated for all models ( <1b>[33m7 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6425121
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 327.8173 , mean = 756.3617 , max = 1159.246
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -635.2394 , mean = 0.1289064 , max = 608.1891
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-linux-x86_64-fedora-clang
Version: 1.2.1
Check: re-building of vignette outputs
Result: ERROR
Error(s) in re-building vignettes:
--- re-building ‘Advanced_tutorial.Rmd’ using rmarkdown
--- finished re-building ‘Advanced_tutorial.Rmd’
--- re-building ‘Basic_tutorial.Rmd’ using rmarkdown
Quitting from Basic_tutorial.Rmd:254-257 [unnamed-chunk-19]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<error/rlang_error>
Error in `rep()`:
! attempt to replicate an object of type 'object'
---
Backtrace:
▆
1. ├─base::plot(fheatmap, text_size = 3)
2. └─fairmodels:::plot.fairness_heatmap(fheatmap, text_size = 3)
3. └─base::ifelse(...)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Error: processing vignette 'Basic_tutorial.Rmd' failed with diagnostics:
attempt to replicate an object of type 'object'
--- failed re-building ‘Basic_tutorial.Rmd’
SUMMARY: processing the following file failed:
‘Basic_tutorial.Rmd’
Error: Vignette re-building failed.
Execution halted
Flavors: r-devel-linux-x86_64-fedora-clang, r-devel-linux-x86_64-fedora-gcc, r-devel-windows-x86_64, r-release-windows-x86_64, r-oldrel-windows-x86_64
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in ‘fairmodels-Ex.R’ failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 1.280546e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-devel-linux-x86_64-fedora-gcc
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [56s/66s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.3).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.145942 , mean = 0.544903 , max = 0.8593951
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8520738 , mean = -2.285049e-05 , max = 0.7923993
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -83.82184 , mean = 744.7284 , max = 1606.464
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -258.5094 , mean = 2.122147e-12 , max = 305.2512
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -83.82184 , mean = 744.7284 , max = 1606.464
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -258.5094 , mean = 2.122147e-12 , max = 305.2512
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6345382
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 0 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 0 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-linux-x86_64-fedora-gcc
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in 'fairmodels-Ex.R' failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 6.648002e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-devel-windows-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running 'testthat.R' [29s]
Running the tests in 'tests/testthat.R' failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.2).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1580515 , mean = 0.5449464 , max = 0.8662081
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8484251 , mean = -6.624993e-05 , max = 0.7898522
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -88.92161 , mean = 758.6242 , max = 1688.851
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -361.9545 , mean = -1.238957e-12 , max = 324.1859
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -88.92161 , mean = 758.6242 , max = 1688.851
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -361.9545 , mean = -1.238957e-12 , max = 324.1859
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 6/13 metrics calculated for all models ( <1b>[33m7 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6459834
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.6.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 2 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at D:\RCompile\CRANpkg\local\4.6\fairmodels.Rcheck\tests\testthat\helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 2 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-devel-windows-x86_64
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in ‘fairmodels-Ex.R’ failed
The error most likely occurred in:
> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.2 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 4.352454e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-patched-linux-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [35s/38s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.3).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1542356 , mean = 0.5449029 , max = 0.8699871
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8534657 , mean = -2.277435e-05 , max = 0.7784863
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.2 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.2 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -32.84055 , mean = 741.3328 , max = 1507.353
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -319.8031 , mean = 7.190836e-14 , max = 304.9299
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.2 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -32.84055 , mean = 741.3328 , max = 1507.353
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -319.8031 , mean = 7.190836e-14 , max = 304.9299
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 11/13 metrics calculated for all models ( <1b>[33m2 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 6/13 metrics calculated for all models ( <1b>[33m7 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6333142
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.2 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.2 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.2 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 1 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-patched-linux-x86_64
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in ‘fairmodels-Ex.R’ failed
The error most likely occurred in:
> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 4.352454e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-release-linux-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running ‘testthat.R’ [34s/41s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.3).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1630211 , mean = 0.5454145 , max = 0.8597198
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8456568 , mean = -0.0005343773 , max = 0.7785638
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -139.6275 , mean = 754.4757 , max = 1620.109
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -335.686 , mean = 5.068602e-13 , max = 343.1502
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -139.6275 , mean = 754.4757 , max = 1620.109
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -335.686 , mean = 5.068602e-13 , max = 343.1502
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6454596
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 0 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at tests/testthat/helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 0 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-release-linux-x86_64
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in 'fairmodels-Ex.R' failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 6.648002e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-release-windows-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running 'testthat.R' [28s]
Running the tests in 'tests/testthat.R' failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.3).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1600242 , mean = 0.5448346 , max = 0.8670448
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8562254 , mean = 4.552168e-05 , max = 0.7849835
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -76.17146 , mean = 755.0365 , max = 1552.098
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -312.9143 , mean = 6.381007e-15 , max = 283.7226
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -76.17146 , mean = 755.0365 , max = 1552.098
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -312.9143 , mean = 6.381007e-15 , max = 283.7226
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6322258
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.5.1 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 0 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at D:\RCompile\CRANpkg\local\4.5\fairmodels.Rcheck\tests\testthat\helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 0 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-release-windows-x86_64
Version: 1.2.1
Check: examples
Result: ERROR
Running examples in 'fairmodels-Ex.R' failed
The error most likely occurred in:
> ### Name: fairness_heatmap
> ### Title: Fairness heatmap
> ### Aliases: fairness_heatmap
>
> ### ** Examples
>
>
> data("german")
>
> y_numeric <- as.numeric(german$Risk) - 1
>
> lm_model <- glm(Risk ~ .,
+ data = german,
+ family = binomial(link = "logit")
+ )
>
> rf_model <- ranger::ranger(Risk ~ .,
+ data = german,
+ probability = TRUE,
+ num.trees = 200,
+ num.threads = 1
+ )
>
> explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1369187 , mean = 0.7 , max = 0.9832426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9572803 , mean = 6.648002e-17 , max = 0.8283475
<1b>[32m A new explainer has been created! <1b>[39m
> explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 9 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.07287302 , mean = 0.6989152 , max = 0.9974848
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.7219256 , mean = 0.001084826 , max = 0.6142332
<1b>[32m A new explainer has been created! <1b>[39m
>
> fobject <- fairness_check(explainer_lm, explainer_rf,
+ protected = german$Sex,
+ privileged = "male"
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
> # same explainers with different cutoffs for female
> fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
+ protected = german$Sex,
+ privileged = "male",
+ cutoff = list(female = 0.4),
+ label = c("lm_2", "rf_2")
+ )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : female: 0.4, male: 0.5
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 10/13 metrics calculated for all models ( <1b>[33m3 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
>
>
> fh <- fairness_heatmap(fobject)
>
> plot(fh)
Error in rep(yes, length.out = len) :
attempt to replicate an object of type 'object'
Calls: plot -> plot.fairness_heatmap -> ifelse
Execution halted
Flavor: r-oldrel-windows-x86_64
Version: 1.2.1
Check: tests
Result: ERROR
Running 'testthat.R' [43s]
Running the tests in 'tests/testthat.R' failed.
Complete output:
> library(testthat)
> library(fairmodels)
>
>
> test_check("fairmodels")
Welcome to DALEX (version: 2.5.3).
Find examples and detailed introduction at: http://ema.drwhy.ai/
Additional features will be available after installation of: ggpubr.
Use 'install_dependencies()' to get all suggested dependencies
Loaded gbm 2.2.2
This version of gbm is no longer under development. Consider transitioning to gbm3, https://github.com/gbm-developers/gbm3
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1703915 , mean = 0.5446966 , max = 0.8693373
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8490232 , mean = 0.000183522 , max = 0.7720914
<1b>[32m A new explainer has been created! <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.004522979 , mean = 0.5448801 , max = 0.8855426
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.8822826 , mean = -5.053611e-13 , max = 0.9767658
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
Fairness object created succesfully
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 4 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 3 in total ( <1b>[31m model type not supported <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -15.90543 , mean = 756.235 , max = 1600.888
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -343.5238 , mean = 1.243839e-12 , max = 247.7049
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
Fairness regression object created succesfully
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from numeric <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[31m model type not supported <1b>[39m )
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -15.90543 , mean = 756.235 , max = 1600.888
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -343.5238 , mean = 1.243839e-12 , max = 247.7049
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[31m not compatible <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[31m y not equal <1b>[39m )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 2 objects ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Performace metric not given, setting deafult ( accuracy )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Fairness Metric not given, setting deafult ( TPR )
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Fairness Metric not given, setting deafult ( TPR )
Performace metric not given, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Creating object with:
Fairness metric: FPR
Performance metric: f1
Fairness data top rows for FPR
group score model
1 African_American 0.35204756 lm
2 Asian 0.04347826 lm
3 Caucasian 0.16393443 lm
4 Hispanic 0.11562500 lm
5 Native_American 0.16666667 lm
6 Other 0.07762557 lm
Performance data for f1 :
1 lm 0.6039853
2 ranger 0.6340996
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Performace metric is NULL, setting deafult ( accuracy )
Creating object with:
Fairness metric: non_existing
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: non_existing
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: auc
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: accuracy
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: precision
Fairness Metric is NULL, setting deafult parity loss metric ( TPR )
Creating object with:
Fairness metric: TPR
Performance metric: recall
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 6172 rows 7 cols
-> target variable : 6172 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 0.1144574 , mean = 0.4551199 , max = 0.995477
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -0.9767658 , mean = 5.053909e-13 , max = 0.8822826
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 8/13 metrics calculated for all models ( <1b>[33m5 NA created<1b>[39m )
<1b>[32m Fairness object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.lm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = -119.546 , mean = 756.4906 , max = 1594.562
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -302.6659 , mean = 3.478115e-13 , max = 332.7938
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[33m changed from character <1b>[39m )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Preparation of a new explainer is initiated
-> model label : ranger ( <1b>[33m default <1b>[39m )
-> data : 1000 rows 3 cols
-> target variable : 1000 values
-> predict function : yhat.ranger will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package ranger , ver. 0.17.0 , task regression ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 361.6527 , mean = 756.1869 , max = 1136.792
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -669.0748 , mean = 0.3037205 , max = 630.6428
<1b>[32m A new explainer has been created! <1b>[39m
Creating fairness regression object
-> Privileged subgroup : character (<1b>[33m from first fairness object <1b>[39m )
-> Protected variable : factor (<1b>[33m from first fairness object <1b>[39m )
-> Fairness objects : 1 object ( <1b>[32m compatible <1b>[39m )
-> Checking explainers : 2 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 3/3 metrics calculated for all models
<1b>[32m Fairness regression object created succesfully <1b>[39m
Creating fairness classification object
-> Privileged subgroup : character (<1b>[32m Ok <1b>[39m )
-> Protected variable : factor (<1b>[32m Ok <1b>[39m )
-> Cutoff values for explainers : 0.5 ( for all subgroups )
-> Fairness objects : 0 objects
-> Checking explainers : 1 in total ( <1b>[32m compatible <1b>[39m )
-> Metric calculation : 13/13 metrics calculated for all models
<1b>[32m Fairness object created succesfully <1b>[39m
changing protected to factor
Preparation of a new explainer is initiated
-> model label : lm ( <1b>[33m default <1b>[39m )
-> data : 15 rows 2 cols
-> target variable : 15 values
-> predict function : yhat.glm will be used ( <1b>[33m default <1b>[39m )
-> predicted values : No value for predict function target column. ( <1b>[33m default <1b>[39m )
-> model_info : package stats , ver. 4.4.3 , task classification ( <1b>[33m default <1b>[39m )
-> predicted values : numerical, min = 7.884924e-12 , mean = 0.4666667 , max = 1
-> residual function : difference between y and yhat ( <1b>[33m default <1b>[39m )
-> residuals : numerical, min = -7.884924e-12 , mean = -5.256659e-13 , max = 7.884915e-12
<1b>[32m A new explainer has been created! <1b>[39m
[ FAIL 3 | WARN 0 | SKIP 0 | PASS 299 ]
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_heatmap.R:2:3'): Test heatmap ──────────────────────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::plot(fairness_heatmap(fobject)) at test_heatmap.R:2:3
2. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(fobject))
3. └─base::ifelse(...)
── Failure ('test_plot_density.R:14:3'): Test plot_density ─────────────────────
plt$labels$x not equal to "probability".
target is NULL, current is character
── Error ('test_plot_fairmodels.R:8:3'): Test plot_fairmodels ──────────────────
Error in `rep(yes, length.out = len)`: attempt to replicate an object of type 'object'
Backtrace:
▆
1. ├─base::suppressWarnings(...) at test_plot_fairmodels.R:8:3
2. │ └─base::withCallingHandlers(...)
3. ├─fairmodels:::expect_s3_class(...)
4. │ ├─testthat::expect(...) at D:\RCompile\CRANpkg\local\4.4\fairmodels.Rcheck\tests\testthat\helper_objects.R:70:20
5. │ └─base::class(object) %in% class
6. ├─fairmodels::plot_fairmodels(fc, type = "fairness_heatmap")
7. └─fairmodels:::plot_fairmodels.fairness_object(fc, type = "fairness_heatmap")
8. └─fairmodels:::plot_fairmodels.default(x, type, ...)
9. └─fairmodels:::plot.fairness_heatmap(fairness_heatmap(x, ...))
10. └─base::ifelse(...)
[ FAIL 3 | WARN 0 | SKIP 0 | PASS 299 ]
Error: Test failures
Execution halted
Flavor: r-oldrel-windows-x86_64