Structural Dimension Selection

Selection problem

An SDR kernel produces an ordered basis, but the structural dimension d must still be selected. risdr supports AIC, BIC, CAIC, ICOMP, CICOMP, predictive cross-validation, a complexity-aware cross-validation family, and a bootstrap ladle diagnostic. ICOMP adds a covariance-complexity penalty to fit (Bozdogan 2000).

Information criteria

library(risdr)

sim <- simulate_risdr_data(
  n = 150,
  p = 18,
  d = 2,
  rho = 0.6,
  seed = 8101
)

fit <- fit_risdr(
  sim$X,
  sim$y,
  sdr_method = "dr",
  cov_method = "oas",
  d_max = 6,
  selector = "cicomp"
)

fit$d_table
#>   d      AIC      BIC     CAIC    ICOMP   CICOMP
#> 1 1 651.5249 660.5568 663.5568 645.5249 663.5568
#> 2 2 648.3686 660.4112 664.4112 640.3713 664.4138
#> 3 3 650.3343 665.3875 670.3875 640.3378 670.3910
#> 4 4 647.2828 665.3467 671.3467 635.3002 671.3641
#> 5 5 643.5549 664.6294 671.6294 629.5790 671.6535
#> 6 6 643.7860 667.8711 675.8711 627.8198 675.9048
fit$d
#> [1] 1

Criterion weights provide a relative summary within the candidate set:

criterion_weights(fit$d_table, criterion = "CICOMP")
#>   d criterion    value      delta      weight
#> 1 1    CICOMP 663.5568  0.0000000 0.580042723
#> 2 2    CICOMP 664.4138  0.8569594 0.377897152
#> 3 3    CICOMP 670.3910  6.8341368 0.019030318
#> 4 4    CICOMP 671.3641  7.8072129 0.011698906
#> 5 5    CICOMP 671.6535  8.0966365 0.010122731
#> 6 6    CICOMP 675.9048 12.3479912 0.001208169

They are not posterior probabilities and do not establish that a candidate set contains the true dimension.

Predictive cross-validation

cv <- select_dimension_cv(
  sim$X,
  sim$y,
  sdr_method = "dr",
  cov_method = "oas",
  d_max = 5,
  v = 5,
  metric = "RMSE",
  seed = 8101
)

cv$selected_d
#> [1] 5
cv$cv_table
#>   d     RMSE      MAE     MAPE          R2 Adjusted_R2 Correlation   RMSE_SD
#> 1 1 2.236412 1.615084 209.2315 -0.08684411  -0.1256600   0.3416631 0.4914911
#> 2 2 2.248237 1.636117 220.3874 -0.08577879  -0.1662068   0.3269974 0.5288902
#> 3 3 2.231080 1.627901 225.7476 -0.09128110  -0.2171982   0.3441692 0.5272284
#> 4 4 2.224662 1.609828 223.9537 -0.08287304  -0.2561327   0.3362836 0.4507436
#> 5 5 2.185786 1.593290 227.2578 -0.01489826  -0.2263354   0.3929539 0.5026319
#>      MAE_SD  MAPE_SD     R2_SD Adjusted_R2_SD Correlation_SD
#> 1 0.3106409 113.7570 0.4238647      0.4390027      0.2728125
#> 2 0.2853035 133.5485 0.4010803      0.4307900      0.2475461
#> 3 0.2944188 140.1272 0.4734029      0.5280263      0.2747317
#> 4 0.2540812 142.9218 0.4323070      0.5014762      0.2518841
#> 5 0.2754011 139.2081 0.3307216      0.3996219      0.1569027

All standardisation and covariance estimation occur inside the training fold. This prevents validation observations from influencing the fitted projection.

Complexity-aware cross-validation

For candidate d, the combined criterion rescales prediction error and an information criterion to [0,1], then uses

\[ \operatorname{CVIC}(d) = \widetilde{\operatorname{RMSE}}(d) + \lambda\widetilde{\operatorname{IC}}(d). \]

cv_icomp <- select_dimension_cv_icomp(
  sim$X,
  sim$y,
  sdr_method = "dr",
  cov_method = "oas",
  d_max = 5,
  v = 5,
  lambda = 1,
  seed = 8101
)

cv_icomp$cv_table
#>   d mean_RMSE mean_MAE     mean_R2 mean_Adjusted_R2 mean_Correlation mean_AIC
#> 1 1  2.236412 1.615084 -0.08684411       -0.1256600        0.3416631 520.1835
#> 2 2  2.248237 1.636117 -0.08577879       -0.1662068        0.3269974 520.3479
#> 3 3  2.231080 1.627901 -0.09128110       -0.2171982        0.3441692 518.5343
#> 4 4  2.224662 1.609828 -0.08287304       -0.2561327        0.3362836 516.9854
#> 5 5  2.185786 1.593290 -0.01489826       -0.2263354        0.3929539 515.2463
#>   mean_BIC mean_CAIC mean_ICOMP mean_CICOMP   sd_RMSE RMSE_scaled BIC_scaled
#> 1 528.5460  531.5460   514.1838    531.5463 0.4914911   0.8106527  0.0000000
#> 2 531.4979  535.4979   512.3507    535.5007 0.5288902   1.0000000  0.4751320
#> 3 532.4718  537.4718   508.5428    537.4803 0.5272284   0.7252778  0.6318866
#> 4 533.7103  539.7103   505.0011    539.7261 0.4507436   0.6225133  0.8312449
#> 5 534.7588  541.7588   501.2807    541.7932 0.5026319   0.0000000  1.0000000
#>   CAIC_scaled CICOMP_scaled     CVBIC    CVCAIC  CVCICOMP
#> 1   0.0000000     0.0000000 0.8106527 0.8106527 0.8106527
#> 2   0.3869557     0.3859120 1.4751320 1.3869557 1.3859120
#> 3   0.5802312     0.5791024 1.3571644 1.3055090 1.3043802
#> 4   0.7994241     0.7982722 1.4537582 1.4219374 1.4207855
#> 5   1.0000000     1.0000000 1.0000000 1.0000000 1.0000000
cv_icomp[c(
  "selected_d_rmse",
  "selected_d_cvbic",
  "selected_d_cvcaic",
  "selected_d_cvcicomp"
)]
#> $selected_d_rmse
#> [1] 5
#> 
#> $selected_d_cvbic
#> [1] 1
#> 
#> $selected_d_cvcaic
#> [1] 1
#> 
#> $selected_d_cvcicomp
#> [1] 1

lambda should be examined through sensitivity analysis. It is a decision weight, not a parameter estimated by the likelihood.

Ladle diagnostic

ladle <- select_dimension_ladle(
  sim$X,
  sim$y,
  sdr_method = "dr",
  cov_method = "oas",
  d_max = 4,
  B = 20,
  seed = 8101
)

ladle$selected_d
#> [1] 1
ladle$ladle_table
#>    d eigen_part stability_part     ladle
#> d1 1 0.07699466      0.1797929 0.2567876
#> d2 2 0.07268661      0.7533782 0.8260648
#> d3 3 0.07050814      0.8449390 0.9154471
#> d4 4 0.06745504      0.9383633 1.0058184

The implemented ladle is a diagnostic combining residual eigenvalue mass with bootstrap subspace instability. It should be reported as a complementary diagnostic rather than treated as an infallible selector.

Decision protocol

A defensible selection report should state:

  1. the candidate range for d;
  2. the SDR and covariance estimators;
  3. the slicing and stabilisation settings;
  4. every criterion examined;
  5. the resampling design and seed;
  6. whether selectors agree;
  7. the sensitivity of substantive conclusions to d.

References

Bozdogan, Hamparsum. 2000. “Akaike’s Information Criterion and Recent Developments in Information Complexity.” Journal of Mathematical Psychology 44 (1): 62–91. https://doi.org/10.1006/jmps.1999.1277.