---
title: "Pricing workflow building blocks"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Pricing workflow building blocks}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

`insurancerating` provides building blocks for common actuarial pricing tasks in
GLM-based tariff analysis. The package does not prescribe a single pricing
method. Instead, it supports practical steps that often appear in insurance
pricing work: portfolio analysis, model interpretation, tariff refinement and
model validation.

This vignette gives a compact overview of those building blocks and how they
can be combined.

```{r}
library(insurancerating)
```

## 1. Start with portfolio experience

A pricing analysis often starts by checking how the observed portfolio behaves
by risk factor. This is useful before modelling, but also later when reviewing
whether fitted relativities are plausible.

`factor_analysis()` summarises exposure, claim frequency, average severity,
risk premium and related metrics by one or more risk factors.

```{r}
fa <- factor_analysis(
  MTPL,
  risk_factors = "zip",
  claim_count = "nclaims",
  claim_amount = "amount",
  exposure = "exposure"
)

head(fa)
```

The output helps answer practical questions such as:

- where exposure is concentrated
- whether observed differences are credible or noisy
- whether a segment is driven by a small number of claims
- which risk factors may need closer modelling or refinement

For numeric variables with long or skewed tails, `outlier_histogram()` can help
inspect extreme observations before fitting severity models or constructing
tariff segments.

```{r}
outlier_histogram(
  MTPL2,
  x = "premium",
  upper = 100,
  density = FALSE
)
```

## 2. Assess large losses

Large claims can dominate severity and risk premium analysis. In capped
severity workflows, it is often useful to assess a cap first, decompose the
historical claim amounts, and then decide how the excess burden should be
allocated. A low threshold increases pricing responsiveness but introduces
volatility. A high threshold improves stability but may understate structural
differences between segments.

```{r, eval = FALSE}
portfolio <- data.frame(
  policy_id = 1:10,
  sector = rep(c("Industry", "Retail"), each = 5),
  claim_count = c(0, 1, 1, 1, 1, 0, 1, 1, 1, 1),
  claim_amount = c(
    0, 25000, 120000, 50000, 175000,
    0, 40000, 90000, 150000, 300000
  ),
  policy_years = rep(1, 10)
)

thresholds <- assess_excess_threshold(
  portfolio,
  claim_amount = "claim_amount",
  thresholds = c(25000, 50000, 100000, 150000),
  exposure = "policy_years",
  group = "sector",
  claim_count = "claim_count"
)

if (requireNamespace("gt", quietly = TRUE)) {
  as_gt(thresholds)
} else {
  thresholds
}
```

After choosing a threshold, `redistribute_excess_loss()` caps the large losses
and allocates the excess burden. The same calculation supports two modelling
workflows:

| Output | Typical redistribution weight | Interpretation | Typical use |
|---|---|---|---|
| `"redistributed_claim"` | Claim count or expected claim count | Excess loss redistributed across claims | One severity model |
| `"excess_loading"` | Earned exposure | Excess risk premium per exposure unit | Separate component of total risk premium |

The total excess loss is preserved in both workflows.

```{r, eval = FALSE}
adjusted <- redistribute_excess_loss(
  portfolio,
  claim_amount = "claim_amount",
  threshold = 100000,
  claim_count = "claim_count",
  risk_factor = "sector",
  redistribution_method = "partial",
  output = "redistributed_claim"
)
```

Portfolio redistribution gives every claim the same excess amount per claim.
Risk-factor redistribution uses only the experience of each group. Partial
redistribution balances portfolio stability, group responsiveness and the
credibility of observed excess experience.

```{r, eval = FALSE}
severity_data <- adjusted[adjusted$claim_count > 0, ]
severity_model <- glm(
  claim_amount_adjusted_average ~ sector,
  weights = claim_count,
  family = Gamma(link = "log"),
  data = severity_data
)
```

The adjusted average claim amount already contains the redistributed cost of
large losses. A separate excess-premium component must therefore not be added
again. This is a concise workflow and generally works well when modelled groups
contain sufficient claims and sparse factor levels have been combined.

The amount allocated to an individual row is not an observed claim amount for
that row. In a sparse sector, a severity model can therefore interpret allocated
portfolio excess as evidence about that sector. Inspect claim volume, combine
sparse levels using an actuarially meaningful hierarchy and validate coefficient
stability over time. A minimum such as 20 claims can be a useful starting point,
but is illustrative and must be selected for the portfolio at hand.

When retained severity and allocated excess should remain conceptually
separate, use earned exposure as the redistribution weight:

```{r, eval = FALSE}
loading_result <- redistribute_excess_loss(
  portfolio,
  claim_amount = "claim_amount",
  threshold = 100000,
  claim_count = "claim_count",
  redistribution_weight = "policy_years",
  risk_factor = "sector",
  redistribution_method = "partial",
  output = "excess_loading"
)

frequency_model <- glm(
  claim_count ~ sector + offset(log(policy_years)),
  family = poisson(link = "log"),
  data = loading_result
)
retained_severity_model <- glm(
  claim_amount_capped ~ sector,
  weights = claim_count,
  family = Gamma(link = "log"),
  data = loading_result[loading_result$claim_count > 0, ]
)

loading_result$predicted_frequency <-
  predict(frequency_model, type = "response") / loading_result$policy_years
loading_result$predicted_retained_severity <- predict(
  retained_severity_model,
  newdata = loading_result,
  type = "response"
)
loading_result$predicted_retained_risk_premium <-
  loading_result$predicted_frequency *
  loading_result$predicted_retained_severity
loading_result$predicted_total_risk_premium <-
  loading_result$predicted_retained_risk_premium +
  loading_result$excess_loading
```

Here `excess_loading` is an amount per policy year. Claim volume can still
determine partial credibility while policy years determine both the allocation
shares and the unit of the loading. This approach avoids treating allocated
portfolio excess as an observed individual claim severity.

## 3. Translate continuous factors into tariff segments

Many tariffs use grouped versions of continuous variables such as age, vehicle
age or insured value. `risk_factor_gam()` can be used to inspect the fitted
shape of a continuous risk factor. `derive_tariff_segments()` can then derive
candidate segment boundaries from that pattern.

```{r}
age_gam <- risk_factor_gam(
  data = MTPL,
  claim_count = "nclaims",
  risk_factor = "age_policyholder",
  exposure = "exposure"
)

age_segments <- derive_tariff_segments(age_gam)
age_segments
```

The derived segments can be added back to the portfolio with
`add_tariff_segments()`.

```{r}
portfolio <- MTPL |>
  add_tariff_segments(age_segments, name = "age_policyholder_segment")

head(portfolio[, c("age_policyholder", "age_policyholder_segment")])
```

These functions are intended to support actuarial judgement, not replace it.
Candidate segment boundaries should still be reviewed for credibility,
stability and practical usability.

## 4. Fit and interpret a GLM

GLMs are widely used in insurance pricing because they provide an interpretable
multiplicative structure. After fitting a model, `rating_table()` expresses the
coefficients in tariff-table form.

```{r}
portfolio$zip <- as.factor(portfolio$zip)

freq_model <- glm(
  nclaims ~ zip + age_policyholder_segment + offset(log(exposure)),
  family = poisson(),
  data = portfolio
)

rt <- rating_table(
  freq_model,
  model_data = portfolio,
  exposure = "exposure"
)

head(rt$df)
```

Observed portfolio experience can be attached to the rating table with
`add_portfolio_experience()`. When `data` is supplied, the function calculates
the observed experience for the rating-table risk factors automatically. This
makes the comparison between model relativities and observed experience
explicit.

```{r}
rt |>
  add_portfolio_experience(
    data = portfolio,
    claim_count = "nclaims",
    exposure = "exposure"
  ) |>
  autoplot(risk_factors = "zip", metric = "frequency")
```

## 5. Refine tariff effects when needed

Raw model output may be statistically valid but still unsuitable for direct
tariff use. Sparse levels, noisy estimates or non-monotonic adjacent effects can
make a tariff hard to explain or maintain.

The refinement workflow makes these adjustments explicit:

```{r, eval = FALSE}
refined_model <- prepare_refinement(freq_model) |>
  add_smoothing(
    model_variable = "age_policyholder_segment",
    source_variable = "age_policyholder",
    weights = "exposure"
  ) |>
  add_restriction(restrictions) |>
  refit()
```

Common refinement tasks include:

- smoothing adjacent tariff levels
- fixing selected coefficients to actuarial or commercial assumptions
- applying sublevel relativities within a broader GLM factor level
- refitting the model while preserving the intended tariff structure

These tools are most useful when the statistical model already captures the main
risk structure and the remaining work is tariff refinement.

## 6. Validate model behaviour

Pricing models should be checked before their output is used in a tariff.
`insurancerating` contains helpers for several common checks:

- `check_overdispersion()` for Poisson frequency models
- `check_residuals()` for simulation-based residual diagnostics using DHARMa
- `bootstrap_performance()` for predictive stability with metrics such as RMSE
- `rating_grid()` to inspect observed rating-grid combinations

For example:

```{r}
check_overdispersion(freq_model)
```

```{r, eval = FALSE}
check_residuals(freq_model) |>
  autoplot()
```

Validation does not make a tariff decision by itself. It gives evidence about
model fit, stability and areas that may need further review.

## Typical workflow

One possible workflow is:

1. Inspect the portfolio with `factor_analysis()` and `outlier_histogram()`.
2. Assess large-loss thresholds with `assess_excess_threshold()` where capped
   severity or excess-loss loadings are relevant.
3. Use `redistribute_excess_loss()` either to create one redistributed severity
   response or to calculate a separate excess loading per exposure unit.
4. Analyse continuous risk factors with `risk_factor_gam()`.
5. Create candidate tariff segments with `derive_tariff_segments()`.
6. Fit GLMs for frequency, severity or pure premium.
7. Interpret coefficients with `rating_table()`.
8. Compare fitted relativities with observed experience using
   `add_portfolio_experience()`.
9. Apply refinement where needed with `prepare_refinement()`, `add_smoothing()`,
   `add_restriction()` or `add_relativities()`.
10. Validate the resulting model with the model-performance helpers.

The exact order and choice of functions depends on the portfolio, product,
data quality and pricing objective.

## Next steps

For a worked example, see:

- [Getting started](getting-started.html)

For coefficient refinement:

- [Refinement building blocks](refinement-workflow.html)

For validation:

- [Model validation](model-validation.html)
