Skip to contents

Check Balance Using Weighted ROC Curves

Source: R/check_auc.R
check_auc.Rd

Computes weighted ROC curves and AUC for evaluating propensity score balance. In causal inference, a weighted ROC curve near the diagonal (AUC around 0.5) indicates good balance between treatment groups.

Usage

check_auc(
  .data,
  .truth,
  .estimate,
  .wts,
  include_observed = TRUE,
  na.rm = TRUE,
  treatment_level = NULL
)

Arguments

.data

A data frame containing the variables.

.truth

The treatment/outcome variable.

.estimate

The propensity score or fitted values.

.wts

Weighting variables (supports tidyselect).

include_observed

Logical. If using .wts, also calculate observed (unweighted) metrics? Defaults to TRUE.

na.rm

A logical value indicating whether to remove missing values before computation. If FALSE (default), missing values in the input will produce NA in the output.

treatment_level

The level of the outcome variable to consider as the treatment/event. If NULL (default), uses the last level for factors or the maximum value for numeric variables.

Value

A tibble with columns:

method

Character. The weighting method ("observed" or weight variable name).

auc

Numeric. The ROC AUC value.

Details

The Area Under the ROC Curve (AUC) provides a single metric for assessing propensity score balance. When propensity scores achieve perfect balance, the weighted distribution of scores should be identical between treatment groups, resulting in an AUC of 0.5 (chance performance).

AUC values significantly different from 0.5 indicate systematic differences in propensity score distributions between groups, suggesting inadequate balance. Values closer to 0.5 indicate better balance achieved by the weighting scheme.

This approach complements traditional balance diagnostics by focusing specifically on the propensity score overlap and balance.

See also

check_balance() for other balance metrics, geom_roc() for plotting ROC curves

Other balance functions: bal_corr(), bal_ks(), bal_smd(), bal_vr(), check_balance()

Examples

# Check balance for propensity scores
check_auc(nhefs_weights, qsmk, .fitted, c(w_ate, w_att))
#> # A tibble: 3 × 2
#>   method     auc
#>   <chr>    <dbl>
#> 1 observed 0.663
#> 2 w_ate    0.502
#> 3 w_att    0.495

# Without observed results
check_auc(nhefs_weights, qsmk, .fitted, w_ate, include_observed = FALSE)
#> # A tibble: 1 × 2
#>   method   auc
#>   <chr>  <dbl>
#> 1 w_ate  0.502