Skip to contents

For each site (cluster), the model is refitted without that site twice on the frozen full-data representation: once with the estimated smoothing parameters (and theta) held at their full-fit values, and once with them re-estimated. The changes in the fitted values of the remaining sites are compared.

Usage

gam_influence(
  model,
  cluster,
  data = NULL,
  which = NULL,
  progress = interactive()
)

Arguments

model

A fitted mgcv::gam() object within the scope described in the README; anything the scope guard recognises as outside it is an error.

cluster

The sites: a column name (in the model frame or in data), or a vector with one value per row of data or per row used by the model. Rows dropped by na.action or subset are matched by row name, and the match is checked against the data values.

data

The data frame used to fit model. Only needed when cluster is not a variable of the model; if NULL, the data named in the model call is used, with a message.

which

Optional subset of site ids to evaluate.

progress

Show a text progress bar.

Value

An object of class gam_influence, a list with

  • sites: one row per site: id, n_obs, fixed, added, total, status, note;

  • hyper: one row per site and hyperparameter: id, parameter, type ("sd" for random-effect terms, reported as the standard deviation; "lambda" for other smooths; "theta"), full, deleted (re-estimated without the site) and ratio. A very large lambda ratio means the smooth became close to a straight line; the exact value is then not meaningful, because the REML score is flat there;

  • info: family, number of rows, and the version numbers used.

What is reported

For each site, three weighted RMS changes in the linear predictor of the other sites' rows (the retained rows):

  • fixed: refit without the site, hyperparameters fixed at their full-fit values. This is what a deletion diagnostic with fixed smoothing parameters sees.

  • total: refit without the site, hyperparameters re-estimated.

  • added: the difference between the two refits, i.e. what re-estimating the hyperparameters adds.

For every retained observation, fixed + added = total exactly (they are changes in the same fitted value). The three columns are RMS summaries of those changes, and RMS values do not add: total is never larger than fixed + added, and equals it only when one change is a non-negative multiple of the other (for example when added is zero). A smaller total therefore only says that the two changes are not perfectly aligned; on its own it does not mean that re-estimation offsets the fixed change. Offsetting shows up as total^2 < fixed^2 + added^2.

Magnitudes use the full fit's IRLS (Fisher) weights, so rows with more information count more, and are on the link scale. On a log link a change of d in the linear predictor multiplies the predicted mean by exp(d), a relative change of exp(d) - 1. When the individual log changes are small, exp(d) - 1 is close to d, so an RMS of 0.02 approximates a weighted RMS relative change of about 2% in the other sites' predicted means. The approximation fails for large individual changes (d = 0.2 is +22%, d = -0.2 is -18%), and a small RMS can still hide a few large changes. There is no null reference, so compare sites with each other rather than with a fixed cut-off.

The hyperparameters are the smoothing parameters mgcv estimated and, for nb() with theta estimated, theta. Smoothing parameters or theta fixed by the user stay fixed in both refits. All results are conditional on the frozen representation: the full fit's model matrix, identifiability constraints and penalties are reused, not rebuilt from the reduced data.

Status

status is "ok" or lists what happened: "rank-deficient" (the site was the only one informing some coefficients; they are dropped from its refits, which leaves the reported changes valid), "lambda held" (a smooth had no data left, so its smoothing parameter was held; see note), "two optima" (see below), "not converged" or "error". Sites whose refits failed have NA magnitudes and a warning is issued.

The REML criterion can have more than one optimum, typically an interior one and a flat plateau where a smoothing parameter is very large (for a thin plate smooth, a nearly straight line). The re-estimated refit is therefore searched for twice, from mgcv's default starting values and from the full-fit values, and the fit with the lower REML score is used. When the two searches end at fits that differ by more than 10% of the larger total change, the status says "two optima" and note gives the other fit's total and how much worse its REML score was. A small difference in REML score means the data barely prefer one fit over the other, so treat that site's added and total with care. Two searches still do not guarantee the best optimum overall.

Examples

library(mgcv)
set.seed(1)
d <- data.frame(site = factor(rep(1:12, each = 6)), x = runif(72))
d$y <- sin(2 * pi * d$x) + rnorm(12, 0, 0.4)[d$site] + rnorm(72, 0, 0.5)
m <- gam(y ~ s(x) + s(site, bs = "re"), data = d, method = "REML")
infl <- gam_influence(m, cluster = "site", progress = FALSE)
infl
#> gamfluence: fixed vs re-estimated deletion, 12 sites
#>   model: gaussian (identity), 72 rows
#> 
#> Largest total change in the other sites' fitted values (RMS, link scale):
#>  id n_obs  fixed   added  total
#>   8     6 0.0576 0.01090 0.0635
#>   9     6 0.0274 0.04670 0.0579
#>   2     6 0.0413 0.01160 0.0442
#>   6     6 0.0380 0.00828 0.0404
#>   1     6 0.0382 0.00307 0.0388
#> 
#> Largest hyperparameter changes when a site is removed:
#>  id parameter  full deleted ratio
#>   9   s(site) 0.351  0.2500 0.710
#>   8      s(x) 0.021  0.0161 0.767
#>   5      s(x) 0.021  0.0264 1.260
#>   6      s(x) 0.021  0.0173 0.825
#>   9      s(x) 0.021  0.0254 1.210
#> 
#> fixed = hyperparameters held; added = what re-estimating them adds.
#> No null reference: compare sites with each other.
top_units(infl, by = "added", n = 3)
#>   id n_obs      fixed      added      total status note
#> 1  9     6 0.02743580 0.04668035 0.05787634     ok <NA>
#> 2  4     6 0.03659087 0.01157631 0.03866023     ok <NA>
#> 3  2     6 0.04125977 0.01155479 0.04416793     ok <NA>