Using R for Significance Tests

How to Test the Statistical Significance of Regression Models

A chi-square test of nested models is a robust means of testing the statistical significance of regression models. For a parametric regression model, the taxon-environment relationship is modeled using three degrees of freedom (a constant, a linear term, and a quadratic term). The following script tests the significance of this model against nested models expressed only as a constant and as a constant and a linear term.

# Conduct chi-square tests on nested parametric models
for (i in 1:length(taxa.names)) {

  print(taxa.names[i])
  resp <- dfmerge[,taxa.names[i]] > 0

  # Fit a model that is only a constant
  modcmp <- glm(resp ~ 1, family = binomial, data = dfmerge)

  # Compare original model with constant model using
  # a chi-square statistic
  modout <- anova(modlist.glm, modcmp, test = "Chi")
  print(modout)

  # Select p < 0.05 as statistically significant
  if (modout[2,"P(>|Chi|)"] < 0.05) {
    print("Model significant compared to constant")
  }

  # Fit a model with only a linear explanatory variable
  modcmp <- glm(resp ~ temp, family = binomial, data = dfmerge)

  # Compare original model with constant model using a
  # chi-square statistic
  modout <- anova(modlist.glm, modcmp, test = "Chi")
  print(modout)
  if (modout[2,"P(>|Chi|)"] < 0.05) {
    print("Model with b2 significant (p < 0.05) improvement over linear model")
  }
}