Create generalized linear regression model by stepwise regression
creates a generalized linear model of a table or dataset array
mdl = stepwiseglm(
tbl using stepwise regression to add or remove predictors,
starting from a constant model.
stepwiseglm uses the last
tbl as the response variable.
stepwiseglm uses forward and backward stepwise regression
to determine a final model. At each step, the function searches for terms to add the
model to or remove from the model, based on the value of the
specifies additional options using one or more name-value pair arguments. For
example, you can specify the categorical variables, the smallest or largest set of
terms to use in the model, the maximum number of steps to take, or the criterion
mdl = stepwiseglm(___,
stepwiseglm uses to add or remove terms.
Generalized Linear Model Using Stepwise Algorithm
Create response data using just three of 20 predictors, and create a generalized linear model using stepwise algorithm to see if it uses just the correct predictors.
Create data with 20 predictors, and Poisson response using just three of the predictors, plus a constant.
rng('default') % for reproducibility X = randn(100,20); mu = exp(X(:,[5 10 15])*[.4;.2;.3] + 1); y = poissrnd(mu);
Fit a generalized linear model using the Poisson distribution.
mdl = stepwiseglm(X,y,... 'constant','upper','linear','Distribution','poisson')
1. Adding x5, Deviance = 134.439, Chi2Stat = 52.24814, PValue = 4.891229e-13 2. Adding x15, Deviance = 106.285, Chi2Stat = 28.15393, PValue = 1.1204e-07 3. Adding x10, Deviance = 95.0207, Chi2Stat = 11.2644, PValue = 0.000790094
mdl = Generalized linear regression model: log(y) ~ 1 + x5 + x10 + x15 Distribution = Poisson Estimated Coefficients: Estimate SE tStat pValue ________ ________ ______ __________ (Intercept) 1.0115 0.064275 15.737 8.4217e-56 x5 0.39508 0.066665 5.9263 3.0977e-09 x10 0.18863 0.05534 3.4085 0.0006532 x15 0.29295 0.053269 5.4995 3.8089e-08 100 observations, 96 error degrees of freedom Dispersion: 1 Chi^2-statistic vs. constant model: 91.7, p-value = 9.61e-20
The starting model is the constant model.
stepwiseglm by default uses deviance of the model as the criterion. It first adds
x5 into the model, as the -value for the test statistic, deviance (the differences in the deviances of the two models), is less than the default threshold value 0.05. Then, it adds
x15 because given
x5 is in the model, when
x15 is added, the -value for chi-squared test is smaller than 0.05. It then adds
x10 because given
x15 are in the model, when
x10 is added, the -value for the chi-square test statistic is again less than 0.05.
modelspec — Starting model
'constant' (default) | character vector or string scalar naming the model | t-by-(p + 1) terms matrix | character vector or string scalar formula in the form
Starting model for
as one of the following:
A character vector or string scalar naming the model.
Value Model Type
Model contains only a constant (intercept) term.
Model contains an intercept and linear term for each predictor.
Model contains an intercept, linear term for each predictor, and all products of pairs of distinct predictors (no squared terms).
Model contains an intercept term and linear and squared terms for each predictor.
Model contains an intercept term, linear and squared terms for each predictor, and all products of pairs of distinct predictors.
Model is a polynomial with all terms up to degree
iin the first predictor, degree
jin the second predictor, and so on. Specify the maximum degree for each predictor by using numerals 0 though 9. The model contains interaction terms, but the degree of each interaction term does not exceed the maximum value of the specified degrees. For example,
'poly13'has an intercept and x1, x2, x22, x23, x1*x2, and x1*x22 terms, where x1 and x2 are the first and second predictors, respectively.
A t-by-(p + 1) matrix, or a Terms Matrix, specifying terms in the model, where t is the number of terms and p is the number of predictor variables, and +1 accounts for the response variable. A terms matrix is convenient when the number of predictors is large and you want to generate the terms programmatically.
A character vector or string scalar Formula in the form
'y ~ terms',
termsare in Wilkinson Notation. The variable names in the formula must be variable names in
tblor variable names specified by
Varnames. Also, the variable names must be valid MATLAB identifiers.
The software determines the order of terms in a fitted model by using the order of terms in
X. Therefore, the order of terms in the model can be different from the order of terms in the specified formula.
comma-separated pairs of
the argument name and
Value is the corresponding value.
Name must appear inside quotes. You can specify several name and value
pair arguments in any order as
'Criterion','aic','Distribution','poisson','Upper','interactions'specifies Akaike Information Criterion as the criterion to add or remove variables to the model, Poisson distribution as the distribution of the response variable, and a model with all possible interactions as the largest model to consider as the fit.
Criterion — Criterion to add or remove terms
'Deviance' (default) |
Criterion to add or remove terms, specified as the comma-separated pair consisting of
'Criterion' and one of these values:
'Deviance'— p-value for an F-test or chi-squared test of the change in the deviance that results from adding or removing the term. The F-test tests a single model, and the chi-squared test compares two different models.
'sse'— p-value for an F-test of the change in the sum of squared error that results from adding or removing the term.
'aic'— Change in the value of the Akaike information criterion (AIC).
'bic'— Change in the value of the Bayesian information criterion (BIC).
'rsquared'— Increase in the value of R2.
'adjrsquared'— Increase in the value of adjusted R2.
The generalized linear model
mdlis a standard linear model unless you specify otherwise with the
For other methods such as
devianceTest, or properties of the
After training a model, you can generate C/C++ code that predicts responses for new data. Generating C/C++ code requires MATLAB Coder™. For details, see Introduction to Code Generation.
Stepwise regression is a systematic method for adding and removing terms from a linear or generalized linear model based on their statistical significance in explaining the response variable. The method begins with an initial model, specified using
modelspec, and then compares the explanatory power of incrementally larger and smaller models.
stepwiseglmfunction uses forward and backward stepwise regression to determine a final model. At each step, the function searches for terms to add to the model or remove from the model based on the value of the
'Criterion'name-value pair argument.
The default value of
'Criterion'for a linear regression model is
'sse'. In this case,
LinearModeluse the p-value of an F-statistic to test models with and without a potential term at each step. If a term is not currently in the model, the null hypothesis is that the term would have a zero coefficient if added to the model. If there is sufficient evidence to reject the null hypothesis, the function adds the term to the model. Conversely, if a term is currently in the model, the null hypothesis is that the term has a zero coefficient. If there is insufficient evidence to reject the null hypothesis, the function removes the term from the model.
Stepwise regression takes these steps when
Fit the initial model.
Examine a set of available terms not in the model. If any of the terms have p-values less than an entrance tolerance (that is, if it is unlikely a term would have a zero coefficient if added to the model), add the term with the smallest p-value and repeat this step; otherwise, go to step 3.
If any of the available terms in the model have p-values greater than an exit tolerance (that is, the hypothesis of a zero coefficient cannot be rejected), remove the term with the largest p-value and return to step 2; otherwise, end the process.
At any stage, the function will not add a higher-order term if the model does not also include all lower-order terms that are subsets of the higher-order term. For example, the function will not try to add the term
X2^2are already in the model. Similarly, the function will not remove lower-order terms that are subsets of higher-order terms that remain in the model. For example, the function will not try to remove
X1:X2^2remains in the model.
You can specify other criteria by using the
'Criterion'name-value pair argument. For example, you can specify the change in the value of the Akaike information criterion, Bayesian information criterion, R-squared, or adjusted R-squared as the criterion to add or remove terms.
Depending on the terms included in the initial model, and the order in which the function adds and removes terms, the function might build different models from the same set of potential terms. The function terminates when no single step improves the model. However, a different initial model or a different sequence of steps does not guarantee a better fit. In this sense, stepwise models are locally optimal, but might not be globally optimal.
stepwiseglmtreats a categorical predictor as follows:
A model with a categorical predictor that has L levels (categories) includes L – 1 indicator variables. The model uses the first category as a reference level, so it does not include the indicator variable for the reference level. If the data type of the categorical predictor is
categorical, then you can check the order of categories by using
categoriesand reorder the categories by using
reordercatsto customize the reference level. For more details about creating indicator variables, see Automatic Creation of Dummy Variables.
stepwiseglmtreats the group of L – 1 indicator variables as a single variable. If you want to treat the indicator variables as distinct predictor variables, create indicator variables manually by using
dummyvar. Then use the indicator variables, except the one corresponding to the reference level of the categorical variable, when you fit a model. For the categorical predictor
X, if you specify all columns of
dummyvar(X)and an intercept term as predictors, then the design matrix becomes rank deficient.
Interaction terms between a continuous predictor and a categorical predictor with L levels consist of the element-wise product of the L – 1 indicator variables with the continuous predictor.
Interaction terms between two categorical predictors with L and M levels consist of the (L – 1)*(M – 1) indicator variables to include all possible combinations of the two categorical predictor levels.
You cannot specify higher-order terms for a categorical predictor because the square of an indicator is equal to itself.
stepwiseglmadds or removes a categorical predictor, the function actually adds or removes the group of indicator variables in one step. Similarly, if
stepwiseglmadds or removes an interaction term with a categorical predictor, the function actually adds or removes the group of interaction terms including the categorical predictor.
''(empty character vector),
Yto be missing values.
stepwiseglmdoes not use observations with missing values in the fit. The
ObservationInfoproperty of a fitted model indicates whether or not
stepwiseglmuses each observation in the fit.
 Collett, D. Modeling Binary Data. New York: Chapman & Hall, 2002.
 Dobson, A. J. An Introduction to Generalized Linear Models. New York: Chapman & Hall, 1990.
 McCullagh, P., and J. A. Nelder. Generalized Linear Models. New York: Chapman & Hall, 1990.