What it means
Penn State's STAT 462 course describes the idea as building a model from candidate predictors by entering and removing them in a stepwise manner until there is no justifiable reason to do either. A fundamental rule is that the candidate list must include every variable that actually predicts the response.
Otherwise, the course warns, the final model is underspecified and misleading. The procedure begins with no predictors.
First the analyst sets an alpha-to-enter level and an alpha-to-remove level. The course notes these are typically larger than the usual 0.05, and that many software packages default to 0.15 for both.
At step one, the method fits each one-predictor model and enters the variable with the smallest p-value, provided it is below the enter level. At step two, it tests each remaining variable alongside the first and adds the best one.
After each addition it steps back and checks whether earlier variables still pass the remove level, and drops any that do not. The process stops when no variable can be added, and the remaining set is the final model.
Variables that look strong alone can lose value once a correlated variable joins, which is why the removal check matters. The Penn State example uses 13 batches of cement and four predictors, where x2 and x4 are strongly correlated with each other, so the order of entry matters and correlated predictors can change the result.
Related strategies exist. OpenIntro Statistics describes backward elimination, which starts with all variables and drops the one with the largest p-value each time.
It also describes forward selection, which adds one variable at a time until none shows strong evidence of importance. Stepwise selection is popular in finance, credit scoring and economics because it is fast with many candidates, but it also has costs: the final p-values and fit can look better than they are, since the data was used to choose the model.
Out-of-sample checks guard against that. Treat the output as a starting point rather than proof that chosen variables cause the outcome.
In practice
Real-world examples.
Example
A fictional analyst studies a fund's monthly return with 8 candidate factors. With an enter level of 0.15, the method adds the factor with the lowest p-value, say 0.001. It then tests the other 7, adds one with p = 0.04, and checks that the first still has p below 0.15.
Example
A fictional lender models loan default using 20 candidate borrower traits. Stepwise selection ends with 5 variables. The lender then tests the model on a separate set of 1,000 loans, since selection on the original data can overstate fit.
Example
A fictional retailer predicts weekly sales from price, ads, weather and holidays. Price and ads are correlated. After holidays enter, the ads p-value rises from 0.03 to 0.22, above the 0.15 remove level, so ads is dropped.
Formula
Calculation
Enter rule: add the candidate with the smallest p-value if p < alpha-to-enter, such as 0.15.
Remove rule: drop a model variable if its p-value > alpha-to-remove, such as 0.15. Stop when no candidate can enter and none must leave.Case study
Seen in the real world.
This case study is fictional and illustrative. Noor, a risk analyst, has 30 candidate variables to predict credit card losses on 5,000 accounts. She sets both alpha levels at 0.15 and runs stepwise selection. The method returns 6 variables. Two of them looked strong alone, but one is dropped after a correlated variable enters.
She then tests the model on 2,000 held-out accounts. Fit on the new accounts is weaker than on the original data. She keeps the model but reports both results, and uses a larger sample next quarter. She also checks that the variables make economic sense.
Watch out
Common mistakes.
- Leaving out a variable that truly predicts the outcome, when the candidate list must include all such variables.
- Reading final p-values as clean evidence, when the data was used to pick the model.
- Skipping a test on new data, when selection can overstate fit.
Questions
People also ask.
What is stepwise regression?
It is a method that builds a regression model by entering and removing predictors one at a time based on significance tests. It stops when no variable can be justifiably added or removed. It is used when there are many candidate predictors.
What are alpha-to-enter and alpha-to-remove?
They are the significance levels used to decide when a variable joins or leaves the model. Penn State says they are typically larger than 0.05. Many packages default both to 0.15.
What is the main risk?
The model may fit the sample better than it fits new data. Missing key variables also leaves a misleading model. Testing on separate data helps.
From the founder's library
Take it further with the book.
Build your financial confidence beyond this definition. Shihan's full-length guide, Accounting Fundamentals, takes the same plain-English approach and turns it into a complete, practical playbook for non-finance managers, business owners and students - with chapter-end quiz answers and presentation slides included.
25% off with code MMHQ25, applied at checkout. Priced in USD - checkout may show the equivalent in your local currency.
View the book and save 25%Related