Ridge regression analysis of collinear data
Mitsaki Evagelia
Supervisor: I. Panaretos
CHAPTER
1
INTRODUCTION
THE PROBLEM OF MULTICOLLINEARITY
2.1 Introduction
2.2 The General Regression Situation
2.3 Multicollinearity2.3.1 Effects of Collinearity
2.4 Detecting Collinearity
2.4.1 Correlation Coefficients
2.4.2 Calculation of |X'X|
2.4.3 Leamer's Method
2.4.4 The Condition Index
2.4.5 Variance Inflation Factors
2.4.6 Variance Decomposition Proportions
2.4.7 The Farrar and Glauber Tests
2.4.8 The Sum of λi-1
2.5 Example
2.6 Remedial Measures
2.6.1 Model Respecification
2.6.2 Variable Selection
2.6.3 Biased Estimation
2.6.4 Prior Information about the Regression Coefficients
2.6.5 Partial Least Squares
2.7 Multicollinearity with Stochastic Regressors
2.8 Multicollinearity and Prediction
RIDGE REGRESSION
3.1 Introduction
3.2 The Parameterized Model
3.3 Hoerl and Kennard's Reasoning
3.4 Properties of the Ridge Estimator
3.5 Mean Squared Error Properties
3.6 Existence Theorems
3.7 Generalized Ridge Estimator
3.8 The Ridge Trace Plot
3.8.1 An Alternative Scaling for the Ridge Trace
3.8.2 Quantification of the Concept of a Stable Region
3.9 Selecting Value of K
3.10 Illustration to Real Data
3.10.1 Body Fat Data
3.10.2 Data Analysis
FURTHER RIDGE THEORY
4.1 Other Interpretations of Ridge Regression
4.1.1 Restricted Least Squares Interpretation
4.1.2 Bayesian Interpretation
4.1.3 An Optimization Problem
4.2 Application of Ridge Regression in Special Cases
4.2.1 Rank Deficient Model
4.2.2 Straight Line Regression with a Small Number of Observations
4.2.3 Models with Lagged Effects
4.2.4 Subset Selection
4.2.5 Logistic Regression
4.2.6 Autocorrelated Disturbances
4.3 A Recent Advance in Ridge Regression
4.3.1 Influence in Ridge Regression
4.3.2 Local Change of Small Perturbations
SIMULATION - APPLICATION
5.1 Description of the Simulation
5.2 The Simulation Results
5.2.1 Mean Squared Error (MSE)
5.2.2 Average k
5.2.3 Conclusions