LOGISTIC REGRESSION: BINARY & MULTINOMIAL
An illustrated tutorial and introduction to binary and multinomial logistic regression using SPSS, SAS, or Stata for examples. Suitable for introductory graduate-level study.
The 2014 edition is a major update to the 2012 edition. Among the new features are these:
The full content is now available from Statistical Associates Publishers. Click here.
Below is the unformatted table of contents.
LOGISTIC REGRESSION Table of Contents Overview 12 Data examples 14 Key Terms and Concepts 15 Binary, binomial, and multinomial logistic regression 15 The logistic model 16 The logistic equation 17 Logits and link functions 19 Saving predicted probabilities 21 The dependent variable 22 The dependent reference default in binary logistic regression 23 The dependent reference default in multinomial logistic regression 24 Factors: Declaring 29 Overview 29 SPSS 29 SAS 31 Stata 32 Factors: Reference levels 33 Overview 33 SPSS 34 SAS 35 Stata 37 Covariates 38 Overview 38 SPSS 38 SAS 39 Stata 40 Interaction Terms 40 Overview 40 SPSS 40 SAS 41 Stata 42 Estimation 43 Overview 43 Maximum likelihood estimation (ML) 43 Weighted least squares estimation (WLS) 44 Ordinary least squares estimation (OLS) 45 A basic binary logistic regression model in SPSS 45 Example 45 SPSS input 45 SPSS output 48 Parameter estimates and odds ratios 48 Omnibus tests of model coefficients 50 Model summary 50 Classification table 51 Classification plot 53 Hosmer-Lemeshow test of goodness of fit 55 Casewise listing of residuals for outliers > 2 standard deviations 56 A basic binary logistic regression model in SAS 57 Example 57 SAS input 58 Reconciling SAS and SPSS output 58 SAS output 59 Parameter estimates 59 Odds ratio estimates 60 Global null hypothesis tests 61 Model fit statistics 62 The classification table 63 The association of predicted probabilities and observed responses table 66 Hosmer and Lemeshow test of goodness of fit 66 Regression diagnostics table 67 A basic binary logistic regression model in STATA 68 Overview and example 68 Data setup 69 Stata input 70 Stata output 70 Parameter estimates 70 Odds ratios 71 Likelihood ratio test of the model 72 Model fit statistics 73 The classification table 74 Classification plot 75 Measures of association 76 Hosmer-Lemeshow test 77 Residuals and regression diagnostics 78 A basic multinomial logistic regression model in SPSS 81 Example 81 Model 83 SPSS statistical output 84 Step summary 86 Model fitting information table 86 Goodness of fit tests 87 Likelihood ratio tests 87 Parameter estimates 88 Pseudo R-square 90 Classification table 91 Observed and expected frequencies 91 Asymptotic correlation matrix 91 A basic multinomial logistic regression model in SAS 92 Example 92 SAS syntax 92 SAS statistical output 93 Overview 93 Model fit 93 Goodness of fit tests 94 Parameter estimates 95 Pseudo R-Square 96 Classification table 97 Observed and predicted functions and residuals 97 Correlation matrix of estimates 98 A basic multinomial logistic regression model in STATA 99 Example 99 Stata data setup 99 Stata syntax 100 Stata statistical output 101 Overview 101 Model fit 101 AIC and BIC 102 Pseudo R-square 103 Goodness of fit test 103 Likelihood ratio tests 104 Parameter estimates 104 Odds ratios/ relative risk ratios 105 Classification table 106 Observed and expected frequencies 107 Asymptotic correlation matrix 107 ROC curve analysis 107 Overview 107 Comparing models 108 Optimal classification cutting points 108 Example 109 SPSS 109 Comparing models 109 Optimal classification cutting points 114 SAS 118 Overview 118 Comparing Models 120 Optimal classification cutting points 122 Stata 124 Overview 124 Comparing Models 126 Optimal classification cutting points 130 Conditional logistic regression for matched pairs 131 Overview 131 Example 131 Data setup 131 Conditional logistic regression in SPSS 132 Overview 132 SPSS input 133 SPSS output 136 Conditional logistic regression in SAS 138 Overview 138 SAS input 139 SAS output 139 Conditional logistic regression in Stata 141 Overview 141 Stata input 141 Stata output 141 More about parameter estimates and odds ratios 143 For binary logistic regression 143 Example 1 143 Example 2 146 For multinomial logistic regression 149 Example 1 149 Example 2 152 Coefficient significance and correlation significance may differ 154 Reporting odds ratios 154 Odds ratios: Summary 156 Effect size 156 Confidence interval on the odds ratio 156 Warning: very high or very low odds ratios 157 Comparing the change in odds for different values of X 157 Comparing the change in odds when interaction terms are in the model 157 Probabilities, logits, and odds ratios 158 Probabilities 158 Relative risk ratios (RRR) 162 More about significance tests 162 Overview 162 Significance of the model 162 SPSS 162 SAS 166 Stata 166 Significance of parameter effects 166 SPSS 166 SAS 170 Stata 170 More about effect size measures 171 Overview 171 Effect size for the model 171 Pseudo R-squared 171 Classification tables 173 Terms associated with classification tables: 177 The c statistic 179 Information theory measures of model fit 180 Effect size for parameters 182 Odds ratios 182 Standardized vs. unstandardized logistic coefficients in model comparisons 182 Stepwise logistic regression 183 Overview 183 Forward selection vs. backward elimination 184 Cross-validation 185 Rao's efficient score as a variable entry criterion for forward selection 186 Score statistic 186 Which step is the best model? 187 Contrast Analysis 188 Repeated contrasts 188 Indicator contrasts 188 Contrasts and ordinality 189 Analysis of residuals 190 Overview 190 Residual analysis in binary logistic regression 190 Outliers 190 The DfBeta statistic 190 The leverage statistic 191 Cook's distance 191 Residual analysis in multinomial logistic regression 191 Assumptions 192 Data level 192 Meaningful coding 193 Proper specification of the model 193 Independence of irrelevant alternatives 193 Error terms are assumed to be independent (independent sampling) 194 Low error in the explanatory variables 194 Linearity 194 Additivity 196 Absence of perfect separation 196 Absence of perfect multicollinearity 196 Absence of high multicollinearity 196 Centered variables 197 No outliers 197 Sample size 197 Sampling adequacy 198 Expected dispersion 198 Frequently Asked Questions 199 How should logistic regression results be reported? 199 Example 199 Why not just use regression with dichotomous dependents? 200 How does OLS regression compare to logistic regression? 201 When is discriminant analysis preferred over logistic regression? 201 What is the SPSS syntax for logistic regression? 202 Apart from indicator coding, what are the other types of contrasts? 204 Can I create interaction terms in my logistic model, as with OLS regression? 207 Will SPSS's binary logistic regression procedure handle my categorical variables automatically? 207 Can I handle missing cases the same in logistic regression as in OLS regression? 208 Explain the error message I am getting about unexpected singularities in the Hessian matrix. 208 Explain the error message I am getting in SPSS about cells with zero frequencies. 209 Is it true for logistic regression, as it is for OLS regression, that the beta weight (standardized logit coefficient) for a given independent reflects its explanatory power controlling for other variables in the equation, and that the betas will change if variables are added or dropped from the equation? 209 What is the coefficient in logistic regression which corresponds to R-Square in multiple regression? 209 Is multicollinearity a problem for logistic regression the way it is for multiple linear regression? 210 What is the logistic equivalent to the VIF test for multicollinearity in OLS regression? Can odds ratios be used? 210 How can one use estimated variance of residuals to test for model misspecification? 211 How are interaction effects handled in logistic regression? 211 Does stepwise logistic regression exist, as it does for OLS regression? 212 What are the stepwise options in multinomial logistic regression in SPSS? 212 May I use the multinomial logistic option when my dependent variable is binary? 215 What is nonparametric logistic regression and how is it more nonlinear? 215 How many independent variables can I have? 216 How do I express the logistic regression equation if one or more of my independent variables is categorical? 217 How do I compare logit coefficients across groups formed by a categorical independent variable? 217 How do I compute the confidence interval for the unstandardized logit (effect) coefficients? 218 Acknowledgments 218 Bibliography 218 Pagecount: 224