
VALIDITY & RELIABILITY 2016 EDITION
Overview
An introductory graduatelevel illustrated tutorial on validity and reliability with numerous worked examples and output using SPSS, SAS, Stata, and ReCal software.
New in the 2016 edition:
The full content is now available from Statistical Associates Publishers. Click here.
Below is the unformatted table of contents.
VALIDITY & RELIABILITY 2016 EDITION Table of Contents Table of Contents Example datasets 9 Validity vs. reliability coefficients 10 VALIDITY OVERVIEW 11 Validity: Historical background 13 Convergent validity 14 Overview: Do items in a scale converge? 14 Cronbach's alpha 16 Simple factor structure 16 Average variance extracted (AVE) 18 Structural equation modeling methods 18 Rasch models 18 Common method variance 19 Monomethod and/or monotrait biases 20 Multitraitmultimethod validation 20 Discriminant validity 21 Overview 21 Correlational methods 21 Factor methods 22 Structural equation modeling method 22 Average variance extracted (AVE) method 22 Criterion validity 23 Overview 23 Types of criterion validity 23 Correlation with an objective measure 23 Correlation with an accepted measure 23 Correlation with other variables 24 Examples 24 Content validity 24 Overview 24 Naming fallacies 25 Example of content validity 25 Ecological validityhh 26 Internal validity 26 Overview 26 Specification bias 27 Hawthorne effect (experimenter expectation) 27 Mortality attrition or rejection bias 27 Selection bias 27 Evaluation apprehension 27 Experimental validity 28 Overview 28 Compensatory equalization of treatments 28 Compensatory rivalry 28 Resentful demoralization 29 Treatment imitation or diffusion 29 Unintended treatments 29 Longitudinal validity 29 Crosssectional limitations 29 Instrumentation change 29 History (intervening events) 30 Maturation 30 Mortality(attrition or rejection effects) 30 Regression toward the mean 30 Test experience 30 External validity 31 Overview 31 Examples 31 Statistical validity 32 Overview 32 Reliability 32 Type I errors and statistical significance 32 Type II Errors and Statistical Power 34 Interaction and nonlinearity 34 Causal ambiguity 34 Fallacies of aggregation 34 Strategies for dealing with fallacies of aggregation 35 Validity Checklist 36 RELIABILITY ANALYSIS 37 Overview 37 Measurement 38 Scores 38 Number of scale items 38 Triangulation 39 Calibration 39 Models 39 In SPSS 39 In SAS 40 In Stata 41 Internal consistency reliability 42 Data 42 Cronbach's alpha 42 Overview 42 Data level 43 Formula 44 Number of items 44 SPSS 44 SAS 52 Stata 54 SpearmanBrown correction for Cronbach's alpha 55 KR20 and KR21 57 Overview 57 SPSS 57 SAS 58 Stata 58 Guttman's splithalf reliability coefficient 59 Overview 59 SPSS 60 SAS 60 Stata 60 Guttman's lower bounds (lambda 1  6) 61 Overview 61 SPSS 62 SAS 62 Stata 62 Greatest lower bound (GLB) 63 Ordinal coefficient alpha and ordinal coefficient theta 63 Overview 63 Computation 65 Spearman's reliability rho 67 Factoranalytic approaches to internal consistency 67 Simple factor structure 67 Confirmatory factor analysis in structural equation modeling 70 Average variance extracted (AVE) 75 Fornell & Larcker criterion 76 Composite reliability (Raykov's rho) 77 Computation of AVE and rho 78 Hancock and Mueller's coefficient H 83 SpearmanBrown splithalf reliability 84 Overview 84 SPSS 85 SAS 87 Stata 88 OddEven Reliability 89 Overview 89 Oddeven reliability in SPSS 89 Oddeven reliability in SAS and Stata 90 Testretest reliability 90 Interrater reliability 91 Overview 91 Percent Agreement 92 Cohen's kappa 94 Overview 94 Data 97 Weighted kappa 97 SPSS 98 SAS 101 Stata 106 ReCal 110 Fleiss' kappa 112 Data 112 Overview 112 SPSS 112 SAS 113 Stata 114 ReCal 114 Scott's pi 116 Overview 116 SPSS, SAS, and Stata 116 ReCal 116 Krippendorff's alpha 116 Overview 116 Example 117 SPSS 117 SAS 123 Stata 125 ReCal 131 Kendall's coefficient of concordance, W 133 Overview 133 Data 134 SPSS 136 SAS 141 Stata 143 Intraclass correlation (ICC) 144 Overview 144 Interpretation 145 ICC in multilevel models 146 Sample size: ICC vs. Pearson r 146 Example data 147 Data setup 147 ICC models and types 150 Single versus average ICC 152 ICC in SPSS 153 ICC in SAS 157 ICC in Stata 157 Other measures of interrater reliability 161 Overview 161 BrennanPrediger coefficient 162 Gwet's AC1 162 Conger's kappa 163 Reliability: Assumptions 163 Items measuring the same underlying dimension 163 Consistent coding 163 Items of the same level of difficulty 163 Data level 164 Same assumptions as for correlation 164 Additivity and Tukey's test 164 SPSS 166 SAS 167 Stata 167 Independence 168 Uncorrelated error 168 Random assignment of subjects 169 Equality of variances 169 Overview 169 SPSS 170 SAS 171 Stata 172 Equality of means and Hotelling's T2 173 Overview 173 SPSS 174 SAS 175 Stata 176 Reliability: Frequently Asked Questions 177 Converting data among statistical packages 177 How many raters do I need to assess reliability? 178 How is reliability related to validity? 179 How is Cronbach's alpha related to factor analysis? 179 Does a high Cronbach's alpha establish unidimensionality? 180 How is reliability related to attenuation in correlation? 180 How should a negative reliability coefficient be interpreted? 181 What is the Krippendorff's alpha macro syntax developed by Hayes? 182 What is Cochran's Q test of equality of proportions for dichotomous items? 188 What is the derivation of intraclass correlation coefficients? 189 What are Method 1 and Method 2 in the SPSS RELIABILITY module? 190 What are BlandAltman plots? 190 Acknowledgments 190 Bibliography 192 Pagecount: 202