Overview A statistical significance coefficient is the chance that a relationship as strong or stronger than the one observed was due to the chance of random sampling. Thus if a correlation coefficient is significant at exactly the .05 level, this means there is 5% chance that a correlation as strong or stronger than the observed one would result from an unusual random sampling of data when in fact the correlation was zero. There are many, many specific significance tests. Common tests are listed below, but in addition each statistical procedure has associated significance tests which are discussed in the respective Statistical Associates "Blue Book" volumes dealing with each procedure.
The full content is now available from Statistical Associates Publishers. Click here.
Below is the unformatted table of contents.
SIGNIFICANCE TESTING Overview 10 Significance Testing 15 Overview 15 Types of significance tests 15 Parametric tests 15 Key Concepts and Terms 16 When significance testing applies 16 Significance and Type I Errors 19 Confidence limits 19 Power and Type II Errors 20 One-tailed vs. two-tailed tests 20 Assumptions of significance testing in general 22 Random sampling 22 Adequate sample size 22 Significance is not importance 22 À priori testing 23 Appropriate alpha significance level 23 Absence of intervening and common anteceding causes 23 Frequently asked questions about significance testing in general 23 How is significance related to effect size? 24 Should I "fail to accept" or should I "reject" the null hypothesis? 24 If cases in my sample are weighted, am I getting accurate significance values? 24 Is significance the same for multistage random samples as for simple random samples? 24 Binomial Test of Significance 26 Overview 26 Key Concepts and Terms 26 Implementing the binomial test in SPSS 27 Selections 27 Normal approximation of the binomial test 30 Assumptions of the binomial test 31 Dichotomous distribution 31 Data distribution 31 Random sampling 31 Student's t-Test of Difference of Means 32 Overview of the t-test 32 SPSS t-test types 32 Key Concepts and Terms 33 Formula 33 Critical value 33 Confidence limits 34 One-sample t-test 34 Example 34 Interpretation 35 Independent sample t-test 35 Example 36 Interpretation (with Levene's test) 36 The independent samples assumption 37 Paired sample t-test for non-independent samples 37 Overview 37 Example 38 Interpretation 38 Assumptions for t-tests 39 Normal distribution 39 Random sampling 39 Similar variances 39 Dependent/independent samples 40 Effect size measures 40 Frequently Asked Questions 40 What are common alternatives to the t-test? 40 What non-parametric test do I use instead of the t-test if my data cannot meet the assumption of normality? 41 Normal Curve Tests of Means and Proportions 42 Overview 42 Key Concepts and Terms 42 Deviation scores 42 Standard deviation 42 Variance 43 Standard error 43 Confidence limits 43 Binomial distribution 44 Normal distribution 44 Normal curve means tests ("hypothesis tests") 46 Confidence interval 47 Manual computation of z values for normal curve tests 49 Assumptions for normal curve tests 50 Interval data 51 Sample size should not be small 51 Homogeneity of variances 51 Random sampling 51 Chi-Square Significance Tests 52 Overview 52 Pearson's chi-square 52 Overview 52 SPSS output 53 Yates correction for continuity 56 Crosstabulation control variables and chi-square 57 Chi-square goodness-of-fit test 58 Likelihood ratio chi-square 61 Mantel-Haenszel (linear by linear) chi-square 62 Assumptions for chi-square tests 63 Random sampling 63 Independence 64 Known distribution 64 Non-directional hypotheses 64 Finite values 64 Normal distribution of deviations 64 Data level 64 Frequently asked questions about chi-square 65 How is Pearsonian chi-square calculated manually for tabular data? 65 My statistics program prints out the chi-square contribution of each cell in the table. Can this be used to establish the significance of each cell? 66 Fisher Exact Test of Significance 67 Overview 67 Key Concepts and Terms 67 Notation 67 Manual computation of Fisher's exact test 68 Assumptions of Fisher's exact test 69 Random sampling 69 Directionality 69 Independent observations 70 Dichotomous level of measurement 70 Fixed marginals 70 Frequently asked questions regarding the Fisher exact test 70 What is Tocher's modification of the Fisher exact test? 70 Is the two-tailed test statistic for Fisher's exact test simply double the one-tailed value? 71 Runs Test of Randomness 72 Overview 72 Key Concepts and Terms 72 Runs 72 Runs test 72 Runs test for serial randomness 73 Cut points 73 Example 73 The data 73 Requesting the runs test 74 Runs test output in SPSS 76 SPSS legacy dialog features 77 Assumptions of the runs test 80 Data order 80 Numeric data 80 Data level 80 Data distribution 80 Frequently Asked Questions 81 One-Sample Kolmogorov-Smirnov Goodness-of-Fit Test 82 Overview 82 Key Concepts and Terms 82 Observed vs. hypothetical distribution 82 Example 84 What is tested 84 SPSS set-up 84 Statistical output for the Kolmogorov-Smirnov test in SPSS 86 Kolmogorov-Smirnov test assumptions 88 Random sampling 88 Level of data 88 Frequently Asked Questions 89 Could I use the Kolmogov-Smirnov goodness-of-fit test to test my data not only against the normal distribution, but also against a series of other distributions? 89 Where is the Kolmogorov-Smirnov exact test found in SPSS? 89 Could goodness-of-fit be assessed graphically instead of by the Kolmogorov-Smirnov test? 90 Tests for two independent samples: 91 The Mann-Whitney U test 91 Overview 91 Key Concepts and Terms 91 Independent samples 91 Data level 91 Computation of Mann-Whitney U 91 A manual example 92 SPSS setup for the Mann-Whitney U test 93 Example 93 SPSS user interface 93 SPSS statistical output for the Mann-Whitney U test 95 The hypothesis test summary table 95 The SPSS model viewer 96 Wilcoxon rank sum W 97 Assumptions of the Mann-Whitney U test 97 Data level 97 Independent samples 97 Random sampling 98 Data pairs 98 Sample size 98 Frequently asked questions: Mann-Whitney U test 98 Where is the Mann-Whitney U exact test found in SPSS? 98 Tests for two independent samples: 100 The Wald-Wolfowitz runs test 100 Overview 100 SPSS inputs for the Wald-Wolfowitz runs test 100 SPSS output for the Wald-Wolfowitz runs test 102 The hypothesis test summary table 102 The model viewer 103 Frequently asked questions: Wald-Wolfowitz runs test 104 Where is the Wald-Wolfowitz exact test found in SPSS? 104 Assumptions of the Wald-Wolfowitz runs test 105 Data level 105 Independent samples 105 Random sampling 105 Data pairs 106 Data distribution 106 Sample size 106 Wald-Wolfowitz median test 106 SPSS inputs for the median test 106 SPSS output for the independent samples median test 107 Tests for two independent samples: 111 The Kolmogorov-Smirnov Z test 111 Overview 111 Computation 111 SPSS inputs for the independent samples Kolmogorov-Smirnov test 111 SPSS output for the independent samples Kolmogorov-Smirnov test 112 The hypothesis test summary table 112 The SPSS model viewer 113 Assumptions of the Kolmogorov-Smirnov test 114 Data level 114 Random sampling 114 Data pairs 115 Data distribution 115 Sample size 115 Frequently asked questions: Kolmogorov-Smirnov test 115 Where is the Kolmogorov-Smirnov exact test found in SPSS? 115 Tests for two independent samples: 117 Moses Extreme Reactions Test 117 Overview 117 Computation 117 Example 118 SPSS inputs for the Moses test 118 SPSS output for the Moses test 120 The hypothesis summary table 120 The SPSS model viewer 121 Assumptions of the Moses extreme reactions test 123 Control group 123 Data level 124 Random sampling 124 Data pairs 124 Data distribution 124 Sample size 124 Frequently asked questions: Moses extreme reactions test 124 Where is the Moses extreme reactions exact test found in SPSS? 124 Tests for more than two independent samples: The Kruskal-Wallis H test 126 Overview 126 Computation 126 Example 127 SPSS input for the Kruskal-Wallis H test 127 SPSS output for the Kruskal-Wallis H test 129 The hypothesis summary table 129 The SPSS model viewer 130 Pairwise comparisons 131 Frequently asked questions: Kruskal-Wallis test 132 Where is the Kruskal-Wallis exact test found in SPSS? 132 Assumptions of the Kruskal Wallis H test 134 Data level 134 Data pairs 135 Data distribution 135 Sample size 135 Random sampling 135 Tests for more than two independent samples: The multiple samples median test 136 Overview 136 Computation 136 Example 136 SPSS input for the median test 136 SPSS output for the median test 138 The hypothesis summary table 138 The SPSS model viewer 139 Pairwise comparisons 140 Frequently asked questions: median test 141 Where is the median exact test found in SPSS? 141 Assumptions of the median test 142 Random sampling 143 Central tendency 143 Data level 143 Data pairs 143 Data distribution 143 Sample size 143 Tests for more than two independent samples: The Jonckheere-Terpstra test 144 Overview 144 Example 144 SPSS input for the Jonkheere-Terpstra test 145 SPSS output for the Jonkheere-Terpstra test 146 The hypothesis summary table 146 The SPSS model viewer 147 Pairwise comparisons 148 Frequently asked questions: Jonkheere-Terpstra test 149 Where is the Jonkheere-Terpstra exact test found in SPSS? 149 Assumptions of the Jonkheere-Terpstra test 151 Random sampling 151 Data level 152 Data distribution 152 Sample size 152 Significance tests for two dependent samples: The McNemar test 153 Overview 153 Key Concepts and Terms 154 Dependent samples 154 Pairs 154 Similar distributions 154 Symmetry 154 Computation 155 Significance tests for two dependent samples: The marginal homogeneity test 167 Overview 167 Example 167 SPSS input for the marginal homogeneity test 167 SPSS output for the marginal homogeneity test 169 The hypothesis summary table 169 The SPSS model viewer 170 Frequently asked questions: Marginal homogeneity test 172 Where is the Marginal homogeneity exact test found in SPSS? 172 Assumptions of the Marginal homogeneity test 173 Data level 173 Data distribution 174 Sample size 174 Random sampling 174 Significance tests for two dependent samples: The sign test 175 Overview 175 Computation 175 Example 175 SPSS input for the sign test 176 SPSS output for the sign test 178 The hypothesis summary table 178 The SPSS model viewer 179 Frequently asked questions: Sign test 180 Where is the sign exact test found in SPSS? 180 Assumptions of the Sign test 182 Data level 182 Data distribution 182 Sample size 183 Random sampling 183 Significance tests for two dependent samples: The Wilcoxon signed-rank test 184 Overview 184 Computation 184 Example 184 SPSS input for the Wilcoxon signed-rank test 185 SPSS output for the Wilcoxon signed-rank test 187 The hypothesis summary table 187 The SPSS model viewer 188 Frequently asked questions: Wilcoxon signed-rank test 190 Where is the sign exact test found in SPSS? 190 Assumptions of the Wilcoxon signed-rank test 191 Data level 192 Data distribution 192 Sample size 192 Random sampling 192 Significance tests for two independent or dependent samples: Hodges-Lehman confidence intervals 193 Overview 193 Example 193 Frequently asked questions: the Hodges-Lehman procedure 194 Is the Hodges-Lehman procedure available in the legacy dialog of SPSS? 194 Is there additional SPSS output for the Hodges-Lehman procedure in the SPSS model viewer? 194 Significance Tests for K Dependent Samples: Friedman Test 195 Overview 195 Computation 195 Example 196 SPSS input for the Friedman test 196 SPSS output for the Friedman test 197 The SPSS model viewer 198 Frequently asked questions: Friedman test 201 Where is the Friedman exact test found in SPSS? 201 Assumptions of the Friedman test 202 Significance Tests for K Dependent Samples: Kendall's W test 204 Overview 204 Computation 204 Example 204 SPSS input for Kendall's W test 204 SPSS output for Kendall's W test 206 The SPSS model viewer 207 Frequently asked questions: Kendall's W test 210 Where is Kendall's W exact test found in SPSS? 210 Assumptions of Kendall's W test 211 Significance Tests for K Dependent Samples: Cochran's Q test 213 Overview 213 Computation 213 Example 213 SPSS input for Cochran's Q test 214 SPSS output for Cochran's Q test 215 The SPSS model viewer 216 Frequently asked questions: Cochran's Q test 219 Where is Cochran's Q exact test found in SPSS? 219 Assumptions of Cochran's Q test 220 Dependent samples 220 Data level 221 Bootstrapping and Resampling 222 Overview 222 Key Concepts and Terms 223 Bootstrapping 223 Resampling 223 Monte Carlo estimation 223 Histogram of z values 224 Bootstrapping in SPSS 224 Overview 224 Exact tests 226 SPSS syntax (command) mode 227 Assumptions 228 Random sampling 228 Nonparametric 229 Sample size 229 Frequently Asked Questions 229 Bibliography 230 Pagecount: 233