In statistics there are two main categories of statistical analysis. One is descriptive statistics. That is one can estimate say mean and standard deviation of a particular variable and describe the data in graphical form to see some pattern. The other is to investigate a specific hypothesis by collecting data of a variable of a considerable size and applying a statistical test method to determine the hypothesis is accepted or rejected with a level of error in prediction. In this context, if one applying the wrong statistical test then the conclusions will be not be correct and he or she may make a significant error in judgment based on the test statistics, which is incorrect given the type of variable and the type of data he is studying, which can be independent, paired or clustered. The type of date determines the designing of the study and the appropriate analysis one must consider.
In designing a study one must consider what questions one must answer, the method of collecting data depending on the type of variable whether it is discrete, categorical or continuous, significant level for the testing of the particular question to be resolved and sample size which can be capable of detecting a meaningful difference. If these are not carefully considered even if a statistical test is appropriately chosen the conclusions derived from the study most probably will be misleading.
In hypothesis testing one must determine whether it is a one sided or two sided test. That is to study changes of a particular variable under examination or increase or decrease. Then one must establish the significance level which is appropriate to the particular study and the field of study. The next step is to select the appropriate test statistic or test statistics and compute the test statistic and determine the degree of freedom if the sample size is small. Then obtain a table value for the statistical test and compare it to the calculated test statistics to derive the p-value which measures the probability the difference or change is due to randomness and the change is significant. If the p-value is
bigger then the difference or change may not be significant given the significant level is chosen correctly.
It is obvious from the discussion it is important to determine the correct significant level to make an objective conclusion statistically. When determining the significant level one must consider how many hypothesizes are being examined, are any interim analysis is planned and how many tests will be ran in total. In the determination of significant level one can also use formal methods such as Bonferroni, Turkey-Cramer, Scheffe's or Duncan-Walker method.
In selecting the appropriate test one must consider the number of samples compared, the independence of subjects in the sample in terms of their relationship or not related and the distribution type. That is whether they are normal or non normal as well as the type of variable studied. The table below will show the appropriate test statistics to be used based on the above consideration matrix.
Type of data number of samples relationship type of distribution statistical test
Binary 1 not applicable binary binomial test
Binary 2 independent binary Chi-square
Test or
Fisher's exact
Test
Binary >2 independent binary Chi-square test
Binary 2 paired binary McNemar's test
Binary >2 related binary Cochran's test
Type of data number of samples relationship type of distribution statistical test
Continuous 1 not applicable normal t-test for mean
Chi-square
Test for
Variance
Continuous 1 not applicable non-normal Wilcoxon
Single-ranked
Test or
Sign test
Continuous 2 independent normal T-test for
Mean F-test
for variance
Continuous 2 independent non-normal Wilcoxon
Rank sum
Test
Continuous 2 paired normal paired t-test
Continuous 2 paired non-normal Wilcoxon
Signed
Ranked
Test or
Sign test
Continuous >2 independent normal One-Way
ANOVA for
Mean and
Bartlett's
Test for
Homogeneity
Of variance
Type of data number of samples relationship type of distribution statistical test
Continuous >2 independent non-normal Kruskal-
Wallis test
Continuous >2 related non-normal Friedman
Rank sum test
