Can I do multiple times of T test?

Dear all:

In my case, there are 7 classes: C1, C2, C3, C4, C5, C6 and C7. In each class, there are 7 students. The dependent variable is the exam score of each student. Now I want to compare the score of class 1 to class 7 (C1 VS C7), class 2 to class 7 (C2 VS C7), class 3 to class 7 (C3 VS C7), ..., and class 6 to class 7 (C6 VS C7). My question is:

Can I do separate t-test of each of them? If so, should I change the confidential level? Or if not, what statistic test I should do?

Thank you.

Answer 1

[Star Strider]

Yes. You are doing single comparisons, not multiple comparisons, so you do not have to do any corrections for multiple comparisons. Use an unpaired t-test, because as I understand your design, your data are independent.

You can also use the Wilcoxon rank sum test (the ranksum) function) if your data do not conform to the assumptions of the t-test.

[Tao Lu]

Thank you for your answer. I think it makes sense :) However, here is another situation: There are 7 difficulty level math exams, and in total 7 students take each of the 7 difficulty level exams. Now I want to compare the exam score of E1 VS E7, E2 VS E7, ..., and E6 VS E7. Could you let me know what kind of t-test I should use? Thank you!

[Star Strider]

My pleasure.

Are the same students sitting the same exam more than once? This then becomes a repeated-measures problem. The samples are no longer independent (this is a paired design), and would probably require a repeated-measures analysis-of-variance. You need to discuss this with a statistician at your university, since the low number of participants and the large number of results make the statistical analysis complicated. I haven’t done a repeated-measures ANOVA in several years, so I would be reluctant to advise you on it.

If your data are independent (different sets of students sitting the same examination once) conform to the assumptions of the t-test (this is a very important consideration), use an unpaired t-test (the ttest2 function).

If they are independent and if you do not know the distribution of scores, or if they are not normally distributed (as the t-test requires), use the ranksum function. The Wilcoxon rank sum test (for unpaired, independent data) compares the medians of the two data, and since it is independent of the distribution of your data, is probably the most appropriate function to use.