multcompare for vartestn?

Hi,

Is there a way to know which of the groups being compared vartestn is the one that differs from the rest? The function multcompare does not admit the stats output from the vartestn function. Thanks.

Answer 1

[Scott MacKenzie]

@Tahariet Sharon It is true that multcompare cannot work with input from vartestn. A frustration with multcompare is that it cannot work as a standalone test; that is, it cannot receive as input a matrix of column data and perform a pairwise comparisons test on that data. That's unfortunate, in my view. However, multcompare can receive as input a stats object created from anova1.

Below is a script that performs a pairwise comparisons test on a matrix of column data. The script loads some test data, creates a stats object from that data, then passes that object in to multcompare. Mission accomplished!

The 16x4 test data are in testdata.txt, which is attached. The file contains 16 marks on 4 tests. The last column of output produced by multcompare is the p-value for the hypothesis test that the corresponding mean difference is not equal to 0 for the pairs tested in that row. As seen below, the marks were significantly different (p < .05) between tests 1 and 3 and tests 2 and 3.

multcompare uses the Tukey-Kramer test by default, but other tests (e.g., Bonferroni) can be specified via the 'CType' option.

% load test data: 16 marks on 4 tests
y = readmatrix('testdata.txt'); % 16x4

% do the anova -- just to get a stats object
[~, ~, stats] = anova1(y, {'t1' 't2' 't3' 't4'}, 'off');

% pass stats object into multcompare (p-value in last column of output)
multcompare(stats, 'display', 'off')

ans = 6×6

1.0000 2.0000 -3.9088 -0.8750 2.1588 0.8711 1.0000 3.0000 -7.5338 -4.5000 -1.4662 0.0013 1.0000 4.0000 -4.8463 -1.8125 1.2213 0.3983 2.0000 3.0000 -6.6588 -3.6250 -0.5912 0.0130 2.0000 4.0000 -3.9713 -0.9375 2.0963 0.8464 3.0000 4.0000 -0.3463 2.6875 5.7213 0.1002

[Tahariet Sharon]

Hi and thanks for the time to respond :)

However, my question was about the vartestn in multcompare, not ANOVA. So I want to test that the columns of data vector x come from normal distributions with the same variance, not that their means differ, and to use multcompare to see which one or which ones of the columns do not meet this criteria.

[Scott MacKenzie]

Hmm, yes, I was overly focusing on multcompare. And I see the issue you note with vartestn that it does not indicate which pairs have variances that different significantly. What about using vartest2 on all possibile pairs? Since the test is intended to compare just two sets of scores, I suppose the Bonferroni correction (or something with the same purpose) should be applied to make the tests more conservative. Here's an example using the "examgrades" data in the documentation for vartest2. Seems the null hypothesis of equal variances is rejected for 6 of the 10 pairs of exam grades, or 5 of 10 with the Bonferroni correction.

load examgrades;
y = grades;

[nRow, nCol] = size(y); % 120x5

% alpha with Bonferroni correction
alpha = 0.05;
nTests = nCol * (nCol - 1) / 2;
alpha2 = alpha / nTests;

C = nchoosek(1:nCol,2);
for i=1:nTests
    [h, p] = vartest2(y(:,C(i,1)), y(:,C(i,2)));
    h2 = p < alpha2; % apply Bonferroni correction
    fprintf('Exam %d vs. Exam %d, p=%6.4f, h=%d, h2=%d\n', C(i,1), C(i,2), p, h, h2);
    
end

Exam 1 vs. Exam 2, p=0.0019, h=1, h2=1
Exam 1 vs. Exam 3, p=0.0822, h=0, h2=0
Exam 1 vs. Exam 4, p=0.8812, h=0, h2=0
Exam 1 vs. Exam 5, p=0.0000, h=1, h2=1
Exam 2 vs. Exam 3, p=0.1661, h=0, h2=0
Exam 2 vs. Exam 4, p=0.0031, h=1, h2=1
Exam 2 vs. Exam 5, p=0.0179, h=1, h2=0
Exam 3 vs. Exam 4, p=0.1120, h=0, h2=0
Exam 3 vs. Exam 5, p=0.0002, h=1, h2=1
Exam 4 vs. Exam 5, p=0.0000, h=1, h2=1

[Tahariet Sharon]

Thanks, I will accept your answer, becuase it is currently the one way to do this :)

Thanks for your time again!