File Exchange

image thumbnail

fdr_bh

version 2.3.0.0 (8.83 KB) by David Groppe
Benjamini & Hochberg/Yekutieli false discovery rate control procedure for a set of statistical tests

82 Downloads

Updated 19 Dec 2015

View License

Executes the Benjamini & Hochberg (1995) procedure for controlling the false discovery rate (FDR) of a family of hypothesis tests. FDR is the expected proportion of rejected hypotheses that are mistakenly rejected (i.e., the null hypothesis is actually true for those tests). FDR is generally a somewhat less conservative/more powerful method for correcting for multiple comparisons than procedures like Bonferroni correction that provide strong control of the family-wise error rate (i.e., the probability that one or more null hypotheses are mistakenly rejected).
This function implements both versions of the Benjamini & Hochberg procedure: the one that assumes independent or positively dependent tests and the one that makes no assumptions about test dependency. The latter procedure (published by Benjamini & Yekutieli in 2001) is always appropriate but is much more conservative than the former. Both procedures are quite simple and require only the p-values of all tests in the family
In addition to correcting p-values for multiple comparisons, this function also returns the multiple comparison adjusted confidence interval coverage for any p-values that remain significant after FDR adjustment. These "FCR-adjusted selected confidence intervals" guarantee that the false coverage-statement rate (FCR) is less than the p-value thredho for signifcance (Benjamini, Y., & Yekutieli, D., 2005).
Benjamini, Y. & Hochberg, Y. (1995) Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B (Methodological). 57(1), 289-300.
Benjamini, Y. & Yekutieli, D. (2001) The control of the false discovery rate in multiple testing under dependency. The Annals of Statistics. 29(4), 1165-1188.
Benjamini, Y., & Yekutieli, D. (2005). False discovery rate–adjusted multiple confidence intervals for selected parameters. Journal of the American Statistical Association, 100(469), 71–81. doi:10.1198/016214504000001907
For a review on false discovery rate control and other contemporary techniques for correcting for multiple comparisons see:
Groppe, D.M., Urbach, T.P., & Kutas, M. (2011) Mass univariate analysis of event-related brain potentials/fields I: A critical tutorial review.
Psychophysiology, 48(12) pp. 1711-1725, DOI: 10.1111/j.1469-8986.2011.01273.x http://www.cogsci.ucsd.edu/~dgroppe/PUBLICATIONS/mass_uni_preprint1.pdf

Comments and Ratings (13)

David Groppe

@ShaneS Currently you need to remove the NaN p-values yourself. For example, you could do something like this h=fdr_bh(pvals(~isnan(pvals)). If I have time, I'll edit the code to automatically deal with this in the future.

ShaneS

Hi, thanks for the toolbox. I want to perform FDR on a matrix of p values. How can I tell it to ignore the NaNs inside?

Thanks!
Small thing: I would convert h (line 202) to a logical so as not to cause unexpected indexing issues when no tests are significant, ie:
h=logical(pvals*0)

imu931

Very helpful, thanks.

Kristin

Works well, thank you.

Greg

Great code, thanks! It would be useful to add the ability for the user to provide a histogram of p-values as input, rather than a list of p-values. This would help in cases where a very large number of hypotheses are being tested.

sundar

Great...Exactly what i was looking for.
Thank you very much

Updates

2.3.0.0

Comments updated to reflect FCR-adjusted CI functionality

2.2.0.0

Previous version would not unzip for some reason.

2.1.0.0

Function now returns FCR-adjusted selected confidence interval coverage

1.5.0.0

Dirk Poot made the computation of adjusted p-values much more efficient. (Dank u Dirk!)

1.4.0.0

Comments updated

1.2.0.0

Now returns FDR-adjusted p-values. Thanks to Yishai Shimoni for inspiring this.

MATLAB Release Compatibility
Created with R2010a
Compatible with any release
Platform Compatibility
Windows macOS Linux