Updated 19 Dec 2015
Executes the Benjamini & Hochberg (1995) procedure for controlling the false discovery rate (FDR) of a family of hypothesis tests. FDR is the expected proportion of rejected hypotheses that are mistakenly rejected (i.e., the null hypothesis is actually true for those tests). FDR is generally a somewhat less conservative/more powerful method for correcting for multiple comparisons than procedures like Bonferroni correction that provide strong control of the family-wise error rate (i.e., the probability that one or more null hypotheses are mistakenly rejected).
This function implements both versions of the Benjamini & Hochberg procedure: the one that assumes independent or positively dependent tests and the one that makes no assumptions about test dependency. The latter procedure (published by Benjamini & Yekutieli in 2001) is always appropriate but is much more conservative than the former. Both procedures are quite simple and require only the p-values of all tests in the family
In addition to correcting p-values for multiple comparisons, this function also returns the multiple comparison adjusted confidence interval coverage for any p-values that remain significant after FDR adjustment. These "FCR-adjusted selected confidence intervals" guarantee that the false coverage-statement rate (FCR) is less than the p-value thredho for signifcance (Benjamini, Y., & Yekutieli, D., 2005).
Benjamini, Y. & Hochberg, Y. (1995) Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B (Methodological). 57(1), 289-300.
Benjamini, Y. & Yekutieli, D. (2001) The control of the false discovery rate in multiple testing under dependency. The Annals of Statistics. 29(4), 1165-1188.
Benjamini, Y., & Yekutieli, D. (2005). False discovery rate–adjusted multiple confidence intervals for selected parameters. Journal of the American Statistical Association, 100(469), 71–81. doi:10.1198/016214504000001907
For a review on false discovery rate control and other contemporary techniques for correcting for multiple comparisons see:
Groppe, D.M., Urbach, T.P., & Kutas, M. (2011) Mass univariate analysis of event-related brain potentials/fields I: A critical tutorial review.
Psychophysiology, 48(12) pp. 1711-1725, DOI: 10.1111/j.1469-8986.2011.01273.x http://www.cogsci.ucsd.edu/~dgroppe/PUBLICATIONS/mass_uni_preprint1.pdf
David Groppe (2023). fdr_bh (https://www.mathworks.com/matlabcentral/fileexchange/27418-fdr_bh), MATLAB Central File Exchange. Retrieved .
MATLAB Release Compatibility
Platform CompatibilityWindows macOS Linux
Inspired: Likelihood ratio test for changes in homeolog expression bias (sample)
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!
Discover Live Editor
Create scripts with code, output, and formatted text in a single executable document.
Comments updated to reflect FCR-adjusted CI functionality
Previous version would not unzip for some reason.
Function now returns FCR-adjusted selected confidence interval coverage
Dirk Poot made the computation of adjusted p-values much more efficient. (Dank u Dirk!)
Now returns FDR-adjusted p-values. Thanks to Yishai Shimoni for inspiring this.