File Exchange

image thumbnail

Reliability study designer

version (2.6 KB) by Joris Meurs
Calculation of the number of required subjects and replications in a reliability assessment


Updated 18 Apr 2016

View Version History

View License

File name : 'Shoukri.m'. This file can be used
for calculating the required number of subjects
at a desired kappa value for reliability
Five input values : 'alfa', 'omega', 'pi', 'rho' and 'kappa'
Three output value : 'k', 'n' and 'N'
alfa : significance level (value between 0 and 1)
omega : desired interval of kappa (value between 0 and 1)
pi : probability of positive rating (value between 0 and 1)
rho : within-subject interclass correlation
kappa : desired coefficient of agreement (value between 0 and 1)

k : number of subjects (value larger than 1)
n : number of replication (value equal to or larger than 1)
N : number of allocations (value larger than 1)
Figure : plot of the number of subjects versus the number of replications
Input syntax : Shoukri(alfa,omega,pi,kappa)
NOTE: No default values are set
[1] : Shoukri, M. M. (2010). Measures of interobserver agreement and reliability: CRC Press.
[2] : Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159-174.
[3] : Shoukri, M., Asyali, M. & Walter, S. (2003). Issues of cost and efficiency in the design of reliability studies. biometrics. 59(4): p. 1107-1112.
[4] : Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement. 20(1): p. 37-46

Developed by Joris Meurs BASc (2015)

Cite As

Joris Meurs (2021). Reliability study designer (, MATLAB Central File Exchange. Retrieved .

Comments and Ratings (2)

Joris Meurs

Requirements are updated now

Carl Witthoft

You need to list the Statistics&MachineLearning Toolbox as a requirement for this. "icdf" is not available in the base package.

MATLAB Release Compatibility
Created with R2014b
Compatible with any release
Platform Compatibility
Windows macOS Linux

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!