risk.validation.correlatedBinomialTest

R2026b

Correlated binomial test

Since R2025a

Syntax

hCorrBinTest = risk.validation.correlatedBinomialTest(Probability,NumEvents,NumTrials,Correlation)

hCorrBinTest = risk.validation.correlatedBinomialTest(Probability,NumEvents,NumTrials,Correlation,Name=Value)

[hCorrBinTest,corrBinTestOutput] = risk.validation.correlatedBinomialTest(___)

Description

hCorrBinTest = risk.validation.correlatedBinomialTest(Probability,NumEvents,NumTrials,Correlation) returns the correlated binomial test result or results, hCorrBinTest, for a given set of probabilities, events, and correlation trials. The output is 1 if the test rejects the null hypothesis at the 95% confidence level, or 0 otherwise. Probability contains numeric values that represent quantities such as probability of default (PD) estimates. For PD models, Correlation represents the asset correlation of a one-factor model.

hCorrBinTest = risk.validation.correlatedBinomialTest(Probability,NumEvents,NumTrials,Correlation,Name=Value) specifies optional name-value arguments. For example, you can set a specific confidence level for the correlated binomial test by using the ConfidenceLevel name-value argument.

[hCorrBinTest,corrBinTestOutput] = risk.validation.correlatedBinomialTest(___) also returns a structure corrBinTestOutput that contains the method used in the correlated binomial test and a table of summary metrics. Specify corrBinTestOutput as the second output argument with any of the input argument combinations in the previous syntaxes.

Examples

Apply Correlated Binomial Test on PD Data

Use the correlatedBinomialTest function to determine whether the correlated binomial test rejects the null hypothesis at a 0.95 confidence level in a probability of default (PD) data set. In this example, you use the credit validation data set, which includes a table, ScorecardValidationData, that contains PD values and their corresponding default status. Before you apply the test, it is common practice to:

• Group the probabilities by deciles.

• Compute the average probability of each group.

• Compute the total number of defaults and loans of each group.

This example assumes the model correlation is 0.1.

Load and display the credit validation data.

load CreditValidationData.mat
head(ScorecardValidationData)

    CreditScore      PD       Default
    ___________    _______    _______

      579.86       0.14182       0   
      563.65       0.17143       0   
      549.52       0.20106       0   
      546.25       0.20845       0   
      485.34       0.37991       0   
      482.07       0.39065       0   
      579.86       0.14182       1   
      451.73         0.494       0   

Group Probabilities by Deciles

Extract the variable PD from the table ScorecardValidationData and group the probabilities by using the risk.validation.groupNumberByQuantile function with the fully qualified namespace risk.validation. Specify deciles as the quantile type.

Probability = ScorecardValidationData.PD;
QuantileType = "deciles";
PDDecileNumber = risk.validation.groupNumberByQuantile(Probability,QuantileType);

Determine Average Probabilities by Group

Next, calculate the average probability for each group by using the groupsummary function.

PDAvgByDecile = groupsummary(Probability,PDDecileNumber,"mean");

Compute Total Defaults and Loans by Group

Extract the variable Default from the table ScorecardValidationData and use this variable as the default indicator. Then use the groupsummary function to compute the sums of defaults and loans for each group.

DefaultIndicator = ScorecardValidationData.Default;
[NumDefaultsByDecile,~,NumLoansByDecile] = groupsummary(DefaultIndicator,PDDecileNumber,"sum");

Apply Correlated Binomial Test

You can then apply the correlatedBinomialTest function with the fully qualified namespace risk.validation for each group to see if the test rejects the null hypothesis. Use the average probabilities and the total number of defaults and loans as input arguments. Then, display a table with the results.

Correlation = 0.1;
[hCorrBinTest,corrBinTestOutput] = risk.validation.correlatedBinomialTest(PDAvgByDecile,NumDefaultsByDecile,NumLoansByDecile,Correlation)

hCorrBinTest = 10×1

     0
     0
     0
     0
     0
     0
     0
     0
     0
     0

corrBinTestOutput = struct with fields:
     Method: "exact"
    Results: [10×10 table]

disp(corrBinTestOutput.Results)

    RejectCorrBinTest    PValue     NumEvents    CriticalValue    ConfidenceLevel    NumTrials    Probability    ObservedProbability    Correlation    EventCorrelation
    _________________    _______    _________    _____________    _______________    _________    ___________    ___________________    ___________    ________________

            0            0.27285        7             12               0.95             36          0.13683            0.19444              0.1            0.043101    
            0            0.19011       11             15               0.95             36          0.19799            0.30556              0.1            0.050533    
            0             0.4731        8             16               0.95             34          0.22656            0.23529              0.1            0.053214    
            0            0.22263       10             14               0.95             27          0.25552            0.37037              0.1            0.055529    
            0            0.25011       16             23               0.95             42           0.2868            0.38095              0.1            0.057635    
            0             0.5342       13             24               0.95             41          0.32664            0.31707              0.1            0.059802    
            0            0.76858       10             23               0.95             36          0.37627            0.27778              0.1            0.061788    
            0            0.58845       11             20               0.95             29          0.40319            0.37931              0.1            0.062566    
            0            0.53888       18             29               0.95             40          0.45525               0.45              0.1            0.063514    
            0            0.53823       22             32               0.95             39          0.56233             0.5641              0.1            0.063273    

Input Arguments

Probability — Probability values

numeric vector with values in the range (0,1)

Probability values, specified as a numeric vector with values in the range (0,1). Probability contains values that indicate quantities such as PD estimates.

NumEvents — Number of events observed

vector of nonnegative integers

Number of events observed, specified as a vector of nonnegative integers. For PD models, NumEvents contains the number of defaults observed.

NumTrials — Number of trials

vector of positive integers

Number of trials, specified as a vector of positive integers. Each element in NumTrials must be greater than or equal to the corresponding elements of NumEvents. For PD models, NumTrials contains the number of loans.

Correlation — Model correlation

numeric vector with values in the range (0,1)

Model correlation, specified as a numeric vector with values in the range (0,1). For PD models, Correlation represents the asset correlation of a one-factor model.

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: hCorrBinTest = risk.validation.correlatedBinomialTest(Probability,NumEvents,NumTrials,Correlation,ConfidenceLevel=0.99) specifies a confidence level of 0.99.

ConfidenceLevel — Confidence level

0.95 (default) | numeric scalar in the range (0,1)

Confidence level of the hypothesis test, specified as a numeric scalar in the range (0,1).

Method — Hypothesis test method

"exact" (default) | "approximate"

Hypothesis test method, specified as either "exact" or "approximate".

• "exact" — The function uses an exact probability based on a binomial distribution.

• "approximate" — The function uses an approximate test based on a normal distribution.

Data Types: string | char

Output Arguments

hCorrBinTest — Hypothesis test results

numeric vector containing values of 1 or 0

Hypothesis test results, returned as a numeric vector containing values of 1 or 0.

• A value of 1 rejects the null hypothesis at the specified confidence level.

• A value of 0 fails to reject the null hypothesis at the specified confidence level.

corrBinTestOutput — Output metrics

structure

Output metrics, returned as a structure with the following fields:

• Method — Method used for the hypothesis test.

• Results — Table with columns:

  • RejectCorrBinTest — Numeric value that indicates whether the null hypothesis was rejected. This column represents the same values as hCorrBinTest.

  • PValue — Numeric value in the range [0,1] representing the p-value for the hypothesis test returned as a scalar in the range [0,1]. A small value indicates that the null hypothesis might not be valid.

  • NumEvents — Numeric value representing the number of events observed. This column is the test statistic for the hypothesis test.

  • CriticalValue — Numeric value representing the minimum number of events at which the test rejects the null hypothesis for the given probability and confidence level.

  • ConfidenceLevel — Numeric value representing the confidence level for the hypothesis test.

  • NumTrials — Numeric value representing the number of trials.

  • Probability — Numeric value representing the probability values.

  • ObservedProbability — Numeric value representing the ratio of NumEvents to NumTrials.

  • Correlation — Numeric value representing the model correlation.

  • EventCorrelation — Numeric value representing the implied correlation between events.

More About

Correlated Binomial Test

The correlated binomial test [1] requires the following inputs:

• Predicted probability, P

• Number of trials, N

• Number of events observed, E

• Correlation parameter, ρ

The null hypothesis states that the probability P is a correct estimate of the event frequency. The risk.validation.correlatedBinomialTest function implements a one-sided version of the test, where the alternative hypothesis states that the probability P underestimates the event frequency. Rejection of the correlated binomial test means that the probability P is statistically small compared to the observed event frequency E / N. The main difference between the correlated and uncorrelated version of the binomial test is that the correlated version relaxes the independence assumption. Following Basel 2005, the correlated binomial test introduces a latent variable A_i for each trial i according to the following 1-factor model:

$$A_i=\sqrt{\rho }Z+\sqrt{1-\rho }{ϵ}_i\text{,}1\le i\le N\text{,}$$

with

$$\begin{array}{l}Z~N\left(0,1\right),\\ {ϵ}_i~N\left(0,1\right)\text{, }{A}_i~N\left(0,1\right)\text{, and corr}\left(Z,{ϵ}_i\right)=0\text{,}1\le i\le N,\\ \text{corr}\left({ϵ}_i,{ϵ}_j\right)=0\text{ and corr}\left(A_i,A_j\right)=\rho \text{,}1\le i<j\le N.\end{array}$$

An event, Y_i, occurs when the latent variable A_i falls below a threshold ɣ, where $$\gamma ={\Phi }^{-1}\left(P\right)$$ and Φ is the cumulative distribution function of the standard normal distribution. If A_i satisfies this threshold criteria, then Y_i = 1; otherwise Y_i = 0. The number of events, E, is then:

$$E=\underset{i=1}{\sum ^N}{Y}_i\text{.}$$

The event correlation is given by:

$$\text{corr}\left(Y_i,Y_j\right)=\frac{{\Phi }_2\left(\gamma ,\gamma ,\rho \right)-P^2}{P\left(1-P\right)}\text{,}1\le i<j\le N\text{.}$$

where Φ₂ is the standard bivariate normal distribution.

The test statistic for the correlated binomial test is the number of events, E. The test has two versions: the exact method and the approximate method. For the exact method, the distribution of the events variable, E, is computed with numerical integration according to the following formula:

$$\text{Prob}\left(E\le k\right)={\displaystyle {\int }_{-\infty }^{\infty }\underset{l=0}{\sum ^k}}\left(\begin{array}{c}N\\ k\end{array}\right)\Phi {\left(\frac{\gamma -\sqrt{\rho }x}{\sqrt{1-\rho }}\right)}^l{\left(1-\Phi \left(\frac{\gamma -\sqrt{\rho }x}{\sqrt{1-\rho }}\right)\right)}^{N-l}\varphi \left(x\right)dx\text{,}$$

where Φ is the standard normal distribution function as before, and the lowercase ϕ function is the standard normal density function. The critical value is given by:

$$E^{*}=\min \left\{k:\text{}\text{ Prob}\left(E\ge k\right)\le 1-\alpha \right\}\text{,}$$

where α is the test confidence level, with a default value of 95%. The critical value, E^*, is the minimum integer value k such that the tail probability of the correlated binomial distribution for values k and above is less than or equal to the complement of the confidence level, 1 − α. Because the correlated binomial distribution is a discrete distribution, the probability that the test statistic is exactly equal to the critical value is positive. The null hypothesis is rejected whenever the test statistic is greater than or equal to the critical value.

For the approximate method, the distribution of the events variable, E, is approximated according to the following formula:

$$\text{Prob}\left(E\le k\right)\approx \Phi \left(\frac{\sqrt{1-\rho }{\Phi }^{-1}\left(\frac{k}{N}\right)-\gamma }{\sqrt{\rho }}\right).$$

The critical value is given by:

$$E_{\text{approx}}^{*}=N\Phi \left(\frac{\sqrt{\rho }{\Phi }^{-1}\left(\alpha \right)+\gamma }{\sqrt{1-\rho }}\right)\text{.}$$

If the inputs to risk.validation.correlatedBinomialTest are vectors of size G, then G statistical tests are run and the function returns the results for each of these tests.

For credit risk applications, P is typically a PD, N is the number of loans, and E is the number of defaults. The correlation parameter ρ can be an average correlation based on borrower-specific correlation parameters used in other 1-factor correlation models such as the asymptotic single risk factor (ASRF) model or a 1-factor credit copula. For more information, see Calculating Regulatory Capital with the ASRF Model. The G groups may be G different credit ratings or deciles based on the PD values of a continuous PD model.

References

[1] Basel Committee on Banking Supervision. “Studies on the Validation of Internal Rating Systems.” Working Paper 14. May, 2005. https://www.bis.org/publ/bcbs_wp14.htm.

Version History

Introduced in R2025a