risk.validation.correlatedBinomialTest

Return correlated binomial test result

Since R2025a

Syntax

hCorrBinTest = risk.validation.correlatedBinomialTest(Probability,NumEvents,NumTrials,Correlation)

hCorrBinTest = risk.validation.correlatedBinomialTest(Probability,NumEvents,NumTrials,Correlation,Name=Value)

[hCorrBinTest,corrBinTestOutput] = risk.validation.correlatedBinomialTest(___)

Description

hCorrBinTest = risk.validation.correlatedBinomialTest(Probability,NumEvents,NumTrials,Correlation) returns the correlated binomial test result, hCorrBinTest, for a given set of probabilities, events, and correlation trials. The output is 1 if the test rejects the null hypothesis at the 95% confidence level, or 0 otherwise. Probability contains numeric values that represent quantities such as probability of default (PD) estimates. For PD models, Correlation represents the asset correlation of a one-factor model.

hCorrBinTest = risk.validation.correlatedBinomialTest(Probability,NumEvents,NumTrials,Correlation,Name=Value) specifies optional name-value arguments. For example, you can set a specific confidence level for the correlated binomial test by using the ConfidenceLevel name-value argument.

[hCorrBinTest,corrBinTestOutput] = risk.validation.correlatedBinomialTest(___) also returns a structure corrBinTestOutput, containing the method used in the correlated binomial test and a table of summary metrics.

Examples

Apply Correlated Binomial Test on PD Data

Use the correlatedBinomialTest function to determine whether the correlated binomial test rejects the null hypothesis at a 0.95 confidence level in a probability of default (PD) dataset. In this example, you use the credit validation data set, which includes a table, ScorecardValidationData, that contains PD values and their corresponding default status. Before you apply the test, it is common practice to:

• Group the probabilities by deciles.

• Compute the average probability of each group.

• Compute the total number of defaults and loans of each group.

This example assumes the model correlation is 0.1.

Load and display the credit validation data.

load CreditValidationData.mat
head(ScorecardValidationData)

    CreditScore      PD       Default
    ___________    _______    _______

      579.86       0.14182       0   
      563.65       0.17143       0   
      549.52       0.20106       0   
      546.25       0.20845       0   
      485.34       0.37991       0   
      482.07       0.39065       0   
      579.86       0.14182       1   
      451.73         0.494       0   

Group Probabilities by Deciles

Extract the variable, PD, from the table ScorecardValidationData and group the probabilities by using the risk.validation.groupNumberByQuantile function with the fully qualified namespace risk.validation. Specify "deciles" as the quantile type.

Probability = ScorecardValidationData.PD;
QuantileType = "deciles";
PDDecileNumber = risk.validation.groupNumberByQuantile(Probability,QuantileType);

Determine Average Probabilities by Group

Next, calculate the average probability for each group by using the groupsummary function.

PDAvgByDecile = groupsummary(Probability,PDDecileNumber,"mean");

Compute Total Defaults and Loans by Group

Extract the variable Default from the table ScorecardValidationData and use this variable as the default indicator. Then use the groupsummary function to compute the sums of defaults and loans for each group.

DefaultIndicator = ScorecardValidationData.Default;
[NumDefaultsByDecile,~,NumLoansByDecile] = groupsummary(DefaultIndicator,PDDecileNumber,"sum");

Apply Correlated Binomial Test

You can then apply the correlatedBinomialTest function with the fully qualified namespace risk.validation for each group to see if the test rejects the null hypothesis. Use the average probabilities and the total number of defaults and loans as input arguments. Then, display a table with the results.

Correlation = 0.1;
[hCorrBinTest,corrBinTestOutput] = risk.validation.correlatedBinomialTest(PDAvgByDecile,NumDefaultsByDecile,NumLoansByDecile,Correlation)

hCorrBinTest = 10×1 logical array

   0
   0
   0
   0
   0
   0
   0
   0
   0
   0

corrBinTestOutput = struct with fields:
     Method: "exact"
    Results: [10×10 table]

disp(corrBinTestOutput.Results)

    RejectCorrBinTest    NumEvents    CriticalValue    PValue     ConfidenceLevel    NumTrials    Probability    ObservedProbability    Correlation    EventCorrelation
    _________________    _________    _____________    _______    _______________    _________    ___________    ___________________    ___________    ________________

          false              7             12          0.27285         0.95             36          0.13683            0.19444              0.1            0.043101    
          false             11             15          0.19011         0.95             36          0.19799            0.30556              0.1            0.050533    
          false              8             16           0.4731         0.95             34          0.22656            0.23529              0.1            0.053214    
          false             10             14          0.22263         0.95             27          0.25552            0.37037              0.1            0.055529    
          false             16             23          0.25011         0.95             42           0.2868            0.38095              0.1            0.057635    
          false             13             24           0.5342         0.95             41          0.32664            0.31707              0.1            0.059802    
          false             10             23          0.76858         0.95             36          0.37627            0.27778              0.1            0.061788    
          false             11             20          0.58845         0.95             29          0.40319            0.37931              0.1            0.062566    
          false             18             29          0.53888         0.95             40          0.45525               0.45              0.1            0.063514    
          false             22             32          0.53823         0.95             39          0.56233             0.5641              0.1            0.063273    

Input Arguments

Probability — Probability values

numeric vector with values in the range (0,1)

Probability values, specified as a numeric vector with values in the range (0,1). Probability contains values that indicate quantities such as PD estimates.

NumEvents — Number of events observed

numeric vector containing nonnegative integers

Number of events observed, specified as a numeric vector containing nonnegative integers. For PD models, NumEvents contains the number of defaults observed.

NumTrials — Number of trials

numeric vector containing positive integers

Number of trials, specified as a numeric vector containing positive integers. Each element in NumTrials must be greater than or equal to the corresponding elements of NumEvents. For PD models, NumTrials contains the number of loans.

Correlation — Model correlation

numeric vector with values in the range [0,1)

Model correlation, specified as a numeric vector with values in the range [0,1). For PD models, Correlation represents the asset correlation of a one-factor model.

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: hBinTest = risk.validation.correlatedBinomialTest(PDAvgByDecile,NumDefaultsByDecile,NumLoansByDecile,ConfidenceLevel=0.95)

ConfidenceLevel — Confidence level

0.95 (default) | numeric scalar in the range (0,1)

Confidence level of the hypothesis test, specified as a numeric scalar in the range (0,1).

Method — Hypothesis test method

"exact" (default) | "approximate"

Hypothesis test method, specified as either "approximate" or "exact".

• "exact" — correlatedBinomialTest uses an exact probability based on numerical integration.

• "approximate" — correlatedBinomialTest uses an approximate test based on an approximation following Vasicek's model.

Output Arguments

hCorrBinTest — Hypothesis test result

logical vector containing 0 or 1

Hypothesis test result, returned as a logical vector.

• A value of 1 rejects the null hypothesis at the specified confidence level.

• A value of 0 fails to reject the null hypothesis at the specified confidence level.

corrBinTestOutput — Output metrics

structure

Output metrics, returned as a structure with the following fields:

• Method — Method used for the hypothesis test.

• Results — Table with columns:

  • RejectCorrBinTest — Logical vector that indicates whether the null hypothesis was rejected. This column represents the same values as hCorrBinTest.

  • NumEvents — Number of events observed. This column is the test statistic for the hypothesis test.

  • CriticalValue — Minimum number of events at which the test rejects the null hypothesis for the given probability and confidence level.

  • PValue — p-value for the hypothesis test returned as a scalar in the range [0,1]. A small value indicates that the null hypothesis might not be valid.

  • ConfidenceLevel — Confidence level for the hypothesis test.

  • NumTrials — Number of trials.

  • Probability — Probability values.

  • ObservedProbability — Ratio of NumEvents to NumTrials.

  • Correlation — Model correlation.

  • EventCorrelation — Implied correlation between events.

More About

Correlated Binomial Test

The correlated binomial test [1] requires the following inputs:

• Predicted probability, P

• Number of trials, N

• Number of events observed, E

• Correlation parameter, ρ

The null hypothesis states that the probability P is a correct estimate of the event frequency. The risk.validation.correlatedBinomialTest function implements a one-sided version of the test, where the alternative hypothesis states that the probability P underestimates the event frequency. Rejection of the correlated binomial test means that the probability P is statistically small compared to the observed event frequency E/N. The main difference between the correlated and the uncorrelated version of the binomial test is that the correlated version relaxes the independence assumption. Following Basel 2005, the correlated binomial test introduces a latent variable A_i for each trial i according to the following 1-factor model:

$$A_i=\sqrt{\rho }Z+\sqrt{1-\rho }{\epsilon }_i,i=1,\dots ,N$$

with

$$\begin{array}{l}Z\sim N(0,1),{\epsilon }_i\sim N(0,1)\forall i,A_i\sim N(0,1)\forall i,\\ \mathrm{corr}(Z,{\epsilon }_i)=0\forall i,\mathrm{corr}({\epsilon }_i,{\epsilon }_j)=0\forall i\ne j,\\ \mathrm{corr}(A_i,A_j)=\rho \forall i\ne j.\end{array}$$

An event, Y_i, occurs when the latent variable A_i falls below a threshold A_i≤ˠ. If A_i satisfies this threshold criteria, then Y_i=1, otherwise Y_i=0, where ˠ=Φ^-1(P) and Φ is the standard normal distribution. The number of events, E, is then:

$$E={\displaystyle \sum _{i=1}^N{Y}_i}.$$

The event correlation is given by:

$$\mathrm{corr}(Y_i,Y_j)=\frac{{\Phi }_2(\gamma ,\gamma ,\rho )-P^2}{P(1-P)}$$

where Φ₂ is the standard bivariate normal distribution.

The test statistic for the correlated binomial test is the number of events, E. There are two versions of the test: the "exact" method and the "approximate" method. For the “exact” method, the distribution of the events variable, E, is computed with numerical integration according to the following formula:

$$\mathrm{Prob}(E\le k)={\displaystyle \underset{-\infty }{\overset{\infty }{\int }}{\displaystyle \sum _{l=0}^k\left(\begin{array}{c}N\\ k\end{array}\right)}}\Phi {\left(\frac{\gamma -\sqrt{\rho }x}{\sqrt{1-\rho }}\right)}^l{\left(1-\Phi \left(\frac{\gamma -\sqrt{\rho }x}{\sqrt{1-\rho }}\right)\right)}^{N-l}\varphi (x)dx$$

where Φ is the standard normal distribution function as before, and the lowercase ϕ function is the standard normal density function. The critical value is given by:

$$E^{*}=\min \left\{k|\mathrm{Prob}(E\ge k\right\}\le 1-\alpha$$

where α is the test confidence level, with a default value of 95%. The critical value, E*, is the minimum integer value k such that the tail probability of the correlated binomial distribution for values k and above is less than or equal to the complement of the confidence level, 1-α. Since this is a discrete distribution, there is a positive probability that the test statistic is exactly equal to the critical value. The null hypothesis is rejected whenever the test statistic is greater than or equal to the critical value. For the “approximate” method, the distribution of the events variable, E, is approximated according to the following formula:

$$\mathrm{Prob}(E\le k)\approx \Phi \left(\frac{\sqrt{1-\rho }{\Phi }^{-1}\left(\frac{k}{N}\right)-\gamma }{\sqrt{\rho }}\right).$$

The critical value is given by:

$$E_{approx}^{*}=N\Phi \left(\frac{\sqrt{\rho }{\Phi }^{-1}(\alpha )+\gamma }{\sqrt{1-\rho }}\right).$$

If the inputs to risk.validation.correlatedBinomialTest are vectors of size G, then G statistical tests are run and the function returns the results for each of these tests.

For credit risk applications, P is typically a PD, N is the number of loans, and E is the number of defaults. The correlation parameter ρ can be an average correlation based on borrower specific correlation parameters used in other 1-factor correlation models such as the asymptotic single risk factor (ASRF) model or a 1-factor credit copula. For more information, see Calculating Regulatory Capital with the ASRF Model. The G groups may be G different credit ratings or deciles based on the PD values of a continuous PD model.

References

[1] Basel Committee on Banking Supervision, “Studies on the Validation of Internal Rating Systems”, Working Paper 14, May, 2005. https://www.bis.org/publ/bcbs_wp14.htm.

Version History

Introduced in R2025a