quantile

Quantile expected shortfall (ES) backtest by Acerbi and Szekely

Syntax

TestResults = quantile(ebts)

[TestResults,SimTestStatistic] = quantile(ebts,Name,Value)

Description

TestResults = quantile(ebts) runs the quantile ES backtest of Acerbi-Szekely (2014).

[TestResults,SimTestStatistic] = quantile(ebts,Name,Value) adds an optional name-value pair argument for TestLevel.

Examples

Run an ES Quantile Test

Create an esbacktestbysim object.

load ESBacktestBySimData
rng('default'); % for reproducibility
ebts = esbacktestbysim(Returns,VaR,ES,"t",...
       'DegreesOfFreedom',10,...
       'Location',Mu,...
       'Scale',Sigma,...
       'PortfolioID',"S&P",...
       'VaRID',["t(10) 95%","t(10) 97.5%","t(10) 99%"],...
       'VaRLevel',VaRLevel);

Generate the ES quantile test report.

TestResults = quantile(ebts)

TestResults=3×10 table
    PortfolioID        VaRID        VaRLevel    Quantile    PValue    TestStatistic    CriticalValue    Observations    Scenarios    TestLevel
    ___________    _____________    ________    ________    ______    _____________    _____________    ____________    _________    _________

       "S&P"       "t(10) 95%"        0.95       reject     0.002       -0.10602         -0.055798          1966          1000         0.95   
       "S&P"       "t(10) 97.5%"     0.975       reject         0       -0.15697         -0.073513          1966          1000         0.95   
       "S&P"       "t(10) 99%"        0.99       reject         0       -0.26561          -0.10117          1966          1000         0.95   

Input Arguments

ebts — esbacktestbysim object

object

esbacktestbysim (ebts) object, which contains a copy of the given data (the PortfolioData, VarData, ESData, and Distribution properties) and all combinations of portfolio ID, VaR ID, and VaR levels to be tested. For more information on creating an esbacktestbysim object, see esbacktestbysim.

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: [TestResults,SimTestStatistic] = quantile(ebts,'TestLevel',0.99)

'TestLevel' — Test confidence level

0.95 (default) | numeric with values between 0 and 1

Test confidence level, specified as the comma-separated pair consisting of 'TestLevel' and a numeric value between 0 and 1.

Data Types: double

Output Arguments

TestResults — Results

table

Results, returned as a table where the rows correspond to all combinations of portfolio ID, VaR ID, and VaR levels to be tested. The columns correspond to the following information:

• 'PortfolioID' — Portfolio ID for the given data

• 'VaRID' — VaR ID for each of the VaR data columns provided

• 'VaRLevel' — VaR level for the corresponding VaR data column

• 'Quantile'— Categorical array with categories 'accept' and 'reject' indicating the result of the quantile test

• 'PValue'— P-value of the quantile test

• 'TestStatistic'— Quantile test statistic

• 'CriticalValue'— Critical value for the quantile test

• 'Observations'— Number of observations

• 'Scenarios'— Number of scenarios simulated to get the p-values

• 'TestLevel'— Test confidence level

SimTestStatistic — Simulated values of test statistic

numeric array

Simulated values of the test statistic, returned as a NumVaRs-by-NumScenarios numeric array.

More About

Quantile Test by Acerbi and Szekely

The quantile test (also known as the third Acerbi-Szekely test) uses a sample estimator of the expected shortfall.

The expected shortfall for a sample Y₁,…,Y_N is:

$$\stackrel{⌢}{ES}(Y)=-\frac{1}{\lfloor N{p}_{VaR}\rfloor }{\displaystyle \sum _{i=1}^{\lfloor N{p}_{VaR}\rfloor }{Y}_{\left[i\right]}}$$

where

N is the number of periods in the test window (t = 1,…,N).

P_VaR is the probability of VaR failure defined as 1-VaR level.

Y_[1],…,Y_[N] are the sorted sample values (from smallest to largest), and $$\lfloor N{p}_{VaR}\rfloor$$ is the largest integer less than or equal to Np_VaR.

To compute the quantile test statistic, a sample of size N is created at each time t as follows. First, convert the portfolio outcomes to X_t to ranks $$U_1=P_1(X_1),\mathrm{...},U_N=P_N(X_N)$$ using the cumulative distribution function P_t. If the distribution assumptions are correct, the rank values U₁,…,U_N are uniformly distributed in the interval (0,1). Then at each time t:

• Invert the ranks U = (U₁,…,U_N) to get N quantiles $$P_t^{-1}(U)=(P_t^{-1}(U_1),\mathrm{...},P_t^{-1}(U_N))$$.

• Compute the sample estimator $$\stackrel{⌢}{ES}(P_t^{-1}(U))$$.

• Compute the expected value of the sample estimator $$E\left[\stackrel{⌢}{ES}(P_t^{-1}(V))\right]$$

  where V = (V₁,…,V_N is a sample of N independent uniform random variables in the interval (0,1). This value can be computed analytically.

Define the quantile test statistic as

$$Z_{quantile}=-\frac{1}{N}{\displaystyle \sum _{t=1}^N\frac{\stackrel{⌢}{ES}(P_t^{-1}(U))}{E[\stackrel{⌢}{ES}(P_t^{-1}(V))]}+1}$$

The denominator inside the sum can be computed analytically as

$$E[\stackrel{⌢}{ES}(P_t^{-1}(V))]=-\frac{N}{\lfloor N_{pVaR}\rfloor }{\displaystyle {\int }_0^1{I}_{1-p}(N-\lfloor N_{pVaR}\rfloor },\lfloor N_{pVaR}\rfloor )P_t^{-1}(p)dp$$

where I_x(z,w) is the regularized incomplete beta function. For more information, see betainc.

Significance of the Test

Assuming that the distributional assumptions are correct, the expected value of the test statistic Z_quantile is 0.

This is expressed as:

$$E[Z_{quantile}]=0$$

Negative values of the test statistic indicate risk underestimation. The quantile test is a one-sided test that rejects the model when there is evidence that the model underestimates risk. (For technical details on the null and alternative hypotheses, see Acerbi-Szekely, 2014). The quantile test rejects the model when the p-value is less than 1 minus the test confidence level.

For more information on simulating the test statistics and computing the p-values and critical values, see simulate.

Edge Cases

The quantile test statistic is well-defined when there are no VaR failures in the data.

However, when the expected number of failures Np_VaR is small, an adjustment is required. The sample estimator of the expected shortfall takes the average of the smallest N_tail observations in the sample, where $$N_{tail}=\lfloor N_{pVaR}\rfloor$$. If Np_VaR < 1, then N_tail = 0, the sample estimator of the expected shortfall becomes an empty sum, and the quantile test statistic is undefined.

To account for this, whenever Np_VaR < 1, the value of N_tail is set to 1. Thus, the sample estimator of the expected shortfall has a single term and is equal to the minimum value of the sample. With this adjustment, the quantile test statistic is then well-defined and the significance analysis is unchanged.

References

[1] Acerbi, C., and B. Szekely. Backtesting Expected Shortfall. MSCI Inc. December, 2014.