Inverse cumulative normal distribution

[Pekis]

Hello,

Here is what seems a good rough approximation of the inverse cumulative normal distribution (Phi^-1(x)) :

Originally based on A logistic approximation to the cumulative normal distribution

It claims a maximum error less than 1.4*10^-4 on all range with this function
Phi(x)=1/(1+e^{-0.07056x^3-1.5976*x})

I simply inverted the function to let Wolfram solve a*y³+b*y-ln(1/x-1)=0 on y (where a=-.07056 and b=-1.5976)

I got a somehow gory formula which can be a bit simplified:
If a=-0.07056 and b=-1.5976 and t(x)=ln(1/x-1)+sqrt(ln²(1/x-1)+4b³/(27a²))
then Phi^-1(x)=(t(x)/(2a))^1/3-(2b³/(27a²t(x)))^1/3

It seems OK with 3 decimals on all range ]0-1[ ... What do you think ?

[Dieter]

(01-12-2016 10:16 PM)Pekis Wrote:  Here is what seems a good rough approximation of the inverse cumulative normal distribution (Phi^-1(x)) :

Sorry, but the link does not work for me. Instead of a google.be link, could you provide the direkt URL of the document you refer to?

(01-12-2016 10:16 PM)Pekis Wrote:  It claims a maximum error less than 1.4*10^-4 on all range with this function
Phi(x)=1/(1+e^{-007056x^3-1.5976*x})

Approximations for the Normal integral and its inverse seem to exist since these functions were first defined. I prefer rational functions for this kind of calculation. But if you want something of this logistic type, here is what I sometimes use:

Phi(x) = 1/(1+e^(0,00063*x⁵ – 0,07359*x³ – 1,59544*x)

This approximation was designed for x≥0 and has an absolute error < 1,9 E–5 while the largest relative error is < 2,9 E–5.
With slightly different coefficients this can also be optimized for both an absolute and relative error within 2,2 E–5.
In this case use 0.000611, –0.07349 and –1.59552.

(01-12-2016 10:16 PM)Pekis Wrote:  I simply inverted the function to let Wolfram solve a*y³+b*y-ln(1/x-1)=0 on y (where a=-.07056 and b=-1.5976)

I got a somehow gory formula which can be a bit simplified:
If a=-0.07056 and b=-1.5976 and t(x)=ln(1/x-1)+sqrt(ln²(1/x-1)+4b³/(27a²))
then Phi^-1(x)=(t(x)/(2a))^1/3-(2b³/(27a²t(x)))^1/3

It seems OK with 3 decimals on all range ]0-1[ ... What do you think ?

At least for the inverse (quantile) I would prefer a much simpler rational approximation. Even the good old Hastings version from the Fifties yields an absolute error within 4,5 E–4 and requires just one log, one root and a few multiplications/additions. For most applications the domain is limited, e.g. to probabilities ≥ 1E–99, so that a different set of coefficients will further reduce the error.

If you are interested in simple approximations to the Normal integral – especially in those that can be inverted to get an approximation for the quantile – I'd recommend this paper: "Very Simply Explicitly Invertible Approximations of Normal Cumulative and Normal Quantile Function" by Alessandro Soranzo and Emanuela Epure, published in Applied Mathematical Sciences, Vol. 8, 2014, no. 87. There also is an approximation for Phi(x) with an accuracy better than 2 E–4 which requires just three or four integer (!) constants. ;-)

BTW, this paper also mentions the two-coefficient approximation you gave (cf. Ref. 11). So I assume your source is S.R. Bowling, M.T. Khasawneh, S. Kaewkuekool, B.R. Cho: "A logistic approximation to the cumulative normal distribution", Journal of Industrial Engineering and Management, 2 (2009).

Dieter

[Pekis]

Quote:Even the good old Hastings version from the Fifties yields an absolute error within 4,5 E–4 and requires just one log, one root and a few multiplications/additions

OK, I had kept in mind approximations with many parameters and multiple pieces ...

Quote:I'd recommend this paper: "Very Simply Explicitly Invertible Approximations of Normal Cumulative and Normal Quantile Function" by Alessandro Soranzo and Emanuela Epure, published in Applied Mathematical Sciences, Vol. 8, 2014, no. 87. There also is an approximation for Phi(x) with an accuracy better than 2 E–4 which requires just three or four integer (!) constants. ;-)

I'll look at it. It seems very interesting.

Thanks !

[Pekis]

Quote:If you are interested in simple approximations to the Normal integral – especially in those that can be inverted to get an approximation for the quantile – I'd recommend this paper: "Very Simply Explicitly Invertible Approximations of Normal Cumulative and Normal Quantile Function" by Alessandro Soranzo and Emanuela Epure, published in Applied Mathematical Sciences, Vol. 8, 2014, no. 87. There also is an approximation for Phi(x) with an accuracy better than 2 E–4 which requires just three or four integer (!) constants. ;-)

I wonder how they found 2^(-22^(1-41^(x/10))) for Phi(x) ... Was it pure brute force ?

[Dieter]

(01-13-2016 09:54 AM)Pekis Wrote:  I wonder how they found 2^(-22^(1-41^(x/10))) for Phi(x) ... Was it pure brute force ?

Ask the author. ;-)

And keep in mind that there is a reason why all these simple approximations require x≥0: As x increases, Phi(x) approaches 1 and the given error near 1E–4 (absolute or relative) makes the results close to useless. Let x=5 and the logistic approximation returns 0,99999995. Plus or minus 0,00014, i.e. the true result can be anything between 0,99986 and 1. #-)

That's why these approximations are useless as x approaches –infinity. The relation Phi(–x) = 1–Phi(x) makes no sense here. 1–Phi(5) would give 5 E–8 instead of the true result 2,8665 E–7.

That's why designing approximations that can handle such cases is a bit more challenging.

Dieter

[Dieter]

(01-12-2016 10:16 PM)Pekis Wrote:  I got a somehow gory formula which can be a bit simplified:
(...)
It seems OK with 3 decimals on all range ]0-1[ ... What do you think ?

I wondered how the inverse of these logistic functions (the one you referred to as well as the one I posted) may perform in terms of accuracy. So I calculated Phi(x) for x=0...5 and evaluated the exact inverse for this result. This was compared to the inital x to get the absolute error. Example: Phi(1,2)=0,88483626. The true quantile for this probability is 1,1995157, so the inverse is off by 0,00048.

Here is what I got:

The inverse of the 3rd order approximation you posted works fine for x=0...2,15. This equals a probability of 0,984. On this interval the error of the inverse is within ±0,0012. There is a negative error extremum at x=1,6...1,7, and at x=2,15 the error has risen to the same positive magnitude. For larger x the error grows, at x~3,09 or p=0,999 it has reached about +0,03.

The inverse of the 5th order approximation I suggested is better, but not by much. It works fine for x=0...2,4. This equals a probability of 0,992. On this interval the error of the inverse is within approx. ±0,0003. There is a negative error extremum near x=2, and at x=2,4 the error has risen to the same positive magnitude. For larger x the error grows, at x=2,6 it is as large as the 3rd order approximation. At x=~3,09 or p=0,999 it has reached about +0,006.

So the inverse of these logistic approximations, even if they could be evaluated without much effort, is only useable if one stays away from probabilities close to 0 or 1.

Dieter