Projection Pursuit Through ϕ-Divergence Minimisation
<p>Graph of the distribution to estimate (red) and of our own estimate (green).</p> "> Figure 2
<p>Graph of the distribution to estimate (red) and of Huber’s estimate (green).</p> "> Figure 3
<p>Graph of the regression of <math display="inline"> <msub> <mi>X</mi> <mn>1</mn> </msub> </math> on <math display="inline"> <msub> <mi>X</mi> <mn>0</mn> </msub> </math> based on the least squares method (red) and based on our theory (green).</p> "> Figure 4
<p>Graph of the regression of <math display="inline"> <msub> <mi>X</mi> <mn>1</mn> </msub> </math> on <math display="inline"> <msub> <mi>X</mi> <mn>0</mn> </msub> </math> based on our theory (green).</p> "> Figure 5
<p>Graph of the estimate of <math display="inline"> <mrow> <mrow> <mo stretchy="false">(</mo> <msub> <mi>x</mi> <mn>0</mn> </msub> <mo>,</mo> <msub> <mi>x</mi> <mn>1</mn> </msub> <mo stretchy="false">)</mo> </mrow> <mo>↦</mo> <msub> <mi>c</mi> <mi>ρ</mi> </msub> <mrow> <mo stretchy="false">(</mo> <msub> <mi>F</mi> <mrow> <mi>G</mi> <mi>u</mi> <mi>m</mi> <mi>b</mi> <mi>e</mi> <mi>l</mi> </mrow> </msub> <mrow> <mo stretchy="false">(</mo> <msub> <mi>x</mi> <mn>0</mn> </msub> <mo stretchy="false">)</mo> </mrow> <mo>,</mo> <msub> <mi>F</mi> <mrow> <mi>E</mi> <mi>x</mi> <mi>p</mi> <mi>o</mi> <mi>n</mi> <mi>e</mi> <mi>n</mi> <mi>t</mi> <mi>i</mi> <mi>a</mi> <mi>l</mi> </mrow> </msub> <mrow> <mo stretchy="false">(</mo> <msub> <mi>x</mi> <mn>1</mn> </msub> <mo stretchy="false">)</mo> </mrow> <mo stretchy="false">)</mo> </mrow> </mrow> </math>.</p> "> Figure 6
<p>Graph of the regression of log of Nokia on Sanofi based on the least squares method (red) and based on our theory (green).</p> ">
Abstract
:1. Outline of the Article
1.1. Huber’s analytic approach
1.2. Huber’s synthetic approach
1.3. Proposal
2. The Algorithm
2.1. The model
Elliptical laws
- with Σ, being a positive-definite matrix and with μ, being a d-column vector,
- with , being referred as the “density generator”,
- with , being a normalisation constant, such that , with .
Choice of g
2.2. Stochastic outline of our algorithm
3. Results
3.1. Convergence results
3.1.1. Hypotheses on f
- (H1)
- :
- (H2)
- :
- (H3)
- : There is a neighbourhood V of ak, and a positive function H, such that, for all
- (H4)
- : There is a neighbourhood V of ak, such that for all ε, there is a η such that for all
- (H5)
- : The function φ is in ) and there is a neighbourhood of such that, for all of , the gradient and the Hessian exist (), and the first order partial derivatives and the first and second order derivatives of are dominated (a.s.) by λ-integrable functions.
- (H6)
- : The function is in a neighbourhood of for all x; and the partial derivatives of are all dominated in by a integrable function .
- (H7)
- : and are finite and the expressions and exist and are invertible.
- (H8)
- : There exists k such that .
- (H9)
- : exists and is invertible.
- (H0)
- : f and g are assumed to be positive and bounded and such that .
3.1.2. Estimation of the first co-vector of f
- is an estimate of as defined in Proposition 3.2 with instead of g,
- is such that , , i.e., .
3.1.3. Convergence study at the step of the algorithm:
3.2. Asymptotic Inference at the step of the algorithm
3.3. A stopping rule for the procedure
3.3.1. Estimation of f
3.3.2. Testing of the criteria
3.4. Goodness-of-fit test for copulas
3.5. Rewriting of the convolution product
3.6. On the regression
3.6.1. The basic idea
3.6.2. General case
4. Simulations
Our Algorithm | |
Projection Study 0 : | minimum : 0.0201741 |
at point : (1.00912,1.09453,0.01893) | |
P-Value : 0.81131 | |
Test : | : : True |
(Kernel Estimation of , ) | 6.1726 |
Our Algorithm | |
Projection Study 0 | minimum : 0.002692 |
at point : (1.01326, 0.0657, 0.0628, 0.1011, 0.0509, 0.1083, | |
0.1261, 0.0573, 0.0377, 0.0794, 0.0906, 0.0356, 0.0012, | |
0.0292, 0.0737, 0.0934, 0.0286, 0.1057, 0.0697, 0.0771) | |
P-Value : 0.80554 | |
Test : | : : True |
H(Est. of , ) | 3.042174 |
Our Algorithm | |
Projection Study 0 : | minimum : 0.0210058 |
at point : (1.001,0.0014) | |
P-Value : 0.989552 | |
Test : | : : True |
(Kernel Estimation of , ) | 6.47617 |
Our Regression | -4.545483 | |
0.0380534 | ||
0.9190052 | ||
0.3103752 | ||
correlation | 0.02158213 | |
Least squares method | -4.34159227 | |
Std Error of | 0.19870 | |
0.06803317 | ||
Std Error of | 0.21154 | |
correlation | 0.04888484 |
Our Algorithm | |
Projection Study 0 : | minimum : 0.010920 |
at point : (1.09,-0.9701) | |
P-Value : 0.889400 | |
Test : | : : True |
(Kernel Estimation of , ) | 5.25077 |
Simulation | a | Std Error of a | b | Std Error of b |
1 | -4.83739 | 0.11149 | -0.95861 | 0.04677 |
2 | -4.56895 | 0.09989 | -0.88577 | 0.04225 |
3 | -4.4926 | 0.1057 | -1.2085 | 0.0452 |
4 | -4.70619 | 0.10350 | -1.04549 | 0.04235 |
5 | -4.40331 | 0.10248 | -1.00890 | 0.0438 |
6 | -4.61757 | 0.09813 | -1.20890 | 0.04649 |
7 | -4.40572 | 0.09172 | -1.16085 | 0.04091 |
8 | -4.39581 | 0.10174 | -1.38696 | 0.04487 |
9 | -4.42780 | 0.10018 | -0.93672 | 0.04066 |
10 | -4.55394 | 0.09923 | -0.98065 | 0.04382 |
- c is the Gaussian copula with correlation coefficient ,
- the Gumbel distribution parameters are and 1 and
- the Exponential density parameter is 2.
Our Algorithm | |
Projection Study number 0 : | minimum : 0.445199 |
at point : (1.0142,0.0026) | |
P-Value : 0.94579 | |
Test : | : : True |
Projection Study number 1 : | minimum : 0.0263 |
at point : (0.0084,0.9006) | |
P-Value : 0.97101 | |
Test : | : : True |
K(Kernel Estimation of , ) | 4.0680 |
Application to real datasets
Our Algorithm | |
Projection Study 0 : | minimum : 0.017345 |
at point : (0.027,3.18) | |
P-Value : 0.890210 | |
Test : | : : True |
K(Kernel Estimation of , ) | 2.7704005 |
Simulation | Std Error of | Std Error of | ||
1 | 3.153694 | 0.230380 | 0.026578 | 0.004236 |
Date | Nokia | Log-of-Nokia | Sanofi | Date | Nokia | Log-of-Nokia | Sanofi |
10/05/10 | 84.75 | 4.44 | 51.62 | 07/05/10 | 81.85 | 4.4 | 48.5 |
06/05/10 | 87.3 | 4.47 | 50.35 | 05/05/10 | 87.75 | 4.47 | 50.95 |
04/05/10 | 87.25 | 4.47 | 50.49 | 03/05/10 | 87.85 | 4.48 | 51.51 |
30/04/10 | 87.8 | 4.48 | 51.66 | 29/04/10 | 87.85 | 4.48 | 51.41 |
28/04/10 | 87.85 | 4.48 | 51.88 | 27/04/10 | 89 | 4.49 | 52.11 |
26/04/10 | 89.2 | 4.49 | 54.09 | 23/04/10 | 90.7 | 4.51 | 53.47 |
22/04/10 | 92.75 | 4.53 | 53.59 | 21/04/10 | 108.4 | 4.69 | 53.95 |
20/04/10 | 108.9 | 4.69 | 54.43 | 19/04/10 | 108.3 | 4.68 | 54.05 |
16/04/10 | 106.8 | 4.67 | 54.04 | 15/04/10 | 109.9 | 4.7 | 54.95 |
14/04/10 | 109.8 | 4.7 | 54.86 | 13/04/10 | 108.3 | 4.68 | 54.67 |
12/04/10 | 109.1 | 4.69 | 55.27 | 09/04/10 | 110.1 | 4.7 | 55.41 |
08/04/10 | 110.7 | 4.71 | 54.96 | 07/04/10 | 113.2 | 4.73 | 55.3 |
06/04/10 | 112.4 | 4.72 | 54.64 | 01/04/10 | 113.3 | 4.73 | 55.16 |
31/03/10 | 112.4 | 4.72 | 55.19 | 30/03/10 | 112.5 | 4.72 | 55.39 |
29/03/10 | 111.8 | 4.72 | 55.49 | 26/03/10 | 112.5 | 4.72 | 55.72 |
25/03/10 | 111.4 | 4.71 | 56.33 | 24/03/10 | 110.2 | 4.7 | 55.95 |
23/03/10 | 109.1 | 4.69 | 56.12 | 22/03/10 | 109.2 | 4.69 | 56.33 |
19/03/10 | 108.5 | 4.69 | 56.57 | 18/03/10 | 108.4 | 4.69 | 56.56 |
17/03/10 | 109.9 | 4.7 | 56.28 | 16/03/10 | 107 | 4.67 | 57.21 |
Date | Nokia | Log-of-Nokia | Sanofi | Date | Nokia | Log-of-Nokia | Sanofi |
15/03/10 | 105.3 | 4.66 | 55.95 | 12/03/10 | 105 | 4.65 | 55.4 |
11/03/10 | 103 | 4.63 | 55.65 | 10/03/10 | 104 | 4.64 | 56.13 |
09/03/10 | 101.5 | 4.62 | 56.17 | 08/03/10 | 100.7 | 4.61 | 55.75 |
05/03/10 | 100.2 | 4.61 | 55.76 | 04/03/10 | 98.7 | 4.59 | 54.81 |
03/03/10 | 99.8 | 4.6 | 55.14 | 02/03/10 | 97.25 | 4.58 | 54.99 |
01/03/10 | 95.85 | 4.56 | 54.82 | 26/02/10 | 95.85 | 4.56 | 53.72 |
25/02/10 | 94.55 | 4.55 | 52.92 | 24/02/10 | 96.3 | 4.57 | 53.92 |
23/02/10 | 96.2 | 4.57 | 54.05 | 22/02/10 | 96.7 | 4.57 | 54.14 |
19/02/10 | 97.3 | 4.58 | 54.71 | 18/02/10 | 96.6 | 4.57 | 54.43 |
17/02/10 | 96.1 | 4.57 | 53.88 | 16/02/10 | 94.95 | 4.55 | 53.56 |
15/02/10 | 93.65 | 4.54 | 53.2 | 12/02/10 | 93.55 | 4.54 | 53.01 |
11/02/10 | 94.6 | 4.55 | 52.52 | 10/02/10 | 95.55 | 4.56 | 52.2 |
09/02/10 | 98.4 | 4.59 | 52.66 | 08/02/10 | 99.2 | 4.6 | 52.98 |
05/02/10 | 99.8 | 4.6 | 51.68 | 04/02/10 | 102.6 | 4.63 | 53.42 |
03/02/10 | 103.9 | 4.64 | 54.06 | 02/02/10 | 103.8 | 4.64 | 53.8 |
01/02/10 | 102.4 | 4.63 | 53.23 | 29/01/10 | 103.6 | 4.64 | 53.6 |
28/01/10 | 101.8 | 4.62 | 52.68 | 27/01/10 | 92.55 | 4.53 | 53.8 |
26/01/10 | 92.7 | 4.53 | 54.42 | 25/01/10 | 91.9 | 4.52 | 53.66 |
22/01/10 | 94.1 | 4.54 | 54.65 | 21/01/10 | 93.7 | 4.54 | 55.28 |
20/01/10 | 92.75 | 4.53 | 56.67 | 19/01/10 | 93.6 | 4.54 | 57.69 |
18/01/10 | 94.55 | 4.55 | 56.67 | 15/01/10 | 93.55 | 4.54 | 56.85 |
14/01/10 | 93.7 | 4.54 | 56.91 | 13/01/10 | 92.5 | 4.53 | 56.18 |
12/01/10 | 92.35 | 4.53 | 55.83 | 11/01/10 | 93 | 4.53 | 56.08 |
5. Critics of the Simulations
6. Conclusions
References
- Friedman, J.H.; Stuetzle, W.; Schroeder, A. Projection pursuit density estimation. J. Amer. Statist. Assoc. 1984, 79, 599–608. [Google Scholar] [CrossRef]
- Huber, P.J. Projection pursuit. Ann. Statist. 1985, 13, 435–525, With discussion. [Google Scholar] [CrossRef]
- Zhu, M. On the forward and backward algorithms of projection pursuit. Ann. Statist. 2004, 32, 233–244. [Google Scholar] [CrossRef]
- Yohai, V.J. Optimal robust estimates using the Kullback-Leibler divergence. Stat. Probab. Lett. 2008, 78, 1811–1816. [Google Scholar] [CrossRef]
- Toma, A. Optimal robust M-estimators using divergences. Stat. Probab. Lett. 2009, 79, 1–5. [Google Scholar] [CrossRef]
- Huber, P.J. Robust Statistics; Wiley: Hoboken, NJ, USA, 1981; republished in paperback, 2004. [Google Scholar]
- Diaconis, P.; Freedman, D. Asymptotics of graphical projection pursuit. Ann. Statist. 1984, 12, 793–815. [Google Scholar] [CrossRef]
- Cambanis, S.; Huang, S.; Simons, G. On the theory of elliptically contoured distributions. J. Multivariate Anal. 1981, 11, 368–385. [Google Scholar] [CrossRef]
- Landsman, Z.M.; Valdez, E.A. Tail conditional expectations for elliptical distributions. N. Am. Actuar. J. 2003, 7, 55–71. [Google Scholar] [CrossRef]
- Van der Vaart, A.W. Asymptotic Statistics. In Cambridge Series in Statistical and Probabilistic Mathematics; Cambridge University Press: Cambridge, MA, USA, 1998; Volume 3. [Google Scholar]
- Broniatowski, M.; Keziou, A. Parametric estimation and tests through divergences and the duality technique. J. Multivariate Anal. 2009, 100, 16–36. [Google Scholar] [CrossRef]
- Vajda, I. χα-divergence and generalized Fisher’s information. In Transactions of the Sixth Prague Conference on Information Theory, Statistical Decision Functions, Random Processes; Czech Technical University in Prague: Prague, Czech, 1971; dedicated to the memory of Antonín Spacek; Academia: Prague, Czech; pp. 873–886. [Google Scholar]
- Black, F.; Scholes, M.S. The pricing of options and corporate liabilities. J. Polit. Econ. 1973, 3, 637–654. [Google Scholar] [CrossRef]
- Saporta, G. Probabilités, Analyse des données et Statistique; Technip: Paris, France, 2006. [Google Scholar]
- Scott, D.W. Multivariate Density Estimation. Theory, Practice, and Visualization; John Wiley and Sons: New York, NY, USA, 1992. [Google Scholar]
- Cressie, N.; Read, T.R.C. Multinomial goodness-of-fit tests. J. Roy. Statist. Soc. 1984, Ser. B 46, 440–464. [Google Scholar]
- Csiszár, I. On topology properties of f-divergences. Studia Sci. Math. Hungar. 1967, 2, 329–339. [Google Scholar]
- Liese, F.; Vajda, I. Convex Statistical Distances. In Teubner-Texte zur Mathematik [Teubner Texts in Mathematics]; B.G. Teubner Verlagsgesellschaft: Leipzig, Germany, 1987; Volume 95. [Google Scholar]
- Pardo, L. Statistical inference based on divergence measures. In Statistics: Textbooks and Monographs; Chapman & Hall/CRC: Boca Raton, FL, USA, 2006; Volume 185. [Google Scholar]
- Zografos, K.; Ferentinos, K.; Papaioannou, T. ϕ-divergence statistics: sampling properties and multinomial goodness of fit and divergence tests. Comm. Statist. Theory Methods 1990, 19, 1785–1802. [Google Scholar] [CrossRef]
- Azé, D. Eléments d’analyse convexe et variationnelle; Ellipse: Minneapolis, MN, USA, 1997. [Google Scholar]
- Touboul, J. Projection pursuit through φ-divergence minimisation. arXiv:0912.2883, 2009. [Google Scholar]
- Bosq, D.; Lecoutre, J.-P. Livre—Theorie De L’Estimation Fonctionnelle; Economica: Hoboken, NJ, USA, 1999. [Google Scholar]
Appendix
A. Reminders
A.1. φ-Divergence
- -
- with the Kullback–Leibler divergence, we associate
- -
- with the Hellinger distance, we associate
- -
- with the distance, we associate
- -
- more generally, with power divergences, we associate , where
- -
- and, finally, with the norm, which is also a divergence, we associate
A.2. Miscellaneous
B. Study of the sample
C. Hypotheses’ discussion
C.1. Discussion of .
C.2. Discussion of .
- We work with the Kullback–Leibler divergence, (0)
- We have , i.e., —we could also derive the same proof with f, and —(1)
D. Proofs
© 2010 by the author; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license http://creativecommons.org/licenses/by/3.0/.
Share and Cite
Touboul, J. Projection Pursuit Through ϕ-Divergence Minimisation. Entropy 2010, 12, 1581-1611. https://doi.org/10.3390/e12061581
Touboul J. Projection Pursuit Through ϕ-Divergence Minimisation. Entropy. 2010; 12(6):1581-1611. https://doi.org/10.3390/e12061581
Chicago/Turabian StyleTouboul, Jacques. 2010. "Projection Pursuit Through ϕ-Divergence Minimisation" Entropy 12, no. 6: 1581-1611. https://doi.org/10.3390/e12061581