Open AccessArticle

A Geometric Approach to Average Problems on Multinomial and Negative Multinomial Models

Mingming Li

^1,2

Huafei Sun

^1,2,* and

Didong Li

School of Mathematics and Statistics, Beijing Institute of Technology, Beijing 100081, China

Beijing Key Laboratory on MCAACI, Beijing Institute of Technology, Beijing 100081, China

Department of Mathematics, Duke University, Durham, NC 27708, USA

Author to whom correspondence should be addressed.

Entropy 2020, 22(3), 306; https://doi.org/10.3390/e22030306

Submission received: 12 February 2020 / Revised: 29 February 2020 / Accepted: 5 March 2020 / Published: 8 March 2020

(This article belongs to the Special Issue Information Geometry III)

Download Versions Notes

Abstract

This paper is concerned with the formulation and computation of average problems on the multinomial and negative multinomial models. It can be deduced that the multinomial and negative multinomial models admit complementary geometric structures. Firstly, we investigate these geometric structures by providing various useful pre-derived expressions of some fundamental geometric quantities, such as Fisher-Riemannian metrics,

α

-connections and

α

-curvatures. Then, we proceed to consider some average methods based on these geometric structures. Specifically, we study the formulation and computation of the midpoint of two points and the Karcher mean of multiple points. In conclusion, we find some parallel results for the average problems on these two complementary models.

Keywords:

structure characterization; average problem; geometric midpoints; Karcher mean

1. Introduction

The concept of an average of a set of points within a given geometric structure abounds in various mathematical research. Significant development has been produced since the conception of the Karcher mean [1]. Among the various structure to be studied, the standard simplex presents itself as an interesting framework since it can be directly connected to the parameter spaces of various probability distributions, such as the two to-be-discussed models—the multinomial and negative multinomial models. There are already some recent works involving the statistical modeling of the probability simplex [2]. In the present work, we provide alternative modelings by considering the multinomial and negative multinomial models with classical methods of information geometry.

As a newly developed theory, information geometry has supplied us with various measures of the discrepancy between any two probability distributions. In addition to some standard distance functions, divergence functions are intended for measuring the asymmetric proximity of probability distributions on an appropriate statistical model. Some geometric quantities, such as Riemannian metric and pair of dual connections, can be readily induced from a divergence function by its higher order derivatives [3,4]. In this way, the geometric structures of various parametric statistical models can be studied specifically [5,6,7], and the investigation about some particular parametric models can also be found in [8,9]. Among these models, we find the multinomial model and negative multinomial model especially interesting, in the sense that they can be seen as a pair of complementary model spaces. The multinomial model is well known as a spherical space of positive constant curvature [10], while the negative multinomial model is found to be a hyperbolic space of negative constant curvature [11].

To be more specific, the motivation of the present paper is twofold. Firstly, we aim at clarifying the complementary geometric structures of the multinomial and negative multinomial models. The main results about these geometric structures involving geometric quantities, such as Fisher-Riemannian metrics,

α

-connections and

α

-curvatures, are collected in Section 3, most of which can be derived in a standard way. In particular, this paper extends the isometric representation results about the multinomial model to those about the negative multinomial model, obtaining new insight into the complementary structures of these two models as illustrated by Table 1. Secondly, the original purpose of the formulation and computation of average problems is approached by utilizing these geometric structures. To this end, we propose a generalized concept of midpoints for two points and a computation scheme of Karcher mean for multiple points. For the midpoints, we generalize the Chernoff points in the literature [12] to some wider parametrized classes. For the Karcher mean, as there are many algorithmic results [13,14] for general manifolds, this paper mainly contributes to addressing some practical issues, such as initial point choice and iteration computation, which yields effective solving methods via the geometric structures within the multinomial model and negative multinomial model. The results about these average methods are presented in Section 4.

2. Preliminaries

For the sake of clarity, we summarize some preliminary knowledge about information divergence functions in this section (more details can be found in [15]).

Given a particular parametric statistical model

M = {p_{θ} | θ \in Θ},

for our purpose, here we mainly consider invariant divergences that satisfy the property of information monotonicity, as mentioned in [15]. A typical kind of invariant divergence is given in the form of the well-defined f-divergence as

D_{f} (p_{θ} ∥ p_{ξ}) = \int_{x \in X} p_{θ} (x) f (\frac{p_{ξ} (x)}{p_{θ} (x)}) d x,

where f is a convex function satisfying

f (1) = 0

and

f^{″} (1) = 1

A commonly used class of f-divergences is given by the

α

-divergence

D^{(α)}

, with

f (u) = \{\begin{matrix} \frac{4}{1 - α^{2}} (1 - u^{\frac{1 + α}{2}}), & α \neq \pm 1, \\ - log u, & α = - 1, \\ u log u, & α = 1 . \end{matrix}

Particularly, for

α = - 1

, the divergence

D^{(- 1)}

is usually called Kullback–Leibler divergence, which we denote by

D_{K L}

; for

α = 0

D^{(0)}

is usually called squared Hellinger distance, which we denote by

H^{2}

; and for

α = 3

D^{(3)}

is often related to the chi-square statistic.

While our later results are mainly related to the

α

-divergence, for comparison, we briefly mention another f-divergence called exponential divergence (see §12.3.6 [6]), which we denote by

E

with

f (u) = \frac{1}{2} {log}^{2} u

For any divergence function D on a statistical manifold, we can construct another divergence function

D^{*}

called dual divergence by swapping the arguments as ([16])

D^{*} (p_{θ} ∥ p_{ξ}) = D (p_{ξ} ∥ p_{θ}) .

The dual divergence of an f-divergence

D_{f}

is again an f-divergence

D_{f}^{*} = D_{f^{*}},

with

f^{*} (u) = u f (\frac{1}{u})

. Particularly, the dual divergence of the

α

-divergence

D^{(α)}

just is

D^{(- α)}

To be mentioned, as a divergence is generally not symmetric, a symmetric divergence

D_{s}

can be constructed from an asymmetric D by averaging it with its dual:

D_{s} = \frac{1}{2} (D + D^{*}) .

A Riemannian metric g can be induced by a divergence D as

g (\frac{\partial}{\partial θ_{i}}, \frac{\partial}{\partial θ_{j}}) = g_{i j} (θ) = {\frac{\partial^{2}}{\partial ξ_{i} \partial ξ_{j}}|}_{ξ = θ} D (p_{θ} ∥ p_{ξ}),

(1)

which is equivalent to the usual Fisher-Riemannian metric in the case when D is an f-divergence.

Furthermore, an affine connection ∇ is induced by the divergence D with connection coefficients

g (\nabla_{\frac{\partial}{\partial θ_{i}}} \frac{\partial}{\partial θ_{j}}, \frac{\partial}{\partial θ_{k}}) = Γ_{i j k} (θ) = - {\frac{\partial^{3}}{\partial θ_{i} \partial θ_{j} \partial ξ_{k}}|}_{ξ = θ} D (p_{θ} ∥ p_{ξ}) .

(2)

Similarly, another affine connection

\nabla^{*}

can be obtained by replacing D with the dual divergence

D^{*}

in the above formula. Thus, with the primal connection ∇ and dual connection

\nabla^{*}

, the statistical manifold admits a dual structure

(M, g, \nabla, \nabla^{*})

. Furthermore, the structure

(M, g, \nabla, \nabla^{*})

is called dually flat if both ∇ and

\nabla^{*}

are flat.

As a well known result [15], the primal connection induced by an f-divergence

D_{f}

is the same as the usual

α

-connection

\nabla^{(α)}

with

α = 3 + 2 f^{‴} (1),

while the induced dual connection is

\nabla^{(- α)}

. Particularly, one can check that the primal connection induced by the

α

-divergence

D^{(α)}

is exactly the

α

-connection

\nabla^{(α)}

3. Geometric Structure of Multinomial and Negative Multinomial Models

3.1. Basic Information Geometric Structure

In this subsection, we present the parametric formulation of the multinomial and negative multinomial model, respectively. Then some basic results about divergences and geometric structures are derived for both models.

3.1.1. Multinomial Model

Consider the multinomial n-dimensional model

M_{N}

consisting of (n + 1)-nominal distributions with probability mass function given by

p_{θ} (x_{0}, x_{1}, \dots, x_{n}) = \frac{N!}{x_{0}! x_{1}! \dots x_{n}!} θ_{0}^{x_{0}} θ_{1}^{x_{1}} \dots θ_{n}^{x_{n}},

(3)

where

\sum_{i = 0}^{n} x_{i} = N

and the parametrization is given by

θ = (θ_{1}, \dots, θ_{n})

with

θ_{0} = 1 - \sum_{i = 1}^{n} θ_{i}

We can rewrite Equation (3) as

p_{θ} (x_{0}, x_{1}, \dots, x_{n}) = \frac{N!}{x_{0}! x_{1}! \dots x_{n}!} exp (\sum_{i = 1}^{n} x_{i} log \frac{θ_{i}}{θ_{0}} + N log θ_{0}) .

By some general knowledge about exponential distribution families [17], we see that the multinomial model

M_{N}

admits the natural parameters

η_{i} = log \frac{θ_{i}}{θ_{0}}, i = 1, \dots, n,

(4)

and the potential function

ψ (θ) = - N log θ_{0} = N log (1 + \sum_{i = 1}^{n} exp η_{i}),

from which we also obtain the expectation parameters

\partial ψ / \partial η_{i} = N {(1 + \sum_{i = 1}^{n} exp η_{i})}^{- 1} exp η_{i} = N θ_{i}, i = 1, \dots, n .

(5)

Next, we have the following result by direct calculation.

Proposition 1.

The divergences introduced in Section 2 are obtained for

M_{N}

as follows:

the Kullback–Leibler divergence

D_{K L} (p_{θ} ∥ p_{ξ}) = N \sum_{i = 0}^{n} θ_{i} log \frac{θ_{i}}{ξ_{i}},

the squared Hellinger distance

H^{2} (p_{θ}, p_{ξ}) = 4 [1 - {(\sum_{i = 0}^{n} \sqrt{θ_{i} ξ_{i}})}^{N}],

the α-divergence

D^{(α)} (p_{θ} ∥ p_{ξ}) = \frac{4}{1 - α^{2}} [1 - {(\sum_{i = 0}^{n} θ_{i}^{\frac{1 - α}{2}} ξ_{i}^{\frac{1 + α}{2}})}^{N}]

and the exponential divergence

E (p_{θ} ∥ p_{ξ}) = \frac{N (N - 1)}{2} {(\sum_{i = 0}^{n} θ_{i} log \frac{θ_{i}}{ξ_{i}})}^{2} + \frac{N}{2} \sum_{i = 0}^{n} θ_{i} {log}^{2} \frac{θ_{i}}{ξ_{i}} .

From any one of these expressions, by using Equation (1), we obtain the Fisher-Riemannian metric matrix as

g_{i j} (θ) = \frac{N}{θ_{0}} + \frac{N}{θ_{i}} δ_{i j},

(6)

where

δ_{i j} = 1

i = j

and 0 otherwise. Then, the inverse matrix of g can also be obtained as

g^{i j} (θ) = \frac{θ_{i}}{N} δ_{i j} - \frac{θ_{i} θ_{j}}{N} .

(7)

Furthermore, by direct verification with the metric expression of Equation (6), we have the following well-known result.

Theorem 1

([18]). An isometry is established between the multinomial model

M_{N}

and the n-sphere within the non-negative orthant of the Euclidean space

R^{n + 1}

by the parametric mapping:

(θ_{0}, θ_{1}, \dots, θ_{n}) \mapsto 2 \sqrt{N} (θ_{0}^{1 / 2}, θ_{1}^{1 / 2}, \dots, θ_{n}^{1 / 2}) .

(8)

Consequently, via this isometry, the Fisher-Riemannian geodesic distance between two parameters θ and ξ of

M_{N}

is given by

d (θ, ξ) = 2 \sqrt{N} arccos (\sum_{i = 0}^{n} \sqrt{θ_{i} ξ_{i}}) .

(9)

Referring to some basic differential geometrical concepts (see [19]), we can derive some further consequent results as follows.

Corollary 1.

The multinomial n-dimensional model

M_{N}

with the Fisher-Riemannian metric is a Riemannian manifold of constant sectional curvature

K = \frac{1}{4 N}

and scalar curvature

S = \frac{n (n - 1)}{4 N}

. Furthermore, for a unit speed geodesic γ in

M_{N}

, the normal Jacobi fields along γ are precisely linear combinations of the vector fields, which are in the form of

J (t) = sin (2 \sqrt{N} t) E (t)

J (t) = cos (2 \sqrt{N} t) E (t)

, where E is any parallel normal vector field along γ.

3.1.2. Negative Multinomial Model

Consider the negative multinomial n-dimensional model

{NM}_{M}

consisting of negative (n + 1)-nominal distributions with probability mass function given by

p_{θ} (x_{1}, \dots, x_{n}) = \frac{Γ (M + x_{1} + \dots + x_{n})}{Γ (M) x_{1}! \dots x_{n}!} θ_{0}^{M} θ_{1}^{x_{1}} \dots θ_{n}^{x_{n}},

(10)

where

M > 0, x_{1}, \dots, x_{n} \geq 0

and the parametrization is given by

θ = (θ_{1}, \dots, θ_{n})

with

θ_{0} = 1 - \sum_{i = 1}^{n} θ_{i}

With the rewritten form of Equation (10) as

p_{θ} (x_{1}, \dots, x_{n}) = \frac{Γ (M + x_{1} + \dots + x_{n})}{Γ (M) x_{1}! \dots x_{n}!} exp (\sum_{i = 1}^{n} x_{i} log θ_{i} + M log θ_{0}),

we find that the negative multinomial model

{NM}_{M}

admits the natural parameters

η_{i} = log θ_{i}, i = 1, \dots, n,

(11)

and the potential function

ψ (θ) = - M log θ_{0} = - M log (1 - \sum_{i = 1}^{n} exp η_{i}),

from which we also obtain the expectation parameters

\partial ψ / \partial η_{i} = M {(1 - \sum_{i = 1}^{n} exp η_{i})}^{- 1} exp η_{i} = M \frac{θ_{i}}{θ_{0}} .

(12)

Again, we derive the following result by direct calculation.

Proposition 2.

The divergences introduced in Section 2 are obtained for

{NM}_{M}

as follows:

the Kullback–Leibler divergence

D_{K L} (p_{θ} ∥ p_{ξ}) = M log \frac{θ_{0}}{ξ_{0}} + M \sum_{i = 1}^{n} \frac{θ_{i}}{θ_{0}} log \frac{θ_{i}}{ξ_{i}},

the squared Hellinger distance

H^{2} (p_{θ}, p_{ξ}) = 4 [1 - {(\frac{\sqrt{θ_{0} ξ_{0}}}{1 - \sum_{i = 1}^{n} \sqrt{θ_{i} ξ_{i}}})}^{M}],

the α-divergence

D^{(α)} (p_{θ} ∥ p_{ξ}) = \frac{4}{1 - α^{2}} [1 - {(\frac{θ_{0}^{\frac{1 - α}{2}} ξ_{0}^{\frac{1 + α}{2}}}{1 - \sum_{i = 1}^{n} θ_{i}^{\frac{1 - α}{2}} ξ_{i}^{\frac{1 + α}{2}}})}^{M}]

and the exponential divergence

E (p_{θ} ∥ p_{ξ}) = \frac{M^{2}}{2} {(log \frac{θ_{0}}{ξ_{0}} + \sum_{i = 1}^{n} \frac{θ_{i}}{θ_{0}} log \frac{θ_{i}}{ξ_{i}})}^{2} + \frac{M}{2} [\sum_{i = 1}^{n} \frac{θ_{i}}{θ_{0}} {log}^{2} \frac{θ_{i}}{ξ_{i}} + {(\sum_{i = 1}^{n} \frac{θ_{i}}{θ_{0}} log \frac{θ_{i}}{ξ_{i}})}^{2}] .

Next, applying Equation (1) to the divergences in the foregoing proposition, we obtain the Fisher-Riemannian metric matrix as

g_{i j} = \frac{M}{θ_{0}^{2}} + \frac{M}{θ_{0} θ_{i}} δ_{i j},

(13)

and its inverse matrix as

g^{i j} = \frac{θ_{0}}{M} (θ_{i} δ_{i j} - θ_{i} θ_{j}) .

(14)

Furthermore, by direct verification with the metric expression of Equation (13), we have the following result parallel to Theorem 1.

Theorem 2.

An isometry is established between the negative multinomial model

{NM}_{M}

and the n-hyperbola within the nonnegative orthant of the Minkowski space

R^{1, n}

by the parametric mapping:

(θ_{0}, θ_{1}, \dots, θ_{n}) \mapsto 2 \sqrt{M} θ_{0}^{- 1 / 2} (1, θ_{1}^{1 / 2}, \dots, θ_{n}^{1 / 2}) .

(15)

Consequently, via this isometry, the Fisher-Riemannian geodesic distance between two parameters θ and ξ of

{NM}_{M}

is given by

d (θ, ξ) = 2 \sqrt{M} arcosh (\frac{1 - \sum_{i = 1}^{n} \sqrt{θ_{i} ξ_{i}}}{\sqrt{θ_{0} ξ_{0}}}) .

(16)

Again, some further consequent results are obtained as follows.

Corollary 2.

The negative multinomial n-dimensional model

{NM}_{M}

with the Fisher-Riemannian metric is a Riemannian manifold of constant sectional curvature

K = - \frac{1}{4 M}

and scalar curvature

S = - \frac{n (n - 1)}{4 M}

. Furthermore, for a unit speed geodesic γ in

{NM}_{M}

, the normal Jacobi fields along γ are precisely linear combinations of the vector fields, which are in the form of

J (t) = sinh (2 \sqrt{M} t) E (t)

J (t) = cosh (2 \sqrt{M} t) E (t)

, where E is any parallel normal vector field along γ.

3.2. Dual Structures

In this section, we derive the

α

-connection coefficients and curvatures of the multinomial and negative multinomial model. While some basic results are equivalent to those in [11], our calculation is performed directly with the original parameters in order to give a clear presentation of the results.

The

α

-connection coefficients can be obtained by applying the calculation of Equation (2) to the

α

-divergence

D^{(α)}

, as mentioned in Section 2. However, an easier derivation of the

α

-connection

\nabla^{(α)}

is given in the form of a linear combination of the mixture connection

\nabla^{(m)} = \nabla^{(- 1)}

and the exponential connection

\nabla^{(e)} = \nabla^{(1)}

. With the mixture connection coefficients obtained from the Kullback–Leibler divergence as

Γ_{i j k}^{(m)} (θ) = - {\frac{\partial^{3}}{\partial θ_{i} \partial θ_{j} \partial ξ_{k}}|}_{ξ = θ} D_{K L} (p_{θ} ∥ p_{ξ}),

(17)

and the exponential connection coefficients obtained from the dual Kullback–Leibler divergence as

Γ_{i j k}^{(e)} (θ) = - {\frac{\partial^{3}}{\partial θ_{i} \partial θ_{j} \partial ξ_{k}}|}_{ξ = θ} D_{K L}^{*} (p_{θ} ∥ p_{ξ}),

(18)

the

α

-connection coefficients are given by

Γ_{i j k}^{(α)} (θ) = \frac{1 + α}{2} Γ_{i j k}^{(e)} (θ) + \frac{1 - α}{2} Γ_{i j k}^{(m)} (θ) .

(19)

Next, the

α

-curvature tensor

R^{(α)}

is defined by

R^{(α)} (X, Y) Z = \nabla_{X}^{(α)} \nabla_{Y}^{(α)} Z - \nabla_{Y}^{(α)} \nabla_{X}^{(α)} Z - \nabla_{[X, Y]}^{(α)} Z,

where

[X, Y]

denotes the Lie bracket of X and Y. By the duality condition (see [20])

X g (Y, Z) = g (\nabla_{X}^{(α)} Y, Z) + g (Y, \nabla_{X}^{(- α)} Z),

one can check that the following identity holds:

g (R^{(α)} (\partial_{i}, \partial_{j}) \partial_{j}, \partial_{i}) = \partial_{i} Γ_{j j i}^{(α)} - \partial_{j} Γ_{i j i}^{(α)} + \sum_{k, l} g^{k l} (Γ_{i j k}^{(α)} Γ_{j i l}^{(- α)} - Γ_{j j k}^{(α)} Γ_{i i l}^{(- α)}),

(20)

where

\frac{\partial}{\partial θ_{i}}

is shortened as

\partial_{i}

At last, the

α

-sectional curvature spanned by two tangent vectors

\partial_{i}

and

\partial_{j}

(

i \neq j

) is determined by

K^{(α)} (\partial_{i}, \partial_{j}) = \frac{g (R^{(α)} (\partial_{i}, \partial_{j}) \partial_{j}, \partial_{i})}{g_{i i} g_{j j} - g_{i j}^{2}} .

(21)

3.2.1. Multinomial Model

For the multinomial model

M_{N}

, by applying Equations (17)–(19) with the Kullback–Leibler divergence in Propositon 1, we have the mixture connection coefficients

Γ_{i j k}^{(m)} (θ) = 0,

the exponential connection coefficients

Γ_{i j k}^{(e)} (θ) = \frac{N}{θ_{0}^{2}} - \frac{N}{θ_{i}^{2}} δ_{i j k}

and the

α

-connection coefficients

Γ_{i j k}^{(α)} (θ) = \frac{1 + α}{2} Γ_{i j k}^{(e)} (θ) .

Furthermore, by Equations (6), (7) and (20), we obtain

g (R^{(α)} (\partial_{i}, \partial_{j}) \partial_{j}, \partial_{i}) = \frac{(1 - α^{2}) N}{4} (\frac{1}{θ_{0} θ_{i}} + \frac{1}{θ_{0} θ_{j}} + \frac{1}{θ_{i} θ_{j}}),

g_{i i} g_{j j} - g_{i j}^{2} = N^{2} (\frac{1}{θ_{0} θ_{i}} + \frac{1}{θ_{0} θ_{j}} + \frac{1}{θ_{i} θ_{j}}) .

Thus, via Equation (21), we recover the following result.

Theorem 3

([11]). The multinomial model

M_{N}

admits constant α-sectional curvature

K^{(α)} = \frac{1 - α^{2}}{4 N} .

(22)

3.2.2. Negative Multinomial Model

For the negative multinomial model

{NM}_{M}

, by applying Equations(17)–(19) with the Kullback–Leibler divergence in Propositon 2, we have the mixture connection coefficients

Γ_{i j k}^{(m)} (θ) = \frac{M}{θ_{0}^{2}} (\frac{2}{θ_{0}} + \frac{δ_{i k} + δ_{j k}}{θ_{k}}) = (g_{i k} + g_{j k}) / θ_{0},

the exponential connection coefficients

Γ_{i j k}^{(e)} (θ) = - \frac{M}{θ_{0} θ_{i}^{2}} δ_{i j k} - \frac{M}{θ_{0}^{2} θ_{i}} δ_{i j} = - g_{i k} δ_{i j} / θ_{i}

and the

α

-connection coefficients

Γ_{i j k}^{(α)} (θ) = \frac{1 + α}{2} Γ_{i j k}^{(e)} (θ) + \frac{1 - α}{2} Γ_{i j k}^{(m)} (θ) .

Furthermore, by Equations (13), (14) and (20), we have

g (R^{(α)} (\partial_{i}, \partial_{j}) \partial_{j}, \partial_{i}) = - \frac{(1 - α^{2}) M}{4 θ_{0}^{2}} (\frac{1}{θ_{0} θ_{i}} + \frac{1}{θ_{0} θ_{j}} + \frac{1}{θ_{i} θ_{j}}),

g_{i i} g_{j j} - g_{i j}^{2} = \frac{M^{2}}{θ_{0}^{2}} (\frac{1}{θ_{0} θ_{i}} + \frac{1}{θ_{0} θ_{j}} + \frac{1}{θ_{i} θ_{j}}) .

Again, via Equation (21), we recover another parallel result.

Theorem 4

([11]). The negative multinomial model

{NM}_{M}

admits constant α-sectional curvature

K^{(α)} = - \frac{1 - α^{2}}{4 M} .

(23)

For clarity, we summarize these results about the complementary geometric structures of the multinomial and negative multinomial models in Table 1.

4. Geometric Average Methods on Multinomial and Negative Multinomial Models

In this section, we present some average methods induced by the geometry of the multinomial and negative multinomial models.

Firstly, we consider the particular case when the to-be-averaged set consists of only two points. In this case, the problem is to find a method of computing a midpoint in some geometric sense. Next, via some techniques related to the Karcher mean, we consider the general case with a set of multiple points.

4.1. Midpoints of Two Points

In this subsection, again within the multinomial and negative multinomial model, we study a particular class of midpoints named Chernoff points. The original Chernoff point, which is motivated by the application of computing the best error exponent for the Bayesian hypothesis testing problem, is determined as the intersection point of an exponential geodesic and a mixture bisector [21]. Furthermore, there are three other generalized Chernoff points proposed by [12].

To present a further generalization, here we formulate the concepts of

α

-geodesic and

α

-bisector determined by two probability distributions

p_{θ^{'}}

and

p_{θ^{″}}

of a parametric statistical model

M

Definition 1.

The α-geodesic is determined by the geodesic equation of the α-connection

\nabla^{(α)}

G^{(α)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ (t)} \in M |θ (0) = θ^{'}, θ (1) = θ^{″}, \nabla_{\dot{θ} (t)}^{(α)} \dot{θ} (t) = 0, t \in [0, 1]\},

where

\dot{θ} (t)

denotes the velocity vector of a curve

θ (t)

Particularly, since an exponential family model is

\pm 1

-flat, as can be directly seen from Equations (22) and (23) for our case, the exponential geodesic for

α = 1

can be determined by the linear interpolation of the natural parameters, while the mixture geodesic for

α = - 1

can be determined by the linear interpolation of the expectation parameters.

Definition 2.

The α-bisector is determined by the equi-divergence identity of the α-divergence

D^{(α)}

{Bi}^{(α)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ} \in M |D^{(α)} (p_{θ} ∥ p_{θ^{'}}) = D^{(α)} (p_{θ} ∥ p_{θ^{″}})\} .

(24)

Particularly, the exponential bisector for

α = 1

is determined in terms of the dual Kullback–Leibler divergence, while the mixture bisector for

α = - 1

is determined in terms of the Kullback–Leibler divergence.

Then, we can generalize the notion of Chernoff points suggested by [12] as follows.

Definition 3.

Two types of generalized Chernoff points are given by the intersection points with parameter α:

C P_{I}^{(α)} (p_{θ^{'}}, p_{θ^{″}}) = G^{(α)} (p_{θ^{'}}, p_{θ^{″}}) \cap {Bi}^{(- α)} (p_{θ^{'}}, p_{θ^{″}}),

C P_{II}^{(α)} (p_{θ^{'}}, p_{θ^{″}}) = G^{(α)} (p_{θ^{'}}, p_{θ^{″}}) \cap {Bi}^{(α)} (p_{θ^{'}}, p_{θ^{″}}) .

Thus, the Chernoff points already proposed in previous works can be recovered by setting

α = \pm 1

in Definition 3.

The existence of these intersection points is assured by the intermediate value property of the determining Equation (24), since replacing

θ

θ^{'}

and

θ^{″}

, respectively, we get two opposite inequalities due to the non-negativity of the

α

-divergence.

While the uniqueness can be proven for an exponential family model if

α = \pm 1

(see [12]), we conjecture it still holds for general cases but this is not pursued in the present paper.

To elucidate what we have mentioned earlier about the application of the original Chernoff point (

C P_{I}^{(1)}

in our notation) to the binary Bayesian hypothesis testing problem, we present here the upper bound of the probability of error of the Bayesian decision suggested by [21] as

exp (- D_{K L} (p_{θ^{*}} ∥ p_{θ^{'}})) = exp (- D_{K L} (p_{θ^{*}} ∥ p_{θ^{″}})),

(25)

where

p_{θ^{*}}

denotes the Chernoff point

C P_{I}^{(1)} (p_{θ^{'}}, p_{θ^{″}})

The overlapping case of the two classes of generalized Chernoff points is given by

α = 0

. For the multinomial and negative multinomial model, by comparing the 0-divergence, i.e., squared Hellinger distance, with the Fisher-Riemannian geodesic distance presented in Section 3.1, we find that the generalized Chernoff point

C P_{I}^{(0)} (p_{θ^{'}}, p_{θ^{″}}) = C P_{II}^{(0)} (p_{θ^{'}}, p_{θ^{″}})

is exactly the unique Fisher-Riemannian geodesic midpoint between

p_{θ^{'}}

and

p_{θ^{″}}

within both models.

Next, we summarize some specific results about Chernoff points for both models as follows.

4.1.1. Multinomial Model

For the multinomial model

M_{N}

, although the geodesic equation of the

α

-geodesic can be explicitly given in general cases, there are simple closed-form geodesic expressions at least for

α = \pm 1

Proposition 3.

The exponential and mixture geodesics connecting two probability distributions

p_{θ^{'}}

and

p_{θ^{″}}

of the multinomial model

M_{N}

are given by

G^{(1)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ (t)} \in M_{N} |θ_{i} (t) = \frac{{(θ_{i}^{'})}^{1 - t} {(θ_{i}^{″})}^{t}}{\sum_{i = 0}^{n} {(θ_{i}^{'})}^{1 - t} {(θ_{i}^{″})}^{t}}, i = 0, \dots, n, t \in [0, 1]\},

G^{(- 1)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ (t)} \in M_{N} |θ_{i} (t) = (1 - t) θ_{i}^{'} + t θ_{i}^{″}, i = 0, \dots, n, t \in [0, 1]\} .

Proof.

As already mentioned, the exponential and mixture geodesics can be easily obtained by the linear interpolation of the natural and expectation parameters by Equations (4) and (5), respectively. □

By using the expressions of

α

-divergences presented in Proposition 1, the

α

-bisectors are directly obtained as follows.

Proposition 4.

The α-bisectors between

p_{θ^{'}}

and

p_{θ^{″}}

within the multinomial model

M_{N}

are given by the following equations:

{Bi}^{(α)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ} \in M_{N} |\sum_{i = 0}^{n} θ_{i}^{\frac{1 - α}{2}} [{(θ_{i}^{'})}^{\frac{1 + α}{2}} - {(θ_{i}^{″})}^{\frac{1 + α}{2}}] = 0\}, α \neq \pm 1,

{Bi}^{(1)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ} \in M_{N} |\sum_{i = 0}^{n} (θ_{i}^{'} - θ_{i}^{″}) log θ_{i} = \sum_{i = 0}^{n} (θ_{i}^{'} log θ_{i}^{'} - θ_{i}^{″} log θ_{i}^{″})\},

{Bi}^{(- 1)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ} \in M_{N} |\sum_{i = 0}^{n} θ_{i} log \frac{θ_{i}^{'}}{θ_{i}^{″}} = 0\} .

Combining the previous two propositions, we have the following result about the determining equations for the four particular Chernoff points with

α = \pm 1

Theorem 5.

The determining equations for the Chernoff points with

α = \pm 1

p_{θ^{'}}

and

p_{θ^{″}}

within the multinomial model

M_{N}

are expressed in terms of the argument

t \in [0, 1]

of the corresponding geodesics as follows:

1.: For $C P_{I}^{(1)} (p_{θ^{'}}, p_{θ^{″}})$ ,

$\sum_{i = 0}^{n} {(θ_{i}^{'})}^{1 - t} {(θ_{i}^{″})}^{t} log \frac{θ_{i}^{'}}{θ_{i}^{″}} = 0;$
2.: For $C P_{I}^{(- 1)} (p_{θ^{'}}, p_{θ^{″}})$ ,

$\sum_{i = 0}^{n} (θ_{i}^{'} - θ_{i}^{″}) log [(1 - t) θ_{i}^{'} + t θ_{i}^{″}] = \sum_{i = 0}^{n} (θ_{i}^{'} log θ_{i}^{'} - θ_{i}^{″} log θ_{i}^{″});$
3.: For $C P_{II}^{(1)} (p_{θ^{'}}, p_{θ^{″}})$ ,

$t = \frac{D_{K L} (p_{θ^{″}} ∥ p_{θ^{'}})}{D_{K L} (p_{θ^{'}} ∥ p_{θ^{″}}) + D_{K L} (p_{θ^{″}} ∥ p_{θ^{'}})};$
4.: For $C P_{II}^{(- 1)} (p_{θ^{'}}, p_{θ^{″}})$ ,

$t = \frac{D_{K L} (p_{θ^{'}} ∥ p_{θ^{″}})}{D_{K L} (p_{θ^{'}} ∥ p_{θ^{″}}) + D_{K L} (p_{θ^{″}} ∥ p_{θ^{'}})} .$

With the Kullback–Leibler divergence

D_{K L}

given by Propositon 1, the two points of second type

C P_{II}^{(1)}

and

C P_{II}^{(- 1)}

are already in an explicit form. Whereas, the two points of first type

C P_{I}^{(1)}

and

C P_{I}^{(- 1)}

are to be solved by using some numerical methods such as simple bisection as suggested in [12].

For the Fisher-Riemannian geodesic midpoint, we have the following result.

Theorem 6.

The Fisher-Riemannian geodesic midpoint

p_{θ^{*}}

between

p_{θ^{'}}

and

p_{θ^{″}}

in the multinomial model

M_{N}

is determined by

θ_{i}^{*} = \frac{{(\sqrt{θ_{i}^{'}} + \sqrt{θ_{i}^{″}})}^{2}}{2 + 2 \sum_{i = 0}^{n} \sqrt{θ_{i}^{'} θ_{i}^{″}}}, i = 0, \dots, n .

Proof.

Let F be the isometry given by Equation (8). Denote the linear midpoint of the two image points

\frac{F (θ^{'}) + F (θ^{″})}{2}

x^{*} \in R^{n + 1}

. Then we normalize

x^{*}

to the n-sphere as

u^{*} = 2 \sqrt{N} x^{*} / {∥ x^{*} ∥}_{e}

, where the Euclidean norm

{∥ x ∥}_{e} = {(\sum_{i = 0}^{n} x_{i}^{2})}^{1 / 2}

is used. At last, the required midpoint is obtained as the inverse image point

θ^{*} = F^{- 1} (u^{*})

. □

To illustrate these notions for the multinomial model

M_{N}

, we present a numerical example as follows. The two parameters

θ^{'}

and

θ^{″}

are taken as the empirical probability vectors of the first and second 100 decimal digits of

π

, respectively. The parameters of these two points and the resulting Chernoff points are summarized in Table 2.

As we can see, the pair of points

C P_{I}^{(1)}

and

C P_{II}^{(1)}

admits a certain similarity between them as both of them lie on the same exponential geodesic, and so does the pair of points

C P_{I}^{(- 1)}

and

C P_{II}^{(- 1)}

as both of them lie on the same mixture geodesic. Whereas, the Fisher-Riemannian geodesic midpoint

C P^{(0)}

can be considered as a medium version among these Chernoff points.

To be mentioned particularly, the upper bound of the probability of error given by Equation (25) is obtained via

C P_{I}^{(1)}

as being equal to

{0.9862}^{N}

. Thus, we can choose sufficiently large N so that the probability of error is less than some threshold value.

4.1.2. Negative Multinomial Model

For the negative multinomial model

{NM}_{M}

, again we can give the geodesic expressions for the exponential and mixture cases.

Proposition 5.

The exponential and mixture geodesics connecting two probability distributions

p_{θ^{'}}

and

p_{θ^{″}}

of the negative multinomial model

{NM}_{M}

are given by

G^{(1)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ (t)} \in {NM}_{M} |θ_{i} (t) = {(θ_{i}^{'})}^{1 - t} {(θ_{i}^{″})}^{t}, i = 1, \dots, n, t \in [0, 1]\},

G^{(- 1)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ (t)} \in {NM}_{M} |θ_{i} (t) = \frac{(1 - t) \frac{θ_{i}^{'}}{θ_{0}^{'}} + t \frac{θ_{i}^{″}}{θ_{0}^{″}}}{\frac{1 - t}{θ_{0}^{'}} + \frac{t}{θ_{0}^{″}}}, i = 1, \dots, n, t \in [0, 1]\} .

Proof.

By using Equations (11) and (12), the exponential and mixture geodesics can be easily obtained by the linear interpolation of the natural and expectation parameters, respectively. □

By using the expressions of

α

-divergences presented in Proposition 2, the

α

-bisectors are directly obtained as follows.

Proposition 6.

The α-bisectors between

p_{θ^{'}}

and

p_{θ^{″}}

within the negative multinomial model

{NM}_{M}

are given by the following equations:

{Bi}^{(α)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ} \in M_{N} |\sum_{i = 1}^{n} θ_{i}^{\frac{1 - α}{2}} [{(\frac{θ_{i}^{'}}{θ_{0}^{'}})}^{\frac{1 + α}{2}} - {(\frac{θ_{i}^{″}}{θ_{0}^{″}})}^{\frac{1 + α}{2}}] = {(θ_{0}^{'})}^{- \frac{1 + α}{2}} - {(θ_{0}^{″})}^{- \frac{1 + α}{2}}\},

{Bi}^{(1)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ} \in {NM}_{M} |\sum_{i = 1}^{n} (\frac{θ_{i}^{'}}{θ_{0}^{'}} - \frac{θ_{i}^{″}}{θ_{0}^{″}}) log θ_{i} = \sum_{i = 0}^{n} (\frac{θ_{i}^{'}}{θ_{0}^{'}} log θ_{i}^{'} - \frac{θ_{i}^{″}}{θ_{0}^{″}} log θ_{i}^{″})\},

{Bi}^{(- 1)} (p_{θ^{'}}, p_{θ^{″}}) = \{p_{θ} \in {NM}_{M} |\sum_{i = 0}^{n} θ_{i} log \frac{θ_{i}^{'}}{θ_{i}^{″}} = 0\} .

Combining the previous two propositions, we have the following result about the determining equations for the four particular Chernoff points with

α = \pm 1

Theorem 7.

The determining equations for the Chernoff points with

α = \pm 1

p_{θ^{'}}

and

p_{θ^{″}}

within the negative multinomial model

{NM}_{M}

are expressed in terms of the argument

t \in [0, 1]

of the corresponding geodesics as follows:

1.: For $C P_{I}^{(1)} (p_{θ^{'}}, p_{θ^{″}})$ ,

$log \frac{θ_{0}^{'}}{θ_{0}^{″}} + \sum_{i = 1}^{n} {(θ_{i}^{'})}^{1 - t} {(θ_{i}^{″})}^{t} (log \frac{θ_{i}^{'}}{θ_{i}^{″}} - log \frac{θ_{0}^{'}}{θ_{0}^{″}}) = 0;$
2.: For $C P_{I}^{(- 1)} (p_{θ^{'}}, p_{θ^{″}})$ ,

$\sum_{i = 1}^{n} (\frac{θ_{i}^{'}}{θ_{0}^{'}} - \frac{θ_{i}^{″}}{θ_{0}^{″}}) log \frac{(1 - t) \frac{θ_{i}^{'}}{θ_{0}^{'}} + t \frac{θ_{i}^{″}}{θ_{0}^{″}}}{\frac{1 - t}{θ_{0}^{'}} + \frac{t}{θ_{0}^{″}}} = \sum_{i = 0}^{n} (\frac{θ_{i}^{'}}{θ_{0}^{'}} log θ_{i}^{'} - \frac{θ_{i}^{″}}{θ_{0}^{″}} log θ_{i}^{″});$
3.: For $C P_{II}^{(1)} (p_{θ^{'}}, p_{θ^{″}})$ ,

$t = \frac{D_{K L} (p_{θ^{″}} ∥ p_{θ^{'}})}{D_{K L} (p_{θ^{'}} ∥ p_{θ^{″}}) + D_{K L} (p_{θ^{″}} ∥ p_{θ^{'}})};$
4.: For $C P_{II}^{(- 1)} (p_{θ^{'}}, p_{θ^{″}})$ ,

$t = \frac{D_{K L} (p_{θ^{'}} ∥ p_{θ^{″}})}{D_{K L} (p_{θ^{'}} ∥ p_{θ^{″}}) + D_{K L} (p_{θ^{″}} ∥ p_{θ^{'}})} .$

As can be seen, the two points of second type

C P_{II}^{(1)}

and

C P_{II}^{(- 1)}

are in the same explicit form as before, except that the Kullback–Leibler divergence

D_{K L}

is given by Proposition 2. And again, the two points of first type

C P_{I}^{(1)}

and

C P_{I}^{(- 1)}

can be solved by using numerical methods.

For the Fisher-Riemannian geodesic midpoint, we have the following result being complementary to the previous one.

Theorem 8.

The Fisher-Riemannian geodesic midpoint

p_{θ^{*}}

between

p_{θ^{'}}

and

p_{θ^{″}}

in the negative multinomial model

{NM}_{M}

is determined by

θ_{i}^{*} = {[\frac{{(θ_{0}^{'})}^{- 1 / 2} {(θ_{i}^{'})}^{1 / 2} + {(θ_{0}^{″})}^{- 1 / 2} {(θ_{i}^{″})}^{1 / 2}}{{(θ_{0}^{'})}^{- 1 / 2} + {(θ_{0}^{″})}^{- 1 / 2}}]}^{2}, i = 1, \dots, n .

Proof.

Let F be the isometry given by Equation (15). Denote the linear midpoint of the two image points

\frac{F (θ^{'}) + F (θ^{″})}{2}

x^{*} \in R^{1, n}

. Then we normalize

x^{*}

to the n-hyperbola as

u^{*} = 2 \sqrt{M} x^{*} / {∥ x^{*} ∥}_{m}

, where the Minkowski norm

{∥ x ∥}_{m} = {(x_{0}^{2} - \sum_{i = 1}^{n} x_{i}^{2})}^{1 / 2}

is used. At last, the required midpoint is obtained as the inverse image point

θ^{*} = F^{- 1} (u^{*})

. □

A numerical illustration for the negative multinomial model

{NM}_{M}

is presented as follows. The two parameters

θ^{'}

and

θ^{″}

are taken as the empirical probability vectors of the decimal digits of

π

within the first and second 10 appearances of “0”, respectively. The parameters of these two points and the resulting Chernoff points are summarized in Table 3.

Again, the pair of points

C P_{I}^{(1)}

and

C P_{II}^{(1)}

admits a certain similarity between them, and so does the pair of points

C P_{I}^{(- 1)}

and

C P_{II}^{(- 1)}

. Whereas, the Fisher-Riemannian geodesic midpoint

C P^{(0)}

serves as a medium version among these Chernoff points.

The upper bound of the probability of error given by Equation (25) is obtained via

C P_{I}^{(1)}

as being equal to

{0.8789}^{M}

. Thus, sufficiently large M need to be chosen so that the probability of error is less than some threshold value.

4.2. Karcher Means of Multiple Points

A natural generalization of the Fisher-Riemannian geodesic midpoint between two points is given by the Karcher mean among multiple points.

Let

M

be a metric space and S be a set of points on

M

. Define a criterion function

f : M \to R

f (x) : = \frac{1}{2 | S |} \sum_{p \in S} d {(x, p)}^{2},

where

d (\cdot, \cdot)

is the distance function and

| S |

is the number of points of S. If the minimizer of the function f exists and is unique, then it is called the Karcher mean of S on

M

. If d is a Riemannian metric on

M

, then the negative gradient vector field with respect to f is found to be the usual average of the corresponding points in the tangent space ([1]):

- \nabla f (x) = \frac{1}{| S |} \sum_{p \in S} {exp}_{x}^{- 1} (p),

(26)

where

{exp}_{x}^{- 1}

is the inverse of the Riemannian exponential map at x. In view of this, the Karcher mean can be alternatively understood as a point at which the above vector field vanishes.

The Karcher mean may be not unique unless all points are located on a geodesically convex region. For example, there are infinitely many geodesic midpoints between two antipodal points on a sphere. However, for model spaces, such as open half-sphere and hyperbolic space, there are existing results to assure the existence and uniqueness of Karcher mean ([22]). Thus, by virtue of Theorem 1 and Theorem 2, we conclude that the concept of a Karcher mean is well-defined on the multinomial and negative multinomial models.

Now, we focus ourselves on the computation of the Karcher mean on these two models. The Karcher mean of two points admits a closed-form expression as the Fisher-Riemannian geodesic midpoint presented previously, but for multiple points, we can only expect to obtain a numerical solution of the Karcher mean.

By virtue of Equation (26), there is a Riemannian gradient iteration algorithm with a locally superlinear convergence in general ([23]):

x_{i + 1} = {exp}_{x_{i}} (\frac{1}{| S |} \sum_{p \in S} {exp}_{x_{i}}^{- 1} (p)) .

However, this general algorithm is difficult to apply in practice unless proper representations of the models are derived. In our case, as we have prepared enough geometric representation results within the multinomial and negative multinomial models in Section 3, we still have to address two practical issuses: the choice of initial points and the computation of Riemannian exponential map

{exp}_{x_{i}}

and its inverse

{exp}_{x_{i}}^{- 1}

4.2.1. Initial Points

Let S be a set of parameters to be averaged in either the model

M_{N}

{NM}_{M}

, we present a heuristic approach motivated by the proof of Theorem 1 and Theorem 2 to provide an initial point choice. The main procedure is presented as follows (here

N = M = 1

is assumed as basic ideas are unchanged up to scale):

Set the average of isometry images $x^{*} : = \frac{1}{| S |} \sum_{θ \in S} F (θ)$ ;
Set the normalized vector $u^{*} : = 2 x^{*} / ∥ x^{*} ∥$ ; the parameter of the initial point is given by $θ^{*} : = F^{- 1} (u^{*})$ .

For the model

M_{N}

, the isometry F is given by Equation (8), and the norm

∥ \cdot ∥

is the Euclidean norm

{∥ \cdot ∥}_{e}

in the proof of Theorem 1. For the model

{NM}_{M}

, the isometry F is given by Equation (15), and the norm

∥ \cdot ∥

is the Minkowski norm

{∥ \cdot ∥}_{m}

in the proof of Theorem 2.

4.2.2. Computation of Riemannian Exponential and its Inverse

Within each of the model

M_{N}

and

{NM}_{M}

(again

N = M = 1

is assumed), the Riemannian exponential and its inverse map can be computed in an easy-to-manipulate way. Except for the isometry F and the norm

∥ \cdot ∥

being given as before, we also need to set

〈 \cdot, \cdot 〉

〈 x, y 〉 = \sum_{i = 0}^{n} x_{i} y_{i}

as the Euclidean inner product for the model

M_{N}

and by

〈 x, y 〉 = x_{0} y_{0} - \sum_{i = 1}^{n} x_{i} y_{i}

as the Minkowski inner product for the model

{NM}_{M}

To compute

{exp}_{θ}^{- 1} (ξ)

, we have the following steps:

Set a tangent vector by orthogonal projection

$x : = F (ξ) - F (θ) 〈 F (ξ), F (θ) 〉 / 4;$
Scale the vector by geodesic distance

$u : = d (θ, ξ) x / ∥ x ∥,$

where the geodesic distance $d (\cdot, \cdot)$ is given by Equation (9) for $M_{N}$ and by Equation (16) for ${NM}_{M}$ .

Thus, we use the resulting vector u to represent

{exp}_{θ}^{- 1} (ξ)

To compute

{exp}_{θ} (u)

, we need to:

Set the angle $α : = ∥ u ∥ / 2$ ;
For $M_{N}$ , express the corresponding point on the sphere

$x : = F (θ) cos α + {2 ∥ u ∥}^{- 1} u sin α,$

for ${NM}_{M}$ , express the corresponding point on the hyperbola

$x : = F (θ) cosh α + {2 ∥ u ∥}^{- 1} u sinh α;$
Set the parameter by isometry $ξ : = F^{- 1} (x)$ .

Thus, the resulting parameter

ξ

is obtained as

{exp}_{θ} (u)

4.2.3. Numerical Example

Now, we test the above algorithm for solving Karcher mean by a numerical example. The data set S is chosen as containing 10 empirical probability vectors from the first to the tenth 100 decimal digits of

π

To illustrate the goodness of each iteration, we present the norm of the negative gradient vector field at each iteration point via Equation (26), as shown in Table 4. Within each model, we present in column (a) the iteration results starting with the initial points chosen as the aforementioned way, while in column (b), the iteration results with initial points chosen as the usual Euclidean means are provided for comparison.

As we can see, all of the four iterations shown here converge rapidly within the first two steps, and our aforementioned choice for initial points is apparently better than the usual choice of Euclidean means. In conclusion, this example, to some extent, shows the effectiveness of our computation scheme for the Karcher mean within the multinomial and negative multinomial models.

5. Conclusions

In this paper, we have studied various information geometric properties based on divergence functions for the multinomial and negative multinomial models. Some pre-derived expressions of fundamental geometric quantities, such as Fisher-Riemannian metric, isometric representation and

α

-curvature, have made it clear that these two models can be put together into a complementary view. With the aid of these geometric structures, we investigate the average problems on these two models. We have proposed the conception of generalized Chernoff points as midpoints of two points and presented some determining equations for them. Then we provided an effective computation scheme for the Karcher mean of multiple points on the multinomial and negative multinomial models.

Author Contributions

Conceptualization, M.L. and H.S.; funding acquisition, H.S.; supervision, H.S.; validation, M.L., H.S. and D.L.; writing—original draft, M.L.; writing—review and editing, D.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Natural Science Foundation of China (Grant No. 61179031).

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

Karcher, H. Riemannian center of mass and mollifier smoothing. Commun. Pure Appl. Math. 1977, 30, 509–541. [Google Scholar] [CrossRef]
Nielsen, F.; Sun, K. Clustering in Hilbert’s projective geometry: The case studies of the probability simplex and the elliptope of correlation matrices. Geom. Struct. Inf. 2019. [Google Scholar] [CrossRef]
Eguchi, S. Second order efficiency of minimum contrast estimations in a curved exponential family. Ann. Stat. 1983, 11, 793–803. [Google Scholar] [CrossRef]
Eguchi, S. A characterization of second order efficiency in a curved exponential family. Ann. Inst. Stat. Math. 1984, 36, 199–206. [Google Scholar] [CrossRef]
Arwini, K.A.; Dodson, C.T. Information Geometry: Near Randomness and Near Independance; Springer: Berlin, Germany, 2008. [Google Scholar]
Calin, O.; Udriste, C. Geometric Modeling in Probability and Statistics; Springer International Publishing: Berlin, Germany, 2014. [Google Scholar]
Amari, S. Information Geometry and Its Applications; Springer: Tokyo, Japan, 2016. [Google Scholar]
Zhang, Z.; Sun, H.; Zhong, F. Information geometry of the power inverse Gaussian distribution. Appl. Sci. 2007, 9, 194–203. [Google Scholar]
Zhong, F.; Sun, H.; Zhang, Z. The geometry of the dirichlet manifold. J. Korean Math. Soc. 2008. [Google Scholar] [CrossRef]
Amari, S. Differential-Geometrical Methods in Statistics; Springer: Berlin, Germany, 1985. [Google Scholar]
Takano, K. Exponential families admitting almost complex structures. SUT J. Math. 2010, 46, 1–21. [Google Scholar]
Nielsen, F. An information-geometric characterization of Chernoff information. IEEE SPL 2013, 20, 269–272. [Google Scholar] [CrossRef]
Ying, S.; Qin, H.; Peng, Y.; Wen, Z. Compute Karcher means on SO(n) by the geometric conjugate gradient method. Neurocomputing 2016, 215, 169–174. [Google Scholar] [CrossRef]
Fiori, S.; Tanaka, T. An algorithm to compute averages on matrix Lie groups. IEEE Trans. Signal Process. 2010, 57, 4734–4743. [Google Scholar] [CrossRef]
Amari, S.; Cichocki, A. Information geometry of divergence functions. Bull. Pol. Acad. Sci. 2010, 58, 183–195. [Google Scholar] [CrossRef] [Green Version]
Ay, N.; Amari, S. A novel approach to canonical divergence within information geometry. Entropy 2015, 17, 8111–8129. [Google Scholar] [CrossRef]
Amari, S. Methods of Information Geometry; Oxford University Press: Oxford, UK, 2000. [Google Scholar]
Kass, R.E.; Vos, P.W. Geom. Found. Asymptot. Inference; Wiley: Hoboken, NJ, USA, 1997. [Google Scholar]
Lee, J.M. Riemannian Manifolds; Springer: Berlin, Germany, 1997. [Google Scholar]
Kurose, T. Dual connections and affine geometry. Math. Z. 1990, 203, 115–121. [Google Scholar] [CrossRef]
Nielsen, F. Hypothesis testing, information divergence and computational geometry. Geom. Sci. Inf. 2013. [Google Scholar] [CrossRef]
Kendall, W.S. Probability, convexity and harmonic maps with small image I: uniqueness and fine existence. Proc. Lond. Math. Soc. 1990, 61, 371–406. [Google Scholar] [CrossRef]
Buss, S.R.; Fillmore, J.P. Spherical averages and applications to spherical splines and interpolation. ACM Trans. Graph. 2001, 20, 95–126. [Google Scholar] [CrossRef]

Table 1. Complementary geometric structures.

Model	Isometric Space	Sectional Curvature	$α$ -Sectional Curvature
$M_{N}$	n-sphere within nonnegative orthant	$\frac{1}{4 N}$	$\frac{1 - α^{2}}{4 N}$
${NM}_{M}$	n-hyperbola within nonnegative orthant	$- \frac{1}{4 M}$	$- \frac{1 - α^{2}}{4 M}$

Table 2. Chernoff points of multinomial model (%).

	0	1	2	3	4	5	6	7	8	9
$θ^{'}$	8.00	8.00	12.00	11.00	10.00	8.00	9.00	8.00	12.00	14.00
$θ^{″}$	11.00	12.00	12.00	8.00	12.00	12.00	7.00	4.00	13.00	9.00
$C P_{I}^{(1)}$	9.49	9.91	12.17	9.53	11.10	9.91	8.06	5.76	12.66	11.41
$C P_{I}^{(- 1)}$	9.52	10.02	12.00	9.48	11.01	10.02	7.99	5.98	12.51	11.47
$C P_{II}^{(1)}$	9.48	9.89	12.17	9.55	11.08	9.89	8.07	5.78	12.65	11.44
$C P_{II}^{(- 1)}$	9.54	10.05	12.00	9.46	11.02	10.05	7.98	5.95	12.51	11.44
$C P^{(0)}$	9.51	9.97	12.08	9.51	11.05	9.97	8.02	5.87	12.58	11.44

Table 3. Chernoff points of negative multinomial model (%).

	0	1	2	3	4	5	6	7	8	9
$θ^{'}$	8.62	8.62	12.93	11.21	9.48	7.76	8.62	6.90	13.79	12.07
$θ^{″}$	10.99	12.09	10.99	6.59	14.29	12.09	6.59	4.40	12.09	9.89
$C P_{I}^{(1)}$	10.99	10.11	11.98	8.73	11.50	9.56	7.60	5.58	12.96	10.99
$C P_{I}^{(- 1)}$	9.73	10.25	12.02	9.04	11.74	9.79	7.67	5.72	12.99	11.05
$C P_{II}^{(1)}$	10.91	10.00	12.04	8.87	11.36	9.43	7.66	5.66	13.01	11.06
$C P_{II}^{(- 1)}$	9.80	10.35	11.96	8.90	11.88	9.92	7.61	5.65	12.94	10.98
$C P^{(0)}$	10.36	10.18	12.00	8.89	11.62	9.67	7.63	5.65	12.98	11.02

Table 4. Iteration for the Karcher mean illustrated by

∥- \nabla f∥ .

Table 4. Iteration for the Karcher mean illustrated by

∥- \nabla f∥ .

	$M_{N}$		${NM}_{M}$
Iteration	(a)	(b)	(a)	(b)
0	$8.21 \times 10^{- 5}$	$1.07 \times 10^{- 2}$	$3.05 \times 10^{- 3}$	$1.85 \times 10^{- 1}$
1	$4.62 \times 10^{- 7}$	$6.28 \times 10^{- 5}$	$1.74 \times 10^{- 4}$	$1.20 \times 10^{- 2}$
2	$2.68 \times 10^{- 9}$	$3.90 \times 10^{- 7}$	$1.01 \times 10^{- 5}$	$7.88 \times 10^{- 4}$

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Li, M.; Sun, H.; Li, D. A Geometric Approach to Average Problems on Multinomial and Negative Multinomial Models. Entropy 2020, 22, 306. https://doi.org/10.3390/e22030306

AMA Style

Li M, Sun H, Li D. A Geometric Approach to Average Problems on Multinomial and Negative Multinomial Models. Entropy. 2020; 22(3):306. https://doi.org/10.3390/e22030306

Chicago/Turabian Style

Li, Mingming, Huafei Sun, and Didong Li. 2020. "A Geometric Approach to Average Problems on Multinomial and Negative Multinomial Models" Entropy 22, no. 3: 306. https://doi.org/10.3390/e22030306

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

A Geometric Approach to Average Problems on Multinomial and Negative Multinomial Models

Abstract

1. Introduction

2. Preliminaries

3. Geometric Structure of Multinomial and Negative Multinomial Models

3.1. Basic Information Geometric Structure

3.1.1. Multinomial Model

3.1.2. Negative Multinomial Model

3.2. Dual Structures

3.2.1. Multinomial Model

3.2.2. Negative Multinomial Model

4. Geometric Average Methods on Multinomial and Negative Multinomial Models

4.1. Midpoints of Two Points

4.1.1. Multinomial Model

4.1.2. Negative Multinomial Model

4.2. Karcher Means of Multiple Points

4.2.1. Initial Points

4.2.2. Computation of Riemannian Exponential and its Inverse

4.2.3. Numerical Example

5. Conclusions

Author Contributions

Funding

Conflicts of Interest

References

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI