Journal of Lie Theory Volume 14 (2004) 427–441 \begin{matrix}c \\ ◯ \end{matrix} 2004 Heldermann Verlag

Separation of Unitary Representations of ℝ × ℝ^{d}

Lobna Abdelmoula, Didier Arnal and Mohamed Selmi

Communicated by J. Ludwig

Abstract. We consider a type I, solvable Lie group G, of the form ℝ × ℝ^{d}. We show that every irreducible unitary representation π of G is characterized by its generalized moment set.

1. Introduction

Let G be a real Lie group with Lie algebra 𝔤, (π, H_{π}) a unitary irreducible representation of G and H_{π}^{∞} the space of C^{∞} vectors for π. Let 𝔤^{∗} be the dual space of 𝔤. In [7], Wildberger defined the moment map Ψ_{π} of π. For all ξ in H_{π}^{∞} \ {0}, the element Ψ_{π}(ξ) in 𝔤^{∗} is defined by:

Ψ_{π}(ξ)(X) := \frac{1}{i} \frac{⟨dπ(X)ξ, ξ⟩}{⟨ξ, ξ⟩}, X ∈ 𝔤.

The moment set I_{π} of the representation π is by definition the closure in 𝔤^{∗} of the image of the moment map:

Ψ_{π} : H_{π}^{∞} \ {0} ⟶ 𝔤^{∗}.

Wildberger gave an explicit description of the moment set I_{π} when G is a connected, simply connected nilpotent Lie group. More precisely, he showed (Theorem 4.2 in [7]) that I_{π} is the closure of the convex hull of the coadjoint orbit O_{π} associated to π via the Kirillov theory, i.e.

I_{π} = \overline{Conv(Ω_{π})}.

This result has been generalized by Arnal and Ludwig [2] for solvable Lie groups. Nevertheless, as shown in [7], the moment set does not characterize the representation even for nilpotent Lie groups.

A. Baklouti, J. Ludwig and M. Selmi extended the moment map to the dual of the complex universal envelopping algebra U(𝔤) of the complexification 𝔤_{ℂ} of 𝔤, as follows:

ISSN 0949–5932 / $2.50 \begin{matrix}c \\ ◯ \end{matrix} Heldermann Verlag

For all A in U(𝔤_{ℂ}) and ξ in H_{π}^{∞} \ {0},

Ψ̃_{π}(ξ)(A) := ℜ𝔢 (\frac{1}{i} \frac{⟨dπ(A)ξ, ξ⟩}{⟨ξ, ξ⟩}) ,

and considered the convex hull J(π) of the image of this generalized moment map Ψ̃_{π}:

J(π) := Conv(Ψ̃_{π}(H_{π}^{∞} \ {0})).

Let U_{n} be the subspace of U(𝔤) consisting of elements of degree less or equal to n. By restriction to U_{n}, we define :

J^{n}(π) = J(π)|_{U_{n}} = {F |_{U_{n}}, F ∈ J(π)}.

In [5], A. Baklouti, J. Ludwig and M. Selmi shown that for all nilpotent Lie group, there exists an integer n such that, for any unitary irreducible representations π and ρ of G, we have

J^{n}(π) = J^{n}(ρ) if and only if π ≃ ρ.

Later on, they generalized this result for an exponential Lie group and, in [4], they shown with D. Arnal, the following result:

Theorem 0. (Separation of unitary representations of exponential Lie groups) Let G = exp(𝔤) be an exponential Lie group. Let π and ρ be two unitary irreducible representations of G. Then:

π ≃ ρ if and only if J(π) = J(ρ).

For exponential groups, there is a bijection between coadjoint orbits and classes of unitary irreducible representations. Thus this result means it is possible to find the coadjoint orbit of a representation π, just by looking at the image of the generalized moment map of π.

Since quantizing a symplectic manifold like a coadjoint orbit is building a Hilbert space and a representation of the algebra of observables, theorem 0 can be viewed as a dequantization of the representation π: we refind directly the coadjoint orbit from the representation and its moment map.

In this paper, we study the simplest solvable, non exponential example: the case of a type I solvable connected and simply connected Lie group of the form G = ℝ × ℝ^{d}. We show that the set J(π), characterizes the irreducible unitary representations of G and we prove the following:

Theorem 1. Let G be a semi-direct product of ℝ by ℝ^{d}, G = ℝ × ℝ^{d}. Suppose G is type I, let π and ρ be two unitary irreducible representations of G. Then:

π ≃ ρ if and only if J(π) = J(ρ).

For type I groups, even for our simple example, we have not only to find the coadjoint orbit associated to a representation π, but also the extension of the inducing character to the non simply connected inducing “small group”.

Thus, we can say that, for non exponential Lie groups, the dequantization procedure is not injective, there is many different “quantum” models for the coadjoint orbit and we have to separate them.

In this article, we get this separation thanks to a case-by-case analysis which is not directly generalizable to a general type I, solvable Lie group. However, we conjecture that theorem 1 holds for any type I, solvable Lie group and we hope that our present proof is the first step in proving a more general result.

2. Preliminaries

The unitary representations of our group are very easy to describe, by using unitary induction.

Let 𝔤 be the Lie algebra of G, 𝔫 its abelian ideal ℝ^{d} and H an element in 𝔤 not belonging to 𝔫. Each irreducible unitary representation π is associated to a coadjoint orbit in 𝔤^{∗}, the dual of 𝔤.

Let f be in 𝔤^{∗}. Its coadjoint orbit is either {f} and the associated representation is the character χ_{f} : exp X ↦ e^{if(X)} , or a two dimensional manifold: denote f_{0} the restriction of f to 𝔫, the coadjoint action of exp(ℝH) on f_{0} is a one-dimensional submanifold of 𝔫^{∗} ≃ ℝ^{d}, which is diffeomorphic to a circle or to a line.

The coadjoint orbit of f is thus the cylinder:

G.f = {ℓ ∈ 𝔤^{∗} such that ℓ|_{𝔫} ∈ exp(ℝH).f_{0}}.

Note G(f) the stabilizer of f in G, 𝔥 = 𝔫 is a real polarization in the point f, stable under the G(f)-adjoint action. Let us put D = G(f) exp(𝔥), this is a closed, normal subgroup of G and the representations associated to f are:

π = π(f, χ_{f}, 𝔥, G),

where χ_{f} is any character of D, with differential if.

Let us remark that the space H_{π} of the representation π is isomorphic to L^{2}(G/D), which is either L^{2}(ℝ) or L^{2}(ℝ/ℤ).

Auslander and Kostant shown in [1] that the representation π does not depend on the choice of the polarization and each unitary representation of G can be obtained in this way.

Since H belongs to 𝔤 \ 𝔫, each element g of G can be written in an unique way as:

g = exp tH n (t ∈ ℝ, n ∈ N).

We complexify the space ℝ^{d} = 𝔫 and we put the matrix of ad_{H} restricted to 𝔫_{ℂ} in its Jordan form. There is four possible types of Jordan blocks:

those associated to the eigenvalue 0, the corresponding subspaces E_{j}(0) are real;

those associated to a real, non zero eigenvalue a, the corresponding subspaces E_{j}(a) are real; those associated to a non real, non purely imaginary eigenvalue a + ib, the corresponding subspaces E_{j}(a + ib) are not real, but a − ib is another eigenvalue and we choose E_{j}(a − ib) = \overline{E_{j}(a + ib)};

those associated to an eigenvalue ib ≠ 0, the corresponding subspaces E_{j}(ib) are not real, but −ib is another eigenvalue and we choose E_{j}(−ib) = \overline{E_{j}(ib)}. In this last case, the real number b will called a period of the group G.

Note p_{j}(λ) the dimensionality of E_{j}(λ) and fix a Jordan basis for ad_{H}:

\begin{matrix}Z_{jk}(0) = X_{jk}(0) \\ Z_{jk}(a) = X_{jk}(a) \\ Z_{jk}(a + ib) = X_{jk}(a + ib) + iY_{jk}(a + ib), \\ Z_{jk}(a − ib) = \overline{Z_{jk}(a + ib)} \\ Z_{jk}(ib) = X_{jk}(ib) + iY_{jk}(ib), \\ Z_{jk}(−ib) = \overline{Z_{jk}(ib)} \end{matrix} \begin{matrix}(1 ≤ k ≤ p_{j}(0)), \\ (1 ≤ k ≤ p_{j}(a)), \\ (1 ≤ k ≤ p_{j}(a + ib), b > 0), \\ (1 ≤ k ≤ p_{j}(ib), b > 0). \end{matrix}

Then:

ad_{H} (Z_{jk}(λ)) = λZ_{jk}(λ) + Z_{j(k+1)}(λ), (k < p_{j}(λ))

and

ad_{H} (Z_{jp_{j}(λ)}(λ)) = λZ_{jp_{j}(λ)}(λ).

Now, for the dual basis Z_{jk}(λ)^{∗}, we have:

ad_{−H}^{∗} (Z_{jk}(λ)^{∗}) = λZ_{jk}(λ)^{∗} + Z_{j(k−1)}(λ)^{∗}, (k > 1),

and

ad_{−H}^{∗} (Z_{j1}(λ)^{∗}) = λZ_{j1}(λ)^{∗}.

Let f_{0} be a point in the dual 𝔫^{∗} of 𝔫,

f_{0} = ∑_{λ, j, k} ζ_{jk}(λ)Z_{jk}(λ)^{∗} (\overline{f_{0}} = f_{0}),

then:

Coad (exp(−tH)) f_{0} = ∑_{λ, j, k} z_{jk}(λ)(t)Z_{jk}(λ)^{∗},

with

z_{jk}(λ)(t) = e^{tλ} (ζ_{jk}(λ) + tζ_{j(k+1)}(λ) + ... + \frac{t^{p_{j}(λ)−k}}{(p_{j}(λ) − k)!}ζ_{jp_{j}(λ)}) .

Now G is supposed to be of type I. Let us call period group of G the subgroup of ℝ generated by the periods b of G. The following is well known.

Proposition 2.1. (Characterization of G-type [6]) The type of G is I if and only if G has a discrete period group.

3. The coadjoints orbits

We have already seen that the coadjoints orbits are:

either points, if and only if H is in the Lie algebra 𝔤(f), then, since H is arbitrary, if and only if 𝔤(f) ⊄ 𝔫, or if and only if:

f_{0} = ∑_{j} ζ_{1j}(0)Z_{1j}(0)^{∗}, f = hH^{∗} + ∑_{j} ζ_{1j}(0)Z_{1j}(0)^{∗},

or diffeomorphic to the cylinder ℝH^{∗} × O, if O is the orbit of f_{0} , O is diffeomorphic to ℝ or to a circle S^{1} .

In order to give a parametrization of the orbits, we choose an ordering on the subspaces E_{jk}(λ). Let us be more precise, we class these spaces as follow:

first the spaces E_{j}(0), then the spaces E_{j}(a) (a real, a ≠ 0), then the spaces E_{j}(a + ib) (a and b reals, a ≠ 0 and b ≠ 0), finally the spaces E_{j}(ib). This induced a natural ordering on the basis (Z_{kj}(λ)^{∗}) of 𝔫_{ℂ}^{∗}.

Each non trivial orbit meets the orthogonal of H. We choose a point f of the orbit in this orthogonal.

\underline{First case}: there exists a non null component ζ_{jk}(0), k > 1. Let j_{0} be the smallest j, k_{0} the largest k possible such that ζ_{jk}(0) ≠ 0. Then:

z_{j_{0}k_{0}}(0)(t) = ζ_{j_{0}k_{0}}(0), z_{j_{0}(k_{0}−1)}(0)(t) = ζ_{j_{0}(k_{0}−1)}(0) + tζ_{j_{0}k_{0}}(0).

We choose, as a base point of the orbit, the only point ℓ orthogonal to H and to Z_{j_{0}(k_{0}−1)}(0). We deduce that the invariant functions which characterize all the orbits of this category having the same j_{0} and k_{0} are the coordinates of ℓ, i.e. the functions:

f ↦ z_{jk}(λ) (−\frac{f(Z_{j_{0}(k_{0}−1)}(0))}{f(Z_{j_{0}k_{0}}(0))}) ((j, k, λ) > (j_{0}, k_{0}, 0)).

For these orbits, O is diffeomorphic to ℝ and G(ℓ) is a subgroup of N.

\underline{Second case}: all the components ζ_{jk}(0), k > 1 vanish, but there exists a non vanishing component ζ_{jk}(a) (a ≠ 0). Let a_{0} be the first a, j_{0} the smallest j, k_{0} the largest k such that ζ_{jk}(a) is non null. Then:

z_{j_{0}k_{0}}(a_{0})(t) = e^{a_{0}t}ζ_{j_{0}k_{0}}(a_{0}).

We choose, as a base point of the orbit, the only point ℓ orthogonal to H and such that |ℓ (Z_{j_{0}k_{0}}(a_{0}))| = 1. Then the invariant functions which characterize all the orbits of this category, having the same a_{0} , j_{0} and k_{0} are the coordinates of ℓ, i.e. the functions:

f ↦ z_{jk}(λ) (−\frac{1}{a_{0}} Log |f(Z_{j_{0}k_{0}}(a_{0})|) ((j, k, λ) > (j_{0}, k_{0}, a_{0})).

For these orbits, O is diffeomorphic to ℝ and G(ℓ) is a subgroup of N.

\underline{Third case}: all the components ζ_{jk}(0), k > 1, are vanishing, as well as all the components ζ_{jk}(a) (a ≠ 0), but there exists a non vanishing component ζ_{jk}(a + ib) (a and b real and non zero). Let a_{0} + ib_{0} the first a + ib (by convention, we take b_{0} > 0), j_{0} the smallest j and k_{0} the largest k such that ζ_{jk}(a + ib) ≠ 0. Then:

z_{j_{0}k_{0}}(a_{0} + ib_{0})(t) = e^{a_{0}t}e^{ib_{0}t}ζ_{j_{0}k_{0}}(a_{0}).

We choose, as a base point of the orbit, the only point ℓ orthogonal to H and such that |ℓ (Z_{j_{0}k_{0}}(a_{0}))| = 1. Then the invariant functions which characterize all the orbits of this category, having the same a_{0} + ib_{0} , j_{0} and k_{0} are the coordinates of ℓ, i.e. the functions:

f ↦ z_{jk}(λ) (−\frac{1}{2a_{0}} Log |f(Z_{j_{0}k_{0}}(a_{0} + ib_{0})|^{2}) ((j, k, λ) > (j_{0}, k_{0}, a_{0} + ib_{0})).

For these orbits, O is diffeomorphic to ℝ and G(ℓ) is a subgroup of N.

\underline{Fourth case}: all the components ζ_{jk}(0), with k > 1, are vanishing, as well as all the components ζ_{jk}(a) (a ≠ 0) and all the components ζ_{jk}(a+ib) (a and b real and non zero), but there exists a non vanishing component ζ_{jk}(ib) with k > 1. Let ib_{0} be the the first ib ( by convention, we take b_{0} > 0), j_{0} the smallest j, k_{0} the largest k > 1 such that ζ_{jk}(ib) ≠ 0. Then:

z_{j_{0}k_{0}}(ib_{0})(t) = e^{ib_{0}t}ζ_{j_{0}k_{0}}(ib_{0}),
z_{j_{0}(k_{0}−1)}(ib_{0})(t) = e^{ib_{0}t} (ζ_{j_{0}(k_{0}−1)}(ib_{0}) + tζ_{j_{0}k_{0}}(ib_{0})) .

We choose, as a base point of the orbit, the only point ℓ orthogonal to H and such the quantity t ↦ |z_{j_{0}(k_{0}−1)}(ib_{0})(t)|^{2} is minimal. Then if we replace f by ℓ, this quantity becomes:

t^{2}|ζ_{j_{0}k_{0}}(ib_{0})|^{2} + |ζ_{j_{0}(k_{0}−1)}(ib_{0})|^{2}

else, the coordinates of ℓ verify:

ζ_{j_{0}k_{0}}(ib_{0})\overline{ζ_{j_{0}(k_{0}−1)}(ib_{0})} + \overline{ζ_{j_{0}k_{0}}(ib_{0})}ζ_{j_{0}(k_{0}−1)}(ib_{0}) = 0.

The invariants functions which characterize the orbits of this category having the same b_{0} , j_{0} and k_{0} are the coordinats of ℓ, i.e. the functions

f ↦ −z_{jk}(λ)\frac{f(Z_{j_{0}k_{0}}(ib_{0}))\overline{f(Z_{j_{0}(k_{0}−1)}(ib_{0}))} + f(Z_{j_{0}(k_{0}−1)}(ib_{0}))\overline{f(Z_{j_{0}k_{0}}(ib_{0}))}}{2 |f(Z_{j_{0}k_{0}}(ib_{0}))|^{2}}

for (j, k, λ) > (j_{0}, k_{0}, ib_{0}). For these orbits, O is diffeomorphic to ℝ and G(ℓ) is a subgroup of N.

\underline{Fifth case}: all the components ζ_{jk}(0), with k > 1, are vanishing, as well as all the components ζ_{jk}(a) (a ≠ 0) and all the components ζ_{jk}(a + ib) (a ≠ 0 and b ≠ 0, a and b real) and all the components ζ_{jk}(ib) (k > 1), but there exists a non vanishing component ζ_{j1}(ib). Then, we have:

f = ∑_{j} ζ_{j1}(0)Z_{j1}(0)^{∗} + ∑_{ib} ∑_{j} ζ_{j1}(ib)Z_{j1}(ib)^{∗}.

We call period group of f, the subgroup of ℝ generated by the numbers b such that there exists j for which ζ_{j1}(ib) ≠ 0. For each b, let j_{0} be the first j such that ζ_{j1}(ib) ≠ 0, by construction, we have:

ζ_{j_{0}1}(−ib) = \overline{ζ_{j_{0}1}(ib)}.

The period group of f can be written as βℤ with β > 0. This implies that Coad(exp(tH))f = f if and only if t = \frac{2nπ}{β} for some integral number n. We choose integral numbers q(b) such that β = ∑_{b} q(b)b. Putting q(−b) = −q(b) and q(b)b = q(−b)(−b), we can suppose q(b) ≥ 0 for any b. Then:

∏_{b} (z_{j_{0}1}(b)(t))^{q(b)} = e^{iβt} ∏_{b} (ζ_{j_{0}1}(b))^{q(b)} .

As a base point in the orbit, we choose the only point ℓ of O for which this quantity is real positive. For orbits in this category, with the same family of b and j_{0} , we identify each orbit with the coordinates of ℓ, i.e. the functions:

f ↦ z_{jk}(λ) (−\frac{1}{β} Arg (∏_{b} (f(Z_{j_{0}1}(b)))^{q(b)})) .

For these orbits, O is diffeomorphic to S^{1} and G(ℓ) is the non connected subgroup exp (\frac{2π}{β} ℤH) exp (𝔤(ℓ)).

Proposition 3.1. (The orbit description) There exist six classes of coadjoints orbits in 𝔤^{∗}: the family of trivial orbits and the five classes of non trivial orbits. Each class is the union of the subclasses C(λ, j_{0}, k_{0}) constituted by the orbits with the same λ, j_{0} and k_{0} .

Each subclass is characterized by the value of polynomial functions, constant on the orbits.

The class of trivial orbits is characterized by the nullity of all the functions f ↦ f(Z_{jk}(λ)) with λ ≠ 0 or k > 1,

a subclass C(0, j_{0}, k_{0}) of the first type is characterized by the vanishing of the functions f ↦ f(Z_{jk}(0)) if j < j_{0} or j = j_{0} and k > k_{0} and the non vanishing of the function f ↦ f(Z_{j_{0}k_{0}}(0)),

a subclass C(a_{0}, j_{0}, k_{0}) of the second type is characterized by the vanishing of the functions f ↦ f(Z_{jk}(0)) and f ↦ f(Z_{jk}(a)) for a < a_{0} or a = a_{0} and j < j_{0} or a = a_{0} , j = j_{0} and k > k_{0} and the non vanishing of the function f ↦ f(Z_{j_{0}k_{0}}(a_{0})), a subclass C(a_{0} + ib_{0}, j_{0}, k_{0}) of the third type is characterized by the vanishing of the functions f ↦ f(Z_{jk}(0)), f ↦ f(Z_{jk}(a)) and f ↦ f(Z_{jk}(a+ib)) for a+ib < a_{0} + ib_{0} or a + ib = a_{0} + ib_{0} and j < j_{0} or a + ib = a_{0} + ib_{0} , j = j_{0} and k > k_{0} and the non vanishing of the function f ↦ f(Z_{j_{0}k_{0}}(a_{0} + ib_{0}))f(Z_{j_{0}k_{0}}(a_{0} − ib_{0})),

a subclass C(ib_{0}, j_{0}, k_{0}) of the fourth type is characterized by the vanishing of the functions f ↦ f(Z_{jk}(0)), f ↦ f(Z_{jk}(a)), f ↦ f(Z_{jk}(a + ib)) and f ↦ f(Z_{jk}(ib)) for ib < ib_{0} and k > 1 or ib = ib_{0} and j < j_{0} and k > 1 or ib = ib_{0} , j = j_{0} and k > k_{0} and the non vanishing of the function f ↦ f(Z_{j_{0}k_{0}}(ib_{0}))f(Z_{j_{0}k_{0}}(−ib_{0})),

a subclass C(ib_{0}, j_{0}, 1) of the fifth type is characterized by the vanishing of the functions f ↦ f(Z_{jk}(0)), f ↦ f(Z_{jk}(a)), f ↦ f(Z_{jk}(a + ib)) and f ↦ f(Z_{jk}(ib)) for k > 1 or ib < ib_{0} or ib = ib_{0} and j < j_{0} or ib = ib_{0} , j = j_{0} and the non vanishing of the function f ↦ f(Z_{j_{0}1}(ib_{0}))f(Z_{j_{0}1}(−ib_{0})).

4. Separation of generic representations

We built in paragraph 2 the unitary irreducible representations π of G. Let ℓ be in 𝔤 and π be the representation associated to ℓ. The space of this representation is L^{2}(O), where the restriction of the orbit O of ℓ to 𝔫^{∗} is parametrized by the map t ↦ f(t) = Coad(exp tH)ℓ.

Thus π is a character if the orbit of ℓ is trivial, or realized in the space L^{2}(ℝ), if the orbit is in the one of the first four type of subclasses, or realized in the space L^{2} (^{ℝ}/_{\frac{2π}{β} ℤ}), if the orbit is in a fifth type subclass.

By construction, if Z is an element of 𝔫, the operator dπ(Z) is the multiplication operator by if(t)(Z).

Especially, if π_{1} and π_{2} are two unitary irreducible representations with the same moment set and if for example, for π_{1} , the function

t ↦ f_{1}(t)(Z_{jk}(λ)) = z_{jk}^{(1)}(λ)(t)

is constant, then the same holds for

t ↦ f_{2}(t)(Z_{jk}(λ)) = z_{jk}^{(2)}(λ)(t)

and these two functions coincide. As a consequence, the subclass of the orbit associated to π_{1} coincides with the subclass of the orbit associated to π_{2} .

Moreover the representations π_{i} (i = 1, 2) are induced from characters defined by the form ℓ_{i} such that, for each Jordan bloc E_{j}(λ) of ad(H),

k_{0} = sup {k such that ℓ_{1}(Z_{jk}(λ)) ≠ 0} = sup {k such that ℓ_{2}(Z_{jk}(λ)) ≠ 0}.

For the separation of two representations which are both in one of the four first type of subclasses, we use the method of [4]. In fact, if π is not in the fifth class, we have π = Ind_{N}^{G} χ_{ℓ}, where:

χ_{ℓ} : N ⟶ ℂ, χ_{ℓ}(exp X) = e^{iℓ(X)}, ∀X ∈ 𝔫.

Lemma 4.1. Let π_{1} and π_{2} be two unitary irreducible representations of G which are both in one of the first four classes. Suppose J(π_{1}) = J(π_{2}). Then we can choose the point ℓ_{i} such that there exists a strictly increasing C^{∞} function h, such that h(0) > 0 and an element u in U(𝔫) such that, for each C^{∞} -vector ϕ_{i} for π_{i} (i = 1, 2),

(dπ_{1}(u)ϕ_{1}) (t) = h(t)ϕ_{1}(t), (dπ_{2}(u)ϕ_{2}) (t) = h(t)ϕ_{2}(t), ∀t.

Proof. We prove the lemma case by case.

If the representations π_{i} = Ind_{N}^{G} χ_{ℓ_{i}} are both in a first type subclass, with our notations, there exists j_{0} and k_{0} > 1 such that:

ζ_{j_{0}k_{0}}^{(1)}(0) = ℓ_{1}(Z_{j_{0}k_{0}}(0)) = ζ_{j_{0}k_{0}}^{(2)}(0) = ℓ_{2}(Z_{j_{0}k_{0}}(0)) ≠ 0.

Then

\frac{1}{i} (dπ_{i}(Z_{j_{0}k_{0}}(0))ϕ_{i}) (t) = ζ_{j_{0}k_{0}}^{(i)}(0)ϕ_{i}(t),

\frac{1}{i} (dπ_{i}(Z_{j_{0}(k_{0}−1)}(0))ϕ_{i}) (t) = (ζ_{j_{0}(k_{0}−1)}^{(i)}(0) + tζ_{j_{0}k_{0}}^{(i)}(0)) ϕ_{i}(t).

Replacing ℓ_{i} by

ℓ′_{i} = Coad (exp −\frac{ζ_{j_{0}(k_{0}−1)}^{(i)}(0)}{ζ_{j_{0}k_{0}}^{(i)}(0)}H) ℓ_{i},

this relation becomes:

\frac{1}{i} (dπ_{i}(Z_{j_{0}(k_{0}−1)}(0))ϕ_{i}) (t) = tζ_{j_{0}k_{0}}^{(i)}(0)ϕ_{i}(t).

The lemma holds for

h(t) = t |ζ_{j_{0}k_{0}}^{(i)}(0)| + 1 and u = \frac{ζ_{j_{0}k_{0}}^{(i)}(0)}{i |ζ_{j_{0}k_{0}}^{(i)}(0)|}Z_{j(k_{0}−1)}(0) + 1.

If the representations π_{i} = Ind_{N}^{G} χ_{ℓ_{i}} are both in a second type class, with our notations, there exists a_{0} , j_{0} and k_{0} such that:

ζ_{j_{0}k_{0}}^{(1)}(a_{0}) = ℓ_{1}(Z_{j_{0}k_{0}}(a_{0})) ≠ 0, ζ_{j_{0}k_{0}}^{(2)}(a_{0}) = ℓ_{2}(Z_{j_{0}k_{0}}(a_{0})) ≠ 0.

Then

\frac{1}{i} (dπ_{i}(Z_{j_{0}k_{0}}(a_{0}))ϕ_{i}) (t) = ζ_{j_{0}k_{0}}^{(i)}(a_{0})e^{a_{0}t}ϕ_{i}(t).

Comparing the signs of ⟨dπ_{i}(Z_{j_{0}k_{0}}(a_{0}))ϕ_{i}, ϕ_{i}⟩, we see that the numbers ζ_{j_{0}k_{0}}^{(i)}(a_{0}) have the same sign ε = ±1. Replacing ℓ_{i} by

ℓ′_{i} = Coad (exp −\frac{1}{a_{0}} Log |ζ_{j_{0}(k_{0}−1)}^{(i)}(a_{0})|H) ℓ_{i},

436 Abdelmoula, Arnal and Selmi we get:

\frac{1}{i} (dπ_{i}(Z_{j_{0}k_{0}}(a_{0}))ϕ_{i}) (t) = εe^{a_{0}t}ϕ_{i}(t).

The lemma holds for h(t) = \frac{a_{0}}{|a_{0}|}e^{a_{0}t} and u = \frac{εa_{0}}{i|a_{0}|}Z_{j_{0}k_{0}}(a_{0}).

If the representations π_{i} = Ind_{N}^{G} χ_{ℓ_{i}} are both in a third type class, there exists a_{0} + ib_{0} , j_{0} and k_{0} such that: ζ_{j_{0}k_{0}}^{(1)}(a_{0}+ib_{0}) = ℓ_{1}(Z_{j_{0}k_{0}}(a_{0}+ib_{0})) ≠ 0, ζ_{j_{0}k_{0}}^{(2)}(a_{0}+ib_{0}) = ℓ_{2}(Z_{j_{0}k_{0}}(a_{0}+ib_{0})) ≠ 0. Put

Z_{j_{0}k_{0}}(a_{0} + ib_{0}) = X_{j_{0}k_{0}}(a_{0} + ib_{0}) + iY_{j_{0}k_{0}}(a_{0} + ib_{0})

and

ζ_{j_{0}k_{0}}^{(i)}(a_{0} + ib_{0}) = ξ_{j_{0}k_{0}}^{(i)} + iη_{j_{0}k_{0}}^{(i)},

we get:

\frac{1}{i} (dπ_{i}(Z_{j_{0}k_{0}}(a_{0} + ib_{0}))ϕ_{i}) (t) = ζ_{j_{0}k_{0}}^{(i)}(a_{0} + ib_{0})e^{(a_{0}+ib_{0})t}ϕ_{i}(t).

Or:

\frac{1}{i} (dπ_{i}(X_{j_{0}k_{0}}(a_{0} + ib_{0}))ϕ_{i}) (t) = e^{a_{0}t} (ξ_{j_{0}k_{0}}^{(i)} cos b_{0}t − η_{j_{0}k_{0}}^{(i)} sin b_{0}t) ϕ_{i}(t)
\frac{1}{i} (dπ_{i}(Y_{j_{0}k_{0}}(a_{0} + ib_{0}))ϕ_{i}) (t) = e^{a_{0}t} (ξ_{j_{0}k_{0}}^{(i)} sin b_{0}t + η_{j_{0}k_{0}}^{(i)} cos b_{0}t) ϕ_{i}(t).

Replacing ℓ_{i} by

ℓ′_{i} = Coad (exp −\frac{1}{a_{0}} Log (ξ_{j_{0}(k_{0}−1)}^{(i) 2} + η_{j_{0}(k_{0}−1)}^{(i) 2}) H) ℓ_{i}

and putting ζ_{j_{0}k_{0}}^{(i)}(a_{0} + ib_{0}) = e^{iθ^{(i)}} , these relations become:

\begin{aligned}\frac{1}{i} (dπ_{i}(X_{j_{0}k_{0}}(a_{0} + ib_{0}))ϕ_{i}) (t) & = e^{a_{0}t} cos(b_{0}t + θ^{(i)})ϕ_{i}(t) \\ \frac{1}{i} (dπ_{i}(Y_{j_{0}k_{0}}(a_{0} + ib_{0}))ϕ_{i}) (t) & = e^{a_{0}t} sin(b_{0}t + θ^{(i)})ϕ_{i}(t). \end{aligned}

Thus:

− (dπ_{i}(X_{j_{0}k_{0}}(a_{0} + ib_{0})^{2} + Y_{j_{0}k_{0}}(a_{0} + ib_{0})^{2})ϕ_{i}) (t) = e^{2a_{0}t}ϕ_{i}(t).

The lemma holds for h(t) = \frac{a_{0}}{|a_{0}|}e^{2a_{0}t} and

u = −\frac{a_{0}}{|a_{0}|} (X_{j_{0}k_{0}}(a_{0} + ib_{0})^{2} − Y_{j_{0}k_{0}}(a_{0} + ib_{0})^{2}) .

If the representations π_{i} = Ind_{N}^{G} χ_{ℓ_{i}} are both in a fourth type subclass, there exists ib_{0} , j_{0} and k_{0} > 1 such that:

ζ_{j_{0}k_{0}}^{(1)}(ib_{0}) = ℓ_{1}(Z_{j_{0}k_{0}}(ib_{0})) = ζ_{j_{0}k_{0}}^{(2)}(ib_{0}) = ℓ_{2}(Z_{j_{0}k_{0}}(ib)) ≠ 0.

Abdelmoula, Arnal and Selmi 437

Then

\frac{1}{i} (dπ_{i}(Z_{j_{0}k_{0}}(ib_{0}))ϕ_{i}) (t) = ζ_{j_{0}k_{0}}^{(i)}(ib_{0})e^{ib_{0}t}ϕ_{i}(t)

and

\frac{1}{i} (dπ_{i}(Z_{j_{0}(k_{0}−1)}(ib_{0}))ϕ_{i}) (t) = e^{ib_{0}t} (ζ_{j_{0}(k_{0}−1)}^{(i)}(ib_{0}) + tζ_{j_{0}k_{0}}^{(i)}(ib_{0})) ϕ_{i}(t).

From these relations, we see:

−⟨dπ_{i}(Z_{j_{0}k_{0}}(ib_{0})\overline{Z_{j_{0}k_{0}}(ib_{0})})ϕ_{i}, ϕ_{i}⟩ = |ζ_{j_{0}k_{0}}^{(i)}|^{2}⟨ϕ_{i}, ϕ_{i}⟩

so |ζ_{j_{0}k_{0}}^{(1)}|^{2} = |ζ_{j_{0}k_{0}}^{(2)}|^{2} .

On the other hand, we have

\begin{aligned}− & (dπ_{i}(Z_{j_{0}(k_{0}−1)}(ib_{0})\overline{Z_{j_{0}k_{0}}(ib_{0})} + Z_{j_{0}k_{0}}(ib_{0})\overline{Z_{j_{0}(k_{0}−1)}(ib_{0})})ϕ_{i}) (t) = \\ & = (2t|ζ_{j_{0}k_{0}}^{(i)}(ib_{0})|^{2} + ζ_{j(k_{0}−1)}^{(i)}(ib_{0})\overline{ζ_{j_{0}k_{0}}^{(i)}(ib_{0})} + ζ_{j_{0}k_{0}}^{(i)}(ib_{0})\overline{ζ_{j_{0}(k_{0}−1)}^{(i)}(ib_{0})}) ϕ_{i}(t). \end{aligned}

Replace ℓ_{i} by

ℓ′_{i} = Coad (exp −\frac{ζ_{j_{0}(k_{0}−1)}^{(i)}(ib_{0})\overline{ζ_{j_{0}k_{0}}^{(i)}(ib_{0})} + ζ_{j_{0}k_{0}}^{(i)}(ib_{0})\overline{ζ_{j_{0}(k_{0}−1)}^{(i)}(ib_{0})}}{2|ζ_{j_{0}k_{0}}^{(i)}(ib_{0})|^{2}}H) ℓ_{i},

this relation becomes:

\begin{matrix}− (dπ_{i}(Z_{j_{0}(k_{0}−1)}(ib_{0})\overline{Z_{j_{0}k_{0}}(ib_{0})} + Z_{j_{0}k_{0}}(ib_{0})\overline{Z_{j_{0}(k_{0}−1)}(ib_{0})})ϕ_{i}) (t) = \\ = 2t|ζ_{j_{0}k_{0}}^{(i)}(ib_{0})|^{2}ϕ_{i}(t). \end{matrix}

The lemma holds for h(t) = 2t|ζ_{j_{0}k_{0}}^{(i)}(ib_{0})|^{2} + 1 and

u = −Z_{j_{0}(k_{0}−1)}(ib_{0})\overline{Z_{j_{0}k_{0}}(ib_{0})} − Z_{j_{0}k_{0}}(ib_{0})\overline{Z_{j_{0}(k_{0}−1)}(ib_{0})} + 1.

Corollary 4.2. (Case of the fourth first type of classes) Let π_{1} and π_{2} be two unitary irreducible representations of G such that J(π_{1}) = J(π_{2}). Suppose π_{1} is not in a fifth type class, then π_{2} is in the same subclass as π_{1} and:

π_{1} ≃ π_{2}.

Proof. Indeed, we can now apply lemma 3.1 of [4].

5. The fifth type classes

In the only remaining case, the two representations π_{1} and π_{2} are in a fifth type class, the only non vanishing quantities ⟨dπ_{i}(Z_{jk}(λ))ϕ_{i}, ϕ_{i}⟩ with λ ≠ 0 are such that λ = ib and k = 1. For them:

\begin{aligned}|ζ_{j1}^{(1)}(ib)|^{2} & = −⟨dπ_{1}(Z_{j1}(ib)\overline{Z_{j1}(ib)}ϕ_{1}, ϕ_{1}⟩ \\ & = −⟨dπ_{2}(Z_{j1}(ib)\overline{Z_{j1}(ib)}ϕ_{2}, ϕ_{2}⟩ = |ζ_{j1}^{(2)}(ib)|^{2}. \end{aligned}

The coadjoint orbits associated to π_{i} are both diffeomorphic to a cylinder with a circular basis.

Moreover, the period groups of ℓ_{1} and ℓ_{2} coincide since:

period group(ℓ_{1}) = ∑_{b,ζ_{j1}^{(1)}(ib)≠0} bℤ = ∑_{b,ζ_{j1}^{(2)}(ib)≠0} bℤ = period group(ℓ_{2}).

Let β > 0 be the generator of this period group, the stabilizer G(ℓ_{i}) contains exp(\frac{2π}{β} ℤH), there is two characters of G(ℓ_{1})N = G(ℓ_{2})N defined by:

χ_{ℓ_{i},μ_{i}} (exp(\frac{2π}{β} nH) exp(X)) = e^{2iπμ_{i}n}e^{iℓ_{i}(X)},

where μ_{i} belongs to [0, 1[, such that:

π_{i} = Ind_{G(ℓ_{i})N}^{G} χ_{ℓ_{i},μ_{i}}.

Lemma 5.1. (Separation of the μ_{i}) With our notations, if J(π_{1}) = J(π_{2}), then μ_{1} = μ_{2} .

Proof. First, we identify the space H_{π_{i}} to L^{2}(S^{1}) by identifying each function Φ : G ⟶ ℂ of H_{π_{i}}, with the function ϕ : S^{1} ⟶ ℂ defined by:

ϕ(t) = e^{iμ_{i}t}Φ(exp \frac{t}{β} H).

Then we have:

\begin{aligned}(dπ_{i}(\frac{H}{β})ϕ) (t) & = − (\frac{d}{dt}ϕ(t) + iμ_{i}ϕ(t)) \\ (dπ_{i}(Z_{j1}(ib))ϕ) (t) & = e^{−i \frac{b}{β} t}ζ_{j1}^{(i)}(ib)ϕ(t). \end{aligned}

Now, let us suppose for instance that 0 ≤ μ_{2} < μ_{1} < 1. We can find δ > 0 such that:

δ < μ_{1} − μ_{2} < 1 − δ.

Let ϕ_{2} be a C^{∞} vector for π_{2} . In the space L^{2}(S^{1}), ϕ_{2} is the sum of its Fourier series:

ϕ_{2} = ∑_{n∈ℤ} c_{n}θ_{n}.

where θ_{n}(t) = e^{int}.

Let A be the element −i(\frac{H}{β} + iμ_{1})^{2} in U(𝔤). We get:

\begin{aligned}\frac{1}{i}⟨dπ_{2}(A)φ_{2}, φ_{2}⟩ & = ⟨− ∑_{n} c_{n} (i(−μ_{2} − n) + iμ_{1})^{2} θ_{n}, ∑_{m} c_{m}θ_{m}⟩ \\ & = ∑_{n} |c_{n}|^{2}(μ_{1} − μ_{2} − n)^{2}. \end{aligned}

But: |μ_{1} − μ_{2} − n| > δ thus:

\frac{1}{i}⟨dπ_{2}(A)φ_{2}, φ_{2} >≥ δ^{2}⟨ϕ_{2}, ϕ_{2}⟩,

for all ϕ_{2} , in H_{π_{2}}^{∞}. The same inequality holds for any convex combination ∑_{r} ν_{r}Ψ_{π_{2}}(ϕ_{2}^{(r)}) of points in the moment set of π_{2} , i.e.

(∑_{r} ν_{r}Ψ_{π_{2}}(ϕ_{2}^{(r)})) (A) = ∑_{r} ν_{r} \frac{1}{i}⟨dπ_{2}(A)φ_{2}^{(r)}, φ_{2}^{(r)}⟩ ≥ δ^{2} ∑_{r} ν_{r} = δ^{2}.

On the other hand, θ_{0} is a C^{∞} vector of the representation π_{1} and we have:

(Ψ_{π_{1}}(θ_{0})) (A) = \frac{1}{i}⟨dπ_{1}(A)θ_{0}, θ_{0}⟩ = 0.

Then, our hypothesis μ_{1} ≠ μ_{2} does not hold.

Moreover, we remark that:

⟨dπ_{1}(u)θ_{0}, θ_{0}⟩ = ⟨dπ_{2}(u)θ_{0}, θ_{0}⟩

for all u in U(𝔤). Indeeed, since the moment sets of π_{1} and π_{2} coincide, then Ψ_{π_{1}}(θ_{0}) is a convex combination:

Ψ_{π_{1}}(θ_{0}) = ∑_{r} ν_{r}Ψ_{π_{2}}(ϕ_{2}^{(r)})

of points in J(π_{2}), the only possibility is ϕ_{2}^{(r)} = c_{r0}θ_{0} for all r, or

Ψ_{π_{1}}(θ_{0}) = Ψ_{π_{2}}(θ_{0}).

Lemma 5.2. (The fifth type classes) Let π_{1} and π_{2} be two unitary irreducible representations of G in a fifth type class. Suppose that J(π_{1}) = J(π_{2}), then: π_{1} ≃ π_{2} .

Proof. As in section three, choose the positif integers q(b) such that:

β = ∑_{b,ζ_{j1}^{(i)}(ib)≠0} q(b)b.

Define now:

B = \frac{1}{∏_{b} |ζ_{j1}^{(i)}(ib)|^{q(b)}} ∏_{b} Z_{j1}(b)^{q(b)}.

If ζ_{j1}^{(i)}(b) = |ζ_{j1}^{(i)}(b)|e^{iα_{j}^{(i)}(b)} we get:

(dπ_{i}(Z_{j1}(b)^{q(b)})ϕ_{i}) (t) = |ζ_{j1}^{(i)}(b)|^{q(b)}e^{iq(b)(α_{j}^{(i)}(b)− \frac{1}{β} tb)}ϕ_{i}(t).

Thus:

(dπ_{i}(B)ϕ_{i}) (t) = e^{i ∑_{b} q(b)α_{j}^{(i)}(b)}e^{−it}ϕ_{i}(t)

and we get two orthogonal basis of L^{2}(S^{1}):

ω_{0}^{(i)} = θ_{0}, ω_{n}^{(i)} = dπ_{i}(B^{n})θ_{0} (n > 0), ω_{n}^{(i)} = dπ_{i}(\overline{B^{−n}})θ_{0} (n < 0).

This defines an intertwining operator for π_{1} and π_{2} , for instance, if n is positive, then, for all u in U(𝔤),

⟨dπ_{1}(u)ω_{n}^{(1)}, ω_{n}^{(1)}⟩ = ⟨dπ_{1}(u)dπ_{1}(B^{n})θ_{0}, dπ_{1}(B^{n})θ_{0}⟩
= ⟨dπ_{1}(\overline{B^{n}}uB^{n})θ_{0}, θ_{0}⟩
= ⟨dπ_{2}(\overline{B^{n}}uB^{n})θ_{0}, θ_{0}⟩
= ⟨dπ_{2}(u)ω_{n}^{(2)}, ω_{n}^{(2)}⟩.

Replacing B by \overline{B}, we get the same result for negative n. By polarization, for all n, m,

⟨dπ_{1}(u)ω_{n}^{(1)}, ω_{m}^{(1)}⟩ = ⟨dπ_{2}(u)ω_{n}^{(2)}, ω_{m}^{(2)}⟩.

Now, let U be the unitary operator of L^{2}(S^{1}), defined by Uω_{n}^{(1)} = ω_{n}^{(2)} , we have

⟨dπ_{2}(u)ω_{n}^{(2)}, ω_{m}^{(2)}⟩ = ⟨dπ_{2}(u)Uω_{n}^{(1)}, Uω_{m}^{(1)}⟩
= ⟨(U^{−1} ∘ dπ_{2}(u) ∘ U)ω_{n}^{(1)}, ω_{m}^{(1)}⟩
= ⟨dπ_{1}(u)ω_{n}^{(1)}, ω_{m}^{(1)}⟩

or U^{−1} ∘ dπ_{2}(u) ∘ U = dπ_{1}(u), this shows that π_{1} and π_{2} are unitary equivalent. The corollary 4.2 and the lemma 5.2 prove our theorem 1.

Acknowledgments. This work was supported by the CMCU contract 00/F 1502. L. Abdelmoula and M. Selmi thank the Institut de Mathématiques de l’Université de Bourgogne and D. Arnal the Faculté des Sciences de Sfax for their kind hospitality during their stay.

References

[1] Auslander L., and B. Kostant, Quantization and representation of solvable Lie groups, Bull. Amer. Math. Soc. 73 (1967), 692–695.

[2] Arnal, D., and J. Ludwig, La convexité de l’application moment d’un groupe de Lie, Journal of Funct. Analysis 105 (1992), 256–300.

[3] Arnal, D., and J.C. Cortet, Star representation of E(2), Letters in Mathematical Physics, 20 (1990), 141–149.

[4] Arnal, D., A. Baklouti, J. Ludwig et M. Selmi, Separation of unitary representations of exponential Lie groups, Journal of Lie Theory, 10 (2000), 399–410.

[5] Baklouti, A., J. Ludwig et M. Selmi, Séparation des représentations unitaires des groupes de Lie nilpotents, in: H.-D. Doebner, V. K. Dobrev, and J. Hilgert, Eds., Proceedings of the II International Workshop, Clausthal, Germany, 17-20 August 1997, Lie theory and its applications in Physics II, Singapore/New Jersey/London: World Scientific, 1998, 75–91.

[6] Bernat, B., N. Conze, M. Duflo, M. Levy-Nahas, M. Raïs, P. Renouard, and M. Vergne, “Représentations des groupes de Lie résolubles”, Dunod, Paris, 1972.

[7] Wildberger, N. J., Convexity and unitary representations of nilpotent Lie groups, Invent. Math. 98 (1989), 281–292.

Lobna Abdelmoula and Mohamed Selmi Faculté de Sciences de Sfax Département de Mathématiques Route de Soukra, 3038 Sfax, Tunisie Mohamed.Selmi@fss.rnu.tn Didier Arnal Institut de Mathématiques de Bourgogne Université de Bourgogne U.F.R. Sciences et Techniques B.P. 47870, F-21078 Dijon Cedex, France didier.arnal@u-bourgogne.fr

Received April 10, 2003 and in final form October 4, 2003