Journal of Lie Theory Volume 14 (2004) 339–370 \begin{matrix}c \\ ◯ \end{matrix} 2004 Heldermann Verlag

Classification of Spherical Nilpotent Orbits in Complex Symmetric Space

Donald R. King

Communicated by F. Knop

Abstract. Let G be the adjoint group of the simple real Lie algebra 𝔤, and let K_{C} → Aut(𝔭_{C}) be the complexified isotropy representation at the identity coset of the corresponding symmetric space. We classify the spherical nilpotent K_{C} orbits in 𝔭_{C} .

1. Introduction

When L is a complex simple Lie group, the spherical nilpotent orbits for the adjoint action of L on its Lie algebra have been determined by Panyushev [22] and McGovern [15]. These orbits are significant in the study of the completely prime primitive ideals in the enveloping algebra of L. For example, McGovern has shown how to associate Dixmier algebras to spherical nilpotent orbits (and their covers) [15]. The Dixmier algebra associated to a spherical orbit has a nice structure owing to the fact that the co-ordinate ring of the orbit is multiplicity free as an L module, i.e., each irreducible finite dimensional representation of L occurs with multiplicity 0 or 1.

The goal of this paper is to classify completely the spherical nilpotent K_{C} -orbits in 𝔭_{C} , the complexified tangent space at the identity coset of the symmetric space formed by a simple group G (of adjoint type) and its maximal compact subgroup K. Here K_{C} is the complexification of K. This classification is contained in Theorems 6.1 (section 6.) and 9.1 (section 9.). Panyushev presented a partial classification of spherical nilpotent K_{C} -orbits in [21]. The classification presented here is especially significant since each spherical nilpotent K_{C} -orbit is diffeomorphic to a nilpotent G-orbit in the Lie algebra of G that is multiplicity free as a Hamiltonian K -space [11].

Many specific spherical nilpotent orbits have been investigated. If G is simple, the number of non-zero minimal nilpotent K_{C} -orbits in 𝔭_{C} is either 1 or 2. These orbits are spherical and have been studied extensively by representation theorists, notably [25]. Other spherical nilpotent K_{C} -orbits are studied in [1] and [16]. One expects spherical nilpotent orbits to play an increasingly prominent role in the representation theory of real simple Lie groups.

ISSN 0949–5932 / $2.50 \begin{matrix}c \\ ◯ \end{matrix} Heldermann Verlag

2. Basic Notation

Throughout this article, we assume that 𝔤 is a real simple Lie algebra with Cartan decomposition 𝔤 = 𝔨 ⊕ 𝔭. θ is the associated Cartan involution. Let 𝔤_{C} , 𝔨_{C} and 𝔭_{C} denote the complexifications of 𝔤, 𝔨 and 𝔭 respectively. θ extends to a complex linear involution on 𝔤_{C} . Let σ denote conjugation on 𝔤_{C} relative to the real form 𝔤. G_{C} is the adjoint group of 𝔤_{C} . G, K, and K_{C} are the connected subgroups of G_{C} corresponding to the Lie algebras 𝔤, 𝔨, and 𝔨_{C} , respectively. Gθ_{C} is the subgroup of G_{C} which is fixed by θ.

3. Kostant-Sekiguchi correspondence

In order to define the Kostant Sekiguchi correspondence, we consider the adjoint actions of G_{C} on 𝔤_{C} , G on 𝔤 and K_{C} on 𝔭_{C} .

Definition 3.1. Let N [𝔤], N [𝔤_{C}], and N [𝔭_{C}] denote the set of nilpotent elements of 𝔤, 𝔤_{C} and 𝔭_{C} respectively. N [𝔤]/G, N [𝔤_{C}]/G_{C} , and N [𝔭_{C}]/K_{C} will denote the orbits (conjugacy classes) in N [𝔤], N [𝔤_{C}], and N [𝔭_{C}] under G, G_{C} and K_{C} respectively.

The Kostant-Sekiguchi correspondence is a special bijection between N [𝔤]/ G and N [𝔭_{C}] / K_{C} . It is defined by means of sl(2)-triples.

Definition 3.2. An ordered triple {Z_{1}, Z_{2}, Z_{3}} of elements in 𝔤_{C} is said to be an sl(2)-triple if the following commutation relations are satisfied:

[Z_{1}, Z_{2}] = 2Z_{2}, [Z_{1}, Z_{3}] = −2Z_{3}, and [Z_{2}, Z_{3}] = Z_{1}.

Two sl(2)-triples {Z_{1}, Z_{2}, Z_{3}} and {Z′_{1}, Z′_{2}, Z′_{3}} are said to be conjugate under a subgroup W of G_{C} if there exists an element w ∈ W such that Z_{i} = w · Z′_{i} for i = 1, 2, 3. (“·” denotes the adjoint action.)

Using the Jacobson-Morosov Theorem, one can prove the following characterization of N [𝔤]/G.

Theorem 3.3. [3] There is a bijection between each of the following sets.

(1) G conjugacy classes of sl(2)-triples of 𝔤

(2) N [𝔤]/G.

The sl(2)-triple {Z_{1}, Z_{2}, Z_{3}} is said to be normal if Z_{1} ∈ 𝔨_{C} , and Z_{2}, Z_{3} ∈ 𝔭_{C} . Normal sl(2)-triples are at the heart of the Kostant-Rallis description of N [𝔭_{C}] / K_{C} .

Theorem 3.4. [13] There is a bijection between each of the following sets.

(1) K_{C} conjugacy classes of normal sl(2)-triples of 𝔤_{C}

(2) N [𝔭_{C}]/K_{C}.

We need to define two notable classes of sl(2)-triples.

Definition 3.5. (Kostant-Sekiguchi triples) An sl(2)-triple {H, E, F } in 𝔤 is said to be a KS-triple in 𝔤 if θ(E) = −F, and hence θ(H) = −H. A normal sl(2)-triple {x, e, f} in 𝔤_{C} is said to be a KS-triple in 𝔤_{C} if f = σ(e).

Sekiguchi established the following facts about KS-triples.

Theorem 3.6. [23] (1) Every sl(2)-triple {H′, E′, F′} in 𝔤 is conjugate under G to a KS-triple in 𝔤. Two KS-triples in 𝔤 are conjugate under G to the same sl(2)-triple in 𝔤 if and only if the KS-triples are conjugate under K.

(2) Every normal sl(2)-triple {x′, e′, f′} in 𝔤_{C} is conjugate under K_{C} to a KS-triple in 𝔤_{C} . Two KS-triples in 𝔤_{C} are conjugate under K_{C} to the same normal sl(2)-triple in 𝔤_{C} if and only if the KS-triples are conjugate under K.

Definition 3.7. Let KS(𝔤) denote the set of KS-triples in 𝔤 and KS(𝔤_{C}) denote the set of KS-triples in 𝔤_{C} . KS(𝔤)/K and KS(𝔤_{C})/K will denote the set of K conjugacy classes in KS(𝔤) and KS(𝔤_{C}) respectively.

Combining Theorems 3.4, 3.3 and 3.6 we have:

Theorem 3.8. There are bijections:

N [𝔤]/G ⟷ KS(𝔤)/K and N [𝔭_{C}]/K_{C} ⟷ KS(𝔤_{C})/K.

The Kostant-Sekiguchi correspondence is a consequence of Theorem 3.8 and the following observation.

Proposition 3.9. There is a bijection: KS(𝔤)/K ⟷ KS(𝔤_{C})/K defined as follows: {H, E, F } ∈ KS(𝔤) is mapped to the {x, e, f} ∈ KS(𝔤_{C}), where

x = i(E − F), e = \frac{E + F + iH}{2} , f = \frac{E + F − iH}{2} .  (1)

The map just defined is K -equivariant.

Let Ω be a conjugacy class in N [𝔤]/G. Let {H, E, F } = {H_{Ω}, E_{Ω}, F_{Ω}} be a representative of the conjugacy class in KS(𝔤) that is associated to Ω by Theorem 3.8. Then set

S(Ω) =^{def} K_{C} · \frac{E + F + iH}{2} .  (2)

We obtain our main result.

Theorem 3.10. (The Kostant-Sekiguchi Correspondence [23]) The mapping S : N [𝔤]/G ⟶ N [𝔭_{C}]/K_{C} , given by Ω ↦ S(Ω) =^{def} O_{Ω} (see formula (2)) is a bijection.

Proof. Combine Theorem 3.8 and Proposition 3.9.

4. Results about spherical nilpotents in 𝔭_{C}

We need several results (mostly due to Panyushev) in order to state necessary and sufficient conditions for O to be K_{C} spherical. Fix a KS-triple {x, e, f} in 𝔤_{C} with e ∈ O. Thus x ∈ i𝔨, σ(e) = f. It follows that the complex subalgebra 𝔞_{C} = Cx + Ce + Cf has a θ-stable real form 𝔞 ⊂ 𝔤. Let 𝔤_{C}(j), 𝔨_{C}(j), and 𝔭_{C}(j) denote the j -eigenspace of ad(x) on 𝔤_{C} , 𝔨_{C} , and 𝔭_{C} respectively.

Since σ(x) = −x, and 𝔨_{C} and 𝔭_{C} are preserved by σ, for all j we have

dim_{C} 𝔨_{C}(j) = dim_{C} 𝔨_{C}(−j); dim_{C} 𝔭_{C}(j) = dim_{C} 𝔭_{C}(−j).

Definition 4.1. We define 𝔨_{C} -height(e) (resp., 𝔭_{C} -height(e)) to be the largest non-negative integer j such that 𝔨_{C}(j) ≠ (0) (resp., 𝔭_{C}(j) ≠ (0)). height(e) is the largest non-negative integer j such that 𝔤_{C}(j) ≠ (0).

Definition 4.2. Let 𝔲 denote the sum of the positive eigenspaces of ad(x) on 𝔤_{C} . Set Z = 𝔲 ∩ 𝔨_{C}/(𝔲 ∩ 𝔨_{C})^{e}.

Lemma 4.3. (1) For each i ≥ 0, as a K_{C}^{{x, e, f}} module, 𝔨_{C}(i)/(𝔨_{C}(i))^{e} is isomorphic to [f, 𝔭_{C}(i + 2)]; and

(2) As K_{C}^{{x, e, f}} modules, Z and ∑_{i≥1} 𝔭_{C}(i + 2) = ∑_{i≥3} 𝔭_{C}(i) are isomorphic.

Proof. For (1) apply the representation theory of sl(2, C). (2) follows from (1).

Let u = 𝔨 + i𝔭. Then u is a compact real form of 𝔤_{C} . τ = σ ∘ θ is the conjugation on 𝔤_{C} with respect to u. Let U be the connected subgroup of G_{C} with Lie algebra u.

Definition 4.4. Let B denote the Killing form of 𝔤_{C} and B_{𝔤} denote the restriction of B to 𝔤. Set ⟨z, w⟩ = −B_{𝔤_{C}}(z, τ(w)). Then ⟨·, ·⟩ is a U invariant, positive definite Hermitian inner product on 𝔤_{C} . If z ∈ 𝔤_{C} , set ∥z∥^{2} = ⟨z, z⟩. Note that if z, w ∈ 𝔤, then ⟨z, w⟩ = −B_{𝔤}(z, θ(w)).

Let 𝔪 be the orthogonal complement of 𝔨_{C}^{{x, e, f}} (relative to ⟨·, ·⟩) inside 𝔨_{C}^{x}. 𝔪 is a K_{C}^{{x, e, f}} module. Note that 𝔪 and 𝔭_{C}(2) are isomorphic as K_{C}^{{x, e, f}} modules.

Recall from [20] the notion of a stabilizer in general position (s.g.p.) for the action of an algebraic group on an irreducible variety.

Definition 4.5. We fix S to be an s.g.p. for the representation of K_{C}^{{x, e, f}} on 𝔪.

That is S is the stabilizer in K{x, e, f}_{C} of a point whose orbit under K{x, e, f}_{C} has maximal dimension. Such a point lies in an open subset of 𝔪 such that the stabilizers of any two points in this subset are conjugate under K_{C}^{{x, e, f}}. Since K_{C}^{{x, e, f}} is reductive, a generic K_{C}^{{x, e, f}} orbit on 𝔪 is closed, so that S is reductive. Also,

dim K_{C}^{{x, e, f}} − dim S = dim orbit of maximal dimension of K_{C}^{{x, e, f}} on 𝔪.

Let 𝔰_{C} denote the Lie algebra of S. Since 𝔰_{C} is stable under σ and τ, it is the complexification of a Lie subalgebra 𝔰_{R} which is contained in 𝔨^{{x, e, f}}. Let S_{R} denote the corresponding connected compact subgroup of K^{{x, e, f}}. S_{0} denotes the identity component of S. B(S) will denote a Borel subgroup of S.

Suppose that X is a variety with K_{C} action and B_{k} is a Borel subgroup of K_{C} .

Definition 4.6. The complex codimension of a generic B_{k} orbit is called the complexity of X, denoted c_{K_{C}}(X) or c(X) (when the reductive group K_{C} is understood).

Remark 4.7. c(X) is also the transcendence degree (over C) of the B_{k} invariant functions in the field of rational functions (with complex coefficients) on X.

Definition 4.8. If X is irreducible, we say that X is spherical for K_{C} if c(X) = 0. That is, some (and hence any) Borel subgroup of K_{C} has a dense orbit on X.

Remark 4.9. If \overline{X} is the Zariski closure of X, then c(\overline{X}) = c(X).

Corollary 4.10. If e ∈ N [𝔭_{C}], then K_{C} · e is spherical ⟺ \overline{K_{C} · e} is spherical.

Proposition 4.11. If e ∈ N [𝔭_{C}] and K_{C} · e is spherical so is each K_{C} orbit in \overline{K_{C} · e}.

Proof. This follows from Corollaire 3.5 in [2].

Proposition 4.12.

c_{K_{C}}(K_{C} · e) = c_{K_{C}^{x}}(Kx_{C}/K_{C}^{{x, e, f}}) + c_{S}(Z).

Proof. This is Theorem 1.2(a) of [22]. (See also Theorem 2.3(a) of [22].)

Corollary 4.13. O is spherical if and only if Kx_{C}/K_{C}^{{x, e, f}} is spherical and a Borel subgroup of S has an open orbit on Z.

Proof. This is Corollary 1.4 of [22].

Lemma 4.14. Suppose 𝔨_{C} -height(e)≤ 3. Then 𝔨_{C}^{{x, e, f}} is a symmetric subalgebra of 𝔨_{C}^{x}.

Proof. This is proved like Proposition 3.3 in [22].

Corollary 4.15. Let {x, e, f} be a normal triple such that height(e)= 2. Then O is spherical for K_{C}.

Proof. Since height(e)= 2 and Z = (0), apply Lemma 4.14 and Corollary 4.13.

Since all spherical nilpotent G_{C} -orbits for 𝔤_{C} of type A_{n} or C_{n} have height 2 (see [22]), we have the following result.

Corollary 4.16. Suppose that 𝔤_{C} is of type A_{n} or C_{n}. If Θ is a spherical nilpotent orbit for G_{C}, and O is a K_{C} orbit in Θ ∩ 𝔭_{C} , then O is spherical for K_{C}.

Panyushev has shown that for a nilpotent e in 𝔤_{C} , height(e)≥ 4 implies that G_{C} · e is not spherical. In addition he has shown that:

Proposition 4.17. If 𝔭_{C} -height(e) > 3, or 𝔨_{C} -height(e) > 4, then K_{C} · e is not spherical.

Proof. This is equivalent to Theorem 5.6 in [21].

Remark 4.18. If O is spherical for K_{C} , then dim_{C} O ≤ dim_{C} B(K_{C}), where B(K_{C}) denotes any Borel subgroup of K_{C} .

Remark 4.19. If e, e′ ∈ N [𝔭_{C}] are conjugate under Gθ_{C} , then K_{C} · e is spherical ⟺ K_{C} · e′ is spherical.

5. Parametrizing nilpotent K_{C} -orbits in 𝔭_{C}

Suppose that 𝔤 is a simple classical real Lie algebra. Because of the Kostant-Sekiguchi correspondence, we will generally use the signed partition description of nilpotent conjugacy classes in N [𝔤]/G (see [3]) to describe the conjugacy classes in N [𝔭_{C}]/K_{C} . The signed partition description is equivalent to the description in terms of “ab − diagrams” given by Ohta ([19]) and others.

Throughout this section and the next {x, e, f} is a KS-triple in 𝔤_{C} . We assume that x ∈ i𝔱, where 𝔱 is a fixed maximal torus of 𝔨. 𝔞_{C} is the sl(2, C) algebra spanned by triple {x, e, f}. For each 𝔤, we will give a recipe for computing x from the signed partition description of K_{C} · e. The idea behind the various recipes is as follows. For each 𝔤 there is a finite dimensional complex vector space V = V (𝔤) carrying the ‘natural’ or ‘basic’ representation of 𝔤. (If 𝔤 is not sl(s, R) or su^{∗}(2n), V is the complex vector space carrying the corresponding 𝔤-invariant bilinear form.) V is a completely reducible 𝔞_{C} -module. An ‘ab’-diagram for the conjugacy class of the nilpotent e ∈ 𝔭_{C} describes a basis for V consisting of eigenvectors of the neutral element x. The eigenvalues of x on this basis determine the values taken by a system of simple roots of 𝔨_{C} on the element x. This gives the weighted Dynkin diagram of x which determines the orbit K_{C} · e. In each case, x is dominant with respect to the system of simple roots of 𝔨_{C} . The recipes below are undoubtedly known to many experts but, to the author’s knowledge, have not been published before. Proofs will be given only for 𝔤 = su(p, q) and 𝔤 = sp(n, R).

5.1. sl(s, R). Let s = 2n or 2n + 1. 𝔨 = so(s). Let E_{i, j} denote the s × s matrix whose (i, j) entry is 1 and whose other entries are 0. We define a torus 𝔱 of 𝔨 to be the real span of the matrices Z_{i} = E_{2i−1, 2i} − E_{2i, 2i−1}, i = 1, …, n. Then the linear functional e_{j}, j = 1, …, n is defined by e_{j}(Z_{i}) = −\sqrt{−1}δ_{ij} (Kronecker delta).

If s = 2n + 1, we label the following set of simple roots for 𝔨:

e_{1} − e_{2}, e_{2} − e_{3}, …, e_{n−1} − e_{n}, e_{n}.

For s = 2n, 𝔨 = so(2n). We label the following set of simple roots for 𝔨:

e_{1} − e_{2}, e_{2} − e_{3}, …, e_{n−2} − e_{n−1}, e_{n−1} − e_{n}, e_{n−1} + e_{n}.

Proposition 5.1. (𝔤 = sl(2n + 1, R), or sl(n. R)) Here is the algorithm for determining the weighted Dynkin diagram of x. Let Λ = m_{1} + m_{2} + m_{3} + … m_{r} be the partition of 2n + 1 or 2n determined by e, with m_{1} ≥ m_{2} ≥ … ≥ m_{r}. Each occurrence of m_{j} corresponds to an m_{j} × m_{j} Jordan block, that is an m_{j} - dimensional 𝔞_{C} -module with basis: v, e · v, e^{2} · v, …, e^{m_{j}−1} · v for some v ∈ V . The eigenvalues of x in this basis are the integers: (m_{j} − 1), …, −(m_{j} − 1).

Case (1) Assume that not all the m_{j} are even. Form the multiset A_{Λ} = A_{Λ}(K_{C} · e) by “joining” all such sequences (m_{j} − 1), …, −(m_{j} − 1) for a given partition of 2n + 1 or 2n. Assume that the elements of A_{Λ} are arranged in descending order. Take the first n non-negative integers from A_{Λ}. These are (respectively) the values e_{1}(x), …, e_{n}(x), which give the weighted Dynkin diagram of x.

Case (2) Assume that all the m_{j} are even. This is possible only for 𝔤 = sl(2n, R). Λ corresponds to two distinct K_{C} nilpotent orbits that are conjugate under Gθ_{C} . To record this fact we label one copy of the partition Λ with a roman numeral “I” and a second copy of Λ with roman numeral “II” (Theorem 9.3.3 in [3]) to distinguish the orbits. Applying the procedure in Case (1) we obtain a weighted Dynkin diagram in which e_{n}(x) = 1. We associate this weighted diagram to the orbit Λ_{I}. We then use the values e_{1}(x), … e_{n−1}(x), e_{n}(x) = −1 to form a second weighted Dynkin diagram. This diagram is assigned to orbit Λ_{II}.

Example 5.2. (a) Λ = 4 + 3 + 2 for sl(9, R). The sequences corresponding to the parts 4, 3, and 2 respectively {3, 1, −1, −3}; {2, 0, −2};and {1, −1}. Thus A_{Λ} = {3, 2, 1, 0, −1, −2, −3}. So e_{1}(x) = 3, e_{2}(x) = 2, e_{3}(x) = 1 and e_{4}(x) = 0. Hence, we obtain the following labels on the simple roots of 𝔨 = so(7): e_{1} − e_{2} = 1, e_{2} − e_{3} = 1, e_{3} − e_{4} = 1, e_{4} = 0.

(b) Λ = 4 + 2 + 2 + 2 for sl(10, R). The sequences corresponding to the parts 4 and 2 are respectively {3, 1, −1, −3} and {1, −1}. Thus,

A_{Λ} = {3, 1, 1, 1, 1, −1, −1, −1, −1, −3}.

So the orbit Λ_{I} has e_{1}(x) = 3, e_{2}(x) = 1, e_{3}(x) = 1, e_{4}(x) = 1, and e_{5}(x) = 1. The orbit Λ_{II} has e_{1}(x) = 3, e_{2}(x) = 1, e_{3}(x) = 1, e_{4}(x) = 1, and e_{5}(x) = −1.

5.2. su^{∗}(2n). We use the notation of Helgason [10], chapter 10 , section 2 for

𝔤 and 𝔨. That is, 𝔤 is the space of 2n × 2n matrices of the form (\begin{matrix}Z_{1} & Z_{2} \\ −Z̄_{2} & Z̄_{1} \end{matrix}) where Z_{1} and Z_{2} are n × n complex matrices and T r(Z_{1}) + T r(Z̄_{1}) = 0. (‘Z̄’ denotes complex conjugation.) 𝔨 is the subspace of skew Hermitian matrices in 𝔤. 𝔨 is isomorphic to sp(n). The space of diagonal matrices in 𝔨 is a maximal torus 𝔱 of 𝔨. The linear functional e_{i} (1 ≤ i ≤ n) on 𝔱 is defined so that its value on the diagonal matrix diag(\sqrt{−1}y_{1}, …, \sqrt{−1}y_{n}, −\sqrt{−1}y_{1}, …, −\sqrt{−1}y_{n}) is \sqrt{−1}y_{i}. Then we have the following set π of simple roots for 𝔨:

e_{1} − e_{2}, e_{2} − e_{3}, …, e_{n−1} − e_{n}, 2e_{n}

Nilpotent conjugacy classes are parametrized by partitions of n.

Proposition 5.3. Suppose Λ = m_{1} + m_{2} + m_{3} + … + m_{r} is a partition of n (m_{1} ≥ m_{2} ≥ … ≥ m_{r}). ”Double” Λ to obtain a partition Λ′ of 2n. That is

Λ′ = m_{1} + m_{1} + m_{2} + m_{2} + m_{3} + +m_{3} + … + m_{r} + m_{r}

Form the multiset A_{Λ′} by joining the integer strings corresponding to each of the integers m_{1}, m_{1}, …, m_{r}, m_{r}. Arrange the integers in A_{Λ′} in descending order and choose the first n non-negative integers in this list. These integers are the values e_{1}(x), e_{2}(x), …, e_{n}(x).

5.3. su(p, q). Let n = p + q and V = C^{n}. Let z = (z_{1}, …, z_{n}) and w = (w_{1}, …, w_{n}) be n-tuples in V . Consider the Hermitian form

⟨z, w⟩_{p, q} = z_{1}w̄_{1} + … + z_{p}w̄_{p} − z_{p+1}w̄_{p+1} − … − z_{p+q}w̄_{p+q}.

(‘z̄’ denotes the complex conjugate of z.) su(p, q) consists of all the trace zero n × n complex matrices which leave the form ⟨·, ·⟩ invariant. Let I_{p, q} be the block diagonal matrix diag(I_{p}, −I_{q}) where I_{p} (resp., I_{q}) is the p×p (resp., q ×q) identity matrix. Let V^{+} (resp. V^{−}) be the +1 (resp. −1) eigenspace of I_{p, q}. 𝔨 = su(p)⊕su(q)⊕T_{1}. Since the restriction of ⟨·, ·⟩ is positive definite on V^{+} and negative definite on V^{−}, 𝔨 can be viewed as trace zero matrices which preserve V^{+} and V^{−}, which we denote s(u(V^{+}) ⊕ u(V^{−})).

Let E_{i, j} denote the n × n matrix defined in subsection 5.. We define a torus 𝔱 of 𝔨 to be the real span of the matrices H_{i} = \sqrt{−1}(E_{i, i} − E_{i+1, i+1}), i = 1, …, n − 1. Then the linear functional e_{j}, for j = 1, …, n, is defined by e_{j}(\sqrt{−1}E_{i, i}) = δ_{ij}.

The neutral element of a nilpotent conjugacy class will be described by giving its values at the following compact simple roots of 𝔨 and at the noncompact root −ψ = −(e_{1} − e_{p+q}):

{e_{1} − e_{2}, e_{2} − e_{3}, …, e_{p−1} − e_{p}} ∪ {e_{p+1} − e_{p+2}, …, e_{p+q−1} − e_{p+q}} (3)

Nilpotent conjugacy classes are parametrized by signed partitions of signature (p, q). See [3].

Proposition 5.4. Let Λ be a signed partition of signature (p, q). See [3]. Let m_{1}, …, m_{d} be the distinct part sizes of Λ arranged in descending order. Let r_{j}^{+} (resp., r_{j}^{−}) be the number of times m_{j} occurs labelled with a ‘+’ (resp., ‘-’) sign.

That is, r_{j}^{+} (resp., r_{j}^{−}) denotes the number of rows of length m_{j} in Λ which begin with ‘+’ (resp., ‘−’). We can write

Λ = (+m_{1})^{r_{1}^{+}}(−m_{1})^{r_{1}^{−}} … (+m_{d})^{r_{d}^{+}}(−m_{d})^{r_{d}^{−}}.

We form two multisets: A_{Λ}^{p} and B_{Λ}^{q}, by performing the following procedure on each row of Λ. Suppose λ is a row of Λ of length m_{j}. Label the first sign in λ with the integer m_{j} − 1, the next sign with the integer m_{j} − 3, etc. The last sign in λ is labelled with the integer −(m_{j} − 1). Each integer labelling a plus sign in λ is placed in A_{Λ}^{p} and each integer labelling a minus sign in λ is placed in B_{Λ}^{q}. By arranging the elements of A_{Λ}^{p} in descending order, we obtain the integers e_{1}(x), …, e_{p}(x). By arranging the elements of B_{Λ}^{q} in descending order, we obtain the integers e_{p+1}(x), …, e_{p+q}(x).

Proof. (Sketch) Each row of type (+m_{j}) in Λ corrresponds to an 𝔞_{C} -module W_{j}^{+} (of dimension m_{j}) with basis: v, e · v, e^{2} · v, …, e^{m_{j}−1} · v for some v ∈ V with e^{m_{j}−1} · v ∈ V^{+}. (And e^{m_{j}−2} · v ∈ V^{−}, e^{m_{j}−3} · v ∈ V^{+}, etc.) Each row of type (−m_{j}) corrresponds to a basis of an m_{j} -dimensional 𝔞_{C} -module W_{j}^{−} with basis: v, e · v, e^{2} · v, …, e^{m_{j}−1} · v for some v ∈ V with e^{m_{j}−1} · v ∈ V^{−}. (And e^{m_{j}−2} · v ∈ V^{+}, e^{m_{j}−3} · v ∈ V^{−}, etc.) The bases corresponding to different rows of Λ may be taken to be orthogonal with respect to ⟨·, ·⟩. Since x ∈ i𝔨 has real eigenvalues and ⟨·, ·⟩ is conjugate linear in the second variable, eigenvectors in V^{+} (resp. V^{−}) for x with distinct eigenvalues are mutually orthogonal relative to the restriction of ⟨·, ·⟩ to V^{+} (respectively V^{−}). Thus by suitable normalization of the vectors in the bases for each row of Λ, we can create an orthonormal basis of each eigenspace in V^{+} (using the form ⟨·, ·⟩) and an orthonormal basis of each eigenspace of V^{−} (using the form -⟨·, ·⟩). In this way we obtain orthonormal bases of V^{+} and V^{−}. The integers in A_{Λ}^{p} (resp., B_{Λ}^{q}) give the eigenvalues of x in an orthonormal basis of V^{+} (resp., V^{−}). We can find an element of K which transforms these orthonormal bases into the standard orthonormal bases of V^{+} and V^{−}. k · x is a diagonal matrix with respect to the new bases. The integers in A_{Λ}^{p} occupy the first p entries along the main diagonal and those in B_{Λ}^{q} occupy the last q positions. Using the Weyl group of K we can rearrange these entries so that k · x becomes dominant with respect to the simple system in equation (3). This establishes the proposition.

Example 5.5. (a) 𝔤 = su(3, 1) and Λ = _{+}^{+ − +} = (+3)(+1)

This is the principal nilpotent. The weighted Dynkin diagram is obtained by evaluating the neutral element at the compact simple roots e_{1} − e_{2}, e_{2} − e_{3} and at −ψ = e_{4} − e_{1}. The integers for the first row of Λ are 2, 0, −2. So 2 and −2 are labeled with +, and 0 is labelled with −. The 0 in the second row of Λ is labelled ‘+’. Therefore, A_{Λ} = {2, −2} and B_{Λ} = {0, 0}. This gives e_{1}(x) = 2, e_{2}(x) = 0, e_{3}(x) = −2, and e_{4}(x) = 0. The weighted Dynkin diagram of x is given by e_{1} − e_{2} = 2, e_{2} − e_{3} = 2, and e_{4} − e_{1} = −2.

(b)(i) 𝔤 = su(2, 2) and Λ = ^{+ − + −} = (+4)

This is one of the principal nilpotents for 𝔤. A_{Λ} = {3, −1} and B_{Λ} = {1, −3}. Therefore, e_{1}(x) = 3, e_{2}(x) = −1, e_{3}(x) = 1, and e_{4}(x) = −3. The weighted Dynkin diagram of x is given by e_{1}−e_{2} = 4, e_{3}−e_{4} = 4, and e_{4}−e_{1} = −6.

(b)(ii) 𝔤 = su(2, 2) and Λ = ^{− + − +} = (−4)

This is the other principal nilpotent for 𝔤. A_{Λ} = {1, −3} and B_{Λ} = {3, −1}. Therefore, e_{1}(x) = 1, e_{2}(x) = −3, e_{3}(x) = 3, and e_{4}(x) = −1. The weighted Dynkin diagram of x is given by e_{1}−e_{2} = 4, e_{3}−e_{4} = 4, and e_{4}−e_{1} = −2.

5.4. so(p, q). so(p, q) is the Lie subalgebra of sl(p+q, R) which leaves invariant the quadratic form:

((x_{1}, …, x_{p+q}), (x_{1}, …, x_{p+q}))_{p, q} = x_{1}^{2} + … x_{p}^{2} − x_{p+1}^{2} − … − x_{p+q}^{2}.

Thus 𝔤 is the space of real matrices of the form (\begin{matrix}A & B \\ B^{⊤} & D \end{matrix}) where A is p × p skew

symmetric, D is q × q skew symmetric, B is p × q and B^{⊤} denotes the transpose

of B. 𝔨 is the subspace of matrices of the form (\begin{matrix}A & 0 \\ 0 & D \end{matrix}) where A is p × p skew

symmetric and D is q × q skew symmetric. So 𝔨 is isomorphic to so(p) ⊕ so(q).

Let I_{p, q}, V^{+} and V^{−} be defined in as subsection 5.. Let s = [\frac{p}{2}] and t = [\frac{q}{2}]. Let E_{i, j} denote the (p + q) × (p + q) matrix defined in subsection 5.. Let 𝔱_{1} be the real span of the matrices Y_{i} = E_{2i−1, 2i}−E_{2i, 2i−1}, i = 1, …, s, and 𝔱_{2} be the real span of the matrices Y′_{i} = E_{p+2i−1, p+2i} −E_{p+2i, p+2i−1}, i = 1, …, t. Then 𝔱 = 𝔱_{1} ⊕ 𝔱_{2} is a maximal torus of 𝔨. Define linear functionals e_{j}, j = 1, …, s + t on 𝔱, as follows.

\begin{matrix}e_{j}(Y_{i}) = −\sqrt{−1}δ_{ij}, e_{j}(𝔱_{2}) = 0, for 1 ≤ i ≤ s, 1 ≤ j ≤ s; \\ e_{j}(Y′_{i}) = −\sqrt{−1}δ_{i, j−s}, e_{j}(𝔱_{1}) = 0 for 1 ≤ i ≤ t, s + 1 ≤ j ≤ s + t. \end{matrix}

We specify π, the following system of simple of roots, for 𝔨 depending on the parity of p and q.

π = {e_{1} − e_{2}, …, e_{s−2} − e_{s−1}} ∪ {e_{s+1} − e_{s+2}, …, e_{s+t−2} − e_{s+t−1}} ∪ π′

where π′ equals

{e_{s−1} − e_{s}, e_{s−1} + e_{s}} ∪ {e_{s+t−1} − e_{s+t}, e_{s+t−1} + e_{s+t}} (p, q even); {e_{s−1} − e_{s}, e_{s−1} + e_{s}} ∪ {e_{s+t−1} − e_{s+t}, e_{s+t}} (p even, q odd); {e_{s−1} − e_{s}, e_{s}} ∪ {e_{s+t−1} − e_{s+t}, e_{s+t−1} + e_{s+t}} (p odd, q even); {e_{s−1} − e_{s}, e_{s}} ∪ {e_{s+t−1} − e_{s+t}, e_{s+t}} (p, q odd).

The nilpotent orbits of so(p, q). are parametrized by signed Young diagrams of signature (p, q) such that rows of even length occur with even multiplicity and have their leftmost boxes labeled ‘+’. Some of these diagrams get roman numerals attached to them as follows. If Λ is such a diagram and all the rows have even length, then Λ corresponds to two K_{C} orbits which are conjugate under Gθ_{C} . Λ_{I} and Λ_{II} will denote the two K_{C} orbits. (This situation is possible only if both p and q are even.) The distinction between Λ_{I} and Λ_{II} is given below in Proposition 5.6. If at least one row of Λ has odd length and all odd rows have an even number of boxes labeled ‘+’, or all odd rows have an even number of boxes labeled ‘-’, then again Λ_{I} and Λ_{II} denote the corresponding K_{C} orbits, which are conjugate under Gθ_{C} (This situation is possible only if at leas! t one of the integers p, q is even.) The distinction between Λ_{I} and Λ_{II} is given below in Proposition 5.6. Thus if p and q are both odd, no numerals are attached to any signed Young diagram.

Proposition 5.6. Let Λ be a signed partition of signature (p, q). Represent Λ using the notation of Proposition 5.4.

(A) Assume that Λ does not have a numeral. We define two multisets: A_{Λ}^{s} and B_{Λ}^{t} by following the rules below. By arranging the elements of A_{Λ}^{s} in descending order, we obtain the integers e_{1}(x), …, e_{s}(x). By arranging the elements of B_{Λ}^{t} in descending order, we obtain the integers e_{s+1}(x), …, e_{s+t}(x).

1. Suppose λ is an odd row (of length m + 1) of Λ with integer labelling:

m, m − 2, …, 2, 0, −2, … − m

In this case, the first integer m is even. Set |λ| equal to the number of integers in the string, |λ|_{+} (resp. |λ|_{−}) is the number of integers labelled with a “+” (resp. “-”) sign.

case(a) Assume that m is labelled by a plus sign. Then, |λ|_{+} = [\frac{m+1}{2}] + 1 and |λ|_{−} = [\frac{m−1}{2}] + 1.

Let c_{m}^{+} denote the number of rows identical to λ in Λ, i.e., the number of rows of length |λ| that begin with a ‘+’ sign. Arrange the c_{m}^{+}|λ|_{+} integers from these rows which are labelled with a ‘+’ sign in descending order, and assign the first [\frac{c_{m}^{+}|λ|_{+}}{2} ] of these integers to A_{Λ}^{s}. Likewise, arrange the c_{m}|λ|_{−} integers from these rows that are labelled with a ‘-’ sign in descending order, and assign the first [\frac{c_{m}^{+}|λ|_{−}}{2} ] to B_{Λ}^{t}.

case(b) Assume that m is labelled by a minus sign. Then, |λ|_{+} = [\frac{m−1}{2}] + 1 and |λ|_{−} = [\frac{m+1}{2}] + 1. Let c_{m}^{−} denote the number of rows identical to λ in Λ, i.e., the number of rows of length |λ| which begin with a ‘-’ sign. Arrange the c_{m}^{−}|λ|_{+} integers from these rows which are labelled with a ‘+’ sign in descending order, and assign the first [\frac{c_{m}^{−}|λ|_{+}}{2} ] of these integers to A_{Λ}^{s}. Likewise, arrange the c_{m}^{−}|λ|_{−} integers from these rows that are labelled with a ‘-’ sign in descending order, and assign the first [\frac{c_{m}^{−}|λ|_{−}}{2} ] of them to B_{Λ}^{t}.

2. Now suppose that λ is an even row with integer labelling m, m − 2, …,−m. Then m is odd. Suppose that there are 2c copies of λ in Λ.

Place c copies of the string: m, m − 2, …, 3, 1 in A_{Λ}^{s} and place c copies of the same string in B_{Λ}^{t}. For example, if c = 1, then assign the string

m, m − 2, m − 4, …, 3, 1.

to A_{Λ}^{s}, and the string

m, m − 2, m − 4, …, 3, 1.

to B_{Λ}^{t}.

3. If after performing the procedures in (1) and (2) on all rows, A_{Λ}^{s} has less that s elements (counted with multiplicity) place enough extra zeroes in A_{Λ}^{s} so that A_{Λ}^{s} has cardinality s. And if B_{Λ}^{t} has less that t elements (counted with multiplicity) place enough extra zeroes in B_{Λ}^{t} so that B_{Λ}^{t} has cardinality t.

We must now take into account the assignment of numerals to some of the signed partitions.

(B) Assume that Λ is a signed partition with roman numeral “I” or “II”. Then there are two possibilities.

1. If all rows of Λ are even then both p and q are even. If we apply the rules for forming A_{Λ}^{s} and B_{Λ}^{t} in part (A) of the proposition, we find that e_{s} = 1 and e_{s+t} = 1. We stipulate that the numeral “I” assigned to Λ will correspond to the unique neutral element coming from the integers e_{1}, …, e_{s+t}. We stipulate that the numeral “II” assigned to Λ will correspond to the unique neutral element coming from the integers e_{1}, …, e_{s−1}, −e_{s}, e_{s+1}, …, e_{s+t−1}, −e_{s+t}.

2. case(a) Suppose Λ has some odd rows, and all odd rows contain an even number of plus signs. Then p must be even. Apply the rules for forming A_{Λ}^{s} and B_{Λ}^{t} in part (A). These rules imply that e_{s} ∈ {1, 2} and e_{s+t} = 0. Label Λ with roman numeral “I”. Consider the neutral element x′ defined by e_{i}(x′) = e_{i} for i ≠ s and e_{s}(x′) = −e_{s}. It will correspond to Λ_{II}.

2. case (b) Suppose Λ has some odd rows, and all odd rows contain an even number of minus signs. Then q must be even. Apply the rules for forming A_{Λ}^{s} and B_{Λ}^{t} in part (A). These rules imply that e_{s} = 0 and e_{s+t} ∈ {1, 2}. Label Λ with roman numeral “I”. Consider the neutral element x′ defined by e′_{i}(x) = e_{i} for i ≠ s + t and e′_{s+t}(x) = −e_{s+t}. It will correspond to Λ_{II}.

The proof of Proposition 5.6 uses ideas similar to those in the proof of Proposition 5.4 and will be omitted.

5.5. so^{∗}(2n). We use the notation of Helgason [10]. 𝔤 is the space of

2n × 2n complex matrices of the form (\begin{matrix}Z_{1} & Z_{2} \\ −Z̄_{2} & Z̄_{1} \end{matrix}) where Z_{1} and Z_{2} are com-

plex n × n-matrices, Z_{1} is skew symmetric and Z_{2} is Hermitian. (‘ ’ denotes complex conjugation.) 𝔨 is the subspace of matrices in 𝔤 where Z_{1} and Z_{2} are real. (Thus Z_{2} is symmetric.) The real linear map from 𝔨 to the complex

n×n matrices given by (\begin{matrix}Z_{1} & Z_{2} \\ −Z̄_{2} & Z̄_{1} \end{matrix}) ↦ Z_{1} + \sqrt{−1}Z_{2} is a Lie algebra isomorphism

onto the space of n × n skew Hermitian matrices. Thus 𝔨 is isomorphic to u(n).

The subspace of matrices in 𝔨 of the form (\begin{matrix}0 & B \\ −B & 0 \end{matrix}), where B is diagonal, is a

maximal torus 𝔱 of 𝔨. If B = diag(y_{1}, …, y_{n}), the linear functional e_{i} (1 ≤ i ≤ n)

on 𝔱 is defined so that its value on the (\begin{matrix}0 & B \\ −B & 0 \end{matrix}) is \sqrt{−1}y_{i}.

𝔨 can also be realized as follows. Let (·, ·) be the usual symmetric form on C^{2n} and let

J = (\begin{matrix}0 & I_{n} \\ −I_{n} & 0 \end{matrix})  (4)

where I_{n} is the n × n identity matrix. Set V^{+} (resp., V^{−}) equal to the i (resp., −i) eigenspace of J. Then, the bilinear form ⟨v, w⟩ = (v, w̄) is a Hermitian inner product on V^{+}, and 𝔨 = u(V^{+}).

For 𝔤 = so^{∗}(2n), the nilpotent orbits are parametrized by signed Young diagrams of size n and any signature in which rows of odd length have their left most boxes labeled ‘+’.

The neutral element of a nilpotent conjugacy class will be described by giving its values at the following simple roots of 𝔨 = u(n) and at the noncompact root −ψ of 𝔤 (where ψ = e_{1} + e_{2}):

e_{1} − e_{2}, e_{2} − e_{3}, …, e_{n−1} − e_{n}, and − ψ = −e_{1} − e_{2}.

Proposition 5.7. Let Λ be a signed partition. Represent Λ using the notation of Proposition 5.4. We form a multiset A_{Λ}. Then, arrange the elements of A_{Λ} in descending order to obtain the integers: e_{1}(x), …, e_{n}(x).

To form A_{Λ}, we label the signs in each row of Λ with the appropriate integers and then proceed as follows.

Suppose λ is an even row of Λ. Then it must begin with a non-negative odd integer m. Suppose m labels a plus sign, then place \underline{two} copies of the integer string

m, m − 4, m − 8, …, −(m − 2)

in A_{Λ}. If m labels a minus sign, then place \underline{two} copies of the integer string

m − 2, m − 6, m − 10, …, −m

in A_{Λ}.

Now suppose that λ is an odd row and is labelled by the integers

m, m − 2, …, 0, −2, … − m

then place this entire string in A_{Λ}.

Proof. The argument is similar to that for Proposition 5.8 below and will be omitted.

5.6. sp(n, R). Let J be the matrix in (4). sp(n, C) is the space of 2n × 2n complex matrices X such that X^{⊤}J + JX = 0. We identify sp(n, R)) with the isomorphic Lie algebra su(n, n) ∩ sp(n, C), where su(n, n) is defined as in subsection 5.4. (See Chapter VI, section 10 of [12].). Using this identification, 𝔨 = sp(n, R) ∩ u(2n) which is isomorphic to u(n). Let 𝔱 be the space of matrices:

\{B = diag(\sqrt{−1}y_{1}, …, \sqrt{−1}y_{n}, −\sqrt{−1}y_{1}, …, −\sqrt{−1}y_{n})|y_{i} ∈ R\}.  (5)

Define the linear functionals e_{j} on 𝔱 by setting e_{j}(B) = \sqrt{−1}y_{j} where B is the diagonal matrix above.

The neutral element of a nilpotent conjugacy class will be described by giving its values at the following simple roots of 𝔨 = u(n) and at −ψ (where ψ = 2e_{1}):

e_{1} − e_{2}, e_{2} − e_{3}, …, e_{n−1} − e_{n}, and − ψ = −2e_{1}.

In addition, let V^{+} (resp. V^{−}) denote the +i (resp., −i) eigenspace of the matrix J. If {·, ·} is the skew symmetric form on C^{2n} which defines sp(n, C), then ⟨v, w⟩ = −\{v, \overline{Jw}\} is the standard positive definite skew Hermitian form on C^{2n}. We will identify 𝔨 with u(V^{+}), defined relative to the restriction of ⟨·, ·⟩ to V^{+}.

Nilpotent orbits are parametrized by signed Young diagrams of size 2n and any signature in which odd length rows appear with even multiplicity and begin with a ‘+’. Even length rows may begin with ‘+’ or ‘-’.

Proposition 5.8. Let Λ be a signed partition. Represent Λ using the notation of Proposition 5.4. We label the signs in each row of Λ with the appropriate integers. We form a multiset A_{Λ} as follows.

(1) Suppose λ is an even row of Λ. Then it must begin with a non-negative odd integer m. Suppose m labels a plus sign, then place the integer string

m, m − 4, m − 8, …, −(m − 2)

in A_{Λ}. If m labels a minus sign, then place the integer string

m − 2, m − 6, m − 10, …, −m

in A_{Λ}.

(2) Suppose that λ is an odd row with integer labelling

m, m − 2, …, 2, 0, −2, … − m

In this case, the first integer m is even, and labels a plus sign.

Suppose that there are exactly 2c copies of λ in Λ, then place c copies of the set {m, m − 2, …, 2, 0, −2, … − m} in A_{Λ}.

Arrange the elements of A_{Λ} in descending order to obtain the integers: e_{1}(x), …, e_{n}(x).

Proof. (Sketch) Suppose that λ is an even length row of type (+m_{j}), then the theory of ‘ab’-diagrams shows that there is an irreducible m_{j} -dimensional 𝔞_{C} - submodule W_{j}^{+} of V with basis: v, e · v, e^{2} · v, …, e^{m_{j}−1} · v for some v ∈ V with e^{m_{j}−1} · v ∈ V^{+} . We have v ∈ V^{−}, e · v ∈ V^{+}, etc. In addition,

\begin{matrix}(1)\{e^{a} · v, e^{b} · v\} = 0 if a + b ≠ m_{j} − 1. \\ (2)\{e^{a} · v, e^{m_{j}−1−a} · v\} = (−1)^{a}α_{m_{j}}, \end{matrix}

where α_{m_{j}} is a nonzero complex number depending on m_{j} alone.

If λ is an even length row of type (−m_{j}) then there is an irreducible m_{j} - dimensional 𝔞_{C} -submodule W_{j}^{−} of V with basis: v, e · v, e^{2} · v, …, e^{m_{j}−1} · v for some v ∈ V and e^{m_{j}−1} · v ∈ V^{−}. We have v ∈ V^{+}, e · v ∈ V^{−}, etc. In addition,

\begin{matrix}(1)\{e^{a} · v, e^{b} · v\} = 0 if a + b ≠ m_{j} − 1. \\ (2)\{e^{a} · v, e^{m_{j}−1−a} · v\} = (−1)^{a}β_{m_{j}}, \end{matrix}

where β_{m_{j}} is a nonzero complex number depending on m_{j} alone.

Since {v, v̄} ≠ 0 and m_{j} is even, W_{j}^{+} and W_{j}^{−} are stable under complex conjugation. See the proof of Proposition 2 in [18].

Clearly \overline{e^{a} · v} is a multiple of e^{m_{j}−1−a} · v. Therefore, ⟨e^{a} · v, e^{b} · v⟩ = 0 unless a = b. Thus, each (+m_{j}), for m_{j} even, contributes mutually orthogonal (relative to ⟨·, ·⟩) eigenvectors

e^{m_{j}−3} · v, e^{m_{j}−7} · v, …, v

for x with eigenvalues m_{j} − 3, m_{j} − 7, …, −(m_{j} − 1) to an eigenbasis of V^{+}. Similarly each (−m_{j}), for m_{j} even, contributes mutually orthogonal eigenvectors

e^{m_{j}−1} · v, e^{m_{j}−5} · v, …, e · v

for x with eigenvalues m_{j} − 1, m_{j} − 5, …, −(m_{j} − 3) to an eigenbasis of V^{+}.

Suppose that λ is an odd length row of type (+m_{j}) in Λ. Such rows occur in pairs. The theory of ‘ab’-diagrams shows that for each such pair, V contains a direct sum of two irreducible m_{j} -dimensional 𝔞_{C} -modules W′_{j} and W′′_{j} with respective bases:

\begin{matrix}v, e · v, e^{2} · v, …, e^{m_{j}−1} · v \\ w, e · w, e^{2} · w, …, e^{m_{j}−1} · w \end{matrix}

where v ∈ V^{+}, e · v ∈ V^{−}, e^{2} · v ∈ V^{+}, etc. and w ∈ V^{−}, e · w ∈ V^{+}, e^{2} · w ∈ V^{−}, etc. Moreover,

(1) {e^{a} · v, e^{b} · v} = 0 = {e^{a} · w, e^{b} · w} for all a and b.
(2) {e^{a} · v, e^{b} · w} = 0 if a + b ≠ m_{j} − 1.
(3) {e^{a} · v, e^{m_{j}−1−a} · w} = (−1)^{a}δ_{m_{j}}.

δ_{m_{j}} is a nonzero complex number depending on m_{j} alone. In this case, W′′_{j} is the complex conjugate of W′_{j}. Each pair of odd rows (+m_{j}) contributes mutually orthogonal eigenvectors (relative to ⟨·, ·⟩) for x:

v, e · w, e^{2} · v, …, e^{m_{j}−2} · w, e^{m_{j}−1} · v

with eigenvalues −(m_{j} − 1), −(m_{j} − 3), …, (m_{j} − 1) to an eigenbasis of V^{+}.

The eigenvectors obtained from distinct even rows and distinct pairs of odd rows of Λ are mutually orthogonal with respect to ⟨·, ·⟩. It is clear that (after normalizing the vectors) steps (1) and (2) of Proposition 5.8 will yield the eigenvalues of x on an orthonormal eigenbasis of V^{+}. This determines x as an element of u(V^{+}). The remaining details are left to the reader.

5.7. sp(p, q). We adopt the notation of Helgason [10] (Chapter X, section 2) for sp(p, q) inside sp(p + q, C). {·, ·} is the skew symmetric form on C^{2p+2q} which defines sp(p+q, C). Let K_{p, q} be the block diagonal matrix diag(−I_{p}, I_{q}, −I_{p}, I_{q}) where I_{p} (resp., I_{q}) is the p×p (resp., q ×q) identity matrix. V^{+} and V^{−} denote the +1 and −1 eigenspaces of K_{p, q}. sp(p, q) is the space of 2(p + q) × 2(p + q) complex matrices X such that X^{⊤}K_{p, q} + K_{p, q}X̄ = 0. 𝔨 = sp(p, q) ∩ u(2p + 2q) which is isomorphic to sp(p) ⊕ sp(q). (See Chapter X, section 2, Lemma 2.1 in [10].) Let 𝔱 be the space of matrices defined as in (5) with n = p + q. Define the linear functionals e_{j}, j = 1, …, p + q on 𝔱 as in subsection 5.

In this case, we choose the following simple system for 𝔨:

{e_{1} − e_{2}, …, e_{p−1} − e_{p}, 2e_{p}} ∪ {e_{p+1} − e_{p+2}, …, e_{p+q−1} − e_{p+q}, 2e_{p+q}}.

Nilpotents in sp(p, q) are parametrized by signed Young tableaus of signature (p, q) in which even rows begin with ‘+’.

Proposition 5.9. Let Λ be a signed partition of signature (p, q). Represent Λ using the notation of Proposition 5.4. We form two multisets of nonnegative integers: A_{Λ}^{p} and B_{Λ}^{q}. By arranging the elements of A_{Λ}^{p} in descending order, we obtain the integers e_{1}(x), …, e_{p}(x). By arranging the elements of B_{Λ}^{q} in descending order, we obtain the integers e_{p+1}(x), …, e_{p+q}(x).

To obtain A_{Λ}^{p} and B_{Λ}^{q}, we first label the signs in each row of Λ with the appropriate integers.

Suppose λ is an even row of Λ. Then it must begin with a non-negative odd integer m. In this case m labels a plus sign. Place one copy of the integer string

m, m − 2, m − 4, …, 1

in A_{Λ}^{p} and one copy of the same string in B_{Λ}^{q}.

Next suppose that λ is an odd row. We have the integer string:

m, m − 2, …, 2, 0, −2, … − m

In this case, the first integer m is even, and the row contains m + 1 integers. If m is labelled with a plus sign there are two subcases:

(a) m ≡ 0 (mod 4), so m = 4k. Then the integer string

m, m, m − 4, m − 4, …, 4, 4, 0

which contains 2k + 1 integers is assigned to A_{Λ}^{p} and the integer string

m − 2, m − 2, m − 6, m − 6, …, 2, 2

which contains 2k integers is assigned to B_{Λ}^{q}. (Thus, if k = 0, no integers are assigned to B_{Λ}^{q}.)

(b) m ≡ 2 (mod 4), so m = 4k + 2. Then the integer string

m, m, m − 4, m − 4, …, 2, 2

which contains 2k + 2 integers is assigned to A_{Λ}^{p} and the integer string

m − 2, m − 2, m − 6, m − 6, …, 4, 4, 0

which contains 2k + 1 integers is assigned to B_{Λ}^{q}.

If m is labelled with a minus sign there are two subcases:

(a) m ≡ 0 (mod 4), so m = 4k. Then the integer string

m − 2, m − 2, m − 6, m − 6, …, 2, 2

which contains 2k integers is assigned to A_{Λ}^{p} and the integer string

m, m, m − 4, m − 4, …, 4, 4, 0

which contains 2k + 1 integers is assigned to B_{Λ}^{q}.

(b) m ≡ 2 (mod 4), so m = 4k + 2. Then the integer string

m − 2, m − 2, m − 6, m − 6, …, 4, 4, 0

which contains 2k + 1 integers is assigned to A_{Λ}^{p} and the integer string

m, m, m − 4, m − 4, …, 2, 2

which contains 2k + 2 integers is assigned to B_{Λ}^{q}.

Proof. One uses V^{+} and V^{−} in a manner similar to that in the proof of Proposition 5.4. The details are left to the reader.

6. The spherical nilpotent K_{C} -orbits in 𝔭_{C} for 𝔤 classical

We will prove the following result in section 8.

Theorem 6.1. If 𝔤 is a simple real classical Lie algebra then the spherical nilpotent K_{C} -orbits in 𝔭_{C} are precisely those corresponding to signed partitions Λ in the following list. The notation for Λ is as in Proposition 5.4. (m, k, k_{1}, k_{2}, r, r_{1}, r_{2} are non negative integers.) sl(s, R): No part size of Λ exceeds 2. su^{∗}(2n): No part size of Λ exceeds 2. su(p, q): Λ is one the following:

(a) (+3)(+2)^{k_{1}}(−2)^{k_{2}}(+1)^{r_{1}}(−1)^{r_{2}} ;(−3)(+2)^{k_{1}}(−2)^{k_{2}}(+1)^{r_{1}}(−1)^{r_{2}}

(b) (+3)^{2}(+1)^{r}; (−3)^{2}(−1)^{r}
(c) (+2)^{k_{1}}(−2)^{k_{2}}(+1)^{r_{1}}(−1)^{r_{2}}.

so(p, q): Λ (after ignoring numerals) is one of the following: (a) (+3)^{m}(+1)^{r}; (−3)^{m}(−1)^{r} (m ≤ 2)

(b) (+3)^{m}(+2)^{k}(+1)^{r_{1}}(−1)^{r_{2}} ; (−3)^{m}(+2)^{k}(+1)^{r_{1}}(−1)^{r_{2}} (m ≤ 1, k even). so^{∗}(2n): Λ is (+3)(+1)^{r} or (+2)^{k_{1}}(−2)^{k_{2}}(+1)^{r}. sp(n, R): No part size of Λ exceeds 2. sp(p, q): Λ is one of the following:

(a) (+3)(+1)^{r_{1}}(−1)^{r_{2}} ; (+3)(+2)(+1)^{r_{1}}(−1)^{r_{2}}

(b) (−3)(+1)^{r_{1}}(−1)^{r_{2}} ; (−3)(+2)(+1)^{r_{1}}(−1)^{r_{2}}
(c) (+2)^{k}(+1)^{r_{1}}(−1)^{r_{2}}.

Remark 6.2. The argument is case by case. Each algebra is treated in one of the subsections of section 8.

The following proposition greatly reduces the task of classifying the spherical nilpotent K_{C} -orbits in 𝔭_{C} for 𝔤 real, classical and simple.

Proposition 6.3. Assume that 𝔤 is real, classical and simple, and we retain the notation of section 5. If K_{C} · e is a spherical nilpotent orbit, and Λ is the corresponding signed partition, then |e_{i}(x)| ≤ 2 for all i. That is, no part size (i.e., row length) of Λ exceeds 3.

Proof. The proof is case by case and depends on Proposition 4.17 and precise information about the 𝔱_{C} -weights of the representation of 𝔨_{C} on 𝔭_{C} to restrict the values of the e_{i}(x). We give details only for 𝔤 = sl(s, R), su^{∗}(2n), su(p, q), and sp(n, R) .

1. 𝔤 = sl(s, R). The weights of 𝔱_{C} in 𝔭_{C} are of the form ±(e_{i} ± e_{j}) (and possibly ±e_{i}) and ±2e_{i}. By Proposition 5.1 we have e_{i}(x) ≥ 0 for all i. Thus 0 ≤ 2e_{i}(x) ≤ 3 by Proposition 4.17. Hence by integrality each e_{i}(x) ≤ 1. By Proposition 5.1 this means that no part size exceeds 2.

2. 𝔤 = su^{∗}(2n). 𝔭_{C} is the representation of 𝔨 = sp(n) on the irreducible submodule of ⋀^{2}C^{2n} of dimension 2n^{2} − n − 1. The non zero 𝔱_{C} weights on 𝔭_{C} are of the form ±(e_{i} ± e_{j}), 1 ≤ i < j ≤ n. The non zero 𝔱_{C} weights on 𝔨_{C} are of the form ±(e_{i} ± e_{j}), 1 ≤ i < j ≤ n, and ±2e_{i}, 1 ≤ i ≤ n.

By Proposition 5.3 we have e_{i}(x) ≥ 0 for all i. By Proposition 4.17, the 𝔨_{C} -height cannot exceed 4, therefore 0 ≤ e_{i}(x) ≤ 2 for all i. Since the 𝔭_{C} height cannot exceed 3, only one e_{i}(x) may equal 2 and the other e_{i}(x) must be 0 or 1.

3. 𝔤 = su(p, q). The weights of 𝔱_{C} in 𝔭_{C} are of the form ±(e_{i} − e_{j}), 1 ≤ i ≤ p, p + 1 ≤ j ≤ p + q. If for some i, e_{i}(x) > 2, then some row of length at least 4 must occur in Λ. If a row of length 4 occurs, then Λ contains either (a)‘+ − +−’ or (b) ‘− + −+’. The corresponding integers are 3, 1, −1, −3. In either case, by Proposition 5.4, there exists i, j, 1 ≤ i ≤ p, p ≤ j ≤ p + q such that |e_{i}(x) − e_{j}(x)| = 6. So 𝔭_{C}(6) ≠ 0, contradicting Proposition 4.17. A similar argument shows that no part size can exceed 4.

4. 𝔤 = sp(n, R). 𝔭_{C} is the sum of the symmetric square of the first fundamental representation of k = u(n) and the symmetric square of the last fundamental representation of k = u(n). So the 𝔱_{C} -weights of 𝔭_{C} have the form ±(e_{i} ± e_{j}), 1 ≤ i, j ≤ n (where i can equal j). Since 𝔭_{C} -height cannot exceed 3, |2e_{i}(x)| ≤ 3 for all i. Since e_{i}(x) is integral this implies |e_{i}(x)| ≤ 1 for all i.

To classify the spherical nilpotent K_{C} -orbits in 𝔭_{C} for 𝔤 real, classical and simple we proceed as follows. First, by Proposition 6.3, we need only consider those signed partitions Λ where no part size exceeds 3. Second, for the corresponding orbit O = O_{Λ} we apply the criteria (a) and (b) of Corollary 4.13. To check criterion (a), i.e., whether Kx_{C}/K{x, e, f}_{C} is spherical: (1) construct 𝔨^{x} and 𝔨^{{x, e, f}}; (2) determine how 𝔨^{{x, e, f}} sits inside 𝔨^{x}; and (3) use the results in [14] to check whether 𝔨^{{x, e, f}} is spherical inside 𝔨^{x}. If O satisfies criterion (a), we show that either: (1) O satisfies criterion (b) of Corollary 4.13; or (2) O lies in the closure of an orbit we have shown to be spherical; or (3! ) the closure of O contains an orbit which we know is not spherical. (See Proposition 4.11.) These techniques handle all the Λ with no part size exceeding 3.

7. Structure of 𝔨^{x} and 𝔨^{{x, e, f}} for special x

7.1. Notation and conventions. In order to test K_{C} · e for sphericality we need to know the structure of 𝔨^{x}, the structure of 𝔨^{{x, e, f}} and how 𝔨^{{x, e, f}} is contained in 𝔨^{x} when the signed partition corresponding to K_{C} · e has no part sizes greater than 3. The reductive components of 𝔨^{x} can be found from the weighted Dynkin diagram of x that can be found from the recipes in section 5.. The structure of 𝔤^{{x, e, f}}, and hence 𝔨^{{x, e, f}} can be found by applying results in [24] and [3].

The ‘ab’-diagram (signed partition) description of K_{C} · e allows one to determine how 𝔨^{{x, e, f}} is embedded in 𝔨^{x}. This information is summarized below for each classical simple real algebra other than sl(s, R) and sp(n, R). We will only sketch the proofs in case 𝔤 is su(p, q). We use the following conventions in stating the results.

First, if 𝔮 is a reductive Lie algebra, then the containment 𝔮 ⊂ 2𝔮 (resp., 𝔮 ⊂ 3𝔮) always refers to the diagonal embedding X ↦ X ⊕ X (resp., X ↦ X ⊕ X ⊕ X). More generally if we have reductive Lie algebras 𝔮_{0}, 𝔮_{1}, and 𝔮_{2} with 𝔮_{0} ⊂ 𝔮_{i} (i = 1, 2). Then, we say that 𝔮_{0} is diagonally embedded in the direct sum 𝔮_{1} ⊕ 𝔮_{2} if we have a Lie algebra homorphism 𝔮_{0} → 𝔮_{1} ⊕ 𝔮_{2} given by X ↦ X ⊕ X.

Suppose 𝔤 is a classical simple real algebra other than sl(s, R) or su^{∗}(2n).

Let V = V (𝔤), be the complex vector space associated to 𝔤 in the corresponding recipe in section 5. In that section subspaces V^{+} and V^{−} were defined for each 𝔤. The notation 𝔮^{+} (resp., 𝔮^{−}) applied to a subalgebra of 𝔤 indicates that 𝔮 consists of linear transformations which preserve the subspace V^{+} (resp., V^{−}) associated to the form on V .(See section 5.) In general, if W is a complex (resp. real) vector space with a positive definite Hermitian (resp. quadratic) form, then u(W) (resp. o(W)) denotes the Lie algebra of the unitary (resp. orthogonal) group preserving the form.

Finally we need

Definition 7.1. Let 𝔤 be a classical simple real algebra other than sl(s, R) or su^{∗}(2n). We set V {j}, (j = 1, 2, 3) equal to the sum of the irreducible j - dimensional 𝔞_{C} -submodules of V , V_{+}{j} (resp., V_{−}{j}) equal to the sum of the irreducible j -dimensional 𝔞_{C} -submodules of V whose highest weight vector (i.e., the kernel of the element e) belongs to V^{+} (resp. V^{−}). If a = 0, ±1, ±2, or ±3, then V {j, a} denotes the a-eigenspace of x in V {j}. V_{+}{j, a} and V_{−}{j, a} have similar meanings.

7.2. su^{∗}(2n). Let Λ = 3^{p} 2^{q} 1^{r}. Then

𝔨^{x} = u(2p) ⊕ u(2q) ⊕ sp(p + r); 𝔨^{{x, e, f}} = sp(p) ⊕ sp(q) ⊕ sp(r)  (6)

with

sp(p) ⊕ sp(r) ⊂ u(2p) ⊕ sp(p + r); sp(q) ⊂ u(2q).  (7)

sp(p) is embedded diagonally in u(2p)⊕sp(p+r), and sp(r) is embedded naturally in sp(p + r).

Lemma 7.2. The embedding sp(p) ⊕ sp(r) ⊂ u(2p) ⊕ sp(p + r) in equation (7) is spherical if and only if p ≤ 1.

Proof. The proof is left to the reader.

7.3. 𝔤 = su(p, q).

Let Λ = (+3)^{m_{1}}(−3)^{m_{2}}(+2)^{n_{1}}(−2)^{n_{2}}(+1)^{r_{1}}(−1)^{r_{2}} . Set 𝔨̃ = u(p) ⊕ u(q). It is more convenient to describe the containment 𝔨̃^{{x, e, f}} ⊂ 𝔨̃^{x}. This is sufficient since 𝔨^{x} (resp., 𝔨^{{x, e, f}}) consists of the trace zero matrices in 𝔨̃^{x} (resp., 𝔨̃^{{x, e, f}}). We have

\begin{matrix}𝔨̃^{x} = 2u(m_{1})^{+} ⊕ u(n_{1})^{+} ⊕ u(n_{2})^{+} ⊕ u(r_{1} + m_{2})^{+} \\ ⊕ 2u(m_{2})^{−} ⊕ u(n_{1})^{−} ⊕ u(n_{2})^{−} ⊕ u(r_{2} + m_{1})^{−} \\ 𝔨̃^{\{x, e, f\}} = u(m_{1}) ⊕ u(m_{2}) ⊕ u(n_{1}) ⊕ u(n_{2}) ⊕ u(r_{1}) ⊕ u(r_{2}) \end{matrix}  (8)

with

\begin{matrix}u(n_{1}) = D(u(n_{1})^{+} ⊕ u(n_{1})^{−}); u(n_{2}) = D(u(n_{2})^{+} ⊕ u(n_{2})^{−}); \\ u(m_{1}) ⊕ u(r_{2}) ⊂ ⊕(2u(m_{1})^{+} ⊕ u(m_{1} + r_{2})^{−}); \\ u(m_{2}) ⊕ u(r_{1}) ⊂ (2u(m_{2})^{−} ⊕ u(m_{2} + r_{1})^{+}). \end{matrix}  (9)

D denotes the diagonal embedding. Note that u(m_{1}) embeds diagonally in 2u(m_{1})^{+} and sits naturally inside u(m_{1} + r_{2})^{−}, while u(r_{2}) sits naturally inside u(m_{1} + r_{2})^{−}. u(m_{2}) embeds diagonally in 2u(m_{2})^{−} and sits naturally inside u(m_{2} + r_{1})^{−}, while u(r_{1}) sits naturally inside u(m_{2} + r_{1})^{−}. If r_{1} = 0, then u(m_{1}) embeds diagonally in 3 copies of u(m_{1}). If r_{2} = 0, then u(m_{2}) embeds diagonally in 3 copies of u(m_{2}).

We will demonstrate the embeddings u(n_{1}) = D(u(n_{1})^{+} ⊕ u(n_{1})^{−}) and u(m_{1}) ⊕ u(r_{2}) ⊂ ⊕(2u(m_{1})^{+} ⊕ u(m_{1} + r_{2})^{−}) in (9).

Proof. The form ⟨·, ·⟩_{p, q} is non degenerate and either positive definite or negative definite on each of the complex subspaces V_{±}{i, j}. If ζ ∈ 𝔨̃^{x}, ζ preserves the eigenspaces of x and the restriction of ζ to each of these subspaces preserves the restriction of the form ⟨·, ·⟩_{p, q}. Thus, ζ corresponds to an element of the direct sum of the Lie algebras in lines (10) and (11) below:

u(V_{+}{3, 2}), u(V_{+}{3, −2}), u(V_{−}{3, 0} + V_{+}{1, 0}), u(V_{+}{2, 1}), u(V_{+}{2, −1})

(10)

u(V_{−}{3, 2}), u(V_{−}{3, −2}), u(V_{+}{3, 0} + V_{−}{1, 0}), u(V_{−}{2, 1}), u(V_{−}{2, −1}).

(11)

Line (10) gives 2u(m_{1})^{+} ⊕ u(m_{2} + r_{1})^{+} ⊕ u(n_{1})^{+} ⊕ u(n_{1})^{−} and line (11) gives 2u(m_{2})^{−} ⊕ u(m_{1} + r_{2})^{−} ⊕ u(n_{2})^{−} ⊕ u(n_{1})^{+}. So we have the first half of (8).

Suppose that ζ ∈ 𝔨^{{x, e, f}}. Clearly, the maps e : V_{±}{3, 0} → V_{∓}{3, 2}, e^{2} : V_{±}{3, −2} → V_{∓}{3, 2}, and e : V_{±}{2, −1} → V_{∓}{2, 1} are isomorphisms. By redefining the restriction of the form ⟨·, ·⟩_{p, q} on each of the spaces V_{∓}{3, 2}, and V_{∓}{2, 1}, we can assume that the maps defined by e and e^{2} are isometries. ζ preserves the following complex subspaces: the m_{1} (resp., m_{2}) dimensional subspace W_{1} (resp., W′_{1}) spanned by the vectors v + (e^{2} · v) and v ∈ V_{+}{3, −2} (resp., v ∈ V_{−}{3, −2}); and the n_{1} (resp., n_{2}) dimensional subspace W_{2} (resp. W′_{2}) spanned by w + e · w , w ∈ V_{+}{2, −1} (resp., w ∈ V_{−}{2, −1}). Thus, u(W_{1}) maps into the diagonal of u(V_{+}{3, 2}) ⊕ u(V_{+}{3, −2}). u(W_{2}) maps into the diagonal of u(V_{+}{2, 1}) ⊕ u(V_{+}{2,−1}). But in addition there is a map from u(W_{1}) into u(V_{−}{3, 0} ⊕ V_{+}{1, 0}) given as follows: T in u(W_{1}) is mapped to e ∘ T ∘ e^{−1} in u(V_{−}{3, 0}). Similar arguments apply to W′_{1} and W′_{2}.

Lemma 7.3. Let u(p)⊕u(r) ⊂ 2u(p)⊕u(r +p) be the embedding in the second line of (9). (a) If p = 1 the embedding is spherical for all r. (b) If p = 2 the embedding is spherical if and only if r = 0. (c) If p ≥ 3, the embedding is not spherical. We leave the proof to the reader.

Corollary 7.4. If m_{1} ≥ 3 or m_{2} ≥ 3, then 𝔨^{{x, e, f}} ⊂ 𝔨^{x} in equation (8) is not a spherical embedding.

7.4. so(p, q). Let Λ = (+3)^{m_{1}}(−3)^{m_{2}}(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} .

\begin{matrix}𝔨^{x} = u(m_{1})^{+} ⊕ u(\frac{n}{2})^{+} ⊕ so(m_{2} + r_{1})^{+} ⊕ u(m_{2})^{−} ⊕ u(\frac{n}{2})^{−} ⊕ so(m_{1} + r_{2})^{−} \\ 𝔨^{\{x, e, f\}} = so(m_{1}) ⊕ so(m_{2}) ⊕ u(\frac{n}{2}) ⊕ so(r_{1}) ⊕ so(r_{2}) \end{matrix} (12)

where

\begin{matrix}u(\frac{n}{2}) = D(u(\frac{n}{2})^{+} ⊕ u(\frac{n}{2})^{−}); \\ so(m_{1}) ⊕ so(r_{2}) ⊂ u(m_{1})^{+} ⊕ so(m_{1} + r_{2})^{−}; \\ so(m_{2}) ⊕ so(r_{1}) ⊂ u(m_{2})^{−} ⊕ so(m_{2} + r_{1})^{+}. \end{matrix}  (13)

so(m_{1}) (resp., so(m_{2})) is embedded diagonally in u(m_{1})^{+} ⊕ so(m_{1} + r_{2})^{−} (resp., u(m_{2})^{−} ⊕ so(m_{2} + r_{1})^{+}). so(r_{2}) (resp., so(r_{1})) is embedded naturally in so(m_{1} + r_{2})^{−} (resp., so(m_{1} + r_{2})^{+}).

Lemma 7.5. (a) If r > 0 and m = 2 or (b) m ≥ 3 the embedding

so(m) ⊕ so(r) ⊂ u(m) ⊕ so(m + r))

in (13) is not spherical. We leave the proof to the reader.

Corollary 7.6. If m_{1} ≥ 3 or m_{2} ≥ 3, then 𝔨^{{x, e, f}} ⊂ 𝔨^{x} in equation (12) is not a spherical embedding.

7.5. so^{∗}(2n). Let Λ = (+3)^{p}(+2)^{q_{1}}(−2)^{q_{2}}(+1)^{r}. Then

\begin{matrix}𝔨^{x} = 2u(p) ⊕ u(p + r) ⊕ u(2q_{1}) ⊕ u(2q_{2}) \\ 𝔨^{\{x, e, f\}} = u(p) ⊕ sp(q_{1}) ⊕ sp(q_{2}) ⊕ u(r). \end{matrix}  (14)

with

u(p) ⊕ u(r) ⊂ 2u(p) ⊕ u(p + r); sp(q_{i}) ⊂ u(2q_{i}), i = 1, 2

where u(p) ⊕ u(r) is embedded in 2u(p) ⊕ u(p + r) as in equation (9).

Lemma 7.7. The containment 𝔨^{{x, e, f}} ⊂ 𝔨^{x} in equation (14) is spherical ⟺ p ≤ 2.

Proof. This follows from Lemma 7.3.

7.6. sp(p, q).

Let Λ = (+3)^{m_{1}}(−3)^{m_{2}}(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} . Then

\begin{matrix}𝔨^{x} = u(2m_{1})^{+} ⊕ u(n)^{+} ⊕ sp(m_{2} + r_{1})^{+} ⊕ u(2m_{2})^{−} ⊕ u(n)^{−} ⊕ sp(m_{1} + r_{2})^{−} \\ 𝔨^{\{x, e, f\}} = sp(m_{1}) ⊕ sp(m_{2}) ⊕ u(n) ⊕ sp(r_{1}) ⊕ sp(r_{2}) \end{matrix} (15)

where

u(n) = D(u(n)^{+} ⊕ u(n)^{−});
sp(m_{1}) ⊕ sp(r_{2}) ⊂ u(2m_{1})^{+} ⊕ sp(m_{1} + r_{2})^{−};
sp(m_{2}) ⊕ sp(r_{1}) ⊂ u(2m_{2})^{−} ⊕ sp(m_{2} + r_{1})^{+}

sp(m_{1}) (resp., sp(m_{2})) is embedded diagonally in u(2m_{1})^{+} ⊕ sp(m_{1} + r_{2})^{−} (resp., u(2m_{2})^{−} ⊕sp(m_{2} +r_{1})^{+}). sp(r_{2}) (resp., sp(r_{1})) is embedded naturally in sp(m_{1} + r_{2})^{−} (resp., sp(m_{2} + r_{1})^{+}).

Lemma 7.8. If m_{1} ≥ 2 or m_{2} ≥ 2, then 𝔨^{{x, e, f}} ⊂ 𝔨^{x} in equation (15) is not a spherical embedding.

Proof. This follows from Lemma 7.2.

8. Proof of Theorem 6.1

8.1. sl(s, R). By the proof of Proposition 6.3 for sl(s, R), if K_{C} ·e is spherical then no part size of the corresponding partition Λ exceeds 3. Indeed we have the following.

Proposition 8.1. (𝔤 = sl(s, R)). K_{C} · e is spherical if and only if in the corresponding partition Λ no part size exceeds 2.

Proof. The proof of Proposition 6.3 implies that no part size in Λ can exceed 2. Corollary 4.16 implies that if no part size exceeds 2, then K_{C} · e is spherical.

8.2. su^{∗}(2n). By the proof of the su^{∗}(2n) case of Proposition 6.3 the signed partition for a spherical nilpotent K_{C} -orbit for su^{∗}(2n) has at most one part of size 3. We now show that in fact no part size can exceed 2.

Lemma 8.2. Suppose 𝔤 = su^{∗}(2n), and Λ = 3^{p}2^{q}1^{r} is the partition of n corresponding to the orbit K_{C} · e. If p > 0 then K_{C} · e is not spherical.

Proof. From the discussion preceding this lemma, if Λ is spherical then p = 0 or 1. Suppose p = 1. By Proposition 5.3, we have that:

e_{1}(x) = 2 = e_{2}(x); e_{3}(x) = 1 = … = e_{2q+2}(x); e_{2q+3}(x) = 0 = … = e_{n}(x).

Since e_{1} + e_{2} is a weight of 𝔱_{C} on 𝔭_{C} , 𝔭_{C} -height(e) > 3. So by Proposition 4.17, K_{C} · e is not spherical.

It follows from the previous lemma and Corollary 4.15, that

Proposition 8.3. If 𝔤 = su^{∗}(2n), then the spherical K_{C} -nilpotent orbits are exactly those corresponding to partitions of n with part sizes not exceeding 2.

8.3. su(p, q). Proposition 6.3 implies the following.

Proposition 8.4. If K_{C} · e is a spherical nilpotent orbit of su(p, q) then the corresponding signed partition has the form

Λ = (+3)^{m_{1}}(−3)^{m_{2}}(+2)^{n_{1}}(−2)^{n_{2}}(+1)^{r_{1}}(−1)^{r_{2}},  (16)

where m_{i}, n_{i} and r_{i} (i = 1, 2) are non negative integers.

Proposition 8.5. If 𝔤 = su(p, q), K_{C} · e is a spherical nilpotent orbit in 𝔭_{C} and Λ is the corresponding signed partition then K_{C} ·e is spherical if and only if Λ has one of the following forms (where n_{1}, n_{2}, r_{1}, r_{2}, r are non-negative integers):

(a) (+3)(+2)^{n_{1}}(−2)^{n_{2}}(+1)^{r_{1}}(−1)^{r_{2}} ; (−3)(+2)^{n_{1}}(−2)^{n_{2}}(+1)^{r_{1}}(−1)^{r_{2}}

(b) (+3)^{2}(+1)^{r} ; (−3)^{2}(−1)^{r}
(c) (+2)^{n_{1}}(−2)^{n_{2}}(+1)^{r_{1}}(−1)^{r_{2}} .

We prove the proposition in a series of lemmas.

Lemma 8.6. (𝔤 = su(p, q)) Let the Λ be the signed partition in equation (16) corresponding to the nilpotent orbit K_{C} · e. If m_{1} ≥ 1 and m_{2} ≥ 1. Then K_{C} · e is not spherical.

Proof. Λ has 𝔭_{C} -height exceeding 3 so the result follows from Proposition 4.17.

Lemma 8.7. (𝔤 = su(p, q)) Let the Λ be the signed partition in equation (16) corresponding to the nilpotent orbit K_{C} · e. If m_{1} ≥ 3 or m_{2} ≥ 3, then K_{C} · e is not spherical.

Proof. This follows from Lemma 7.4 and Corollary 4.13.

Lemma 8.8. (𝔤 = su(p, q)) Let the signed partition corresponding to the nilpotent orbit K_{C} ·e be Λ = (+2)^{n_{1}}(−2)^{n_{2}}(+1)^{r_{1}}(−1)^{r_{2}} . Then K_{C} ·e is spherical.

Proof. This follows from Corollary 4.15.

Lemma 8.9. (𝔤 = su(p, q)) Let the Λ be the signed partition in equation (16) corresponding to the nilpotent orbit K_{C} · e, where m_{1} or m_{2} is positive. Then Λ is spherical if and only if one of the following is true:

(a) m_{1} = 1, m_{2} = 0 .

(a)′ m_{2} = 1, m_{1} = 0 .
(b) m_{1} = 2, m_{2} = n_{1} = n_{2} = r_{2} = 0
(b)′ m_{2} = 2, m_{1} = n_{1} = n_{2} = r_{1} = 0

Proof. By Lemma 8.6 and Lemma 8.7, Λ spherical implies that either m_{1} ∈ {1, 2} and m_{2} = 0 or m_{2} ∈ {1, 2} and m_{1} = 0.

(a) We first consider the case m_{1} = 1, m_{2} = 0. Then p = n_{1} + n_{2} + r_{1} + 2 and q = n_{1} + n_{2} + r_{2} + 1. After applying Proposition 5.4 we find that e_{j}(x) = 0 for n_{1} + 2 ≤ j ≤ n_{1} + r_{1} + 1 and p + n_{2} + 1 ≤ j ≤ p + n_{2} + r_{2} + 1, and the non-zero e_{j}(x) are:

\begin{matrix}e_{1}(x) = 2, e_{j}(x) = 1 (2 ≤ j ≤ n_{1} + 1), e_{j}(x) = −1 (n_{1} + r_{1} + 2 ≤ j ≤ p − 1), \\ e_{p}(x) = −2, e_{j}(x) = 1 (p + 1 ≤ j ≤ p + n_{2}), e_{j}(x) = −1 (p + n_{2} + r_{2} + 2 ≤ j). \end{matrix}

By equation (8)

\begin{matrix}𝔨^{x} = s(2u(1)^{+} ⊕ u(n_{1})^{+} ⊕ u(n_{2})^{+} ⊕ u(r_{1})^{+} ⊕ u(n_{2})^{−} ⊕ u(n_{1})^{−} ⊕ u(r_{2} + 1)^{−}) \\ 𝔨^{\{x, e, f\}} = s(u(1) ⊕ u(n_{1}) ⊕ u(n_{2}) ⊕ u(r_{1}) ⊕ u(r_{2})). \end{matrix} ; (17)

The containment relations following equation (8) imply that K{x, e, f}_{C} is spherical in Kx_{C} . In this case 𝔰_{R} = s(u(1) ⊕ t(n_{1}) ⊕ t(n_{2}) ⊕ u(r_{1}) ⊕ u(r_{2} − 1)) where t(n_{1}) and t(n_{2}) are maximal tori inside the subalgebras u(n_{1}) and u(n_{2}) respectively inside the parentheses in the expression for 𝔨^{{x, e, f}} in equation (17 ).

If Δ(Z) denotes the 𝔱_{C} -weights of Z, we also have:

Δ(Z) = {e_{1} − e_{p+n_{2}+r_{2}+2}, …, e_{1} − e_{p+q}; e_{p+1} − e_{p+q}, …, e_{p+n_{2}} − e_{p+q}}.

Let V_{1} denote the span of the eigenvectors in Z with the first n_{1} weights in Δ(Z) and let V_{2} denote the span of the eigenvectors with last n_{2} weights in Δ(Z). One can show that for i = 1, 2, V_{i} is an irreducible representation of u(n_{i}) of dimension n_{i}. It is known that the maximal torus of U(n_{i}) has a dense orbit on V_{i}. It follows that B(S) has a dense orbit on Z.

(b) Finally, we will consider the case m_{1} = 2, m_{2} = 0. (m_{1} = 0, m_{2} = 2 is similar.)

We have Λ = (+3)^{2}(+2)^{n_{1}}(−2)^{n_{2}}(+1)^{r_{1}}(−1)^{r_{2}} . From equation (8) we get

\begin{matrix}𝔨^{x} = s(2u(2)^{+} ⊕ u(n_{1})^{+} ⊕ u(n_{2})^{+} ⊕ u(r_{1})^{+} ⊕ u(n_{2})^{−} ⊕ u(n_{1})^{−} ⊕ u(r_{2} + 2)^{−}) \\ 𝔨^{\{x, e, f\}} = s(u(2) ⊕ u(n_{1}) ⊕ u(n_{2}) ⊕ u(r_{1}) ⊕ u(r_{2})). \end{matrix}

By Lemma 7.3, Kx_{C}/K{x, e, f}_{C} is spherical if and only if r_{2} = 0. It follows that

Remark 8.10. If r_{2} > 0 and n_{1}, n_{2}, r_{1} are arbitrary, the following nilpotent orbit is not spherical:

Λ = (+3)^{2}(+2)^{n_{1}}(−2)^{n_{2}}(+1)^{r_{1}}(−1)^{r_{2}}.

From now on assume that r_{2} = 0.

By applying Proposition 5.4 to Λ = (+3)^{2}(+2)^{n_{1}}(−2)^{n_{2}}(+1)^{r_{1}} , we obtain the values of the e_{i}(x). Using these values we find Δ(Z). We obtain the decomposition Δ(Z) = Δ_{1} ∪ Δ_{2} ∪ Δ_{3} ∪ Δ_{4} where

Δ_{1} = {e_{1} − e_{n_{1}+2n_{2}+r_{1}+7}, …, e_{1} − e_{2n_{1}+2n_{2}+r_{1}+6}},
Δ_{2} = {e_{2} − e_{n_{1}+2n_{2}+r_{1}+7}, …, e_{2} − e_{2n_{1}+2n_{2}+r_{1}+6}},
Δ_{3} = {e_{n_{1}+n_{2}+r_{1}+5} − e_{n_{1}+n_{2}+r_{1}+3}, …, e_{n_{1}+2n_{2}+r_{1}+4} − e_{n_{1}+n_{2}+r_{1}+3}},
Δ_{4} = {e_{n_{1}+n_{2}+r_{1}+5} − e_{n_{1}+n_{2}+r_{1}+4}, …, e_{n_{1}+2n_{2}+r_{1}+4} − e_{n_{1}+n_{2}+r_{1}+4}}.

We also have

\begin{matrix}dim_{C} Z = 2n_{1} + 2n_{2} \\ 𝔰_{R} = s(2u(1) ⊕ t(n_{1}) ⊕ t(n_{2}) ⊕ u(r_{1})) \end{matrix}

u(r_{1}) acts trivially on Z. Therefore if B(S) has an open orbit on Z, we must have n_{1} + n_{2} + 1 ≥ 2n_{1} + 2n_{2}, i.e., n_{1} + n_{2} ≤ 1. But if n_{1} > 0 or n_{2} > 0, Λ = (+3)^{2}(+2)^{n_{1}}(−2)^{n_{2}}(+1)^{r_{1}} contains either (+3)^{2}(+2)^{n_{1}−1}(−2)^{n_{2}}(+1)^{r_{1}+1}(−1) or (+3)^{2}(+2)^{n_{1}}(−2)^{n_{2}−1}(+1)^{r_{1}+1}(−1) in its closure (by Theorem 2 of [19]). Neither of the latter nilpotent orbits is spherical by Remark 8.10. Therefore Λ is not spherical when n_{1} > 0 or n_{2} > 0. If n_{1} + n_{2} = 0, then Z = 0 and we conclude that Λ = (+3)^{2}(+1)^{r_{1}} is spherical. We have shown that cases (a) and (b) of our proposition are the only possibilities when m_{1} > 0. Likewise by considering Λ = (−3)^{2}(+2)^{n_{1}}(−2)^{n_{2}}(+1)^{r_{1}}(−1)^{r_{2}} and applying similar arguments we can show that cases (a)′ and (b)′ are the only ones possible when m_{2} > 0.

Proposition 8.5 follows from Propositions 8.4, Lemmas 8.6, 8.7, 8.8, and 8.9.

8.4. so(p, q). Proposition 6.3 implies the following.

Proposition 8.11. (𝔤 = so(p, q)) If K_{C} ·e is spherical, then the corresponding signed partition Λ has the form

Λ = (+3)^{m_{1}}(−3)^{m_{2}}(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}},

where m_{1}, m_{2}, n, r_{1}, r_{2} are non-negative integers.

For convenience, in the proofs below we consider only nilpotents whose signed partitions are without numerals or have roman numeral “I”. Nilpotents whosed signed partitions have roman numeral “II” are handled by remark 4.19.

Lemma 8.12. (𝔤 = so(p, q)) If K_{C} · e is the orbit corresponding to any of the following signed partitions with or without numerals then K_{C} · e is spherical:

(a) Λ = (+3)^{m}(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} , 1 ≥ m ≥ 0, n ≥ 0, r_{1}, r_{2} ≥ 0;

(a)′ Λ = (−3)^{m}(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} , 1 ≥ m ≥ 0, n ≥ 0, r_{1}, r_{2} ≥ 0;
(b) Λ = (+3)^{2}(+1)^{r_{1}} , r_{1} ≥ 0;
(c) Λ = (−3)^{2}(−1)^{r_{2}} , r_{2} ≥ 0.

Proof. For (a), we use an argument similar to that for part (a) of Lemma 8.9. For (b) and (c), we use an argument similar to that for part (b) of Lemma 8.9.

Lemma 8.13. (𝔤 = so(p, q)) If K_{C} · e corresponds to any of the following signed partitions with or without numerals then K_{C} · e is not spherical:

(a) Λ = (+3)^{2}(+1)^{r_{1}}(−1)^{r_{2}} , r_{1} ≥ 0, r_{2} ≥ 1;

(b) Λ = (−3)^{2}(+1)^{r_{1}}(−1)^{r_{2}} , r_{1} ≥ 1, r_{2} ≥ 0;
(c) Λ = (+3)^{p_{1}}(−3)^{p_{2}}(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} , p_{1} ≥ 1, p_{2} ≥ 1;

Proof. (a) Here 𝔤 = so(r_{1} + 4, r_{2} + 2). By Proposition 5.6: e_{1}(x) = 2 = e_{2}(x), and e_{j}(x) = 0 for all other j. If {x, e, f} is the corresponding triple, then:

𝔨^{x} = u(2)^{+} ⊕ so(r_{1})^{+} ⊕ so(r_{2} + 2)^{−}; 𝔨^{{x, e, f}} = so(2) ⊕ so(r_{1}) ⊕ so(r_{2}).

By Lemma 7.5 Kx_{C} / K{x, e, f}_{C} is spherical if and only if r_{2} = 0. (b) is proven similarly.

(c) This follows from Proposition 4.17.

Lemma 8.14. (𝔤 = so(p, q)) If K_{C} · e corresponds to any of the following signed partitions with or without numerals then K_{C} · e is not spherical:

(a) Λ = (+3)^{m}(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} , for m ≥ 2, and n ≥ 2 (n even).

(a)′ Λ = (−3)^{m}(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} , for m ≥ 2, and n ≥ 2 (n even).
(b) Λ = (+3)^{m}(+1)^{r_{1}}(−1)^{r_{2}} , for m ≥ 3.
(b)′ Λ = (−3)^{m}(+1)^{r_{1}}(−1)^{r_{2}} , for m ≥ 3.

Proof. Assume that Λ has no numeral attached. In case (a) or (a)′, if n ≥ 2 then the closure of K_{C} · e contains an orbit K_{C} · e′ where K_{C} · e′ is of type (a) or (b) in Lemma 8.13. (See [9].) Now apply Proposition 4.11. The assertions about (b) and (b)′ follow from Corollary 7.6.

Proposition 8.15. If 𝔤 = so(p, q), then K_{C} · e is spherical ⟺ the corresponding signed partition (after ignoring numerals) is of the form:

(a) Λ = (+3)^{m}(+1)^{r_{1}} , m ≤ 2, r_{1} ≥ 0;

(b) Λ = (−3)^{m}(−1)^{r_{2}} , m ≤ 2, r_{2} ≥ 0;

(c) Λ = (+3)^{m}(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} , 1 ≥ m ≥ 0, n ≥ 0 (n even), r_{1} ≥ 0, r_{2} ≥ 0;

(d) Λ = (−3)^{m}(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} , 1 ≥ m ≥ 0, n ≥ 0 (n even), r_{1} ≥ 0, r_{2} ≥ 0.

Proof. The result follows from Proposition 8.11 and Lemmas 8.12, 8.13, and 8.14.

8.5. so^{∗}(2n). Proposition 6.3 implies the following.

Proposition 8.16. (𝔤 = so^{∗}(2n)) If K_{C} ·e is spherical, then the corresponding signed partition Λ has the form

Λ = (+3)^{p}(+2)^{q_{1}}(−2)^{q_{2}}(+1)^{r},

where p, q_{1}, q_{2}, r are non-negative integers.

Lemma 8.17. (𝔤 = so^{∗}(2n)) Let Λ be the signed partition for K_{C} · e. (a) For all integers r ≥ 0, if Λ = (+3)(+1)^{r} , then K_{C} ·e is spherical. (b) For all integers, q_{1} ≥ 0, q_{2} ≥ 0, r ≥ 0, if Λ = (+2)^{q_{1}}(−2)^{q_{2}}(+1)^{r}, then K_{C} · e is spherical.

Proof. For (a) first apply Proposition 5.7 to obtain (with n = r + 3) to conclude that e_{1}(x) = 2, e_{n}(x) = −2, and e_{j}(x) = 0 for 1 < j < n. By (14) 𝔨^{x} = 2u(1) ⊕ u(r + 1) and 𝔨^{{x, e, f}} = u(1) ⊕ u(r). By Lemma 7.7 Kx_{C} / K{x, e, f}_{C} is spherical. Since Z = 0, Λ is spherical by Corollary 4.13. (b) follows from Corollary 4.15.

Lemma 8.18. (𝔤 = so^{∗}(2n)) For all integers p ≥ 2, q_{1} ≥ 0, q_{2} ≥ 0, and r ≥ 0, if Λ = (+3)^{p} (+2)^{q_{1}} (−2)^{q_{2}} (+1)^{r} is the signed partition for K_{C} · e, then K_{C} · e is not spherical.

Proof. Applying Proposition 5.7, we find that e_{1}(x) = e_{2}(x) = 2. Since e_{1} +e_{2} is a weight of 𝔱_{C} on 𝔭_{C} , we can apply Proposition 4.17.

Lemma 8.19. (𝔤 = so^{∗}(2n)) For all integers q_{1} ≥ 1, q_{2} ≥ 1, r ≥ 0, if the signed partition for K_{C} · e is

Λ = (+3)(+2)^{q_{1}}(−2)^{q_{2}}(+1)^{r},

then K_{C} · e is not spherical.

Proof. In this case n = 2q_{1} + 2q_{2} + r + 3. If x is the neutral element corresponding to Λ = (+3)(+2)^{q_{1}}(−2)^{q_{2}}(+1)^{r}, then:

\begin{matrix}𝔨^{x} = u(2q_{1}) ⊕ u(2q_{2}) ⊕ u(r + 1) \\ 𝔨^{\{x, e, f\}} = u(1) ⊕ u(r) ⊕ sp(q_{1}) ⊕ sp(q_{2}). \end{matrix}

Kx_{C} / K{x, e, f}_{C} is spherical. Applying Proposition 5.7, we find that e_{2q_{1}+2}(x) = … = e_{2q_{1}+r+2}(x) = 0 and that

Δ(Z) = {e_{1} + e_{2}, …, e_{1} + e_{2q_{1}+1}, −(e_{n} + e_{2q_{1}+r+3}), …, −(e_{n} + e_{n−1})}.

Using an argument similar to that for part (b) of Lemma 8.9, it can be shown that B(S) does not have a dense orbit in Z.

Proposition 8.20. If 𝔤 = so^{∗}(2n), then the nilpotent e ∈ 𝔭_{C} is spherical ⟺ the corresponding signed partition is of the form (1) Λ = (+3)(+1)^{r} or (2) Λ = (+2)^{q_{1}}(−2)^{q_{2}}(+1)^{r}.

Proof. Proposition 8.16 and Lemmas 8.17–8.19.

8.6. sp(n, R).

Proposition 8.21. If 𝔤 = sp(n, R), the spherical nilpotent K_{C} nilpotent orbits in 𝔭_{C} are precisely those corresponding to signed partitions with part sizes not exceeding 2. That is, Λ corresponds to a spherical nilpotent orbit if and only if Λ = (+2)^{q_{1}}(−2)^{q_{2}}1^{r} for nonnegative integers q_{1}, q_{2}, and r.

Proof. Corollary 4.16 and the proof of Proposition 6.3.

8.7. sp(p, q). Proposition 6.3 implies the following.

Proposition 8.22. (𝔤 = sp(p, q)) If K_{C} ·e is spherical, then the corresponding signed partition Λ has the form

Λ = (+3)^{m_{1}}(−3)^{m_{2}}(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}},  (18)

where m_{1}, , m_{2}, n, r_{1}, r_{2} are non-negative integers.

If Λ is as in (18), then by applying Proposition 5.9 we find that e_{j}(x) = 0 for 2m_{1} + n + 1 ≤ j ≤ p and p + 2m_{2} + n + 1 ≤ j ≤ p + q and the non-zero e_{j}(x) are:

\begin{matrix}e_{1}(x) = 2 = … = e_{2m_{1}}(x), e_{2m_{1}+1}(x) = 1 = … = e_{2m_{1}+n}(x), \\ e_{p+1}(x) = 2 = … = e_{p+2m_{2}}(x), e_{p+2m_{2}+1}(x) = 1 = … = e_{p+2m_{2}+n}(x), \end{matrix} (19)

where p = 2m_{1} + m_{2} + n + r_{1} and q = m_{1} + 2m_{2} + n + r_{2}.

Corollary 8.23. (𝔤 = sp(p, q)) If the nilpotent orbit K_{C} · e is spherical and corresponds to the signed partition Λ in equation (18) , then either, (a) m_{1} = 0 = m_{2} = 0, (b) m_{1} = 1, m_{2} = 0, or (c) m_{1} = 0, m_{2} = 1.

Proof. By Lemma 7.8 and Corolary 4.13, we must have m_{1} ≤ 1 and m_{2} ≤ 1. If m_{1} = m_{2} = 1, then apply Proposition 4.17.

Lemma 8.24. (𝔤 = sp(p, q)) Let Λ be the signed partition corresponding to K_{C} · e.

(a) If Λ = (+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} , where n ≥ 0, r_{1} ≥ 0, r_{2} ≥ 0, then K_{C} · e is spherical.

(b) If Λ = (+3)(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} or Λ = (−3)(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} , where 0 ≤ n ≤ 1 and r_{1}, r_{2} ≥ 0, then K_{C} · e is spherical.

(c) If Λ = (+3)(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} or Λ = (−3)(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} , where n > 1 and r_{1}, r_{2} ≥ 0, then K_{C} · e is not spherical.

Proof. (a) follows from Corollary 4.15.

For (b) and (c), first note that if Λ = (+3)(+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} , then p = n + r_{1} + 2, and q = n + r_{2} + 1.(15) implies that:

\begin{matrix}𝔨^{x} = u(2)^{+} ⊕ u(n)^{+} ⊕ sp(r_{1})^{+} ⊕ u(n)^{−} ⊕ sp(r_{2} + 1)^{−} \\ 𝔨^{\{x, e, f\}} = sp(1) ⊕ u(n) ⊕ sp(r_{1}) ⊕ sp(r_{2}). \end{matrix}

By Lemma 7.2, Kx_{C} / K{x, e, f}_{C} is spherical.

Now suppose that n = 0. Equation (19) implies that 0 ≤ e_{j}(x) + e_{k}(x) ≤ 2 for all 1 ≤ j ≤ p and p + 1 ≤ k ≤ p + q. Therefore, Z = 0. Since Z = 0, K_{C} · e is spherical by Corollary 4.13. The proof of sphericality for Λ = (−3)(+1)^{r_{1}}(−1)^{r_{2}} is similar.

If n > 0, an argument similar to that in part (b) of Lemma 8.9 completes the proof.

Proposition 8.25. If 𝔤 = sp(p, q), then K_{C} · e is spherical ⟺ the corresponding signed partition is one of the following (for n ≥ 0, r_{1} ≥ 0, r_{2} ≥ 0):

(a) Λ = (+3)(+1)^{r_{1}}(−1)^{r_{2}} or (+3)(+2)(+1)^{r_{1}}(−1)^{r_{2}} ;

(b) Λ = (−3)(+1)^{r_{1}}(−1)^{r_{2}} or (−3)(+2)(+1)^{r_{1}}(−1)^{r_{2}} ;
(c) Λ = (+2)^{n}(+1)^{r_{1}}(−1)^{r_{2}} .

Proof. Combine Proposition 8.22, Corollary 8.23, and Lemma 8.24.

9. The spherical nilpotent K_{C} -orbits in 𝔭_{C} for 𝔤 exceptional

The following conventions hold throughout this section. We will use the notation of Helgason [10] to describe the simple real exceptional algebra 𝔤. We use the tables of D – okovic in [4] and [5] to describe the K_{C} -nilpotent conjugacy classes of the exceptional simple algebras and for information about the neutral ement x, 𝔨^{x}, and 𝔨^{{x, e, f}} associated to each conjugacy class.

Theorem 9.1. If 𝔤 is a simple real exceptional Lie algebra then the spherical nilpotent K_{C} -orbits in 𝔭_{C} are the following ones as listed in [4] and [5].

EI: classes 1, 2 and 3.

EII: classes 1, 2, 3, 4 and 5.

EIII: classes 1, 2, 3, 4, 5, 6, 7, 8, and 9.

EIV : class 1.

EV : classes 1, 2, 3, 4, 5, 8 and 9. EV I: classes 1, 2, 3, 4, and 5. EV II: classes: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, and 12. EV III: classes 1, 2, 3 and 6. EIX: classes 1, 2, 3, 4, and 5. F I: classes 1, 2, 3, 4, and 5. F II: classes 1 and 2. GI: classes 1 and 2.

The proof is case by case. Detailed arguments will only be given for 𝔤 not equal to EIV , F I, F II or GI.

9.1. EI (𝔨 = sp(4)). By Remark 4.18 and Proposition 4.17, the K_{C} classes 1, 2 and 3 are the only ones that could be spherical. Classes 1 and 2 must be spherical by Corollary 4.15.

For class 3, 𝔨^{x} = su(3) ⊕ 2u(1), 𝔨^{{x, e, f}} = su(2) ⊕ u(1). Therefore, Kx_{C}/K{x, e, f}_{C} is spherical. 𝔰_{R} = 2u(1). Z = 𝔭_{C}(3) and dim Z = 1. By constructing 𝔰_{R} and Z one shows directly that 𝔰_{R} acts nontrivially on Z. Since S_{0} = B(S), this shows that B(S) has a dense orbit on Z. So class 3 is spherical.

9.2. EII (𝔨 = su(6) ⊕ su(2)). By Remark 4.18 and Proposition 4.17, the K_{C} classes 1-7, are the only ones that could be spherical. Classes 1, 2 and 3 are spherical by Corollary 4.15.

For class 6, 𝔨^{x} = su(6) ⊕ u(1) and 𝔨^{{x, e, f}} = 2su(3). Kx_{C}/K{x, e, f}_{C} is not spherical. For class 7, 𝔨^{x} = su(4) ⊕ su(2) ⊕ 2u(1) and 𝔨^{{x, e, f}} = 2su(2) ⊕ 2u(1). By explicit construction of 𝔨^{x} and 𝔨^{{x, e, f}} one finds that 𝔨^{{x, e, f}} is contained in the su(4) ⊕ 2u(1) component of 𝔨^{x} hence Kx_{C}/K{x, e, f}_{C} is not spherical. It follows that neither class 6 nor class 7 is spherical.

Finally consider classes 4 and 5. For class 4, 𝔨^{x} = 2su(3) ⊕ 2u(1), and 𝔨^{{x, e, f}} = su(3) ⊕ u(1); for class 5, 𝔨^{x} = 2su(2) ⊕ 4u(1), and 𝔨^{{x, e, f}} = su(2) ⊕ 2u(1). By Lemma 4.14, 𝔨^{{x, e, f}} is a symmetric subalgebra of 𝔨^{x}, so condition (a) of Corollary 4.13 is satisfied by class 4 and class 5. For class 4 and class 5, 𝔰_{R} = 3u(1). One verifies directly that in each case 𝔰_{R} acts non trivially on Z = 𝔭_{C}(3) which is one dimensional. Therefore, B(S), the complex torus corresponding to 𝔰_{C} , has a dense orbit on Z. So condition (b) of Corollary 4.13 is satisfied and both classes are spherical.

9.3. EIII (𝔨 = so(10) ⊕ R). By Remark 4.18 and Proposition 4.17, the K_{C} classes 1-9, are the only ones that could be spherical. In fact, classes 1, 2, 3, 4 and 5 are spherical since the height of each of the corresponding G_{C} classes is 2.

For class 9, 𝔨^{x} = so(8) ⊕ 2u(1), and 𝔨^{{x, e, f}} = (G_{2})_{−14}. So K{x, e, f}_{C} is spherical in Kx_{C} . Since Z = 0 , class 9 is spherical.

By Figure 5 in [6], the closure of class 9 contains the closure of classes 6, 7 and 8 (and classes 1-5). Therefore, the spherical K_{C} nilpotent classes of EIII are classes 1–9.

9.4. EV (𝔨 = su(8)). By Remark 4.18 and Proposition 4.17, the K_{C} classes 1-6, 8 and 9 are the only ones that could be spherical. Classes 1, 2, 3 and 4 are spherical because each of the corresponding G_{C} classes has height 2.

For class 6, we have 𝔨^{x} = su(6) ⊕ 2u(1) and 𝔨^{{x, e, f}} = 2su(3). So Kx_{C}/K{x, e, f}_{C} is not spherical.

For class 8 and for class 9, we have 𝔨^{x} = 2su(3) ⊕ 3u(1) and 𝔨^{{x, e, f}} = su(3) ⊕ u(1). Class 8 and class 9 are conjugate under Gθ_{C} .

Consider class 8. By constructing 𝔨^{x} and 𝔨^{{x, e, f}} one checks that the su(3) summand of 𝔨^{{x, e, f}} is embedded diagonally in the 2su(3) summand of 𝔨^{x} and that Kx_{C}/K{x, e, f}_{C} is spherical. We also have 𝔰_{R} = 3u(1), a maximal torus in 𝔨^{{x, e, f}}. Now 𝔰_{C} is a complex three dimensional torus. One can find a basis for Z consisting of noncompact root vectors for a maximal torus of 𝔨 that contains 𝔰_{R}. Using this information, one verifies that [𝔰_{C}, Z] = Z. So B(S) = S_{0} has a dense orbit on Z, making class 8 spherical. By Remark 4.19, class 9 is spherical.

Figure 6 in [8] shows that class 5 lies in the closure of each of the following classes: 8 and 9. Their sphericality implies that of class 5.

9.5. EV I (𝔨 = so(12) ⊕ su(2)). By Remark 4.18 and Proposition 4.17, the K_{C} classes 1–6, 8 and 9 are the only ones that could be spherical. In fact, classes 1, 2 and 3 are spherical since the height of each of the corresponding G_{C} classes is 2.

Since Z = 0 for classes 6 and 8, it suffices to check whether K{x, e, f}_{C} is spherical in Kx_{C} . By [14], for n ≥ 3, su(n) is only spherical in so(2n) when n is odd. For class 6, 𝔨^{x} = so(12) ⊕ u(1), and 𝔨^{{x, e, f}} = su(6). For class 8, 𝔨^{x} = so(8)⊕su(2)⊕u(1)⊕u(1), and 𝔨^{{x, e, f}} = s(u(4)⊕u(2)) = su(4)⊕su(2)⊕u(1). Thus for classes 6 and 8 sphericality fails. Since class 8 is not spherical and the closure of class 9 contains class 8 (by results in [7]), class 9 can’t be spherical.

For class 4, 𝔨^{x} = su(6) ⊕ 2u(1), and 𝔨^{{x, e, f}} = sp(3) ⊕ u(1). For class 5, 𝔨^{x} = su(4) ⊕ su(2) ⊕ 3u(1), and 𝔨^{{x, e, f}} = sp(2) ⊕ su(2) ⊕ u(1). So for class 4 and class 5, K{x, e, f}_{C} is spherical in Kx_{C} . In each case, 𝔰_{R} = 3su(2) ⊕ u(1) and 𝔰_{R} contains a maximal torus of 𝔨^{{x, e, f}}. One checks directly that this torus acts non trivially on the one dimensional space Z. This implies that B(S) has an open orbit on Z. Thus classes 4 and 5 are spherical.

9.6. EV II (𝔨 = e_{6} ⊕R). By Remark 4.18 and Proposition 4.17, the K_{C} classes 1–12 are the only ones that could be spherical. In fact, classes 1–9 are spherical since the height of each of the corresponding G_{C} classes = 2.

Since Z = 0 for class 10, it suffices to check whether K{x, e, f}_{C} is spherical in Kx_{C} . We have 𝔨^{x} = su(6)⊕2u(1), and 𝔨^{{x, e, f}} = su(5)⊕u(1). By constructing these subalgebras, one checks that sphericality holds for the corresponding pair of reductive groups. Thus class 10 is spherical.

For classes 11 and 12, 𝔨^{x} = su(4) ⊕ su(2) ⊕ 3u(1), and 𝔨^{{x, e, f}} = su(4) ⊕ u(1) = u(4), so K{x, e, f}_{C} is spherical in Kx_{C} . Consider either class 11 or class 12. In each case Z = 𝔭_{C}(3). 𝔰_{R} = 4u(1), a maximal torus in 𝔨{x, e, f}_{C}. A. Noel constructs this maximal torus in [17]. One verifies directly that the intersection of this maximal torus with su(4) does not act trivially on Z. This shows that 𝔭_{C}(3) (which has dimension 4) is not a trivial su(4)-module. Since, the dimension of the smallest non-trivial su(4) module is 4 this shows that 𝔭_{C}(3) is a 4 dimensional irreducible representation of u(4). It follows that B(S) = S_{0} has a dense open orbit on 𝔭_{C}(3) . This establishes the sphericality of classes 11 and 12.

9.7. EV III (𝔨 = so(16)). By Remark 4.18 and Proposition 4.17, the K_{C} classes 1–4 and 6 are the only ones that could be spherical. In fact classes 1 and 2 are spherical since each has height = 2.

For class 6, 𝔨^{{x, e, f}} = u(4) is diagonally embedded in 𝔨^{x} = u(4) ⊕ u(4), so that K{x, e, f}_{C} is spherical inside Kx_{C} . 𝔰_{R} = 4u(1), and an argument similar to that for classes 11 and 12 in EV II shows that B(S) has a dense orbit on 𝔭_{C}(3). So class 6 is spherical. Since class 3 lies in the closure of class 6, class 3 is also spherical.

For class 4, 𝔨^{x} = so(12) ⊕ su(2) ⊕ u(1), and 𝔨^{{x, e, f}} = su(6) ⊕ su(2). Therefore, so K{x, e, f}_{C} is not spherical in Kx_{C} . Hence class 4 is not spherical.

9.8. EIX (𝔨 = e_{7} ⊕ su(2)). By Remark 4.18 and Proposition 4.17, the K_{C} classes 1–6, 8 and 9 are the only ones that could be spherical. In fact classes 1, 2, and 3 are spherical because they have height equal to 2.

Since Z = 0 for classes 6 and 8 for sphericality, it suffices to check whether K{x, e, f}_{C} is spherical in Kx_{C} . For class 6, 𝔨^{x} = (E_{7})_{−133} ⊕ u(1), and 𝔨^{{x, e, f}} = (E_{6})_{−78}, so sphericality fails. For class 8, 𝔨^{x} = so(12)⊕su(2)⊕u(1), and 𝔨^{{x, e, f}} = so(10) ⊕ u(1). One verifies that 𝔨^{{x, e, f}} is contained in the so(12) ⊕ su(2) component of 𝔨^{x}. But K{x, e, f}_{C} is not spherical in Kx_{C} . Therefore, class 8 is not spherical. The closure of class 9 contains class 8. Hence by Proposition 4.11, class 9 is not spherical.

For class 4, 𝔨^{x} = E_{6(−78)} ⊕ 2u(1), and 𝔨^{{x, e, f}} = F_{4(−52)} ⊕ u(1). So sphericality holds for the reductive groups. 𝔰_{R} = so(8) ⊕ u(1). One checks that the maximal torus in S acts non-trivially on the one dimensional space Z which establishes the sphericality of class 4. A similar argument establishes the sphericality of class 5.

References

[1] Adams, J., J.-S. Huang, and D. A. Vogan, Jr., Functions on the Model Orbit in E_{8}, Rep. Theory 2 (1998), 224–263.

[2] Brion, M., and D. Luna, Sur la structure locale des variétés sphériques, Bull. Soc. Math. Fr. 115 (1987), 211–226.

[3] Collingwood, D. H., and W. M. McGovern, “Nilpotent Orbits in Semisimple Lie Algebras,” Van Nostrand Reinhold, 1993.

[4] D – okovic, D. Ž., Classification of nilpotent elements in simple exceptional real Lie algebras of inner type and description of their centralizers, J. Alg. 112 (1988), 503–524.

[5] —, Classification of nilpotent elements in simple real Lie algebras e_{6(6)} and e_{6(−26)} and description of their centralizers, J. Alg. 116 (1988), 196–207.

[6] —, The closure diagrams for nilpotent orbits of real forms of E_{6}, J. Lie Theory 11 (2001), 381–413.

[7] —, The closure diagrams for nilpotent orbits of the real forms EVI and EVII, Rep. Theory 5 (2001), 17–42.

[8] —, The closure diagrams for nilpotent orbits of the real forms EV, Rep. Theory (to appear).

[9] —, The closure ordering of nilpotent orbits of the complex symmetric pair SO_{p+q}, SO_{p} × SO_{q})), Canadian J. Math. (to appear).

[10] S. Helgason, “Differential Geometry, Lie Groups, and Symmetric Spaces,” Academic Press, New York, 1978.

[11] King, D. R., Spherical nilpotent orbits and the Kostant-Sekiguchi Correspondence, Trans. Amer. Math. Soc. 354 (2002), 4909–4920.

[12] Knapp, A., “Lie Groups Beyond an Introduction,” Progress in Mathematics, Volume 140, Birkhäuser, Boston, 1996.

[13] Kostant, B., and S. Rallis, Orbits and representations associated with symmetric spaces, Amer. J. Math. 93 (1971), 753–809.

[14] Krämer, M., Sphärische Untergruppen in kompakten zusammenhängenden Liegruppen, Comp. Math. 38 (1979), 129–153.

[15] McGovern, W., Rings of regular functions on nilpotent orbits II, Comm. Alg. 22 (1994), 765–772.

[16] Nishiyama, K., H. Ochiai, and K. Taniguchi, “Bernstein degree and associated cycles of Harish Chandra modules: Hermitian symmetric space, ” Kyushu University Preprint Series, 1999-16, Fukuoka, 1999.

[17] Noel, A., Classification of Admissible Nilpotent Orbits in Simple Exceptional Real Lie Algebras of Inner Type, Rep. Theory 5 (2001), 455–493.

[18] Ohta, T., Classification of Admissible Nilpotent Orbits in the Classical Real Lie Algebras, J. Alg. 136 (1991), 290–333.

[19] —, The closures of nipotent orbits in the classical symmetric pairs and their singularities, Tohoku Math. J. 43 (1991), 161–211.

[20] Dmitri I. Panyushev, Complexity and rank of homogeneous spaces, Geometriae Dedicata 34 (1990), 249–269.

[21] —, On spherical nilpotent orbits and beyond, Ann. Inst. Fourier (Grenoble) 49 (1999), 1453–1476.

[22] —, Complexity and nilpotent orbits, Manuscripta Math. 83 (1994), 223– 237.

[23] Sekiguchi, J., Remarks on real nilpotent orbits of a symmetric pair, J. Math. Soc. Japan 39 (1987), 127–138.

[24] Springer, T. A., and R. Steinberg, Conjugacy Classes, in: A. Borel, R. Carter et al, Eds., “Seminar on Algebraic Groups and Related Finite Groups,” Lecture Notes in Mathematics (Springer) 131 (1970), 167–266.

[25] Vogan Jr., D. A., Singular unitary representations, in: J. Carmona and M. Vergne, Eds., “Non-commutative Harmonic Analysis and Lie groups,” Lecture Notes in Mathematics (Springer) 880 (1987), 506–535.

Donald R. King Department of Mathematics 567 Lake Hall Northeastern University Boston MA 02115 donking@neu.edu

Received March 25, 2002 and in final form October 28, 2003