Journal of Lie Theory Volume 12 (2002) 535–538 C ◯2002 Heldermann Verlag

Two Observations on Irreducible Representations of Groups

Jorge Galindo, Pierre de la Harpe, and Thierry Vust^{∗}

Communicated by A. Valette

Abstract. For an irreducible representation of a connected affine algebraic group G in a vector space V of dimension at least 2, it is shown that the intersection of any orbit π(G)x (with x∈V ) and any hyperplane of V is non-empty. The question is raised to decide whether an analogous fact holds for irreducible continuous representations of connected compact groups, for example of SU(2). Keywords and phrases: Irreducible representations, orbits, algebraic groups, compact groups Subject Classification: 22 E 45

By definition, a linear representation π : G ⟶ GL(V) of a group G in a vector space V is irreducible if, for any vector x ≠ 0 in V and for any hyperplane H of V , the orbit π(G)x does not lie inside H. The purpose of this note is to record how irreducibility may imply other geometrical properties of the orbits, either in general as in the most elementary Proposition 1 below about “affine irreducibility”, or for representations of algebraic groups as in Proposition 2. We provide also examples which show that Proposition 2 has no analogue for noncompact semisimple real Lie groups, but we leave open the question to decide if it has for compact semisimple Lie groups.

Proposition 1. Let G be a group, V a vector space over some field, and π : G ⟶ GL(V) an irreducible linear representation distinct from the unit representation. If A is an affine subspace of V which is invariant by G, then A = 0 or A = V .

Proof. If an affine subspace A is π(G)-invariant, so is the linear space H of differences of vectors in A, so that H is one of 0 or V , and the same holds for A.

Proposition 2. Let G be a connected algebraic group over some algebraically closed field 𝕂, let V a finite dimensional vector space of dimension at least

ISSN 0949–5932 / $2.50 ^{_{C} ◯} Heldermann Verlag two, and let π : G ⟶ GL(V) be a rational irreducible representation. For any linear hyperplane H of V and any x ∈ V , the intersection of H with the orbit X = π(G)x is non empty. Proof. Consider first the case of a group G which is semisimple. Choose a linear form f ≠ 0 on V such that H = ker(f). Define a regular function ϕ : G ⟶ 𝕂 by ϕ(g) = f(π(g)x).

∗ The authors acknowledge support from the Swiss National Science Foundation.

Assume (ab absurdo) that the intersection of H and X is empty. Then ϕ does not have any zero on G. A theorem of Rosenlicht (see below) implies that there exists a constant c ≠ 0 such that cϕ is a group homomorphism G ⟶ 𝕂^{∗}; this implies that ϕ is constant since G is perfect. Thus X is contained in an affine hyperplane of V . The affine hull of X is non-trivial and invariant by G; this is absurd by Proposition 1, so that the proposition is proved in the semisimple case.

Consider now the general case. Let R_{u} denote the unipotent radical of G. By a theorem of Kolchin (see e.g. 4.8 in [1]), the subspace V^{u} = {v ∈ V | π(r)v = v for all r ∈ R_{u}} is not reduced to zero. This space being π(G)- invariant, because R_{u} is normal in G, and π being irreducible, we have V^{u} = V . Consequently, we may replace G by G/R_{u}, namely we may assume that G is reductive.

Let T denote the solvable radical of G, which is a torus (11.21 in [1]). Let V = ⊕V^{χ} denote the weight space decomposition of the restriction π|T, where V^{χ} = {v ∈ V | π(t)v = χ(t)v for all t ∈ T } for χ ∈ Hom(T, 𝕂^{∗}). We may choose ψ ∈ Hom(T, 𝕂^{∗}) such that V^{ψ} ≠ {0}. Since T is normal in G and since the abelian group Hom(T, 𝕂^{∗}) is finitely generated (8.2 in [1]), there is a natural action of the connected group G on Hom(T, 𝕂^{∗}) and this action is trivial. Hence V^{ψ} is π(G)-invariant, and indeed is equal to V by irreducibility of π. Thus π coincides on T with some ψ ∈ Hom(T, 𝕂^{∗}).

Now G is a product of its derived group DG and of T, and DG is semisimple (14.2 in [1]). Thus the equality π|T = ψ ⊗ id_{V} and the irreducibility of π imply that the restriction of π to the semisimple group DG is irreducible. This ends the proof of the reduction of the general case to the semisimple case.

Reminder of Rosenlicht’s result [6]. If Y, Z are two irreducible affine algebraic varieties, any scalar-valued function on the product Y × Z which is regular and without zero is a product of a regular function on Y by a regular function on Z. Thus, if ϕ is a regular function without zero on a linear algebraic group G, there exist regular functions ψ, χ such that ϕ(gh) = ψ(g)χ(h) for all g, h ∈ G. Set c = ϕ(1)^{−1} and let φ denote the function cϕ; the previous relation implies that φ = ψ(1)^{−1}ψ = χ(1)^{−1}χ and that φ(gh) = φ(g)φ(h) for all g, h ∈ G, namely that φ is a character on G, by which we mean here a homomorphism of groups G ⟶ 𝕂^{∗}. For an exposition of Rosenlicht’s result, see [4]; see also [2]. Corollary. Let G be a reductive connected complex Lie group, let V a finite dimensional complex vector space of dimension at least two, and let π : G ⟶ GL(V) be an irreducible holomorphic representation. For any linear hyperplane H of V and any x ∈ V , the intersection of H with the orbit X = π(G)x is non empty. Proof. This is a straightforward consequence of Proposition 2, since a connected reductive complex Lie group G has a unique algebraic structure, and a holomorphic representation of such a group is necessarily algebraic. See e.g. Theorem 6.4 of Chapter 1 and Theorem 2.8 of Chapter 4 in [5].

Observations. There are no analogues of Proposition 2 for finite groups and for simple connected real Lie groups, as the following examples show.

(i) If G is a finite group, G-orbits in V \ {0} and hyperplanes are generically disjoint.

(ii) Let π be the 2-dimensional irreducible representation of the group SL_{2}(ℝ) in the space ℂ^{2} . For a vector x ∈ ℝ^{2} , x ≠ 0, and the linear span H of the vector (1, i) ∈ ℂ^{2} , the SL_{2}(ℝ)-orbit of x and the hyperplane H are disjoint.

For another example, consider the 3-dimensional irreducible representation of SL_{2}(ℝ) in the space V of homogeneous polynomials of degree 2 with complex coefficients in 2 variables ξ, η. If x ∈ V is the polynomial ξη, its SL_{2}(ℝ)-orbit X is a surface of equation ρ^{2} −4στ = 1 (with respect to appropriate coordinates (ρ, σ, τ) on V ), and its intersection with the complex hyperplane of equation σ = \sqrt{i} τ is empty.

(iii) Consider more generally an integer n ≥ 2, the connected component G of the group SO(n, 1), and the natural irreducible representation π of G in ℂ^{n+1} . For a non-zero vector x ∈ ℝ^{n+1} inside and a real hyperplane H_{0} ⊂ ℝ^{n+1} outside the light cone, it is clear that the orbit π(G)x is disjoint from H_{0} ; it follows that π(G)x is also disjoint from the complexified hyperplane H_{0} ⊗_{ℝ} ℂ in ℂ^{n+1} .

Question. In which situations does some Proposition 2 hold? what about a connected compact group and an irreducible continuous representation? what about the irreducible representation of SU(n)? of SU(2)? We spell out explicitely the last particular case of the question:

Let π_{k} be the natural representation of SU(2) in the space P_{k} of complex polynomials in two variables which are homogeneous of degree k, for some k ≥ 1, let H be a complex hyperplane in P_{k} and let P ∈ P_{k}; is it always true that π_{k}(SU(2))P ∩ H ≠ Ø?

Remarks. (i) Let G be a compact topological group, V an Hermitian space, and π : G ⟶ U(V) an irreducible continuous unitary representation distinct from the unit representation. It is known [3] that, for any vector x ∈ V of norm 1, the diameter max_{g∈G} ∥π(g)x − x∥ of the orbit π(G)x is strictly larger than \sqrt{2}.

(ii) Let G be a compact connected topological group, V a finite dimensional real vector space, π : G ⟶ GL(V) an irreducible continuous representation distinct from the unit representation, X = π(G)x the G-orbit of a vector x ≠ 0 in V , and H an hyperplane of V , say H = ker(f) for some linear form f ≠ 0 on V . Then the intersection of H and X is non empty.

Indeed, define as above ϕ : G ⟶ ℝ by ϕ(g) = f(π(g)x). If X ∩H = Ø, then ϕ is either strictly positive or strictly negative on G, so that ∫_{G} ϕ(g)dg = f(y) ≠ 0 for y = ∫_{G} π(g)x dg, and in particular y ≠ 0; but this is impossible because y is π(G)-invariant by invariance of the Haar measure dg on G.

We are grateful to Gus Lehrer and Alain Valette for useful comments on the observations of the present Note.

References

[1] Borel, A. “Linear algebraic groups”, second enlarged edition, Springer-Verlag New York etc., 1991.

[2] Broughton, S. A., A note on characters of algebraic groups, Proc. Amer. Math. Soc. 89 (1983), 39–40.

[3] Deutsch, A. and A. Valette, On diameters of orbits of compact groups in unitary representations, J. Austral. Math. Soc. Ser. A 59 (1995), 308– 312.

[4] Knop, F., H. Kraft, and T. Vust, The Picard group of a G-variety, in: H. Kraft, P. Slodowy, and T.A. Springer, Editors, ,,Algebraische Transformationsgruppen und Invariantentheorie“, DMV Sem. 13, Birkhäuser, Basel, 1989, 77–87.

[5] Gorbatsevich, V. V., A. L. Onishchik, and E. B. Vinberg, “Structure of Lie groups and Lie algebras,” Lie groups and Lie algebras III, Encyclopaedia of Math. Sciences, Vol. 41, Springer-Verlag Berlin etc., 1994.

[6] Rosenlicht, M., Toroidal algebraic groups, Proc. Amer. Math. Soc. 12 (1961), 984–988.

Jorge Galindo Departamento de Matemáticas Universidad Jaume I 8029-AP, Castellón Spain jgalindo@mat.uji.es Pierre de la Harpe Section de Mathématiques Université de Genève C.P. 240, CH-1211 Genève 24 Switzerland Pierre.delaHarpe@math.unige.ch

Thierry Vust

Section de Mathématiques Université de Genève C.P. 240, CH-1211 Genève 24 Switzerland Thierry.Vust@math.unige.ch

Received July 1, 2001 and in final form September 9, 2001