Journal of Lie Theory Volume 6 (1996) 249–269 C ◯1996 Heldermann Verlag

On invariants of a set of matrices

E. B. Vinberg

Communicated by K. H. Hofmann

Abstract. For any connected reductive linear group H⊂SL_{n}(k) it is proved that the algebra of invariants of a set of matrices X_{1},...,X_{p}∈H (p⩾2) under simultaneous conjugations by matrices of N(H) is the normalization of the algebra generated by the traces tr X_{i1} ...X_{ik} .

It is a classical fact that the algebra of invariants of a set of matrices with respect to simultaneous conjugations is generated by the traces of their products.

Let now H ⊂ SL_{n}(k) be a reductive algebraic linear group. Consider the algebra of invariants of a set of matrices

(X_{1}, …, X_{p}) ∈ H^{p} = \underbrace{H × ⋯ × H}_{p}

with respect to simultaneous conjugations by the matrices of the normalizer N(H) of H in SL_{n}(k). Is it true that this algebra is generated by the traces of products of X_{1}, …, X_{p}?

Making use of the classical invariant theory, one can easily show that it is true for any classical linear group

H = SL_{n}(k), SO_{n}(k), Sp_{n}(k).

(Except for the case SO_{n}(k), n even, the action of N(H) on H reduces to that of H. In the exceptional case it reduces to the action of O_{n}(k).)

An analogous question can be asked for the field of rational invariants. It was proved in my preprint [5] that the answer to this question is positive for any connected reductive linear group H and p ⩾ 2. For p = 1, the answer is in general negative (see a counterexample in Section 4).

As for the original question, the answer is in general negative, even for H connected and p ⩾ 2 (see a counterexample in Section 4). It was conjectured in [5] that under these assumptions the algebra of invariants is the normalization of the algebra generated by the traces. The main purpose of this paper is to prove this conjecture. In fact, we prove it in greater generality.

ISSN 0949–5932 / $2.50 ^{_{C} ◯} Heldermann Verlag

A portion of this work was done during my visit to the Tata Institute of Fundamental Research in January–February of 1993. I am grateful to this institute for its hospitality. The completion of this research was made possible by the grant MQZ300 from the International Science Foundation and and the Russian Government, and by the grant 95-01-00783a from the Russian Foundation of Fundamental Research.

I am grateful to E. A. Tevelev, who suggested a simpler proof of Proposition 8 in the case when G_{0} is a torus.

1. Statement of the main results.

The ground field k is supposed to be algebraically closed and of characteristic zero. For an action of a reductive algebraic group H on an affine algebraic variety X the algebra of G-invariant polynomials (respectively, the field of G-invariant rational functions) on X is denoted by k[X]^{G} (respectively, by k(X)^{G}). The categorical quotient of the action G : X, i.e., the spectrum of k[X]^{G}, is denoted by X//G, and the canonical morphism X → X//G defined by the embedding k[X]^{G} ⊂ k[X] is denoted by π_{G}.

We use some standard facts of invariant theory, which can be found, for example, in [2]. In particular, it is a standard fact that each fiber of π_{G} contains exactly one closed G-orbit.

Let a reductive algebraic group G act on G^{p} by simultaneous conjugations. It is proved in [3] that the orbit of a p-tuple

g = (g_{1}, …, g_{p}) ∈ G^{p}

is closed iff the algebraic subgroup (topologically) generated by g is reductive.

For any reductive algebraic subgroup H ⊂ G we have the following commutative diagram of natural morphisms:

(1)  \begin{matrix}H^{p} & ⟶^{π_{H}} & H^{p}//H & ⟶^{π_{N(H)/H}} & H^{p}//N(H) \\ ↓ & & ↓^{ψ} & & ↓^{φ} \\ G^{p} & ⟶^{π_{G}} & G^{p}//G & ↩ & H^{p}//G \end{matrix}

where N(H) is the normalizer of H in G and

(2)  H^{p}//G ≑ \overline{π_{G}(H^{p})}(= \overline{φ(H^{p}//N(H))} = \overline{ψ(H^{p}//H)}).

Since the group of automorphisms of H defined by N(H) contains the group of inner automorphisms of H as a subgroup of finite index (see Corollary 2 to Proposition 3), the morphism π_{N(H)/H} is finite.

Now we formulate the main results of the paper.

Theorem 1. The morphism ψ is finite (and, in particular, π_{G}(H^{p}) is closed in G^{p}//G).

An interpretation of this theorem in terms of varieties of characters of finitely generated groups will be given in Section 16.

Theorem 2. If H is connected and p ⩾ 2, the morhism φ is birational.

Very often, but not always, the morphism φ is birational for p = 1 as well.

Corollary 2a. If H is connected and p ⩾ 2, the morphism φ is the normalization.

In other words, the algebra k[H^{p}]^{N(H)} is the normalization of the restriction to H^{p} of the algebra k[G^{p}]^{G}. Applying this to G = SL_{n}(k), we obtain that for any connected reductive group H ⊂ SL_{n}(k) and any p ⩾ 2 the algebra k[H^{p}]^{N(H)} is the normalization of the algebra generated by the traces of products.

In the case G = SL_{n}(k) Theorem 2 was proved in the preprint [5]. In the general case the proof is the same. It will be given in the next section.

Theorem 2 can be generalized to non-connected subgroups. To this end, we need some modification of H^{p}.

Denote by H̃^{p} the union of connected (=irreducible) components of H^{p} containing a generating set of H. Note that any connected component of H^{p} has the form H_{1} × ⋯ × H_{p}, where H_{1}, …, H_{p} are some connected components of H. As we shall prove (see Corollary 1 of Proposition 8), for p ⩾ 2 H_{1} × ⋯ × H_{p} ⊂ H̃^{p} iff H_{1}, …, H_{p} generate the group H/H_{0} . Obviously, H̃^{p} is invariant under N(H).

The diagram (1) can be re-written with replacing H^{p} by H̃^{p}. Denote by φ̃ the morphism replacing φ in the modified diagram.

Theorem 2′. The morphism φ̃ is birational.

Corollary 2′ a. The morphism φ̃ is the normalization.

2. Proof of Theorem 2

To prove the theorem, we need some preparatory results on generating sets of reductive groups.

Unless stated otherwise, we understand the term “generate” in the topological sense. So “an algebraic group G is generated by a p-tuple g ∈ G^{p} ” means that the subgroup algebraically generated by g is (Zariski) dense in G.

Proposition 1. Any algebraic torus T is generated by one element. Moreover, the set of all generating elements is dense in T. Proof. An element t generates T iff there is no nontrivial character of T vanishing at t. For such t, one can take any element, whose coordinates are different primes. Such elements constitute a dense subset in T.

Corollary 1a. For any p ⩾ 1 the set of p-tuples generating T is dense in T^{p}.

Proposition 2. Any connected reductive algebraic group G is generated by two elements. Moreover, the set of all generating pairs is dense in G^{2} .

Proof. Let t ∈ G be a semisimple element generating a maximal torus T of G. It follows from the root decomposition of the tangent algebra of G with respect to T that there exist only finitely many algebraic subgroups of G containing T. If an element h does not belong to any of maximal algebraic subgroups containing T, then the pair (t, h) generates G. The set of pairs of such type is dense in G^{2} .

Corollary 2a. For any p ⩾ 2 the set of p-tuples generating G is dense in G^{p}.

Proof of Theorem 2. Let us note that if h ∈ H^{p} generates H and, for some g ∈ G, we have ghg^{−1} ∈ H^{p}, then gHg^{−1} ⊂ H, which implies g ∈ N(H).

Denote by H_{gen}^{p} the set of all p-tuples generating H. Under the hypotheses of Theorem 2, H_{gen}^{p} is dense in H^{p}. It follows that π_{G}(H_{gen}^{p}) is dense in H^{p}//G. For any p-tuple h ∈ H^{p} generating a reductive subgroup, the fiber φ^{−1}(π_{G}(h)) consists of the points π_{N(H)}(h′), where h′ ∈ H^{p} is G-equivalent to h. It follows from what was proved above that if h ∈ H_{gen}^{p}, then φ^{−1}(π_{G}(h)) consists of a single point. Now Theorem 2 is implied by the following

Lemma 1. Let φ: X → Y be a dominant morphism of irreducible algebraic varieties. If there exists a dense subset Z ⊂ Y such that for any y ∈ Z the fiber φ^{−1}(y) consists of a single point, then φ is birational.

Proof. If φ is not birational, then there exists an open subset Y_{0} ⊂ Y such that for any y ∈ Y_{0} the fiber φ^{−1}(y) contains more than one point. Since Y_{0} ∩ Z ≠ ∅, this contradicts our condition.

The proof of Theorem 2′ is similar. It will be given in Section 8.

3. An intermediate result

Let us now prove that the morphism ψ is quasi-finite, i.e., that all of its fibers are finite. This means that, for any closed G-orbit O in G^{p}, the intersection O ∩ H^{p} decomposes into finitely many (closed) H -orbits.

To prove this, we apply the following lemma, going back to R. W. Richardson. Its proof can be found in [1].

Lemma 2. Let an algebraic group G act on an algebraic variety X. Suppose H ⊂ G is an algebraic subgroup and Y ⊂ X is an H -invariant subvariety. If for any y ∈ Y

(3)  𝔤(y) ∩ T_{y}(Y) = 𝔥(y),

then, for any G-orbit O, the intersection O ∩ Y decomposes into finitely many H -orbits, each of them being closed in O ∩ Y . (Here 𝔤 and 𝔥 are the tangent algebras of G and H, respectively, and T_{y}(Y) is the tangent space of Y at y.)

Proposition 3. Let G be a reductive group, and H a reductive subgroup of G. Then for any G-orbit O in G^{p}, the intersection O ∩ H^{p} decomposes into finitely many H -orbits, each of them being closed in O ∩ H^{p}.

Proof. We apply Lemma 2 to the action G: G^{p} and the subvariety H^{p} ⊂ G^{p}. For a p-tuple h = (h_{1}, …, h_{p}) ∈ H^{p}, we have

T_{h}(G^{p}) = 𝔤h_{1} ⊕ … 𝔤h_{p},
T_{h}(H^{p}) = 𝔥h_{1} ⊕ … 𝔥h_{p},

and, for any ξ ∈ 𝔤,

ξ(h) = (ξh_{1} − h_{1}ξ, …, ξh_{p} − h_{p}ξ) = ((ξ − Ad(h_{1})ξ)h_{1}, …, (ξ − Ad(h_{p})ξ)h_{p}).

So ξ(h) ∈ T_{h}(H^{p}) iff

(4)  ξ − Ad(h_{i})ξ ∈ 𝔥, i = 1, …, p.

Let 𝔪 be an Ad(H)-invariant complementary subspace of 𝔥 in 𝔤. If

ξ = η + ζ (η ∈ 𝔥, ζ ∈ 𝔪),

then (4) implies that

ζ − Ad(h_{i})ζ = 0, i = 1, …, p,

whence

ξ(h) = η(h) ∈ 𝔥(h).

So the condition (3) is fulfilled in our situation and Lemma 2 gives the desired result.

Corollary 3a. The morphism ψ is quasi-finite.

Corollary 3b. The group of automorphisms of H defined by N(H) is a finite extension of the group of inner automorphisms of H.

Proof. Take any p-tuple h = (h_{1}, …, h_{p}) generating H. It follows from the proposition that the N(H)-orbit of h decomposes into finitely many H -orbits. Since any automorphism a of H is uniquely defined by a(h), this implies the assertion.

4. Some counterexamples

The following example shows that for p = 1 (and H connected) the morphism φ need not be birational.

Let

G = SL_{5}(k), H = SL_{2}(k) ⊕ SO_{3}(k).

Since H has no outer automorphisms, the action of N(H) on H coincides with that of H. It h_{1} ∈ SL_{2}(k) has eigenvalues λ, λ^{−1} and h_{2} ∈ SO_{3}(k) has eigenvalues μ, μ^{−1}, 1, then h = h_{1} ⊕ h_{2} has eigenvalues λ, λ^{−1}, μ, μ^{−1}, 1. The same eigenvalues has h′ = h′_{1} ⊕ h′_{2} , where h′_{1} ∈ SL_{2}(k) has eigenvalues μ, μ^{−1} and h′_{2} ∈ SO_{3}(k) has eigenvalues λ, λ^{−1}, 1. If λ, μ ≠ ±1 and μ ≠ λ, λ^{−1} , then the H -orbits of h and h′ are closed and distinct from each other, while their G-orbits coincide.

The next example shows that even if p ⩾ 2 (and H connected) the morphism φ need not be injective.

Let

G = SL_{5}(k), H = R(SL_{2}(k)),

where R is the 5-dimensional irreducible representation of SL_{2}(k). As in the preceding example, H has no outer automorphisms, so the action of N(H) on H^{p} coincides with that of H. If u ∈ SL_{2}(k) has eigenvalues λ, λ^{−1} , then R(u) has eigenvalues λ^{2}, λ^{−2}, λ, λ^{−1}, 1. Let λ = exp \frac{2πi}{5} . Then R(u) and R(u^{2}) have the same eigenvalues, so their G-orbits coincide. At the same time their H -orbits are closed and distinct from each other. The same is true for the p-tuples

(R(u), …, R(u)), (R(u^{2}), …, R(u^{2})) ∈ G^{p}

for any p.

5. Toric subvarieties

In order to handle nonconnected groups, we need some generalization of maximal tori. We follow ideas of Gantmacher [2]. For more details see [3].

Let G be a reductive group, G_{0} its connected component containing e, and G_{1} some connected component of G.

Any coset S of a torus in G, lying in the centralizer of this torus, will be called a toric subvariety. Its normalizer (respectively, centralizer) in G_{0} will be denoted by N_{0}(S) (respectively, Z_{0}(S)). The group W_{0}(S) = N_{0}(S)/Z_{0}(S) will be called the Weyl group of S. Note that the group W_{0}(S) is finite, because it is an algebraic group of automorphisms of the subgroup generated by S, which is a direct product of a torus and a finite cyclic group.

Proposition 4. 1) Any two maximal toric subvarieties of G_{1} are G_{0} - conjugate.

2) Any semisimple element of G is contained in a maximal toric subvariety.

3) Two elements of a maximal toric subvariety S are G_{0} -conjugate iff they are W_{0}(S)- conjugate. One can easily show (and it follows from the first assertion of the proposition) that any maximal toric subvariety of a connected reductive group is a maximal torus. In this partial case the proof of the proposition is well-known. In the general case only small modifications are needed.

Sketch of the proof. Let S be a maximal toric subvariety contained in G_{1} , and g ∈ S. We have S = gT, where T is a maximal torus in the centralizer Z_{0}(g) of g in G_{0} . An element g ∈ S is called regular, if Z_{0}(g) is a finite extension of T. Obviously, the subvariety S is uniquely determined by any regular element of it. Considering the weight decomposition of the tangent algebra 𝔤 of G with respect to the subgroup generated by S, we see that the regular elements constitute a non-empty open subset in S. Denote it by S^{reg} . A standard computation shows that the map

G_{0} × S^{reg} → G_{1}, (g, s) ↦ gsg^{−1},

is smooth and hence its image is open in G_{1} . As usually, this implies the first assertion of the proposition. The second one is trivial. The proof of the third one does not differ from the usual proof for the case when S is a maximal torus.

Corollary 4a. Let S ⊂ G_{1} be a maximal toric subvariety. Then the natural morphism

S//W_{0}(S) ⟶ G_{1}//G_{0}

is an isomorphism.

The following consequence of this theory will be used in the proof of Theorem 1.

Proposition 5. The restriction of the morphism π_{G}: G → G//G to any toric subvariety S ⊂ G is a finite morphism. Proof. Let S_{1} be a maximal toric subvariety containing S. If S (and S_{1} ) is contained in a connected component G_{1} of G, then the restriction of π_{G} on S is the composition of the following finite morphisms:

S ↪ S_{1} → S_{1}/W_{0}(S_{1}) ˜→ G_{1}//G_{0} → G//G_{0} → G//G.

Let now S be a maximal toric subvariety of G, which is a coset of a torus T. It is known [3] that the centralizer of T in G_{0} is a maximal torus of G.

Consider the weight decomposition 𝔤 = ∑_{α} 𝔤_{α} of the Lie algebra 𝔤 with respect to the subgroup generated by S. We have 𝔤_{0} = 𝔱 (the tangent algebra of T). The non-zero weights vanishing on T are called imaginary roots. Denote the set of imaginary roots by Δ_{im} . The sum 𝔱 + ∑_{α∈Δ_{im}} 𝔤_{α} is a Cartan subalgebra

of 𝔤. The weights not vanishing on T are called real roots. Denote the set of real roots by Δ_{re} . It is known [3] that dim 𝔤_{α} = 1 for any α ∈ Δ_{re} .

Proposition 6. If G is semisimple, there are only finitely many maximal algebraic subgroups of G containing S. Proof. Let H ⊂ G be such a subgroup. Since H ⊂ N(H_{0}), we have N(H_{0}) = G or H.

In the first case H_{0} is a normal subgroup of G_{0} . Since H_{0} ⊃ T, the centralizer of H_{0} in G_{0} is abelian. Hence H_{0} = G_{0} , and there are only finitely many possibilities for H.

In the second case H is completely defined by its tangent algebra 𝔥, which is invariant under S, contains T, and coincides with its normalizer in 𝔤. Let us prove that there are only finitely many such subalgebras in 𝔤.

Let 𝔥 be such a subalgebra, and

Δ_{re}(𝔥) = {α ∈ Δ_{re}: 𝔤_{α} ⊂ 𝔥}.

We have

𝔥 = 𝔱 + ∑_{α∈Δ_{im}}(𝔥 ∩ 𝔤_{α}) + ∑_{α∈Δ_{re}(𝔥)} 𝔤_{α}.

Since 𝔥 coincides with its normalizer in 𝔤,

𝔥 ∩ 𝔤_{α} = {ξ ∈ 𝔤_{α}: [ξ, 𝔤_{β}] = 0 for all β ∈ Δ_{re}(𝔥) such that β + α ∉ Δ_{re}(𝔥)}

for α ∈ Δ_{im} . It follows that 𝔥 is completely defined by Δ_{re}(𝔥).

6. Splitting of nonconnected reductive groups

Let G be any reductive group.

Proposition 7. There exists a finite subgroup F ⊂ G such that G = G_{0}F. Proof. For an algebraic group H, let Aut H denote the group of all automorphisms and Int H the group of inner automorphisms of H. It is well-known that if H is a connected semisimple group, then Aut H is a semidirect product Int H ⋋ A(H), where A(H) is some finite group.

Put (G_{0}, G_{0}) = H and consider the subgroup of G, consisting of the elements g such that the automorphism h ↦ ghg^{−1} of H belongs to A(H). It contains representatives of all cosets of H (and the more of all cosets of G_{0} ) and its connected component is contained in the centralizer of H and hence is a torus. Thereby the proof reduces to the case when G_{0} is a torus.

Let G_{0} = T be a torus, and G/G_{0} = Γ. The extention T ⊂ G is described by a cocycle c ∈ Z^{2}(Γ, T). Let T_{fin} denote the subgroup of elements of finite order of T. Since in the group T/T_{fin} each equation x^{n} = a has a unique solution, H^{2}(Γ, T/T_{fin}) = 0. Therefore the cocycle c is equivalent to a cocycle c′ ∈ Z^{2}(Γ, T_{fin}). In fact all the values of c′ belong to a (finitely generated and hence) finite subgroup Δ ⊂ T. This means that there exists a finite subgroup F ⊂ G containing representatives of all cosets of T and intersecting T in Δ.

In the notation of the proposition, G is isomorphic to a quotient group of the semidirect product G_{0} ⋋ F with respect to some finite normal subgroup.

7. Generators of non-connected reductive groups

To prove Theorem 2′, we need the following generalization of Proposition 2.

Proposition 8. Let G_{1} and G_{2} be (possibly coinciding) connected components of a reductive group G. Then there are elements g_{1} ∈ G_{1} and g_{2} ∈ G_{2} generating an algebraic subgroup of finite index in G. Moreover, the set of all such pairs is dense in G_{1} × G_{2} .

Proof. We may assume that the group G/G_{0} is generated by G_{1} and G_{2} . Under this condition we are to prove that there are elements g_{1} ∈ G_{1} and g_{2} ∈ G_{2} generating G, and the set of all such pairs is dense in G_{1} × G_{2} .

Denote by Z the center of G_{0} . It is easy to see that an algebraic subgroup H ⊂ G coincides with G iff it projects both onto G/Z and G/(G_{0}, G_{0}). In such a way the proof of the theorem reduces to two partial cases when G is semisimple or G_{0} is a torus.

Let first G be semisimple. Take any maximal toric subvariety S ⊂ G_{1} . The subgroup generated by S is a direct product of a torus T and a finite cyclic group C. Let c be the generator of C lying in S. Then for any element t ∈ T generating T, the element s = tc generates T × C. The set of all such elements s is dense in S.

Let s be chosen as above. According to Proposition 6 there are only finitely many maximal algebraic subgroups of G containing s. Note that none of them can contain G_{2} . If h ∈ G_{2} does not belong to any of these subgroups, the pair (s, h) generates G. Obviously, the set of all pairs (s, h) of such type is dense in G_{1} × G_{2} .

Let now G_{0} = T be a torus. In view of Proposition 7 it suffices to consider the case G = T ⋋ Γ, where Γ is some finite group. In this case, we have G_{1} = T γ_{1}, G_{2} = T γ_{2} , where γ_{1}, γ_{2} ∈ Γ.

Consider the homomorphism of the free group on two generators to G/T, taking the i-th generator to G_{i}. Let w_{1}, …, w_{n} be generators of its kernel. Then for g_{1} ∈ G_{1}, g_{2} ∈ G_{2} the intersection of T with the algebraic subgroup generated by g_{1} and g_{2} is the algebraic subgroup generated by w_{1}(g_{1}, g_{2}), …, w_{n}(g_{1}, g_{2}).

For i = 1, …, n, the map

φ: T^{2} → T, (t_{1}, t_{2}) ↦ w_{i}(t_{1}γ_{1}, t_{2}γ_{2}),

is a homomorphism of algebraic tori. The elements t_{1}γ_{1} and t_{2}γ_{2} generate G iff φ_{1}(t_{1}, t_{2}), …, φ_{n}(t_{1}, t_{2}) generate T. Note that if (t_{1}, t_{2}) generates T^{2} , then φ_{i}(t_{1}, t_{2}) generates φ_{i}(T^{2}) for each i. So it suffices to prove that the subgroup T′ generated by φ_{1}(T^{2}), …, φ_{n}(T^{2}) coincides with T.

It is easy to see that T′ is a normal subgroup of G. Passing to the quotient group, we may assume that T′ = {e}. Under this condition we are to prove that T = {e}.

The condition T′ = {e} means that for any t_{1}, t_{2} ∈ T the intersection of T with the subgroup generated by t_{1}γ_{1} and t_{2}γ_{2} reduces to {e}. For any γ ∈ Γ denote by c(γ) the (unique) element of T such that c(γ)γ belongs to this subgroup. Then c is a 1-cocycle on Γ with values in T. By definition c(γ_{1}) = t_{1} , c(γ_{2}) = t_{2} . If follows that dim Z^{1}(Γ, T) = 2 dim T, whence

dim H^{1}(Γ, T) ⩾ dim Z^{1}(Γ, T) − dim C^{0}(Γ, T) = dim T.

But the group H^{1}(Γ, T) is finite. Hence dim T = 0.

Corollary 8a. Let p ⩾ 2 and some connected components G_{1}, …, G_{p} of G generate the group G/G_{0} . Then G_{1} × ⋯ × G_{p} contains a generating p-tuple of G; moreover, the set of all such p-tuples is dense in G_{1} × ⋯ × G_{p}.

Corollary 8b. The set of generating p-tuples is dense in G̃^{p} for any p. (The definition of G̃^{p} is given in Section 1.)

Proof. For p ⩾ 2 this follows immediately from Corollary 1. For p = 1 the only case is to be considered when G is a direct product of a torus and a finite cyclic group. This case was handled in the proof of Proposition 8.

Remark 1. One can prove that, if G is semisimple, the set of generating ptuples is open in G^{p}. But we do not need this fact.

Now we can prove Theorem 2′ by repeating the proof of Theorem 2 given in Section 2, replacing φ by φ̃ and H^{p} by H̃^{p}.

8. Stable actions

In order to prove Theorem 1 we need some preliminary results. In this section we recall the notion of a stable action.

Let a reductive group G act on an affine variety X. We admit X to be reducible but require that G act transitively on the set of irreducible components of X (so X//G is irreducible). Such a G-variety will be called G-irreducible.

Let m_{G} be the maximal (=typical) dimension of orbits of the action G : X, and X^{deg} the union of orbits of lesser dimension. Obviously, X^{deg} is a proper G-invariant closed subvariety of X.

We put X//G = Y .

Proposition 9. The following properties of the action G : X are equivalent:

(a) there exists a nonempty open subset Ỹ ⊂ Y such that for any y ∈ Ỹ the fiber π_{G}^{−1}(y) consists of a single orbit;

(b) there exists an invariant non-empty open subset X̃ ⊂ X consisting of closed orbits;

(c) there is a closed orbit of dimension m_{G};

(d) π_{G}(X^{deg}) ≠ Y .

These equivalences are well-known: see [1] and references there. Nevertheless we give their proof for convenience of the reader. Proof. (a) ⇒ (b). Take X̃ = π_{G}^{−1}(Ỹ).

(b) ⇒ (c) is obvious.

(c) ⇒ (d). If O is a closed orbit of dimension m_{G}, then π_{G}(O) ∉ π_{G}(X^{deg}).

(d) ⇒ (a). Note that π_{G}(X^{deg}) is a closed subvariety of Y . Let Ỹ be the complement of this subvariety. Then for any y ∈ Ỹ the fiber π_{G}^{−1}(y) consists of orbits of the maximal dimension, hence of a single orbit.

An action, satisfying the equivalent conditions of Proposition 9, is called stable. In the case k[X]^{G} = k the stability of the action means its transitivity.

For a stable action, orbits in general position are separated by invariants, which implies that

(6)  dim X//G = dim X − m_{G}.

9. The dimension of H^{p}//G

For the action G: G^{p} considered above, all the stabilizers contain the center Z(G) of G. Moreover, if a p-tuple generates G, then its stabilizer coincides with Z(G) and its orbit is closed. So the action of G on any G-irreducible component of G^{p} lying in G̃^{p} is stable and applying formula (6) yields that all irreducible components of G̃^{p}//G have the dimension equal to

(7)  dim G̃^{p}//G = (p − 1) dim G + dim Z(G).

Let now H be a connected reductive subgroup of G. It was proved in Section 3, that the natural dominant morphism H^{p}//H → H^{p}//G is quasi-finite. Hence

(8)  dim H̃^{p}//G = dim H̃^{p}//H = (p − 1) dim H + dim Z(H),

where Z(H) denotes the center of H.

10. Fields of definition of reductive groups

The results presented in this section are, of course, known to specialists, but I have not found an appropriate reference.

Let as above k be an algebraically closed field of characteristic zero, and K an algebraically closed extension of k.

Any algebraic k -variety X (respectively, algebraic k -group G) defines in a natural way an algebraic K -variety X(K) (respectively, an algebraic K - group G(K)). Such an algebraic K -variety (respectively, an algebraic K -group) is called defined over k.

Any morphism X → Y of algebraic k -varieties (respectively, a homomorphism G → H of algebraic k -groups, an action of an algebraic k -group G on an algebraic k -variety X) defines in a natural way a morphism X(K) → Y (K) (respectively, a homomorphism G(K) → H(K), an action of G(K) on X(K)). Such a morphism (respectively, a homomorphism, an action) is called defined over k.

Two homomorphism φ, ψ : G → H of algebraic groups will be called equivalent, if there exists h ∈ H such that

ψ(g) = hφ(g)h^{−1}

for any g ∈ G.

Proposition 10. 1) Any reductive K -group is isomorphic to a K -group defined over k.

2) Any homomorphism of reductive K -groups defined over k is equivalent to a homomorphism defined over k.

3) Any reductive subgroup of an algebraic K -group defined over k is conjugate to a subgroup defined over k. Before proving the proposition we prove some lemmas.

Lemma 3. For a reductive k -group G, any linear representation of G(K) is equivalent to a representation defined over k. Proof. It suffices to prove the assertion for irreducible representations. Decompose the regular representation of G into a sum of irreducible representations. Tensoring this decomposition by K, we obtain a decomposition of the regular representation of G(K) into a sum of irreducible representations, each of them is defined over k. Since the regular representation of a reductive group contains all its irreducible representations, we obtain the desirable result.

Lemma 4. For a reductive k -group G, any normal algebraic subgroup of G(K) is defined over k. Proof. Since any normal algebraic subgroup of an algebraic group is the kernel of some linear representation of the group, the assertion follows from Lemma 4.

Lemma 5. Let an algebraic k -group G act on an algebraic k -variety X. If the group G(K) acts on X(K) with finitely many orbits, then all these orbits are defined over k. Proof. We may assume that G acts transitively on the set of irreducible components of X (and hence G(K) acts transitively on the set of irreducible components of X(K)). Let dim X = m. Then the (only) open orbit of G(K) in X(K) is the complement of the subvariety

X(K)^{deg} = {x ∈ X(K): dim G(K)x < m} =
= {x ∈ X(K): dim 𝔤(K)(x) < m}

which is defined over k. Hence the open orbit is also defined over k. Proceeding by induction on dim X, we obtain the desirable result.

Proof of Proposition 10. We start with the second assertion. Let G and H be reductive k -groups and φ: G(K) → H(K) a homomorphism. We may assume that H ⊂ GL_{n}(k), so H(K) ⊂ GL_{n}(K). Then the composition of φ and the latter embedding yields a linear representation

ρ: G(K) → GL_{n}(K),

which is by Lemma 3 equivalent to a representation defined over k.

Let the group G be generated by a p-tuple g = (g_{1}, …, g_{p}) ∈ G^{p}. The variety O of all representations equivalent to ρ, can be identified with the GL_{n}(K)-orbit of the p-tuple

ρ(g) = φ(g) ∈ GL_{n}(K)^{p},

a representation σ ∈ O being identified with the p-tuple σ(g). Since the orbit O contains a k -point, it is defined over k.

The intersection O ∩H(K)^{p} is, of course, also defined over k. According to Proposition 3, it decomposes into finitely many H(K)-orbits. Each of them is by Lemma 5 defined over k. In particular, the H(K)-orbit of φ(g) is defined over k and, consequently, contains a k -point. This means that the homomorphism φ is equivalent to a homomorphism defined over k.

Now we prove the first assertion of the proposition. Lemma 4 implies that if a reductive K -group is defined over k, then any quotient group of it is also defined over k. Any reductive K -group is isomorphic to a quotient group of a group of the form

(9)  G = (H × T) ⋋ F,

where H is a simply connected semisimple group, T is a torus, F is a finite group and ⋋ denotes a semidirect product (see Section 6). So it suffices to prove the assertion for groups of the form (9).

It follows from the classification of semisimple groups that any simply connected semisimple K -group is isomorphic to a group defined over k. The same is, of course, true for tori. So we may assume that the group H × T = G_{0} is defined over k.

Finally, the homomorphism φ: F → Aut G_{0} , defining the semidirect product in (9), by the above is equivalent to a homomorphism defined over k. Hence, the group G is isomorphic to a group defined over k.

The third assertion of the proposition follows from the first two ones.

11. The embeddings G^{p}//G ⊂ G^{q}//G

For any group G, let us identify a p-tuple (g_{1}, …, g_{p}) ∈ G^{p} with the (p+1)-tuple (g_{1}, …, g_{p}, e) ∈ G^{p+1} . In such a way we get G-equivariant embeddings

(10)  G^{1} ⊂ G^{2} ⊂ ⋯ ⊂ G^{p} ⊂ G^{p+1} ⊂ …

Let now G be a reductive algebraic group. Then the embeddings (10) are closed and give rise to closed embeddings

G^{1}//G ⊂ G^{2}//G ⊂ ⋯ ⊂ G^{p}//G ⊂ G^{p+1}//G ⊂ …

For a reductive subgroup H ⊂ G we have

H^{1}//G ⊂ H^{2}//G ⊂ ⋯ ⊂ H^{p}//G ⊂ H^{p+1}//G ⊂ …

Proposition 11. (H^{q}//G) ∩ (G^{p}//G) = H^{p}//G for q ⩾ p.

Proof. It suffices to prove that

(H^{p+1}//G) ∩ (G^{p}//G) = H^{p}//G.

Consider the G-equivariant projection

ρ: G^{p+1} → G^{p}, (g_{1}, …, g_{p}, g_{p+1}) ↦ (g_{1}, …, g_{p}).

It defines a projection

ρ_{G}: G^{p+1}//G → G^{p}//G.

Since ρ_{G}(H^{p+1}//G) ⊂ H^{p}//G and ρ_{G} is the identity map on G^{p}//G, we get

(H^{p+1}//G) ∩ (G^{p}//G) ⊂ H^{p}//G.

The opposite inclusion is evident.

Let us now prove the analogous property for H̃^{p} ’s (see the notation in Section 1). Obviously, we have

H̃^{1}//G ⊂ H̃^{2}//G ⊂ ⋯ ⊂ H̃^{p}//G ⊂ H̃^{p+1}//G ⊂ …

Proposition 12. (H̃^{q}//G) ∩ (G^{p}//G) = H̃^{p}//G for q ⩾ p ⩾ 2. Let H_{0} be the connected component of H, containing e, and H_{1} some other connected component. We put

H_{1}//G = \overline{π_{G}(H_{1})}.

Lemma 6. H_{1}//G ∌ π_{G}(e).

Proof. Let S be a maximal toric subvariety of H_{1} . Then π_{G}(H_{1}) = π_{G}(S). According to Proposition 5, π_{G} is finite on S. Hence π_{G}(S) is closed and H_{1}//G = π_{G}(S). Since S consists of semisimple elements distinct from e, π_{G}(S) ∌ e.

Proof of Proposition 12. It suffices to prove that

(H̃^{p+1}//G) ∩ (G^{p}//G) = H̃^{p}//G.

Consider the G-equivariant projection

σ: G^{p+1} → G, (g_{1}, …, g_{p}, g_{p+1}) ↦ g_{p+1}.

It defines a projection

σ_{G}: G^{p+1}//G → G//G.

Let H′ = H \ H_{0} . In view of Corrollary 1 to Proposition 8

H̃^{p+1} ⊂ (H̃^{p} × H_{0}) ∪ (H^{p} × H′)

and hence

H̃^{p+1}//G = \overline{π_{G}(H̃^{p+1})} ⊂ \overline{π_{G}(H̃^{p} × H_{0})} ∪ \overline{π_{G}(H^{p} × H′)}.

By Lemma 6 we have

σ_{G}\overline{(π_{G}(H^{p} × H′))} ⊂ \overline{σ_{G}π_{G}(H^{p} × H′)} = \overline{π_{G}(H′)} ∌ π_{G}(e),

which implies that

\overline{π_{G}(H^{p} × H′)} ∩ (G^{p}//G) = ∅

Thus

(H̃^{p+1}//G) ∩ (G^{p}//G) = \overline{π_{G}(H̃^{p} × H_{0})} ∩ (G^{p}//G).

Now we can do as in the proof of Proposition 11. Namely, since

ρ_{G}(\overline{π_{G}(H̃^{p} × H_{0})}) ⊂ H̃^{p}//G

and ρ_{G} is the identity map on G^{p}//G, we get

(H̃^{p+1}//G) ∩ (G^{p}//G) ⊂ H̃^{p}//G.

The opposite inclusion is evident.

12. The action Aut F_{p}: G^{p}

Let F_{p} be the free group on generators u_{1}, …, u_{p}, and Γ_{p} = Aut F_{p} the group of its automorphisms. For any group G, one can define a right action of Γ_{p} on G^{p} as follows. Let γ ∈ Γ_{p}. Then

γ(u_{i}) = w_{i}(u_{1}, …, u_{p}),

where w_{1}, …, w_{p} are some words in p letters. We put

(g_{1}, …, g_{p})^{γ} = (w_{1}(g_{1}, …, g_{p}), …, w_{p}(g_{1}, …, g_{p})).

Obviously, this action commutes with the action of G by simultaneous conjugations. If H ⊂ G is a subgroup, then the subset H^{p} ⊂ G^{p} is invariant under Γ_{p}.

We have natural embeddings

\begin{matrix}F_{1} ⊂ F_{2} ⊂ ⋯ ⊂ F_{p} ⊂ F_{p+1} ⊂ …, \\ Γ_{1} ⊂ Γ_{2} ⊂ ⋯ ⊂ Γ_{p} ⊂ Γ_{p+1} ⊂ …, \end{matrix}

where γ ∈ Γ_{p} is extended to an automorphism of F_{p+1} by γ(u_{p+1}) = u_{p+1} . Obviously, the embedding G^{p} ⊂ G^{p+1} is Γ_{p} -equivariant.

Let now G be a reductive algebraic group. Then any element of Γ_{p} acts as an automorphism of the algebraic variety G^{p}, so the action of the group Γ_{p} on G^{p} induces its action on G^{p}//G. Note that G̃^{p}, and hence G̃^{p}//G, is invariant under Γ_{p}. The embedding G^{p}//G ⊂ G^{p+1}//G is Γ_{p} -equivariant.

If H ⊂ G is a reductive subgroup, then H^{p}//G and H̃^{p}//G are invariant under Γ_{p}.

Proposition 13. Assume that g = (g_{1}, …, g_{p}) ∈ G^{p} (topologically) generates G. Then for sufficiently large q ⩾ p the closure of the Γ_{q} -orbit of g coincides with G̃^{q}.

Proof. Denote the Γ_{q} -orbit of g by O and its closure by Ō. Obviously, any element of O generates G and hence belongs to G̃^{q}. Thus, Ō ⊂ G̃^{q}. We are to prove that Ō ⊃ G̃^{q} for sufficiently large q.

Note that the group Γ_{q} contains (and is generated by) the following two types of transformations of G^{q}:

(T1) multiplication of some component by some word in other components; (T2) permutation of the components.

Let q = p + r. Applying to g transformations of type (T1), we can get any q -tuple of the form

(g_{1}, …, g_{p}, h_{1}, …, h_{r}),

where h_{1}, …, h_{r} belong to the subgroup Γ algebraically generated by g. Since Γ is dense in G, this implies that

(g_{1}, …, g_{p}, h_{1}, …, h_{r}) ∈ Ō

for any h_{1}, …, h_{r} ∈ G.

If h_{1}, …, h_{r} generate G, we can prove in the same way that

(f_{1}, …, f_{p}, h_{1}, …, h_{r}) ∈ Ō

for any f_{1}, …, f_{p} ∈ G. Since the set of generating r-tuples is dense in G̃^{r} (Corollary 8b of Proposition 8), we obtain that

(11)  G^{p} × G̃^{r} ⊂ Ō.

Now take r ⩾ 2 large enough for the following condition to be satisfied: Any generating set of ⩾ r elements of the group G/G_{0} contains a generating subset of r elements. (For example, we can take r = max{2, |G/G_{0}|}.) Then (11) and Corollary 8a of Proposition 8 imply that G̃^{q} ⊂ Ō.

Corollary 13a. If h = (h_{1}, …, h_{p}) ∈ G^{p} generates a reductive subgroup H, then for sufficiently large q ⩾ p the Γ_{q} -orbit of π_{G}(h) is dense in H̃^{q}//G.

13. Approximation of subgroups

Let H and F be reductive subgroups of a reductive group G.

Lemma 7. The following conditions are equivalent:

(a) π_{G}(f) ∈ H̃^{p}//G for some generating set f = (f_{1}, …, f_{p}) of F;

(b) F̃^{p}//G ⊂ H̃^{p}//G for all p ⩾ 2.
(c) F̃^{p}//G ⊂ H̃^{p}//G for some p such that F̃^{p} ≠ ∅.

Proof. (a) ⇒ (b). According to Corollary 13a of Proposition 13 the Γ_{q} -orbit of π_{G}(f) is dense in F̃^{q}//G for sufficiently large q. Since H̃^{q}//G is Γ_{q} -invariant and closed in G^{q}//G, this implies that F̃^{q}//G ⊂ H̃^{q}//G. But then (b) follows in view of Proposition 12.

(b) ⇒ (c) is evident.

(c) ⇒ (a) is also evident.

If the equivalent properties of Lemma 7 are satisfied, we shall say that the subgroup F is approximated by H and write F ≺ H. Obviously, this relation is transitive. It is also clear that if F ⊂ H and F H_{0} = H (i.e., F intersects each connected component of H), then F ≺ H. (We shall see later that the converse is also true up to conjugacy.)

If follows from formula (8) applied for sufficiently large p that if F ≺ H then

dim F ⩽ dim H.

Lemma 8. If F ≺ H, then (F, F) ≺ (H, H).

Proof. Let f = (f_{1}, …, f_{p}) be a generating set of F. Then π_{G}(f) ∈ \overline{π_{G}(H_{1} × ⋯ × H_{p})}, where H_{1}, …, H_{p} are connected components of H such that H_{1} × ⋯ × H_{p} ⊂ H̃^{p}.

Let v_{1}, …, v_{r} (r ⩾ 2) be some words in the commutator subgroup of the free group on p generators. Define a G-equivariant morphism θ: G^{p} → (G, G)^{r} by

θ(g_{1}, …, g_{p}) = (v_{1}(g_{1}, …, g_{p}), …, v_{r}(g_{1}, …, g_{p})).

It induces a morphism θ_{G}: G^{p}//G → (G, G)^{r}//G. The words v_{1}, …, v_{r} can be choosen in such a way that θ(f) generate (F, F) and θ(h) generate

(H, H)/(H, H)_{0} for h ∈ H_{1} × ⋯ × H_{p}.

Then θ(H_{1} × ⋯ × H_{p}) ⊂ { \atop (H, H)}^{r} and

\begin{aligned}π_{G}(θ(f)) & = θ_{G}(π_{G}(f)) ∈ θ_{G}(\overline{π_{G}(H_{1} × ⋯ × H_{p})}) ⊂ \overline{π_{G}(θ(H_{1} × ⋯ × H_{p}))} \\ & ⊂ { \atop (H, H)}^{r}//G, \end{aligned}

so (F, F) is approximated by (H, H).

Lemma 9. If F ≺ H and dim F = dim H, then dim Z(F) ⩾ dim Z(H). (We shall see below that in fact such a situation is impossible, unless F = H.)

Proof. This is an immediate consequence of Lemma 8 and the following lemma.

Lemma 10. For any reductive group G,

(12)  dim(G, G) + dim Z(G) = dim G.

Proof. Consider the group G/(G, G_{0}). Its center contains G_{0}/(G, G_{0}) and hence has finite index. By a theorem of Schur the commutator subgroup of G/(G, G_{0}) is finite. This means that (G, G) is a finite extension of (G, G_{0}). (In fact (G, G_{0}) = (G, G)_{0} .)

Consider now the decomposition of the tangent algebra 𝔤 of G into the direct sum of the subspace 𝔤^{G} of Ad(G)-invariant elements and the subspace 𝔤_{G} spanned by the elements Ad(g)ξ−ξ, (g ∈ G, ξ ∈ 𝔤). Clearly, 𝔤^{G} is the tangent algebra of Z(G). The subspace 𝔤_{G} contains [𝔤, 𝔤] and can be characterized as the least ideal of 𝔤 such that Ad(G) acts trivially on the corresponding quotient algebra. On the other hand, (G, G_{0}) is the least G-invariant algebraic subgroup of G_{0} such that G acts trivially on the corresponding quotient group. Hence 𝔤_{G} is the tangent algebra of (G, G_{0}) or, which is the same, of (G, G). The equality (12) follows, since

dim 𝔤^{G} + dim 𝔤_{G} = dim 𝔤.

14. Structure of quasi-finite morphisms

We recall the structure of quasi-finite morphisms given by the principal Zariski theorem.

Any quasi-finite morphism ψ: X → Y of irreducible algebraic varieties admits a decomposition

(13)  ψ = ψ_{1}ψ_{0},

where ψ_{0}: X → Z is an open embedding and ψ_{1}: Z → Y a finite morphism. If X is normal, one may assume Z to be normal as well, and under this condition the decomposition (13) is unique. Moreover, for any other decomposition ψ = ψ̃_{1}ψ̃_{0} of type (13) we have

ψ̃_{0} = νψ_{0}, ψ_{1} = ψ̃_{1}ν,

where ν: Z → Z̃ is a normalization, whose restriction to ψ_{0}(X) is an open embedding.

Suppose now that X is an affine variety. Then S_{0} = Z \ ψ_{0}(X) is a divisor in Z. Moreover, if ψ (and hence ψ_{1} ) is dominant, S = ψ_{1}(S_{0}) is a divisor in Y . It follows from the above description of all decompositions of type (13) that the divisor S does not depend on the choice of such a decomposition (that is, we need not suppose Z to be normal). We shall call S the singular divisor of ψ.

For our purposes it is convenient to extend this definition to reducible varieties. Let X be an unmixed normal affine variety (the normality means, in particular, that the irreducible components of X do not intersect), and ψ: X → Y a quasi-finite dominant morphism. We define the singular divisor of ψ as the union of the singular divisors of the induced morphisms

ψ_{i}: X_{i} → \overline{ψ(X_{i})},

where X_{i} runs over the irreducible components of X.

Let X, Y , ψ be as above, and X′ an unmixed normal closed subvariety of X. Then the induced morphism

ψ′: X′ → Y′ = \overline{ψ(X′)}

is also quasi-finite and dominant. Denote by S′ its singular divisor.

Lemma 11. S′ ⊂ S.

Proof. If suffices to consider the case when X and X′ are irreducible. In this case, any decomposition ψ = ψ_{1}ψ_{0} of type (13) gives rise to a decomposition ψ′ = ψ1^{′}ψ0^{′},where ψ0^{′}: X′ → Z′ = \overline{ψ_{0}(X′)} and ψ1^{′}: Z′ → Y′ are the restrictions of ψ_{0} and ψ_{1} , respectively. Obviously, ψ0^{′} is an open embedding and ψ1^{′} a finite morphism. It follows that

S′ = ψ1^{′}(Z′ \ ψ0^{′}(X′)) ⊂ ψ_{1}(Z \ ψ_{0}(X)) = S.

15. Proof of Theorem 1

Since the restriction of a finite morphism to any closed subvariety is also finite, it suffices to prove the theorem for sufficiently large p. In particular, we may (and shall) assume that p ⩾ 2.

In view of Corollary 8a to Proposition 8, H^{p} is the disjoint union of (closed) subsets F̃^{p}, where F runs over the subgroups of finite index in H. Consequently, it suffices to prove that for any reductive sugroup H ⊂ G the natural morphism

ψ̃: H̃^{p}//H → H̃^{p}//G

is finite. We already proved in Section 3 that it is quasi-finite. Let S_{p} ⊂ H̃^{p}//G denotes its singular divisor (see the preceding section). We are to prove that S_{p} = ∅.

Since the group Γ_{p} naturally acts on all the involved varieties, the divisor S_{p} is Γ_{p} -invariant.

Suppose that S_{p} ≠ ∅, and take any irreducible component T of S_{p}. Let K = k(T) and K̄ be the algebraic closure of K. We shall think of elements of K̄ as of algebraic functions on T.

Let t be the K -point of H̃^{p}//G defined by the restriction of functions to T. Take any semisimple point

f(t) = (f_{1}(t), …, f_{p}(t)) ∈ G^{p}(K̄),

projecting to t. Let F (t) be the (reductive) subgroup of G(K̄) generated by f(t). According to Proposition 8 there exists a reductive subgroup F ⊂ G such that F (t) is conjugate to F (K̄) in G(K̄). It follows that

t ∈ F̃^{p}(K̄)//G(K̄) = (F̃^{p}//G)(K̄),

whence

T ⊂ F̃^{p}//G.

According to Corollary 13a of Proposition 13, Γ_{q} -orbit of π_{G(K̄)}(f(t)) is dense in (F̃^{q}//G)(K̄) for sufficiently large q. Since π_{G(K̄)}(f(t)) ∈ S_{p}(K̄) ⊂ S_{q}(K̄) and S_{q}(K̄) is Γ_{q} -invariant, we get

(F̃^{q}//G)(K̄) ⊂ S_{q}(K̄),

which implies that

(14)  F̃^{q}//G ⊂ S_{q} ⊂ H̃^{q}//G.

In particular, F is approximated by H, and

F̃^{p}//G ⊂ H̃^{p}//G.

If F̃^{p}//G contains an irreducible component of H̃^{p}//G, then it contains a point of the form π_{G}(h), where h ∈ H̃^{p} generates H. Since the Γ_{q} -orbit of such point is dense in H̃^{q}//G for sufficiently large q, we get F̃^{q}//G = H̃^{q}//G, which contradicts (14). Hence, F̃^{p}//G is a divisor in H̃^{p}//G, that is

(15)  dim F̃^{p}//G = dim H̃^{p}//G − 1.

By formula (8) we have

dim H̃^{p}//G = (p − 1) dim H + dim Z(H),
dim F̃^{p}//G = (p − 1) dim F + dim Z(F).

Note that dim Z(F) ⩽ rk G. Therefore, if p ⩾ rk G + 3, the equality (15) is impossible, unless dim F = dim H. But Lemma 10 shows that (15) is also impossible if dim F = dim H. We came to a contradiction, which proves that S_{p} = ∅. Corollary 1 of Theorem 1. Let H and F be reductive subgroups of a reductive group G. The subgroup F is approximated by H iff it is conjugate to a subgroup of H intersecting all connected components of H.

Proof. The ”if” part is clear due to Corollary 1 to Proposition 8.

Assume now that F ≺ H, and let f = (f_{1}, …, f_{p}), p ⩾ 2, be a generating set of F. Then Theorem 1 shows that π_{G}(f) ∈ π_{G}(H̃^{p}). This means that f is conjugate to a p-tuple of H̃^{p}, and hence F is conjugate to a subgroup of H, projecting onto H/H_{0} .

16. Varieties of characters of finitely generated groups

One can look at Theorem 1 from another point of view. A p-tuple g = (g_{1}, …, g_{p}) ∈ G^{p} can be identified with the representation F_{p} → G taking the i-th generator of F_{p} to g_{p}. In such a way the variety R(F_{p}, G) of all representations of F_{p} into G is identified with G^{p}. If Γ is any group with p generators, one can consider Γ as a quotient group of F_{p}, and thereby the variety R(Γ, G) of representations of Γ into G is identified with a closed G-invariant subvariety of R(F_{p}, G).

The quotient X(Γ, G) = R(Γ, G)//G is called the variety of G-characters of Γ. It is naturally identified with a closed subvariety of X(F_{p}, G). In view of the description of invariants for the action SL_{n}(k): SL_{n}(k)^{p} (see the introduction) the elements of X(Γ, SL_{n}(k)) are interpreted as the usual characters of (unimodular) n-dimensional linear representations of Γ. (In the case of GL_{n}(k), one should consider the inverse determinant of the representation together with its character.)

It is just a reformulation of Theorem 1 that, for a reductive subgroup H ⊂ G, the natural morphism X(F_{p}, H) → X(F_{p}, G) is finite. Moreover, since the restriction of a finite morphism to a closed subvariety is finite as well, we obtain Corollary 2 of Theorem 1. Let Γ be a finitely generated group, and H a reductive subgroup of a reductive group G. Then the natural morphism X(Γ, H) → X(Γ, G) is finite. In particular, for any reductive subgroup H ⊂ SL_{n}(k) the set of characters of representations of Γ into H is closed in X(Γ, SL_{n}(k)).

References

[1] Popov, V. L. and E. B. Vinberg, “Invariant theory,” in: Algebraic geometry IV, Encyclopaedia of Mathematical Sciences, Springer-Verlag Berlin etc., 55 (1994), 123–278.

[2] Gantmakher, F. R., Canonical representation of automorphisms of a complex semisimple Lie group, Mat. Sb., Nov. Ser. 5 (1939), 101–144.

[3] Gorbatsevich, V. V., A. L. Onishchik, and E. B. Vinberg, Structure of Lie groups and Lie algebras, in: Lie groups and Lie algebras III, Encyclopaedia of Mathematical Sciences, Springer-Verlag Berlin etc., 41 (1994), 1–248.

[4] Richardson, R. W., Conjugacy classes of n-tuples in Lie algebras and algebraic groups, Duke Math. J. 57 (1988), 1–35.

[5] Vinberg, E. B., On invariants of a set of matrices, Preprint Tata Institute of Fundamental Research, 1993.

Chair of Algebra Moscow University 119899 Moscow Russia vinberg@ebv.pvt.msu.su

Received October 23, 1995 and in final form October 4, 1996