Journal of Convex Analysis Volume 7 (2000), No. 1, 19–45

Rank-One Connections at Infinity and Quasiconvex Hulls

Kewei Zhang

Department of Mathematics, Macquarie University, Sydney, NSW 2109, Australia. e-mail: kewei@ics.mq.edu.au

Received August 24, 1998

We define p-rank-one connections at infinity for an unbounded set K in M^{N×n} and show that the quasiconvex hull Q_{p}(K) may be bigger than K if K has a p-rank-one connection, where Q_{p}(K) is the zero set of the quasiconvex relaxation of the p-distance function to K. We examine some examples and compare Q_{p}(K) with Q_{p}(K) - a more restrictive quasiconvex hull of K.

1. Introduction

For a compact set K ⊂ M^{N×n}, the quasiconvex hull records the locations of all possible microstructures generated by K in the variational approach to martensitic phase transformation [8, 9, 23, 27, 33, 38]. Related notions such as the rank-one convex hull are also useful for the study of microstructures. There are three different but connected definitions of the quasiconvex hull and the rank-one convex hull.

Definition 1.1 ([27]). Let K ⊂ M^{N×n} be a non-empty closed set. The quasiconvex hull Q(K) of K is defined by

Q(K) = {X ∈ M^{N×n}, f(X) ≤ sup_{Y ∈K} f(Y), f : M^{N×n} → ℝ quasiconvex}.

The rank-one convex hull R(K) is defined by

R(K) = {X ∈ M^{N×n}, f(X) ≤ sup_{Y ∈K} f(Y), f : M^{N×n} → ℝ rank-one convex}.

Definition 1.2 ([38]). For any 1 ≤ p < ∞, the p-quasiconvex hull and p-rank one convex hull of K are defined by

Q_{p}(K) = {X ∈ M^{N×n}, Q dist^{p}(X, K) = 0},
R_{p}(K) = {X ∈ M^{N×n}, R dist^{p}(X, K) = 0},

where dist(·, K) is the Euclidean distance function to K and Q dist^{p}(·, K) and R dist^{p}(·, K) are the quasiconvex relaxation and rank-one convex relaxation of dist^{p}(·, K) respectively.

It was established in [38] that for 1 ≤ p < ∞, Q(K) = Q_{p}(K) whenever K is compact. The advantage of Definition 1.2 is that the quasiconvex hull is determined by the zero set of a single quasiconvex function.

When K is bounded, quasiconvex hulls arise naturally from the study of singular solutions of certain elliptic systems [28], the stability problem of the conformal set [32, 33], and the

ISSN 0944-6532 / $ 2.50 \begin{matrix}c \\ ◯ \end{matrix} Heldermann Verlag

20 K. Zhang / Rank-one connections at infinity and quasiconvex hulls

study of minimizing problems with linear growth quasiconvex functions [14]. However, for unbounded K the the situation is quite different [32, 33, 35, 39, 24, 11]. In [33], Yan defined the p-quasiconvex hull for an unbounded set as

Definition 1.3 ([33]). Let K ⊂ M^{N×n} be non-empty and closed. The p-quasiconvex hull Q_{p}(K) of K is defined by

Q_{p}(K) = {X ∈ M^{N×n}, f(X) ≤ sup_{Y ∈K} f(Y),

f : M^{N×n} → ℝ quasiconvex, and 0 ≤ f(X) ≤ C_{f}(1 + |X|^{p})}

The p-rank-one convex hull R_{p}(K) of K is defined by

R_{p}(K) = {X ∈ M^{N×n}, f(X) ≤ sup_{Y ∈K} f(Y),

f : M^{N×n} → ℝ rank-one convex, and 0 ≤ f(X) ≤ C_{f}(1 + |X|^{p})}

where C_{f} ≥ 0 is a constant depending on f.

It is well-known that when f is quasiconvex, it is rank-one convex [21, 4, 10], while the converse is not true [29]. Therefore we have

R(K) ⊂ Q(K),  R_{p}(K) ⊂ Q_{p}(K),  R_{p}(K) ⊂ Q_{p}(K).

From the definitions above, we also see that

Q(K) ⊂ Q_{p}(K) ⊂ Q_{p}(K), R(K) ⊂ R_{p}(K) ⊂ R_{p}(K),

for all 1 ≤ p < ∞.

Since quasiconvex hulls arise naturally from minimizing problems, let us examine briefly how they might otherwise be defined. Consider the minimizing problem inf I(u) subject to certain boundary conditions, where

I(u) = ∫_{Ω} F (Du(x))dx.  (1.1)

We are interested in the situation when inf I(u) = 0 with F satisfying F (P) = 0, P ∈ K and F (P) > 0, P ∉ K. The quasiconvex hull of (K, F) could be defined as

Q_{F} (K) = {P ∈ M^{N×n}, inf_{u|_{∂Ω}=P x} ∫_{Ω} F (Du)dx = 0}.  (1.2)

Therefore, we see that Q(K), given by Definition 1.1, is the smallest quasiconvex hull while Q_{p}(K) is the smallest among all Q_{F} (K) with F (P) has p-th order growth at infinity. However, for general K, it is hard to find enough quasiconvex functions to estimate Q(K) or Q_{p}(K), while the p-distance function dist^{p}(·, K) characterizes the geometry of K and can always be defined. The study of Q_{p}(K) depends only on the behaviour of the quasiconvex function Q dist^{p}(·, K). This function has a very interesting property which is not shared by other type of nonnegative quasiconvex functions vanishing on K, manely:

Q dist^{p}(·, K) = Q dist^{p}(·, Q_{p}(K)),

for all 1 ≤ p < ∞. A matrix P is in Q_{p}(K) if there is a sequence (ϕ_{j}) in C_{0}^{∞}, such that

lim_{j→∞} ∫_{Ω} dist^{p}(P + Dϕ_{j}(x), K)dx = 0.

Hence for every ϵ > 0, P + Dϕ_{j}(x) stays in the ϵ-neighbourhood of K except on a small set H_{j} whose measure tends to 0 as j → 0. For functions F other than those of distance type, if F (P) = 0, P ∈ K and F (P) > 0, P ∉ K, the behaviour of the minimizing sequences is more difficult to characterize. Therefore, the study of Q_{p}(K) could also be considered as the first step toward the understanding of Q(K) and Q_{p}(K).

In this paper, we define the notion of rank-one connections at infinity, and use it as a tool to study the quasiconvex hull Q_{p}(K) without knowing too many details of K. It is well known that rank-one connections are the simplest type of algebraic conditions on a closed set K which are sufficient to ensure a nontrivial quasiconvex hull Q_{p}(K). Rankone connections at infinity are a natural generalization of the notion of the rank-one connection. We have

Definition 1.4. Suppose K ⊂ M^{N×n} is closed and unbounded. K has a p-rank-one connection at infinity if there exists a sequence (A_{j}) in K with eigenvalues of A_{j}^{T} A_{j}

λ_{j}^{(1)} ≤ λ_{j}^{(2)} ≤ ⋯ ≤ λ_{j}^{(n−1)} ≤ λ_{j}^{(n)} =: Λ_{j},

such that

(i) Λ^{j} → ∞ as j → ∞;

(ii) Let λ_{j} = λ_{j}^{(1)} + ⋯ + λ_{j}^{(n−1)}, then

\frac{λ_{j}}{(Λ_{j})^{1/p}} → 0, as j → ∞;

(iii) There exists a rank-one matrix A_{0} such that

\frac{A_{j}}{\sqrt{Λ_{j}}} → A_{0}, as j → ∞.

When p = 1 and K has a 1-rank-one connection at infinity, we simply say that K has a rank-one connection at infinity. For simplicity we say ‘p-rank-one connection’ rather than ‘p-rank-one connection at infinity’.

Intuitively, a rank-one connection at infinity just means that the greatest eigenvalue of A_{j}^{T} A_{j} goes to infinity faster than all other eigenvalues. Hence the limit of A_{j}/\sqrt{Λ_{j}} is a rank-one matrix.

Before we give examples of sets with rank-one connections at infinity, we state the main theorem of this paper.

Theorem 1.5. Suppose K is closed, unbounded and has a p-rank-one connection at infinity. Then

{P + tA_{0}; P ∈ K, t ≥ 0} ⊂ R_{p}(K) ⊂ Q_{p}(K),  (1.3)

where A_{0} is the rank-one matrix given by Definition 1.4.

This result shows what type of extra points will be in R_{p}(K) and Q_{p}(K). If K already contains the set in (1.3), this theorem does not provide any information on R_{p}(K) or Q_{p}(K). The following result is a direct consequence of the proof of Theorem 1.5.

Proposition 1.6. Let S_{+}, S_{−} and S be given by

S_{+} = {P ∈ M^{2×2}, P^{T} = P, det P = 1, and P is positive definite}
S_{−} = {P ∈ M^{2×2}, P^{T} = P, det P = 1, and P is negative definite},
S = S_{+} ∪ S_{−}.

Then

R_{p}(S_{+}) = Q_{p}(S_{+}) = C(S_{+}), R_{p}(S_{−}) = Q_{p}(S_{−}) = C(S_{−}), R_{p}(S) = Q_{p}(S) = C(S)

for all p ≥ 1, where C(K) is the closed convex hull of set K.

Remark 1.7. Since the structure of S_{+} is simple, we can show that at least for p ≥ 2, Q_{p}(S_{+}) = S_{+}. Let

f(P) = dist^{2}(P, C(S_{+})) + | det P − 1|.

Then F is quasiconvex (in fact, polyconvex), F ≥ 0 and F^{−1}(0) = S_{+}.

Theorem 1.5 does not apply to the quasiconformal set R_{+}SO(n) where the eigenvalues do not satisfy assumption (ii) of Definition 1.4 [32, 33, 22].

We also have

Proposition 1.8. Let S_{+}^{(n)} be given by

S_{+}^{(n)} = {P ∈ M^{n×n}, P^{T} = P, det P = 1, and P is positive definite},

Then

R_{p}(S_{+}^{(n)}) = Q_{p}(S_{+}^{(n)})

= C(S_{+}^{(n)}) = {P ∈ M^{n×n}, P^{T} = P, det P ≥ 1, and P is positive definite}

for all p ≥ 1.

We establish Theorem 1.5 by using an upper bound of the rank-one convex relaxation of the p-distance function from a two point set, Q dist^{p}(·, {A, B}) based on the translation method [13]. The bound recovers the explicit quasiconvex relaxation when p = 2, obtained by Kohn [17].

In Section 2, we give some preliminaries and establish an upper bound of the quasiconvex relaxation of dist^{p}(·, {A, B}) which is essential for the proof of Theorem 1.5. We prove Theorem 1.5 and Proposition 1.6 in Section 3. In Section 4, we apply Theorem 1.5 to various situations. We will study quasiconvex hulls for closed, connected and unbounded sets in M^{2×2} without rank-one connections. We will also study the quasiconvex hulls related to singular solutions of elliptic systems proposed in [28], and quasimonotone mappings.

2. Preliminaries

Throughout the rest of this paper Ω is a bounded open subset of ℝ^{n}. We denote by M^{N×n} the space of real N × n matrices with the ℝ^{Nn} metric; hence the norm of P ∈ M^{N×n} is defined by | P |= (tr P^{T} P)^{1/2}, where tr is the trace operator and P^{T} is the transpose of P. We denote by diag(a_{1}, a_{2}, ⋯, a_{n}) an n × n diagonal matrix with diagonal entries a_{1}, a_{2}, ⋯, a_{n}. The inner product of two matrices in M^{N×n} is P · Q = tr P^{T} Q. For an n × n matrix P, denote by adj P the transpose of the cofactors of P. O(n) is the set of all n × n orthogonal matrices while SO(n) ⊂ O(n) is the set of all orthogonal matrices with determinant 1. For a compact subset K ⊂ M^{N×n}, let C(K) be the convex hull of K. We write C_{0}(Ω) for the space of continuous functions ϕ : Ω → ℝ having compact support in Ω, and define C_{0}^{1}(Ω) = C^{1}(Ω) ∩ C_{0}(Ω). If 1 ≤ p ≤ ∞ we denote by L^{p}(Ω; ℝ^{N}) the Banach space of mappings u : Ω → R^{N}, u = (u_{1}, ⋯, u_{N}), such that u_{i} ∈ L^{p}(Ω) for each i, with norm ∥u∥_{L^{p}(Ω;ℝ^{N})} = ∑_{i=1}^{N} ∥u_{i}∥_{L^{p}(Ω)}. Similarly, we denote by W^{1,p}(Ω, ℝ^{N}) the usual Sobolev space of mappings u ∈ L^{p}(Ω; ℝ^{N}) all of whose distributional derivatives \frac{∂u_{i}}{∂x_{j}} = D_{j}u_{i}, 1 ≤ i ≤ N, 1 ≤ j ≤ n, belong to L^{p}(Ω). W^{1,p}(Ω, ℝ^{N}) is a Banach space under the norm

∥u∥_{W^{1,p}(Ω,ℝ^{N})} = ∥u∥_{L^{p}(Ω;ℝ^{N})} + ∥Du∥_{L^{p}(Ω;M^{N×n})},

where Du = (D_{j}u_{i}), and we define, as usual, W_{0}^{1,p}(Ω; ℝ^{N}) to be the closure of C_{0}^{∞}(Ω; ℝ^{N}) in the topology of W^{1,p}(Ω; ℝ^{N}).

Weak and weak ∗ convergence of sequences are written as ⇀ and ⇀^{∗}, respectively. If H ⊂ M^{N×n}, P ∈ M^{N×n}, then we write H + P to denote the set {P + Q : Q ∈ H}, jH = {jQ, Q ∈ H} for an integer j > 0. We define the distance function for a set K ⊂ M^{N×n} by

f(P) = dist(P, K) := inf_{Q∈K} | P − Q | .

Definition 2.1 ([21, 4, 10, 1]). A continuous function f : M^{N×n} → ℝ is quasiconvex if

∫_{U} f(P + Dϕ(x)) dx ≥ f(P) meas(U)

for every P ∈ M^{N×n}, ϕ ∈ C_{0}^{1}(U; ℝ^{N}), and every open bounded subset U ⊂ ℝ^{n} .

For a given function, we can consider its quasiconvexification (quasiconvex relaxation):

Definition 2.2 ([10]). Suppose f : M^{N×n} → ℝ is a continuous function. The quasiconvexification of f is defined by

sup{g ≤ f; g quasiconvex }

and will be denoted denoted by Qf.

Proposition 2.3 ([10]). Suppose f : M^{N×n} → ℝ is continuous, then

Qf(P) = inf_{ϕ∈C_{0}^{∞}(Ω;ℝ^{N})} \frac{1}{meas(Ω)} ∫_{Ω} f(P + Dϕ(x)) dx,  (2.1)

where Ω ⊂ ℝ^{n} is a bounded domain. In particular the infimum in (2.1) is independent of the choice of Ω.

We use the following theorem concerning the existence and properties of Young measures from Tartar [30]. For results in a more general context and their proofs, the reader is referred to [5, 31].

Theorem 2.4. Let (z^{(j)}) be a bounded sequence in L^{∞}(Ω; ℝ^{s}). Then there exist a subsequence (z^{(ν)}) of (z^{(j)}) and a family {ν_{x}}_{x∈Ω} of probability measures on ℝ^{s}, depending measurably on x ∈ Ω, such that

f(z^{(ν)}) ⇀^{∗} ⟨ν_{x}, f(·)⟩ in L^{∞}(Ω)

for every continuous function f : ℝ^{s} → ℝ.

We say that ν_{x} is a trivial Young measure at x ∈ Ω if ν_{x} = δ_{A} for some A ∈ ℝ^{s}, where δ_{A} is the Dirac mass at A.

Suppose that Ω ⊂ ℝ^{n}. A family of parametrized measures {ν_{x}}_{x∈Ω} is called a Young measure limit of gradients [7, 18, 19], if it is generated by a sequence of gradients Du_{j} with (u_{j}) bounded in W^{1,p}(Ω, ℝ^{N}).

We will need a consequence of the following theorem, but will not introduce the more general notion of normal integrals to which it applies (see [12, page 234]).

Theorem 2.5 (The measurable selection theorem (see [12, page 236])). Let B be a compact subset of ℝ^{p} and g a Carathéodory function of Ω × B. Then, there exists a measurable mapping ũ : Ω → B such that for all x ∈ Ω:

g(x, ũ(x)) = min_{a∈B}{g(x, a)}.

A direct consequence of Theorem 2.5 is the following:

Proposition 2.6. Let B ⊂ ℝ^{p} be a compact subset and let u : Ω → ℝ^{p} be an integrable mapping. Then there exists a measurable mapping ũ : Ω → B such that for all x ∈ Ω

|u(x) − ũ(x)| = dist(u(x), B).

The following simple result will provide us with a guide for the estimates in the proof of Theorem 1.5.

Lemma 2.7. Let a < 0 < b and 0 < λ < 1 be fixed numbers. Define

f(x) = min{|x − a|^{p}, |x − b|^{p}} + λ|x|^{p}.

Then

Cf(0) ≤ λ\frac{b|a|^{p} − a|b|^{p}}{b − a} .

Proof. The proof is very simple. Let θ = b/(b−a), then, 0 < θ < 1 and θa+(1−θ)b = 0. We then have

Cf(0) ≤ θCf(a) + (1 − θ)Cf(b) ≤ θf(a) + (1 − θ)f(b)

= θλ|a|^{p} + (1 − θ)|b|^{p} = λ\frac{b|a|^{p} − a|b|^{p}}{b − a} .

We conclude this section by examining the structure of closed connected sets in M^{2×2} without rank-one connections.

It was established in [26] that if K ⊂ M^{2×2} is closed, connected and has no rank-one connections, then det(A − B) > 0 for all A, B ∈ K, A ≠ B, or det(A − B) < 0 for all A, B ∈ K, A ≠ B. Such sets are characterized in [36].

Let E_{∂} = span{E_{1}, E_{2}}, E ¯∂ = span{E_{3}, E_{4}} be the pair of two dimensional subspaces of M^{2×2} where

E_{1} = (\begin{matrix}\frac{1}{\sqrt{2}} & 0 \\ 0 & \frac{1}{\sqrt{2}} \end{matrix}) , E_{2} = (\begin{matrix}0 & \frac{1}{\sqrt{2}} \\ −\frac{1}{\sqrt{2}} & 0 \end{matrix}) , E_{3} = (\begin{matrix}\frac{1}{\sqrt{2}} & 0 \\ 0 & \frac{−1}{\sqrt{2}} \end{matrix}) , E_{4} = (\begin{matrix}0 & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & 0 \end{matrix}) .

Notice that E_{∂} and E ¯∂ are orthogonal to each other and E_{1}, …, E_{4} is an orthonormal basis of M^{2×2}. Let P_{E_{∂}} and P_{E ¯∂} be the orthogonal projections from M^{2×2} to E_{∂} and E ¯∂ respectively. It is easy to check that for any ϕ ∈ C_{0}^{∞}(Ω, ℝ^{2}), with Ω ⊂ ℝ^{2} open,

∫_{Ω} |P_{E_{∂}}(Dϕ(x))|^{2}dx = ∫_{Ω} |P_{E ¯∂}(Dϕ(x))|^{2}dx.

The following was established in [36, Theorem 3.2].

Lemma 2.8. Suppose K ⊂ M^{2×2}, and det(P − Q) > 0 for all P, Q ∈ K, P ≠ Q. Then

K = {A + f(A), A ∈ P_{E_{∂}}(K)}

where E_{∂} is the two dimensional subspace defined in Lemma 3.1, P_{E_{∂}}(K) is the image of K under P_{E_{∂}} and f : P_{E_{∂}}(K) → E ¯∂ satisfies

| f(A) − f(B) |<| A − B |  (2.2)

for all A, B ∈ K, A ≠ B.

We call the mapping f obtained in Theorem 3.2 the Lipschitz mapping associated to K. Notice that without loss of generality, we may assume that det(P − Q) > 0 for P, Q ∈ K, P ≠ Q by multiplying K by J if necessary, where

J = (\begin{matrix}1 & 0 \\ 0 & −1 \end{matrix}) .

Lemma 2.9 (see [37, Lemma 2.9] and its proof). Suppose f : E_{∂} → E ¯∂ is a Lipschitz mapping with Lipschitz constant k ≤ 1. Let K = {P + f(P), P ∈ E_{∂}}. Then

|P_{E ¯∂}(A) − f(P_{E_{∂}}(A))| ≤ \sqrt{2} dist(A, K)

for all A ∈ M^{2×2}.

3. Proof of the main results

Proof of Theorem 1.5. Since for any P ∈ M^{N×n} and f : M^{N×n} → ℝ+, we have

Rf(P) ≤ inf{θf(P +A)+(1−θ)f(P +B), 0 ≤ θ ≤ 1, θA+(1−θ)B = 0, rank(A−B) = 1}.

Let K_{j} = {P, A_{j}}. Since an everywhere finite rank-one convex function is continuous, we estimate the value of R dist^{p}(·, K_{j}) at B_{j} = P + t(A_{j} − P)/|A_{j} − P | instead of at P + tA_{0}, because it is easy to see that B_{j} → P + tA_{0} as j → ∞. This follows from the facts that |A_{j}| → ∞, P is fixed and A_{j}/|A_{j}| → A_{0} as j → ∞.

We consider two different situations.

Case (1) If there exists a sequence of A_{j}, such that rank(A_{j} − P) = 1. then obviously

R dist^{p}(P + t(A_{j} − P)/|A_{j} − P |, K) ≤ R dist^{p}(P + t(A_{j} − P)/|A_{j} − P |, K_{j}) = 0.

Passing to the limit j → ∞, we have P + t(A_{j} − P)/|A_{j} − P | → P + tA_{0}. Hence, R dist^{p}(P + tA_{0}, K) = 0.

Case (2) If for sufficiently large j, rank(A_{j} − P) > 1 we choose

U_{j} = α(A_{j} − P)v ⊗ v, V_{j} = β(A_{j} − P)v ⊗ v

with α, β to be determined, where v is a unit eigenvector corresponding to the greatest eigenvalue μ_{j} of (A_{j} − P)^{T} (A_{j} − P) . Obviously, rank(U_{j} − V_{j}) = 1. Let us estimate dist^{p}(B_{j} + U_{j}, K_{j}).

dist^{p}(B_{j} + U_{j}, K_{j}) = min{|B_{j} + U_{j} − P |^{p}, |B_{j} + U_{j} − A_{j}|^{p}}

= min \{ \begin{matrix}|\frac{t(A_{j} − P)}{|A_{j} − P |} + α(A_{j} − P)v ⊗ v|^{p} , \\ |\frac{t(A_{j} − P)}{|A_{j} − P |} + α(A_{j} − P)v ⊗ v − (A_{j} − P)| & p \end{matrix}\}

(3.1)

Now, let E_{j} be the one dimensional subspace of M^{N×n} spanned by A_{j} − P. We denote by P_{E_{j}} and P_{E_{j}^{⊥}} the orthogonal projection from M^{n×N} to E_{j} and its orthogonal complement

E_{j}^{⊥} respectively. We have, in (3.1) that

min \{ \begin{matrix}|\frac{t(A_{j} − P)}{|A_{j} − P |} + α(A_{j} − P)v ⊗ v|^{p} , \\ |\frac{t(A_{j} − P)}{|A_{j} − P |} + α(A_{j} − P)v ⊗ v − (A_{j} − P)| & p \end{matrix}\}

= min \{ \begin{matrix}|\frac{t(A_{j} − P)}{|A_{j} − P |} + P_{E_{j}}(α(A_{j} − P)v ⊗ v) + P_{E_{j}^{⊥}}(α(A_{j} − P)v ⊗ v)|^{p} , \\ |\frac{t(A_{j} − P)}{|A_{j} − P |} + P_{E_{j}}(α(A_{j} − P)v ⊗ v) + P_{E_{j}^{⊥}}(α(A_{j} − P)v ⊗ v) − (A_{j} − P)| & p \end{matrix}\}

≤ 2^{p−1} [min \{ \begin{matrix}|\frac{t(A_{j} − P)}{|A_{j} − P |} + P_{E_{j}}(α(A_{j} − P)v ⊗ v)|^{p} , \\ |\frac{t(A_{j} − P)}{|A_{j} − P |} + P_{E_{j}}(α(A_{j} − P)v ⊗ v) − (A_{j} − P)| & p \end{matrix}\}

+ |P_{E_{j}^{⊥}}(α(A_{j} − P)v ⊗ v)|^{p}] .

(3.2)

K. Zhang / Rank-one connections at infinity and quasiconvex hulls 27

Let us calculate P_{E_{j}}((A_{j} − P)v ⊗ v) and |P_{E_{j}^{⊥}}((A_{j} − P)v ⊗ v)|. We have

\begin{matrix}P_{E_{j}}((A_{j} − P)v ⊗ v) = (\frac{(A_{j} − P)}{|A_{j} − P |} · (A_{j} − P)v ⊗ v) \frac{(A_{j} − P)}{|A_{j} − P |} \\ = \frac{μ_{j}}{|A_{j} − P |^{2}} (A_{j} − P). \end{matrix}  (3.3)

Since

|(A_{j} − P)v ⊗ v|^{2} = |P_{E_{j}}((A_{j} − P)v ⊗ v)|^{2} + |P_{E_{j}^{⊥}}((A_{j} − P)v ⊗ v)|^{2},  (3.4)

while

|(A_{j} − P)v ⊗ v|^{2} = μ_{j}  and  |P_{E_{j}}((A_{j} − P)v ⊗ v)|^{2} = \frac{μ_{j}^{2}}{|A_{j} − P |^{2}} .  (3.5)

Consequently,

|P_{E_{j}^{⊥}}((A_{j} − P)v ⊗ v)|^{2} = μ_{j} − \frac{μ_{j}^{2}}{|A_{j} − P |^{2}} = \frac{μ_{j}(|A_{j} − P |^{2} − μ_{j})}{|A_{j} − P |^{2}}.

Therefore

\frac{|P_{E_{j}^{⊥}}((A_{j} − P)v ⊗ v)|^{2}}{|P_{E_{j}}((A_{j} − P)v ⊗ v)|^{2}} = \frac{\frac{μ_{j}(|A_{j}−P |^{2}−μ_{j})}{|A_{j}−P |^{2}}}{\frac{μ_{j}^{2}}{|A_{j}−P |^{2}}} = \frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} .

We then have

|P_{E_{j}^{⊥}}((A_{j} − P)v ⊗ v)|^{p} = (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} |P_{E_{j}}((A_{j} − P)v ⊗ v)|^{p}.  (3.6)

Now we substitute (3.6) into (3.2), dist^{p}(B_{j} + U_{j}, K_{j})

≤ 2^{p−1} min \{ \begin{matrix}|\frac{t(A_{j} − P)}{|A_{j} − P |} + P_{E_{j}}(α(A_{j} − P)v ⊗ v)|^{p} , \\ |\frac{t(A_{j} − P)}{|A_{j} − P |} + P_{E_{j}}(α(A_{j} − P)v ⊗ v) − (A_{j} − P)| & p \end{matrix}\}

+ 2^{p−1} |P_{E_{j}^{⊥}}(α(A_{j} − P)v ⊗ v)|^{p}

= 2^{p−1} min \{ \begin{matrix}|\frac{t(A_{j} − P)}{|A_{j} − P |} + P_{E_{j}}(α(A_{j} − P)v ⊗ v)|^{p} , \\ |\frac{t(A_{j} − P)}{|A_{j} − P |} + P_{E_{j}}(α(A_{j} − P)v ⊗ v) − (A_{j} − P)| & p \end{matrix}\}

+ 2^{p−1} (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} |P_{E_{j}}(α(A_{j} − P)v ⊗ v)|^{p}

\begin{matrix}= 2^{p−1} min \{ \begin{matrix}|\frac{t(A_{j} − P)}{|A_{j} − P |} + \frac{αμ_{j}}{|A_{j} − P |^{2}} (A_{j} − P)|^{p} , \\ |\frac{t(A_{j} − P)}{|A_{j} − P |} + \frac{αμ_{j}}{|A_{j} − P |^{2}} (A_{j} − P) − (A_{j} − P)| & p \end{matrix}\} \\ + 2^{p−1} (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} |\frac{αμ_{j}}{|A_{j} − P |^{2}} (A_{j} − P)|^{p} \\ = 2^{p−1}|A_{j} − P |^{p} [min \{ \begin{matrix}|\frac{αμ_{j}}{||A_{j} − P |^{2}} − (−\frac{t}{|A_{j} − P |})|^{p} , \\ |\frac{αμ_{j}}{|A_{j} − P |^{2}} − (1 − \frac{t}{|A_{j} − P |})| & p \end{matrix}\} \\ + (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} |\frac{αμ_{j}}{|A_{j} − P |^{2}} |^{p}] . \end{matrix}  (3.7)

If we replace α by β, we can obtain a similar estimate of dist^{p}(B_{j} + V_{j}, K_{j}). Now for large j > 0,

(\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} < 1.

We may use Lemma 2.7 to estimate R dist^{p}(B_{j}, K_{j}) by careful choice of α, β and θ_{j} .

We choose α, such that

\frac{αμ_{j}}{|A_{j} − P |^{2}} − (−\frac{t}{|A_{j} − P |}) = 0,

that is,

α = −\frac{t|A_{j} − P |}{μ_{j}},

so that

U_{j} = −\frac{t|A_{j} − P |}{μ_{j}}(A_{j} − P)v ⊗ v.

Similarly, we take β, such that

\frac{βμ_{j}}{|A_{j} − P |^{2}} − (1 − \frac{t}{|A_{j} − P |}) = 0,

which gives

β = (1 − \frac{t}{|A_{j} − P |}) \frac{|A_{j} − P |^{2}}{μ_{j}} .

Similarly, we choose

V_{j} = (1 − \frac{t}{|A_{j} − P |}) \frac{|A_{j} − P |^{2}}{μ_{j}} (A_{j} − P)v ⊗ v.

We see that 0 is a convex combination of U_{j} and V_{j} if

1 − \frac{t}{|A_{j} − P |} > 0,  or, equivalently  \frac{t}{|A_{j} − P |} < 1.

This is possible for large j > 0 because |A_{j}| → ∞ as j → ∞ and t > 0 is fixed. For U_{j} and V_{j} given above, let us find θ_{j}, such that 0 < θ_{j} < 1 and θ_{j}U_{j} + (1 − θ_{j})V_{j} = 0. This is equivalent to

−\frac{t|A_{j} − P |}{μ_{j}}θ_{j} + (1 − θ_{j}) (1 − \frac{t}{|A_{j} − P |}) \frac{|A_{j} − P |^{2}}{μ_{j}} = 0

which implies

θ_{j} = 1 − \frac{t}{|A_{j} − P |}, 1 − θ_{j} = \frac{t}{|A_{j} − P |}.

Obviously, 0 < θ_{j} < 1 for large j > 0. Now we have

R dist^{p}(B_{j}, K) ≤ dist^{p}(B_{j}, K_{j}) ≤

θ_{j} dist^{p}(B_{j} + U_{j}, K_{j}) + (1 − θ_{j}) dist^{p}(B_{j} + V_{j}, K_{j})

≤ 2^{p−1}|A_{j} − P |^{p} [θ_{j} (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} |\frac{αμ_{j}}{|A_{j} − P |^{2}} |^{p}

+(1 − θ_{j}) (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} |\frac{βμ_{j}}{|A_{j} − P |^{2}} |^{p}]

= 2^{p−1}|A_{j} − P |^{p} (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} (\frac{μ_{j}}{|A_{j} − P |^{2}} )^{p}

(θ_{j}|α|^{p} + (1 − θ_{j})|β|^{p})

≤ 2^{p−1}|A_{j} − P |^{p} (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} (\frac{μ_{j}}{|A_{j} − P |^{2}} )^{p}

\begin{matrix}[(\frac{t|A_{j} − P |}{μ_{j}})^{p} + \frac{t}{|A_{j} − P |}] \\ = 2^{p−1}t^{p} (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} \\ + 2^{p−1}t (\frac{|A_{j} − P |^{2}}{μ_{j}} )^{(p−1)/2} (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}^{1/p}} )^{p/2} \\ ≤ 2^{p−1}t^{p} (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} \\ + 2^{p−1}tn^{(p−1)/2} (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}^{1/p}} )^{p/2} =: I_{1} + I_{2}, \end{matrix}  (3.8)

where we have used the facts that 1 − t/|A_{j} − P | < 1 and \frac{|A_{j}−P |^{2}}{μ_{j}} ≤ n. The second

inequality follows from the fact that μ_{j} is the greatest eigenvalue of (A_{j} − P)^{T} (A_{j} − P).

From (3.8) we see that the conclusion will follow if we can prove that

lim_{j→∞} \frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}^{1/p}} = 0.  (3.9)

Let us consider the polar decomposition of A_{j} as

A_{j} = R|A_{j}| = RQ^{T} diag(\sqrt{λ_{j}^{(1)}}, ⋯, \sqrt{λ_{j}^{(n−1)}}, \sqrt{Λ_{j}})Q,

where R is a max{N, n} × n matrix with columns orthogonal to one another and with

the norm of each column 1, Λ_{j} is the greatest eigenvalue of A_{j}^{T} A_{j}, |A_{j}| = \sqrt{A_{j}^{T} A_{j}} and Q

is an n × n orthogonal matrix such that

|A_{j}| = Q^{T} diag(\sqrt{λ_{j}^{(1)}}, ⋯, \sqrt{λ_{j}^{(n−1)}}, \sqrt{Λ_{j}})Q.

Now, |A_{j} − P |^{2} = |A_{j}|^{2} − 2 tr(A_{j}^{T} P) + |P |^{2} and

tr(A_{j}^{T} P) = tr(Q^{T} diag(\sqrt{λ_{j}^{(1)}}, ⋯, \sqrt{λ_{j}^{(n−1)}}, \sqrt{Λ_{j}})QR^{T} P)
= tr(diag(\sqrt{λ_{j}^{(1)}}, ⋯, \sqrt{λ_{j}^{(n−1)}}, \sqrt{Λ_{j}})QR^{T} P Q^{T}).

Here we have used the simple fact that tr Q^{T} CQ = tr C when Q is orthogonal and C is a square matrix. If we let QR^{T} P Q^{T} = B and let b_{ii} be the (i, i) entry of B for i = 1, 2, ⋯, n, we have

tr(A_{j}^{T} P) = tr(diag(\sqrt{λ_{j}^{(1)}}, ⋯, \sqrt{λ_{j}^{(n−1)}}, \sqrt{Λ_{j}})B) = (∑_{i=1}^{n−1} \sqrt{λ_{j}^{(i)}}b_{ii}) + \sqrt{Λ_{j}}b_{nn}. (3.10)

Now we estimate μ_{j} from below. Letting v = Q^{T} (0, ⋯, 0, 1)^{T}, we have

μ_{j} ≥ |(A_{j} − P)v|^{2} = |A_{j}v|^{2} − 2(v^{T} A_{j}^{T} P v) + |P v|^{2}

= Λ_{j} − 2(0, ⋯, 0, 1)QQ^{T} diag(\sqrt{λ_{j}^{(1)}}, ⋯, \sqrt{λ_{j}^{(n−1)}}, \sqrt{Λ_{j}})QR^{T} P Q^{T} (0, ⋯, 0, 1)^{T} + |P v|^{2}

= Λ_{j} − 2(0, ⋯, 0, 1) diag(\sqrt{λ_{j}^{(1)}}, ⋯, \sqrt{λ_{j}^{(n−1)}}, \sqrt{Λ_{j}})B(0, ⋯, 0, 1)^{T} + |P v|^{2}

= Λ_{j} − 2\sqrt{Λ_{j}}b_{nn} + |P v|^{2} ≥ Λ_{j} − 2\sqrt{Λ_{j}}b_{nn},

so that

μ_{j} ≥ Λ_{j} − 2\sqrt{Λ_{j}}b_{nn},  (3.11)

and |b_{nn}| ≤ |P |. Combining (3.10) and (3.11), we see that

\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}^{1/p}}

≤ \frac{|A_{j}|^{2} − 2 (∑_{i=1}^{n−1} \sqrt{λ_{j}^{(i)}}b_{ii}) + \sqrt{Λ_{j}}b_{nn} + |P |^{2} − (Λ_{j} − 2\sqrt{Λ_{j}}b_{nn}}{(Λ_{j} − 2\sqrt{Λ_{j}}b_{nn})^{1/p}}

≤ \frac{|A_{j}|^{2} − Λ_{j}}{Λ_{j}^{1/p}} (\frac{Λ_{j}}{Λ_{j} − 2\sqrt{Λ_{j}}|P |})^{1/p}

+ \frac{2(∑_{i=1}^{n−1} λ_{j}^{(i)})^{1/2}(∑_{i=1}^{n−1} b_{ii}^{2})^{1/2} + |P |^{2}}{(Λ_{j} − 2\sqrt{Λ_{j}}|P |)^{1/p}}

≤ (\frac{λ_{j}}{Λ_{j}^{1/p}} + \frac{2λ_{j}^{1/2}|P | + |P |^{2}}{Λ_{j}^{1/p}} ) (\frac{Λ_{j}}{Λ_{j} − 2\sqrt{Λ_{j}}|P |})^{1/p} .

(3.12)

Since

lim_{j→∞} Λ_{j} = +∞,  lim_{j→∞} \frac{λ_{j}}{Λ_{j}^{1/p}} = 0 and  lim_{j→∞} \frac{Λ_{j}}{Λ_{j} − 2\sqrt{Λ_{j}}|P |} = 1,

we see that (3.9) follows. Therefore in (3.8), I_{2} → 0, as j → ∞, where

I_{2} = 2^{p−1}tn^{(p−1)/2} (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}^{1/p}} )^{p/2} .

From (3.11) we also see that μ_{j} → ∞ as j → ∞. This together with (3.9) implies

I_{1} = 2^{p−1}t^{p} (\frac{|A_{j} − P |^{2} − μ_{j}}{μ_{j}} )^{p/2} → 0, as j → ∞.

therefore, finally we have,

R dist^{p}(P + tA_{0}, K) ≤ lim sup_{j→∞} R dist^{p}(P + t\frac{A_{j} − P}{|A_{j} − P |}, K_{j}) = 0.

The conclusion follows.

Proof of Proposition 1.6. For A ∈ S_{+}, there is a rotation R ∈ M^{2×2}, such that A = R^{T} diag(λ, 1/λ)R. Let us first examine diagonal matrices in S_{+}. We have, for every t > 0, A(t) = diag(t, 1/t) ∈ S_{+}. When t > 1, the greatest eigenvalue of A^{T} (t)A(t) is t^{2}, the other eigenvalue is 1/t^{2}. We see that for every t > 1, and p ≥ 1,

lim_{t→∞} \frac{1/t^{2}}{(t^{2})^{1/p}} = 0,  and  lim_{t→∞} A(t)/\sqrt{t^{2}} = diag(1, 0).

From Theorem 1.5, we see that

{P + s diag(1, 0), P ∈ S_{+}, s > 0} ⊂ R_{p}(S_{+}).

Similarly, we see that for every rotation R,

{P + sR^{T} diag(1, 0)R, P ∈ S_{+}, s > 0} ⊂ R_{p}(S_{+}).

Therefore C(S_{+}) ⊂ R_{p}(S_{+}), since we always have R_{p}(S_{+}) ⊂ Q_{p}(S_{+}) ⊂ C(S_{+}). The first conclusion follows. The proof for S_{−} is similar.

Now for R_{p}(S). Since S_{+} and S_{−} are subsets of S, the p-rank-one connections of S_{+} and S_{−} are also p-rank-one connections of S and therefore

{P + sR^{T} diag(±1, 0)R, P ∈ S, s > 0} ⊂ R_{p}(S),

for every orthogonal matrix R. Therefore C(S) ⊂ R_{p}(S). The proof is complete.

Proof of Proposition 1.8. Let

K = {P ∈ M^{n×n}, P^{T} = P, det P ≥ 1, and P is positive definite}.

We first prove that K ⊂ R_{p}(S_{+}^{(n)}). Then we show that K is convex so that C(S_{+}^{(n)}) ⊂ K. Since R_{p}(S_{+}^{(n)}) ⊂ C(S_{+}^{(n)}), the conclusion then follows.

In a similar manner to the proof of Proposition 1.6, if we take

A(t) = diag(t, 1/t^{1/(n−1)}, ⋯, 1/t^{1/(n−1)}),

we see that det A(t) = 1 for all t > 0, and if we apply Theorem 1.5 and follow the proof of Proposition 1.6, we see that

{P + sR^{T} diag(1, 0, ⋯, 0)R, P ∈ S_{+}^{(n)}, s ≥ 0, R ∈ O(n)} ⊂ R_{p}(S_{+}^{(n)}).

Now, for every A ∈ K \ S_{+}^{(n)}, let λ_{1} ≥ ⋯, ≥ λ_{n} be its eigenvalues. There is a rotation R_{0}, such that A = R_{0}^{T} diag(λ_{1}, ⋯, λ_{n})R_{0}. Since λ_{1}λ_{2} ⋯ λ_{n} > 1, λ_{1} > 1/(λ_{2} ⋯ λ_{n}). Let λ = 1/(λ_{2} ⋯ λ_{n}) and s = λ_{1} − 1/(λ_{2} ⋯ λ_{n}) > 0, we have

A = R_{0}^{T} diag(λ, λ_{2} ⋯, λ_{n})R_{0} + sR_{0}^{T} diag(1, 0, ⋯, 0)R_{0},

hence

A ∈ {P + sR^{T} diag(1, 0, ⋯, 0)R, P ∈ S_{+}^{(n)}, s ≥ 0, R ∈ O(n)} ⊂ R_{p}(S_{+}^{(n)}).

It is well known that K is convex. In fact, let A, B ∈ K, and we let

a^{n} = det A ≥ 1, b^{n} = det B ≥ 1

and we assume that 1 ≤ a ≤ b. Let 0 < t < 1, and let σ_{1} ≥ ⋯ ≥ σ_{n} be the eigenvalues of C = \sqrt{A^{−1}}B\sqrt{A^{−1}}, we see that

det C = σ_{1} ⋯ σ_{n}b^{n}/a^{n} ≥ 1,

and

det(tA + (1 − t)B) = a^{n} det(tI + (1 − t)C) ≥ det(tI + (1 − t)C).

We see that tA + (1 − t)B ∈ K if det(tI + (1 − t)C) ≥ 1. This follows essentially from the arithmetic-geometric inequality. We have

det(tI + (1 − t)C) = det[diag(t + (1 − t)σ_{1}, ⋯, t + (1 − t)σ_{n})] = [(t + (1 − t)σ_{1}] ⋯ [(t + (1 − t)σ_{n}]

= ∑_{k=0}^{n} ( ∑_{1≤i_{1}<···<i_{k}≤n} σ_{i_{1}} ⋯ σ_{i_{k}}) t^{n−k}(1 − t)^{k}.

(3.13)

From the arithmetic-geometric means inequality,

\begin{matrix}∑_{1≤i_{1}<···<i_{k}≤n} σ_{i_{1}} ⋯ σ_{i_{k}} ≥ \binom{n}{k} ( ∏_{1≤i_{1}<···<i_{k}≤n} σ_{i_{1}} ⋯ σ_{i_{k}})\frac{1}{\binom{n}{k}} \\ = \binom{n}{k} (σ_{1} ⋯ σ_{n})\frac{k}{\binom{n}{k}} ≥ \binom{n}{k} \end{matrix}  (3.14)

which follows from the fact that σ_{1} ⋯ σ_{n} = det C ≥ 1. Combining (3.13) and (3.14), and applying the binomial theorem, we have

det(tI + (1 − t)C) ≥ ∑_{k=0}^{n} \binom{n}{k}t^{n−k}(1 − t)^{k} = (t + (1 − t))^{n} = 1.

Hence tA + (1 − t)B ∈ K, K is convex. The proof is complete.

Remark 3.1. For p ≥ n, we know that Q_{n}(S_{+}^{(n)}) = S_{+}^{(n)} because the following quasiconvex function f : M^{n×n} → R_{+} vanishes only on S_{+}^{(n)}:

f(P) = dist^{n}(P, C(S_{+}^{(n)})) + | det(P) − 1|.

4. Connected sets in M^{2×2} without rank-one connections

In this section, we study quasiconvex hulls for connected, closed, and unbounded subsets of M^{2×2} which do not have rank-one connections. We give conditions such that the corresponding quasiconvex hulls are larger than the sets themselves. We consider this type of sets because they have very simple representations [36] so we can translate the conditions for rank-one connections at infinity into conditions on the assymptotic behaviours of certain Lipschitz mappings at infinity.

Theorem 4.1. Let K ⊂ M^{2×2} with 0 ∈ K be an unbounded closed set without rankone connections and f : P_{E_{∂}}(K) → E ¯∂ be its associated Lipschitz mapping (the case f : P_{E ¯∂} → E_{∂} is similar) satisfying

|f(A) − f(B)| < |A − B|, A, B ∈ P_{E_{∂}}(K), A ≠ B.

Then Khas a p-rank-one connection at infinity if and only if

lim sup_{|A|→∞, A∈P_{E_{∂}} (K)} \frac{|P | − |f(A)|}{|P |^{1/p}} = 0.  (4.1)

When (4.1) holds for some 1 ≤ p < ∞, Q_{p}(K) ≠ K.

Remark 4.2. The condition 0 ∈ K is a normalization assumption for the function f which does not affect the generality of the result because Q_{p}(K + A) = Q_{p}(K) + A for all 1 ≤ p < ∞. However, the condition 0 ∈ K implies |f(A)| < |A|.

In [37], the Lipschitz condition |f(A) − f(B)| ≤ c(p)|A − B| for a certain constant 0 < c(p) < 1, was assumed to ensure that Q_{p}(K) = K [37]. This Lipschitz condition also implies

lim sup_{|A|→∞, A∈P_{E_{∂}} (K)} \frac{|f(A)|}{|A|} ≤ c(p) < 1

so that

lim sup_{|A|→∞, A∈P_{E_{∂}} (K)} \frac{|A| − |f(A)|}{|A|^{1/p}} = +∞.

However, since c(p) is obtained as a bound of a certain singular integral, we do not know the value of c(p) explicitly (except for p = 2 where c(2) can be chosen as any positive number strictly less than 1). I do not know whether Q_{2}(K) = K if there exists α > 0 such that

lim sup_{|A|→∞, A∈P_{E_{∂}} (K)} \frac{|P | − |f(A)|}{|P |^{1/2}} ≥ α  (4.2)

Notice that when p = 1, (4.1) is equivalent to

lim sup_{|A|→∞, A∈P_{E_{∂}} (K)} \frac{|f(A)|}{|A|} = 1.

We would like to know whether

lim sup_{|A|→∞, A∈P_{E_{∂}} (K)} \frac{|f(A)|}{|A|} < 1,

would imply Q_{1}(K) = K. In [39], Q_{1}(K) = K was established under the condition that f ≡ 0, or, equivalently, K ⊂ E_{∂}. However, we have the following result for p = 2.

Theorem 4.3. Let K ⊂ M^{2×2} with 0 ∈ K, be an unbounded, closed and connected set without rank-one connections. Let f : P_{E_{∂}}(K) → E ¯∂ be its associated Lipschitz mapping (the case f : P_{E ¯∂} → E_{∂} is similar) satisfying

|f(A) − f(B)| < |A − B|.

Then

lim sup_{|A|→∞, A∈P_{E_{∂}} (K)} \frac{|f(A)|}{|A|} = α < 1  (4.3)

implies Q_{2}(K) = K.

Remark 4.4. Theorem 4.3 improves upon Theorem 4.1 in [37] where a much stronger assumption

|f(A) − f(B)| ≤ k|A − B|

with 0 < k < 1 is used.

Proof of Theorem 4.1. Let us find the eigenvalues Λ, λ (with λ ≤ Λ) for a general element (A + f(A))^{T} (A + f(A)) and noticing that A and f(A) are orthogonal to each other. We have

Λ(A) + λ(A) = tr(A + f(A))^{T} (A + f(A)) = |A|^{2} + |f(A)|^{2},

Λ(A)λ(A) = det(A + f(A))^{T} (A + f(A)) = (det(A + f(A)))^{2} = \frac{1}{4}(|A|^{2} − |f(A)|^{2})^{2}.

Therefore,

Λ(A) = \frac{1}{2} (|A|^{2} + |f(A)|^{2} + \sqrt{(|A|^{2} + |f(A)|^{2})^{2} − (|A|^{2} − |f(A)|^{2})^{2}})

= \frac{1}{2} (|A| + |f(A)|)^{2} .

Similarly,

λ(A) = \frac{1}{2} (|A| − |f(A)|)^{2} ,

so that

\begin{matrix}\frac{λ(A)}{(Λ(A))^{1/p}} = (\frac{|A| − |f(A)|}{(|A| + |f(A)|)^{1/p}} )^{2} \\ = (\frac{|A| − |f(A)|}{|A|^{1/p}})^{2} \frac{1}{(1 + \frac{|f(A)|}{|A|})^{2/p}} . \end{matrix}  (4.4)

From Definition 1.4, we see that K has a p-rank-one conncetion at infinity if

lim sup_{|A|→∞, A∈P_{E_{∂}} (K)} \frac{λ(A)}{(Λ(A))^{1/p}} = 0.  (4.5)

Since 1 ≤ 1 + |f(A)|/|A| ≤ 2, we have, combining (4.3) and (4.5) that

lim sup_{|A|→∞, A∈P_{E_{∂}} (K)} \frac{|A| − |f(A)|}{|A|^{1/p}} = 0.

Hence we have a sequence (A_{j} + f(A_{j})), such that

lim_{j→∞} \frac{|A_{j}| − |f(A_{j})|}{|A_{j}|^{1/p}} = 0,

lim_{j→∞} \frac{A_{j} + f_{(}A_{j})}{\sqrt{Λ(A_{j})}} = A_{0} + B_{0},

where A_{0} ∈ E_{∂}, B_{0} ∈ E ¯∂ respectively, rank(A_{0} + B_{0}) = 1 and |A_{0} + B_{0}| = 1. From Theorem 1.5, we have

{A + f(A) + t(A_{0} + B_{0}), A + f(A) ∈ K, t ≥ 0} ⊂ Q_{p}(K).

Therefore Q_{p}(K) has rank-one connections, which implies that Q_{p}(K) ≠ K.

Proof of Theorem 4.3. We extend f to be defined on E_{∂} such that |f(A) − f(B)| ≤ |A − B|, while on P_{E_{∂}}(K) we have the strict inequality |f(A) − f(B)| < |A − B|. Let P ∈ Q_{2}(K). There exists a sequence ϕ_{j} ∈ C_{0}^{∞}(U, R^{2}) such that

lim_{j→∞} ∫_{U} dist^{2}(P + Dϕ_{j}, K)dx = 0,

where U ⊂ R^{2} is the unit square.

As in the proof of [37, Th.4.1], we first consider

I_{j} = ∫_{U}[|P_{E ¯∂}(Dϕ_{j})|^{2} − |f(P_{E_{∂}}(P + Dϕ_{j})) − f(P_{E_{∂}}(P))|^{2}]dx,  (4.6)

to show that Dϕ_{j} is bounded in L^{2}(U). Since |f(A)|/|A| → α < 1 as |A| → ∞, there exists M > 0, whenever |A| ≥ M, |f(A)|/|A| ≤ (1 + α)/2. Now, on the one hand, since

∫_{U} |P_{E ¯∂}(Dϕ_{j})|^{2}dx = ∫_{U} |P_{E_{∂}}(Dϕ_{j})|^{2}dx,  (4.7)

we have

\begin{matrix}I_{j} = ∫_{U}[|P_{E_{∂}}(Dϕ_{j})|^{2} − |f(P_{E_{∂}}(P + Dϕ_{j})) − f(P_{E_{∂}}(P))|^{2}]dx \\ ≥ ∫_{\{x∈U, |P_{E_{∂}} (Dϕ_{j})(x)|≥M+|P |\}}[|P_{E_{∂}}(Dϕ_{j})|^{2} − |f(P_{E_{∂}}(P + Dϕ_{j})) − f(P_{E_{∂}}(P))|^{2}]dx \\ ≥ ∫_{\{x∈U, |P_{E_{∂}} (Dϕ_{j})(x)|≥M+|P |\}} |P_{E_{∂}}(Dϕ_{j})|^{2}dx \\ − ∫_{\{x∈U, |P_{E_{∂}} (Dϕ_{j})(x)|≥M+|P |\}} (\frac{(1 + α)^{2}(1 + ϵ)}{4}|P_{E_{∂}}(Dϕ_{j})|^{2}) dx \\ − C(ϵ) ∫_{\{x∈U, |P_{E_{∂}} (Dϕ_{j})(x)|≥M+|P |\}} (|P_{E_{∂}}(P)| + |f(P_{E_{∂}}(P))|)^{2} dx \\ = (1 − \frac{(1 + α)^{2}(1 + ϵ)}{4}) ∫_{\{x∈U, |P_{E_{∂}} (Dϕ_{j})(x)|≥M+|P |\}} |P_{E_{∂}}(Dϕ_{j})|^{2}dx \\ − C(ϵ)(|P_{E_{∂}}(P)| + |f(P_{E_{∂}}(P))|)^{2} \\ = μ ∫_{\{x∈U, |P_{E_{∂}} (Dϕ_{j})(x)|≥M+|P |\}} |P_{E_{∂}}(Dϕ_{j})|^{2}dx − C(ϵ, P), \end{matrix} (4.8)

where ϵ > 0 is a positive integer such that μ = 1 − (1 + α)^{2}(1 + ϵ)/4 > 0; C(ϵ) > 0, C(ϵ, P) > 0 are positive constants.

On the other hand, we may extend f to be defined on E_{∂} such that |f(A)−f(B)| ≤ |A−B|.

From Lemma 2.9 we have

|P_{E ¯∂}(A) − f(P_{E_{∂}}(A))| ≤ \sqrt{2} dist(A, K)  (4.9)

for every A ∈ M^{2×2}, so that

I_{j} = ∫_{U} |P_{E ¯∂}(Dϕ_{j})|^{2} − |f(P_{E_{∂}}(P + Dϕ_{j})) − f(P_{E_{∂}}(P))|^{2}]dx

= ∫_{U}[|P_{E ¯∂}(Dϕ_{j})|^{2}dx

− ∫_{U} |[f(P_{E_{∂}}(P + Dϕ_{j})) − P_{E ¯∂}(P + Dϕ_{j})] + [P_{E ¯∂}(Dϕ_{j})] + [P_{E ¯∂}(P) − f(P_{E_{∂}}(P))]|^{2} dx

≤ − ∫_{U}[P_{E ¯∂}(Dϕ_{j})] · ([f(P_{E_{∂}}(P + Dϕ_{j})) − P_{E ¯∂}(P + Dϕ_{j})] + [P_{E ¯∂}(P) − f(P_{E_{∂}}(P))]) dx

≤ 2\sqrt{2} ∫_{U} |P_{E ¯∂}(Dϕ_{j})|[dist(P + Dϕ_{j}, K) + dist(P, K)]dx

≤ \frac{μ}{2} ∫_{U} |P_{E ¯∂}(Dϕ_{j})|^{2}dx + C(μ) ∫_{U}[dist^{2}(P + Dϕ_{j}, K) + dist^{2}(P, K)]dx

= \frac{μ}{2} ∫_{U} |P_{E_{∂}}(Dϕ_{j})|^{2}dx + C(μ) ∫_{U}[dist^{2}(P + Dϕ_{j}, K) + dist^{2}(P, K)]dx

(4.10)

≤ \frac{μ}{2} ∫_{\{x∈U, |P_{E_{∂}} (Dϕ_{j})(x)|≥M+|P |\}} |P_{E_{∂}}(Dϕ_{j})|^{2}dx

+ \frac{μ}{2} ∫_{\{x∈U, |P_{E_{∂}} (Dϕ_{j})(x)|<M+|P |\}}(M + |P |)^{2}dx

+ C(μ) [dist^{2}(P, K) + ∫_{U} dist^{2}(P + Dϕ_{j}, K)] dx,

where we have used (4.9) and the inequality ab ≤ ϵa^{2} + C(ϵ)b^{2} for real numbers a, b and ϵ > 0 and C(ϵ) is a constant. Combining (4.8) and (4.10), we obtain

∫_{{x∈U, |P_{E_{∂}} (Dϕ_{j})(x)|≥M+|P |}} |P_{E_{∂}}(Dϕ_{j})(x)|^{2}dx ≤ C

for some constant C > 0 independent of j, because ∫_{U} dist^{2}(P +Dϕ_{j}, K)dx → 0 as j → ∞. Therefore P_{E_{∂}}(Dϕ_{j}) is bounded in L^{2}(U, M^{2×2}). Then (4.7) implies that Dϕ_{j} is bounded in L^{2}(U, M^{2×2}).

We may assume, up to a subsequence, that ϕ_{j} ⇀ 0 in W^{1,2}(U, R^{2}) because we can extend ϕ_{j} periodically outside U and then define ψ_{j} = \frac{1}{j}ϕ(jx). It is then easy to see that ψ_{j} ⇀ 0 in W^{1,2}(U, R^{2}) and

lim_{j→∞} ∫_{U} dist^{2}(P + Dψ_{j}, K)dx = 0,

so that (ψ_{j}) satisfies our requirements.

Now we show that Dϕ_{j} → 0 almost everywhere in U. Similar to the proof of the boundedness of Dϕ_{j} in L^{2}, we introduce

I_{j,k} = ∫_{U}[|P_{E ¯∂}(Dϕ_{j} − Dϕ_{k})|^{2} − |f(P_{E_{∂}}(P + Dϕ_{j})) − f(P_{E_{∂}}(P + Dϕ_{k}))|^{2}]dx,

motivated from [26]. We use the so-called ‘biting’ Young-measures [19] to study the limit of I_{j,k}. Since Dϕ_{j} is bounded in L^{2}, there exists a sequence of measurable sets U_{m} ⊂ U, U_{m+1} ⊂ U_{m} and meas(U_{m}) → 0, a subsequence of Dϕ_{j} (we still denote it by using subscript j) and a family of Young measures {ν_{x}}_{x∈U}, such that for every Carathéodory function f : U × M^{2×2} satisfying |f(x, A)| ≤ a(x) + C|A|^{2},

lim_{j→∞} ∫_{U\U_{m}} f(x, Dϕ_{j}(x))dx = ∫_{U\U_{m}} (∫_{M^{2×2}} f(x, λ)dν_{x}) dx.

for each fixed m > 0.

For I_{j,k}, we have, on the one hand,

\begin{matrix}I_{j,k} = ∫_{U}[|P_{E_{∂}}(Dϕ_{j} − Dϕ_{k})|^{2} − |f(P_{E_{∂}}(P + Dϕ_{j})) − f(P_{E_{∂}}(P + Dϕ_{k}))|^{2}]dx \\ ≥ ∫_{U\U_{m}} |P_{E_{∂}}(Dϕ_{j} − Dϕ_{k})|^{2} − |f(P_{E_{∂}}(P + Dϕ_{j})) − f(P_{E_{∂}}(P + Dϕ_{k}))|^{2}]dx, \end{matrix} (4.11)

because f is a 1-Lipschitz function. If we pass to the limits, j → ∞, then k → ∞, we have

lim inf_{k→∞} lim inf_{j→∞} I_{j,k}

≥ lim_{k→∞} lim_{j→∞} ∫_{U\U_{m}}[|P_{E_{∂}}(Dϕ_{j} − Dϕ_{k})|^{2}

− |f(P_{E_{∂}}(P + Dϕ_{j})) − f(P_{E_{∂}}(P + Dϕ_{k}))|^{2}]dx

= ∫_{U\U_{m}} ∫_{M^{2×2}} ∫_{M^{2×2}}[|P_{E_{∂}}(P + λ) − P_{E_{∂}}(P + τ)|^{2}

− |f(P_{E_{∂}}(P + λ)) − f(P_{E_{∂}}(P + τ))|^{2}]dν_{x}(λ)dν_{x}(τ)dx.

(4.12)

On the other hand (c.f. the calculations in (4.10)), we can estimate I_{j,k} from above by

I_{j,k} ≤ 2\sqrt{2} ∫_{U} |P_{E ¯∂}(Dϕ_{j} − Dϕ_{k})|[dist(P + Dϕ_{j}, K) + dist(P + Dϕ_{k}, K)]dx

≤ 2\sqrt{2} (∫_{U} |P_{E ¯∂}(Dϕ_{j} − Dϕ_{k})|^{2}dx)^{1/2}

(∫_{U} dist^{2}(P + Dϕ_{j}, K)dx)^{1/2} (∫_{U} dist^{2}(P + Dϕ_{k}, K)dx)^{1/2} .

(4.13)

Since ∫_{U} |P_{E ¯∂}(Dϕ_{j} − Dϕ_{k})|^{2}dx is bounded and

lim_{j→∞} ∫_{U} dist^{2}(P + Dϕ_{j}, K)dx = 0, lim_{k→∞} ∫_{U} dist^{2}(P + Dϕ_{k}, K)dx = 0,

we let j → ∞, then k → ∞,

lim sup_{k→∞} lim sup_{j→∞} I_{j,k} ≤ 0.

Combining (4.12) and (4.13) we see that

\begin{matrix}∫_{U} ∫_{M^{2×2}} ∫_{M^{2×2}}[|P_{E_{∂}}(P + λ) − P_{E_{∂}}(P + τ)|^{2} \\ − |f(P_{E_{∂}}(P + λ)) − f(P_{E_{∂}}(P + τ))|^{2}]dν_{x}(λ)dν_{x}(τ)dx = 0 \end{matrix}  (4.14)

Since the integrand in (4.14) equals 0 if and only if P_{E_{∂}}(λ) = P_{E_{∂}}(τ), we see that P_{E_{∂}}(Dϕ_{j}) → 0 almost everywhere. Now, since

0 ≤ |P_{E ¯∂}(P + Dϕ_{j}) − f(P_{E_{∂}}(P + Dϕ_{j}))| ≤ \sqrt{2} dist(P + Dϕ_{j}, K) → 0,

almost everywhere, we see that P_{E ¯∂}(Dϕ_{j}) converges almost everywhere in U. The limit is 0 almost everywhere because P_{E ¯∂}(Dϕ_{j}) ⇀ 0 in L^{2}.

The conclusion follows from the fact that

0 = lim_{j→∞} ∫_{U} dist^{2}(P + Dϕ_{j}, K)dx

≥ lim_{j→∞} ∫_{U\U_{m}} dist^{2}(P + Dϕ_{j}, K)dx = ∫_{U\U_{m}} dist^{2}(P, K)dx,

for each fixed m, hence dist^{2}(P, K) = 0, P ∈ K.

Remark 4.5. The second part of the proof of Theorem 4.3 shows that if we do not assume condition (4.3), and instead we require that the sequence Dϕ_{j} is bounded, we can still show that up to a subsequence Dϕ_{j} converges almost everywhere. In the terminology of Yan [33], the set is W^{1,2}-weakly stable. However, when we study quasiconvex hulls, the sequence comes from a minimization problem. Therefore we have to establish boundedness of the sequence separately.

5. Quasiconvex hulls of some graphs in M^{2n×2}

In this section, we examine the quasiconvex hulls for a class of sets in M^{2n×2} proposed in [28] for the study of singular solutions for elliptic systems in two-dimensional spaces. We examine two cases. The first one is when the system is of sublinear growth, so that the quasiconvex hull contains extra matrices. The second is when the system is of linear growth and the corresponding constitutive functions are strongly quasimonotone. In this case we can prove that Q_{2}(K) = K.

We examine the quasiconvex hulls of the unbounded set designed by Šverák [28] in searching of singular solutions of the Euler-Lagrange equations of variational problems with quasiconvex integrands. Let F : M^{n×2} → ℝ be smooth and quasiconvex, then the weak solution of div DF (Du) = 0 is equivalent to a system of first order equations Dv = DF (Du)J where

J = (\begin{matrix}0 & 1 \\ −1 & 0 \end{matrix}) .

The method suggested in [28] is to find nontrivial Young measures for the n×2 dimensional graph K ⊂ M^{2n×2} defined by

G_{F} = \{(\begin{matrix}X \\ DF (X)J \end{matrix}) , X ∈ M^{n×2}\} .  (5.1)

Let us examine a simple example first, which connects the Lipschitz graphs in M^{2×2} without rank-one connections studied earlier in this section and monotone gradient mappings from ℝ^{2} to ℝ^{2}.

A mapping f : ℝ^{2} → ℝ^{2} is strictly monotone increasing (decreasing) if for all x, y ∈ ℝ^{2}, x ≠ y, (f(x) − f(y)) · (x − y) > 0 ((f(x) − f(y)) · (x − y) < 0 respectively). We have

Proposition 5.1. Suppose F : ℝ^{2} → ℝ is of class C^{1}. Then DF : ℝ^{2} → ℝ^{2} is a strictly monotone mapping if and only if G_{F} ⊂ M^{2×2} defined in (5.1) with n = 1 does not have rank-one connections.

The following result is a simple application of Theorem 1.5.

Theorem 5.2. Suppose F : M^{n×2} → ℝ is of class C^{1} and |DF (X)| ≤ C(1 + |X|^{σ}) for some 0 ≤ σ < 1. Then

\{(\begin{matrix}Y \\ DF (X)J \end{matrix}) , X, Y ∈ M^{n×2}\} ⊂ Q_{1}(G_{F}).

If we further assume that 0 ≤ σ < 1/2, then

\{(\begin{matrix}Y \\ DF (X)J \end{matrix}) , X, Y ∈ M^{n×2}\} ⊂ Q_{2}(G_{F}).

In the next result, we use the notion of strong quasimonotonicity of a gradient mapping to study quasiconvex hulls. We give conditions such that Q_{2}(G_{F}) = G_{F}. An important class of mappings in the study of existence problem for elliptic equations and systems are the so-called pseudo-monotone mappings [25]. Let A : W_{0}^{1,2} → (W_{0}^{1,2})^{∗} be a continuous map. If u_{j} ⇀ u in W_{0}^{1,2} and

lim inf_{j→∞}⟨A(u_{j}), u_{j} − u⟩ ≤ 0

imply that u_{j} → u strongly in W_{0}^{1,2}, the operator A is called pseudo-monotone. For elliptic systems, a class of mappings A : M^{N×n} → M^{N×n} called quasimonotone mappings was introduced to solve elliptic systems under a weaker monotonicity condition [34] (also see [15]) for the regularity problem). Recently, R. Landes [20] showed that for elliptic systems, quasimonotonicity is also a necessary condition for pseudo-monotonicity. For a gradient mapping DF (P) with F : M^{N×n} → ℝ, quasimonotonicity of DF implies quasiconvexity of F while the converse is not true [16].

Theorem 5.3. Suppose F : M^{n×2} → R is of class C^{1} such that

|DF (X) − DF (Y)| ≤ C|X − Y |

for some c_{0} > 0 and DF is strongly quasimonotonic in the sense that

∫_{Ω} DF (P + Dϕ(x)) · Dϕ(x)dx ≥ c_{0} ∫_{Ω} |Dϕ(x)|^{2}dx,

for some constant c > 0 and for every open set Ω ⊂ ℝ^{2}, for every P ∈ M^{n×2} and every ϕ ∈ C_{0}^{1}(Ω, ℝ^{n}). Then

Q_{2}(G_{F}) = G_{F}

and G_{F} can only support trivial gradient Young measures.

Proof of Proposition 5.1. The statement that G_{F} is closed, unbounded and does not have rank-one connections is equivalent to the statement that G_{F} is a Lipschitz graph from P_{E_{∂}}(G_{F}) to E ¯∂, or from P_{E ¯∂}(G_{F}) to E_{∂}. Without loss of generality, we may assume that the first situation occurs. The statement is also equivalent to

|P_{E ¯∂} (\binom{X}{DF (X)J}) − P_{E ¯∂} (\binom{X}{DF (X)J})|

< |P_{E_{∂}} (\binom{X}{DF (X)J}) − P_{E_{∂}} (\binom{X}{DF (X)J})| .

This is equivalent to

[(x_{1} − y_{1}) − (\frac{∂F (X)}{∂x_{1}} − \frac{∂F (Y)}{∂x_{1}})]^{2} + [(x_{2} − y_{2}) − (\frac{∂F (X)}{∂x_{2}} − \frac{∂F (Y)}{∂x_{2}})]^{2}

< [(x_{1} − y_{1}) + (\frac{∂F (X)}{∂x_{1}} − \frac{∂F (Y)}{∂x_{1}})]^{2} + [(x_{2} − y_{2}) + (\frac{∂F (X)}{∂x_{2}} − \frac{∂F (Y)}{∂x_{2}})]^{2} ,

which is equivalent to

(DF (X) − DF (Y)) · (X − Y) > 0,

hence DF is monotone increasing.

Proof of Theorem 5.2. Let us calculate the eigenvalues Λ(X) ≥ λ(X) of

\binom{X}{DF (X)J}^{T} \binom{X}{DF (X)J}.

We obtain

Λ(X)

= \frac{1}{2} (|X|^{2} + |DF (X)|^{2} + \sqrt{(|X|^{2} + |DF (X)|^{2}) − 4 det(X^{T} X + J^{T} DF (X)^{T} DF (X)J)}) ,

λ(X)

= \frac{1}{2} (|X|^{2} + |DF (X)|^{2} − \sqrt{(|X|^{2} + |DF (X)|^{2}) − 4 det(X^{T} X + J^{T} DF (X)^{T} DF (X)J)}) .

We have

\frac{λ(X)}{Λ(X)} = 4\frac{det (\frac{(X^{T} X+J^{T} DF (X)^{T} DF (X)J)}{|X|^{2}+|DF (X)|^{2}})}{1 + \sqrt{1 − 4 det (\frac{(X^{T} X+J^{T} DF (X)^{T} DF (X)J)}{|X|^{2}+|DF (X)|^{2}})}}.

Noticing that |DF (X)| ≤ C(1 + |X|^{σ}) and 0 < σ < 1, we have, for any rank-one matrix X_{0} with unit norm |X_{0}| = 1,

lim_{t→+∞} \frac{λ(tX_{0})}{Λ(tX_{0})} = 4\frac{det(X_{0}^{T} X_{0})}{1 + \sqrt{1 − 4 det(X_{0}^{T} X_{0})}} = 0.

From Theorem 1.5 we see that

\{\binom{X}{DF (X)J} + s\binom{X_{0}}{0}, X ∈ M^{n×2}, s ≥ 0\} ⊂ R_{1}(G_{F}).

Since for any closed set K such that G_{F} ⊂ K ⊂ R_{p}(G_{F}), we have R_{p}(K) = R_{p}(G_{F}), we see that for any X_{1}, ⋯, X_{m} with X_{i} a rank-one matrix and |X_{i}| = 1, i = 1, ⋯, m, we have

\{\binom{X}{DF (X)J} + ∑_{i=1}^{m} s_{i}\binom{X_{i}}{0}, X ∈ M^{n×2}, s_{i} ≥ 0\} ⊂ R_{1}(G_{F}).

Notice that for any X, Y ∈ M^{n×2}, Y can be represented by

Y = X + ∑_{i=1}^{m} s_{i}X_{i}

for some rank-one matrices X_{i} with norm 1 and s_{i} ≥ 0, 0 ≤ m ≤ 2n. Therefore

\{\binom{Y}{DF (X)J}, X, Y ∈ M^{n×2}\} ⊂ R_{1}(G_{F}).

The proof is complete.

If σ < 1/2, we have

\frac{λ(X)}{Λ(X)^{1/2}} = 4\begin{matrix}det & (\frac{(X^{T} X+J^{T} DF (X)^{T} DF (X)J)}{(|X|^{2}+|DF (X)|^{2})^{3/4}}) \\ [1 + \sqrt{1 − 4 det (\frac{(X^{T} X+J^{T} DF (X)^{T} DF (X)J)}{|X|^{2}+|DF (X)|^{2}})}] & 3/2 \end{matrix} .

As in the above proof, let X_{0} be a rank-one matrix with norm 1, we see that

lim_{t→+∞} \frac{λ(tX_{0})}{(Λ(tX_{0}))^{1/2}} = 0

if and only if

lim_{t→∞} det (\frac{(tX_{0}^{T} tX_{0} + J^{T} DF (tX_{0})^{T} DF (tX_{0})J)}{(|tX_{0}|^{2} + |DF (tX_{0})|^{2})^{3/4}}) = 0.

This follows from the growth condition |DF (X)| ≤ C(1 + |X|^{σ}) with 0 ≤ σ < 1/2.

Proof of Theorem 5.3. Let \binom{A}{B} ∈ Q_{2}(G_{F}) and let \binom{ϕ_{j}}{ψ_{j}} be a minimizing sequence in C_{0}^{∞} such that

lim_{j→∞} ∫_{D} dist^{2} (\binom{A + Dϕ_{j}}{B + Dψ_{j}}, G_{F} ) dx = 0.

Since for each fixed j, Dϕ_{j} and Dψ_{j} are bounded in L^{∞}, we can apply the measurable selection lemma to find an measurable mapping X_{j} : D → M^{2×n} such that

dist^{2} (\binom{A + Dϕ_{j}(x)}{B + Dψ_{j}(x)}, G_{F} ) = |A + Dϕ_{j}(x) − X_{j}(x)|^{2} + |B + Dψ_{j}(x) − DF (X_{j}(x))J|^{2}

and

lim_{j→∞} ∫_{D} |A + Dϕ_{j}(x) − X_{j}(x)|^{2}dx = 0,  lim_{j→∞} ∫_{D} |B + Dψ_{j}(x) − DF (X_{j}(x))J|^{2}dx = 0.

Let A + Dϕ_{j}(x) − X_{j}(x) = n_{j}(x), B + Dψ_{j}(x) − DF (X_{j}(x))J = m_{j}(x), then n_{j} → 0, m_{j} → 0 strongly in L^{2}.

Now we have X_{j}(x) = A + Dϕ_{j}(x) − n_{j}(x) so that

B + Dψ_{j}(x) = DF (A + Dϕ_{j}(x) − n_{j}(x))J + m_{j}(x).

Applying the curl operator on both sides of the above equality in the sense of distributions we have

− div DF (A + Dϕ_{j}(x) − n_{j}(x)) + curl m_{j}(x) = 0.

Since n_{j}, m_{j} ∈ L^{2}, we have

∫_{D}[DF (A + Dϕ_{j}(x) − n_{j}(x)) · Dϕ_{j}(x) − m_{j}(x) · D̃ϕ_{j}(x)]dx = 0,  (5.2)

where

D̃ϕ_{j} = (m_{12} \frac{∂ϕ_{j1}}{∂x_{1}} − m_{11} \frac{∂ϕ_{j1}}{∂x_{2}} ) + (m_{22} \frac{∂ϕ_{j2}}{∂x_{1}} − m_{21} \frac{∂ϕ_{j2}}{∂x_{2}} ) .

Since DF is strongly quasimonotone and satisfies Lipschitz condition, we have in (5.2)

∫_{D} DF (A + Dϕ_{j}(x)) · Dϕ_{j}(x)
= ∫_{D}[DF (A + Dϕ_{j}(x)) − DF (A + Dϕ_{j}(x) − n_{j}(x))] · Dϕ_{j}(x)dx
+ ∫_{D} m_{j}(x) · D̃ϕ_{j}(x)]dx,

so that

c_{0} ∫_{D} |Dϕ_{j}|^{2}dx ≤ C ∫_{D} |n_{j}||Dϕ_{j}(x)|dx + ∫_{D} |m_{j}||Dϕ_{j}|dx.

Hence there exists a constant C_{1} > 0 depending only on c_{0} and C, such that

∫_{D} |Dϕ_{j}|^{2}dx ≤ C_{1} ∫_{D}(|n_{j}|^{2} + |m_{j}|^{2})dx → 0

as j → ∞. Therefore Dϕ_{j} → 0 strongly in L^{2}. Since

B + Dψ_{j}(x) = DF (A + Dϕ_{j}(x) − n_{j}(x))J + m_{j}(x),

we may pass to the limit j → ∞ in this equality, so that

Dψ_{j} → DF (A)J − B.

Since ψ_{j} has compact support in D, we see that B = DF (A)J. Thus

\binom{A}{B} ∈ G_{F} .

The proof is complete.

44 K. Zhang / Rank-one connections at infinity and quasiconvex hulls

Remark 5.4. Instead of requiring that DF is a Lipschitz mapping, we could require that DF (X) is coercive in the sense that DF (X) · X ≥ C|X|^{2} − C_{1}. In that situation one could apply the method used in [34] to establish a similar result.

Acknowledgements. I would like to thank Z. Iqbal, S. Müller and J. Kristensen for helpful discussions and suggestions. This work was completed during my visit to Max-Planck Institute for Mathematics in the Sciences, Leipzig. I would like to thank Stefan Müller for invitation.

References

[1] E. Acerbi, N. Fusco: Semicontinuity problems in the calculus of variations, Arch. Rational Mech. Anal. 86 (1984) 125–145.

[2] E. J. Balder: A general approach to lower semicontinuity and lower closure in optimal control theory, SIAM J. Control and Optimization 22 (1984) 570–597.

[3] H. Berliocchi, J. M. Lasry: Intégrandes normales et mesures paramétrées en calcul des variations, Bull. Soc. Math. France 101 (1973) 129–184.

[4] J. M. Ball: Convexity conditions and existence theorems in nonlinear elasticity, Arch. Rational Mech. Anal. 63 (1977) 337–403.

[5] J. M. Ball: A version of the fundamental theorem of Young measures, in: Partial Differential Equations and Continuum Models of Phase Transitions, edited by M. Rascle, D. Serre and M. Slemrod, Springer-Verlag (1989) 207–215.

[6] J. M. Ball: Sets of gradients with no rank-one connections, J. Math. Pures et Appl. 69 (1990) 241–259.

[7] K. Bhattacharya, N. B. Firoozye, R. D. James, R. V. Kohn: Restrictions on microstructures, Proc. Royal Soc. Edinburgh A 124 (1994) 843–878.

[8] J. M. Ball, R. D. James: Fine phase mixtures as minimizers of energy, Arch. Rational Mech. Anal. 100 (1987) 13–52.

[9] J. M. Ball, R. D. James: Proposed experimental tests of a theory of fine microstructures and the two-well problem, Phil. Royal Soc. Lon. 338A (1992) 389–450.

[10] B. Dacorogna: Direct Methods in the Calculus of Variations, Springer-Verlag, 1989.

[11] B. Dacorogna: On rank one convex functions which are homogeneous of degree one, Preprint (1994).

[12] I. Ekeland, R. Temam: Convex Aanlysis and Variational Problems, North-Holland, 1976.

[13] N. B. Firoozye: Optimal use of the translation method and relaxation of variational problems, Comm. Pure Appl. Math. 44 (1991) 643–678.

[14] I. Fonseca, S. Müller: Relaxation of quasiconvex integrals in BV (Ω, ℝ^{p}) for integrands f(x, u, Du), Arch. Rational Mech. Anal. 123 (1993) 1–49.

[15] M. Fuchs: Regularity theorems for nonlinear systems of partial differential equations under natural ellipticity conditions, Analysis 7 (1987) 83–93.

[16] C. Hamburger: Quasimonotonicity, regularity and duality for nonlinear systems of partial differential equations, Ann. Mat. Pura Appl. 169 (1995) 321–354.

[17] R. V. Kohn: The relaxation of a double-well energy, Cont. Mech. Therm. 3 (1991) 981–1000.

[18] D. Kinderlehrer, P. Pedregal: Characterizations of Young measures generated by gradients, Arch. Rational Mech. Anal 115 (1991) 329–365.

[19] D. Kinderlehrer, P. Pedregal: Gradient Young measures generated by sequences in Sobolev spaces, J. Geom. Anal. 4 (1994) 59–90.

[20] R. Landes: Quasimonotone versus pseudomonotone, Proc. Royal Soc. Edin. 126A (1996) 705–717.

[21] C. B. Morrey jun.: Multiple Integrals in the Calculus of Variations, Springer, 1966.

[22] S. Müller, V. Šverák, B.-S. Yan: Sharp stability results for almost conformal maps, preprint.

[23] S. Müller: Variational Models for Microstructure and Phase Transitions, Max-Planck Inst. Leipzig, Lecture Notes No. 2, 1998.

[24] S. Müller: On quasiconvex functions which are homogeneous of degree one, Indiana Math. J. 41 (1992) 295–300.

[25] J. Necas: Introduction to the Theory of Nonlinear Elliptic Equations, Wiley, 1986.

[26] V. Šverák: On Tartar’s conjecture, Ann. Inst. H-Poincaré, Anal. Non Lin. 10 (1993) 1405– 1412.

[27] V. Šverák: On the problem of two wells, in: Microstructure and Phase Transitions, IMA Vol. Math. Appl. 54, edited by D. Kinderlehrer, R. D. James, M. Luskin and J. L. Ericksen, Springer (1993) 183–189.

[28] V. Šverák: Lower semicontinuity of variational integrals and compensated compactness, in: Proc. ICM 1994 (2), edited by S. D. Chatterji, Birkhäuser (1995) 1153–1158.

[29] V. Šverák: Rank one convexity does not imply quasiconvexity, Proc. Royal Soc. Edin. 120A (1992) 185–189.

[30] L. Tartar: Compensated compactness and partial differential equations, in: Nonlinear Analysis and Mechanics: Heriot-Watt Symposium Vol.IV, edited by R. Knops, Pitman (1979) 136–212.

[31] M. Valadier: Young measures, in: Methods of Nonconvex Analysis, Springer Lecture Notes in Math. 1446 (1990) 152–188.

[32] B.-S. Yan: Remarks on the set of quasi-conformal matrices in higher dimensions, Preprint (1994).

[33] B.-S. Yan: Remarks on W^{1,p}-stability of the conformal set in higher dimensions, Ann. Inst. H-Poincaré, Anal. Non Lin. 13 (1996) 691–705.

[34] K.-W. Zhang: On the Dirichlet problem for a class of quasilinear elliptic systems of PDEs in divergence form, in: Partial Differential Equations, Proc. Tianjin, 1996, edited by S. S. Chern, Springer Lecture Notes in Math. 1036 (1988) 262–277.

[35] K.-W. Zhang: A construction of quasiconvex functions with linear growth at infinity, Ann. Sc. Norm. Sup. Pisa Serie IV, 19 (1992) 313–326.

[36] K.-W. Zhang: On connected subsets of M^{2×2} without rank-one connections, Proc. Royal Soc. Edin. 127A (1997) 207–216.

[37] K.-W. Zhang: On non-negative quasiconvex functions with unbounded zero sets, Proc. Royal Soc. Edin. 127A (1997) 411–422.

[38] K.-W. Zhang: Quasiconvex functions, SO(n) and two elastic wells, Ann. Inst. H-Poincaré, Anal. Non Lin. 14 (1997) 759–785.

[39] K.-W. Zhang: On some quasiconvex functions with linear growth, J. Convex Anal. 5 (1998) 133–146.