PUBLICATIONS DE L’INSTITUT MATHÉMATIQUE Nouvelle série, tome 97(111) (2015), 89–102 DOI: 10.2298/PIM140923001J

ON THE CONVERGENCE OF A CLASS OF THE REGULARIZATION METHODS FOR ILL-POSED QUADRATIC MINIMIZATION PROBLEMS WITH CONSTRAINT

Milojica Jaćimović, Izedin Krnić, and Oleg Obradović

Abstract. We study a class of regularization methods for solving least-squares ill-posed problem with a convex constraint. Convergence and convergence rate results are proven for the problems which satisfy so called power source condition. All the results are obtained under the assumptions that, instead of exact initial data, only their approximations are known.

1. Introduction

We consider an ill-posed quadratic minimization problem

(1.1)  J(u) = \frac{1}{2}∥Au − f∥^{2} → inf, u ∈ U,

where A : H → F is a bounded linear operator mapping between Hilbert spaces H and F, U ⊆ H is a closed convex set and f ∈ F is fixed. We will deal with this problem assuming that the sets U_{∗} and U_{∞} of the solutions of the given problem and of the corresponding problem without constraints

(1.2)  J(u) = \frac{1}{2}∥Au − f∥^{2} → inf, u ∈ H,

are nonempty. In this case the problems

(1.3)  ∥u∥^{2} → inf, u ∈ U_{∗}, ∥u∥^{2} → inf, u ∈ U_{∞}

have unique solutions, which we will denote by u_{∗} and u_{∞} and call normal solutions of problems (1.1) and (1.2).

Let us observe that problem (1.1) is equivalent to the variational inequality

find u ∈ U such that ⟨A^{∗}Au − A^{∗}f, v − u⟩ ⩾ 0, ∀v ∈ U.

while the operator equation (which is equivalent to the problem (1.2)) A^{∗}Au = A^{∗}f, is a specific case of the previous variational inequality for U = H.

2010 Mathematics Subject Classification: Primary 47A52; Secondary 65F22.

Key words and phrases: quadratic programming, ill-posed problems, regularization.

Communicated by Gradimir Milovanović.

Problems (1.1) and (1.2) in the literature (see [9, 11, 12, 18]) are regularly studied under the assumptions that, instead of the exact operator A and instead of the element f, one actually deals with their approximations A_{η} ∈ L(H, F), f_{δ} ∈ F, such that

(1.4)  ∥A − A_{η}∥ ⩽ η, ∥f − f_{δ}∥ ⩽ δ, ∥A_{η}∥^{2} ⩽ a,

where η, delta > 0 are small positive real numbers and a > 0.

In general, problems (1.1) and (1.2) are ill-posed. Therefore, it is necessary to apply some methods of regularization [1, 5, 18, 19] that will produce good and stable sequence of approximate solutions of the problems.

Methods of regularization considered here are based on a modification of the family of regularizing functions from [18]. Note that the Tikhonov method and the iterated Tikhonov method of regularization belong to this class of methods.

The bounds of the accuracy of the regularization methods for solving ill-posed problems (1.1) and (1.2) are usually obtained for classes of problems defined by certain conditions concerning their normal solutions. The conditions that we discuss here and that were discussed in the cited papers belong to the class of so-called source conditions or sourcewise representable conditions. The well known conditions of this type are power source conditions that were widely used in [18] for obtaining the estimates of the convergence rate of regularization methods for solving linear operator equations.

Our main goal is to construct the methods of regularization of some classes of ill-posed problems (1.1) (minimization of quadratic function with convex constraints), with the convergence rate that can be compared with the convergence rate of regularization of linear operator equation (see for example [18]). However, it is not possible to obtain such convergence rates without additional conditions. In [11] an example was constructed that shows the rate of convergence, depends on the boundary of the set U ⊆ R^{2}, and it can be arbitrarily slow.

The paper is organized as follows. In Section 2 we present and discuss the roles and the meanings of the source conditions for problems (1.1) and (1.2) and describe one class of regularization methods. The key relation of the method for solving (1.2 is presented in two forms: as a variational inequality and as a problem of minimization. The proposed methods are inspired by the regularization methods for solving linear operator equation without constraints, studied in [1,5,17,18] and developed e.g., in [3,7,8,13–18], in a way that is different from the one presented here. Taking into account the presence of the constraints, for the functions that define the regularization method, we had to include new conditions, and to use new techniques other than those used in [9–12].

The main results of the paper are contained in Section 3 where we prove convergence of the regularization method for ill-posed problems (1.1). In two theorems we show that the typical (power) source condition or projected source condition is sufficient for the convergence rate (of regularized solutions of (1.1) to normal solution) of the same order as in the case of minimization problem (1.2) without constraints.

ON THE CONVERGENCE OF A CLASS OF THE REGULARIZATION METHODS ... 91

2. Source conditions and regularization method

2.1. Power source conditions and unconstrained regularization. The first source conditions we consider here is the so-called power source condition or power sourcewise representable condition for problem (1.2), which can be presented in the form

(2.1)  u_{∞} = |A|^{p}h_{∗}, where h_{∗} ∈ H, |A|^{p} = (A^{∗}A)^{p/2}, p > 0.

The previous condition seems quite natural, if we have in mind that u_{∞} ∈ \overline{R(A^{∗})}, where R(A^{∗}) is the range of the operator A and \overline{R(A^{∗})} its closure in norm of the space H. It means that the solution u_{∞} is densely surrounded by the elements from R(A^{∗}).

In [1,5,15,18] as approximations of the normal solution u_{∞} of problem (1.2), the following was used

(2.2)  u_{α} = g_{α} (A_{η}^{∗}A_{η})A_{η}^{∗}f_{δ}

that are generated by the family of Borel measurable functions g_{α} : [0, a] ↦ ℝ satisfying the conditions

(2.3)

sup_{0⩽t⩽a} |g_{α} (t)| ⩽ M/α, M > 0,

sup_{0⩽t⩽a} t^{p}|1 − tg_{α} (t)| ⩽ γ_{p}α^{p}, α > 0, γ_{p} = const, 0 ⩽ p ⩽ p_{0}, 0 < p_{0}.

Here, a is the constant from (1.4). Number p_{0} is called qualification of the family {g_{α} : α > 0} and it has an important role. For example, (see [19, p. 100]) if the conditions (1.4), (2.3) and (2.1) are satisfied, then for

α = α(η, δ) = d(η + δ)^{\frac{2}{p+2}} , d = const > 0,

one has

∥u_{α} − u_{∞} ∥ ⩽ const(η + δ)^{\frac{p}{p+2}} , 0 < p ⩽ 2p_{0}.

2.2. Projected source condition and constrained regularization. At first, we need to address the question of the meaning of the source condition [15]

(2.4)  u_{∗} = |A|^{p}h_{∗}, where h_{∗} ∈ H, |A|^{p} = (A^{∗}A)^{p/2}, p > 0.

When studying the problem (1.1) of minimization with constraints, it is necessary to have in mind that, in general, the normal solution u_{∗} does not satisfy the condition

(2.5)  u_{∗} ∈ \overline{R(A)^{∗}} = N (A)^{⊥} ,

that was fulfilled for the problem without constraints (in this case u_{∗} = u_{∞} ). This fact was important for the justification of the source condition (2.1). Hence, in case (1.1) of minimization with constraints, it is necessary to describe a class of the problems whose normal solutions satisfy the condition (2.4) or to find a more adequate source condition with similar meaning. We will follow these steps to prove a lemma.

Generally, in what follows, we will denote by Π_{X} (x) (or by Π_{X} x) the metric projection of the point x on a closed convex set X ⊆ H.

Lemma 2.1. Normal solution u_{∗} of problem (1.1) belongs to \overline{Π_{U}(R(A^{∗}))} .

Proof. The function F_{α} (u) = \frac{1}{2}∥Au − f∥^{2} + \frac{1}{2}α∥u∥^{2} is strongly convex, such that the problem F_{α} (u) ↦ inf, u ∈ U has a unique solution, which we denote by v_{α}. It is well known that v_{α} → u_{∗} as α → 0. The corresponding variational inequality ⟨F′_{α}(v_{α}), u − v_{α} ⟩ ⩾ 0, ∀u ∈ U can be written in the form ⟨v_{α} − (A^{∗}f − A^{∗}Au_{α})/α, u − v_{α} ⟩ ⩾ 0, ∀u ∈ U. It is well known ( [17] or [19, p. 183]) that then v_{α} = Π_{U}((A^{∗}f − A^{∗}Au_{α})/α) ∈ Π_{U}(R(A^{∗})). From here, allowing that α → 0, we obtain that u_{∗} ∈ \overline{Π_{U}(R(A^{∗}))} . □

In what follows we will also study the accuracy of the regularization methods on the class of problems of the type (1.1) with normal solutions satisfying the so-called projected source condition:

(2.6)  u_{∗} = Π_{U}(|A|^{p}h_{∗}), h_{∗} ∈ H, p > 0.

Note that the source conditions for the linear operator equations with constraints, that are coordinated with the Tikhonov method of regularization were considered in [11].

The following example indicates one class of problems (1.1) whose normal solutions satisfy (2.5).

Example 2.1. If U ⊆ H is a closed convex which has exactly one hyperplane of support at each boundary point and the conditions u_{∗} ∈ Π_{U} \overline{(R(A^{∗}))} and U_{∗} ∩ U_{∞} = ∅ , are satisfied, then (2.5) holds.

Proof. From U_{∗}∩U_{∞} = ∅ it follows that u_{∗} belongs to the boundary of the set U_{∗} and that J′(u_{∗}) = A^{∗}Au_{∗} − A^{∗}f ≠ 0. Let u_{∗} = Π_{U}(h), h ∈ \overline{R(A^{∗})} and denote c = u_{∗} − h. Then ⟨c, u − u_{∗}⟩ = ⟨u_{∗} − h, u − u_{∗}⟩ ⩾ 0, ∀u ∈ U. It follows from here that the hyperplane H_{c} := {x ∈ H : ⟨c, x⟩ = ⟨c, u_{∗}⟩} is a hyperplane of support of U at u_{∗}. From the inequality ⟨A^{∗}Au_{∗}−A^{∗}f, u−u_{∗}⟩ = ⟨J′(u_{∗}), u−u_{∗}⟩ ⩾ 0, ∀u ∈ U, it follows that H_{J′(u_{∗})} = {x ∈ H : ⟨J′(u_{∗}), x⟩ = ⟨J′(u_{∗}), u_{∗}⟩} is also a hyperplane of support of U at u_{∗}. Having in mind that U has only one supporting hiperplane at u_{∗}, we can now conclude that the vector A^{∗}Au_{∗} − A^{∗}f = J′(u_{∗}) is collinear with c : J′(u_{∗}) = A^{∗}Au_{∗} − A^{∗}f = γc, γ ≠ 0. It means that c ∈ R(A^{∗}) ⊆ \overline{R(A^{∗})}. Now u_{∗} ∈ \overline{R(A^{∗})} immediately follows from here and from equality u_{∗} = c + h, h ∈ \overline{R(A^{∗})}. □

Now we can describe a class of regularization methods whose accuracy we study on a class of the problems whose normal solutions satisfy (2.1). Later, we will also consider condition (2.6).

Suppose that the Borel measurable functions g_{α} : [0, a] ↦ ℝ, a > 0 satisfy the conditions

(2.7)  1 − tan_{α} (t) ⩾ 0, t ∈ [0, a],
(2.8)  \frac{1}{t + βα} ⩽ g_{α} (t) ⩽ \frac{1}{βα} , t ∈ [0, a], β > 0,

(2.9)

∃p_{0} > 0 such that ∀p ∈ [0, p_{0}] sup_{0⩽t⩽a}t^{p}(1−tan_{α} (t)) ⩽ γα^{p}, α > 0, where γ = γ(p_{0}).

Notice that, in this case, the functions satisfying conditions (2.7)–(2.9) also satisfy conditions (2.3). In addition, the family of the functions g_{α} (t) = (t + α)^{−1} and g_{α} (t) = ∑ _{j=0}^{m−1} \frac{α^{j}}{(t+α)^{j}} = t^{−1} (1−(1+t)^{−m} ) (that defines the Tikhonov methods of regularization and its iterated variant) also satisfy these conditions.

We prove some properties of the operators g_{α} (A_{η}^{∗}A_{η}) and g_{α}^{−1} (A_{η}^{∗}A_{η}), where the function t ↦ 1/g_{α} (t) is denoted by (g_{α} (t))^{−1} .

Firstly, let us observe that g_{α} (A_{η}^{∗}A_{η}) : H ↦ H is a self-adjoint positive definite operator. Hence there exist g_{α}^{1/2} (A_{η}^{∗}A_{η}) and g_{α}^{−1/2} (A_{η}^{∗}A_{η}) and they are also selfadjoint and positive definite operators.

Lemma 2.2. If conditions (2.7) and (2.8) are satisfied, then the following estimates hold:

∥g_{α} (A_{η}^{∗}A_{η})∥ ⩽ 1/βα, ∥g_{α} (A_{η}^{∗}A_{η}) − A_{η}^{∗}A_{η}∥ ⩽ αβ, (2.10)

max{∥A_{η}u∥^{2}; βα∥u∥^{2}} ⩽ ⟨g_{α}^{−1} (A_{η}^{∗}A_{η})u, u⟩ ⩽ ∥A_{η}u∥^{2} + βα∥u∥^{2}, (2.11)

∥A_{η}g_{α} (A_{η}^{∗}A_{η})A_{η}^{∗}∥ ⩽ 1, ∥g_{α}^{1/2} (A_{η}^{∗}A_{η})A_{η}^{∗}∥ ⩽ 1. (2.12)

Proof. The first inequality in (2.10) follows from (2.8):

∥g_{α} (A_{η}^{∗}A_{η})∥ ⩽ sup_{0⩽t⩽a} g_{α} (t) ⩽ 1/βα.

while the second one from ∥g_{α}^{−1} (A_{η}^{∗}A_{η}) − A_{η}^{∗}A_{η}∥ ⩽ sup_{0⩽t⩽a} |(g_{α} (t))^{−1} − t| ⩽ αβ. Let us prove the first inequality in (2.11). Based on (2.8), we have that βα ⩽ (g_{α} (t))^{−1} ⩽ t + βα. Therefore, we have βα∥u∥^{2} ⩽ ⟨g_{α}^{−1} (A_{η}^{∗}A_{η})u, u⟩. From 1 − tan_{α} (t) ⩾ 0 it follows that t ⩽ g_{α}^{−1} (t), wherefrom we obtain ⟨g_{α}^{−1} (A_{η}^{∗}A_{η})u, u⟩ ⩾ ⟨A_{η}^{∗}A_{η}u, u⟩ = ∥A_{η}u∥^{2}. Furthermore, the second inequality in (2.11) is a consequence of the inequality (g_{α} (t))^{−1} ⩽ t + βα.

Let us prove (2.12). Using the equality A_{η}g_{α} (A_{η}^{∗}A_{η})A_{η}^{∗} = A_{η}A_{η}^{∗}g_{α} (A_{η}A_{η}^{∗}), we have ∥A_{η}g_{α} (A_{η}^{∗}A_{η})A_{η}^{∗}∥ = ∥A_{η}A_{η}^{∗}g_{α} (A_{η}A_{η}^{∗})∥ ⩽ sup_{0⩽t⩽a} tg_{α} (t) ⩽ 1. The second inequality in (2.12) is a consequence of the previous one:

∥g_{α}^{1/2} (A_{η}^{∗}A_{η})A_{η}^{∗}u∥^{2} = ⟨g_{α}^{1/2} (A_{η}^{∗}A_{η})A_{η}^{∗}u, g_{α}^{1/2} (A_{η}^{∗}A_{η})A_{η}^{∗}u⟩
⟨g_{α} (A_{η}^{∗}A_{η})A_{η}^{∗}u, A_{η}^{∗}u⟩ = ⟨A_{η}g_{α} (A_{η}^{∗}A_{η})A_{η}^{∗}u, u⟩ ⩽ ∥u∥^{2}.

Consequently, ∥g_{α}^{1/2} (A_{η}^{∗}A_{η})A_{η}^{∗}∥ ⩽ 1. □

As an approximation of the normal solution of problem (1.1), one can take the unique solution u_{α} ∈ U of the variational inequality

(2.13)  ⟨g_{α}^{−1} (A_{η}^{∗}A_{η})u_{α} − A_{η}^{∗}f_{δ}, u − u_{α} ⟩ ⩾ 0, ∀u ∈ U,

where the functions g_{α} satisfy conditions (2.7)–(2.9).

In the cases of Tikhonov regularization and the Tikhonov iterated regularization, variational inequalities (2.13) become ⟨(A_{η}^{∗}A_{η} + αI)u_{α} − A_{η}^{∗}f_{δ}, u − u_{α} ⟩ ⩾ 0, ∀u ∈ U, and ⟨(A_{η}^{∗}A_{η} + αI)u_{n,α} − αu_{n−1,α} − A_{η}^{∗}f_{δ}, u − u_{n,α} ⟩ ⩾ 0, ∀u ∈ U, n = 1, …, m, u_{0,α} = 0, u_{α} = u_{m,α} .

Problem (2.13) has a unique solution, since the operator G_{α} (u) = g_{α}^{−1} (A_{η}^{∗}A_{η})u− Aη^{∗}f_{δ} is strongly monotone:

⟨G_{α} (u) − G_{α} (v), u − v⟩ = ⟨g_{α}^{−1} (A_{η}^{∗}A_{η})(u − v), u − v⟩ ⩾ βα∥u − v∥^{2}.

Note that u_{α}^{∗} = g_{α} (A_{η}^{∗}A_{η})A_{η}^{∗}f_{δ} in (2.2) (which is an approximate solution of problem (1.2)), is a unique solution of the equation G_{α} (u) = 0.

It is easy to see that G_{α} is a potential operator: G(u) = T′_{α}(u), where

T_{α} (u) := ∥g_{α}^{−1/2} (A_{η}^{∗}A_{η})u − g_{α}^{1/2} (A_{η}^{∗}A_{η})A_{η}^{∗}f_{δ}∥^{2}.

Consequently, variational inequality (2.13) is equivalent to the minimization problem T_{α} (u) → inf, u ∈ U.

3. Convergence of the regularization methods

In this section we prove the convergence u_{α} → u_{∗} as α → 0 and derive an estimate of the rate of convergence in the case when condition (2.4) (or (2.6) is satisfied. Firstly, we prove some auxiliary results.

Lemma 3.1. If conditions (1.4) and (2.8) are satisfied, and if \frac{η}{α} → 0 as α → 0, then

\frac{1}{βα} ∥A_{η}^{∗}A_{η}(I − Π\overline{R(A^{∗})} )u_{∗} ∥ → 0 as α → 0, (3.1)

\frac{1}{βα} ∥(g_{α}^{−1} (A_{η}^{∗}A_{η}) − βα)(I − Π\overline{R(A^{∗})} )u_{∗} ∥ → 0 as α → 0. (3.2)

\frac{1}{βα} ∥(g_{α}^{−1} (A_{η}^{∗}A_{η}) − A_{η}^{∗}A_{η})∥ ⩽ C, C = const. (3.3)

Proof. Taking into account that the equality A(I − Π\overline{R(A^{∗})} )u_{∗} = 0, (3.1) can be obtained as an immediate consequence of (1.4) and of the lemma, namely, bearing in mind the conditions of the lemma, we can obtain

\frac{1}{βα} ∥A_{η}^{∗}A_{η}(I − Π\overline{R(A^{∗})} )u_{∗} ∥ = \frac{1}{βα} ∥A_{η}^{∗}(A_{η} − A)(I − Π\overline{R(A^{∗})} )u_{∗} ∥

⩽ \frac{η}{βα} ∥A_{η}^{∗}∥∥(I − Π\overline{R(A^{∗})} )u_{∗} ∥ → 0 as α → 0.

Convergence (3.2) is also a consequence of (1.4) and of the conditions of the lemma. From βα ⩽ (g_{α} (t))^{−1} ⩽ t+βα it follows 0 ⩽ (g_{α} (t))^{−1} −βα ⩽ t. Therefore, we have

\frac{1}{βα} ∥(g_{α}^{−1} (A_{η}^{∗}A_{η}) − βαI)(I − Π\overline{R(A^{∗})} )u_{∗} ∥ ⩽ \frac{1}{βα} ∥A_{η}^{∗} ∥∥A_{η}(I − Π\overline{R(A^{∗})} )u_{∗} ∥

= \frac{1}{βα} ∥A_{η}^{∗}∥∥(A_{η} − A)(I − Π\overline{R(A^{∗})} )u_{∗} ∥ ⩽ \frac{aη}{βα} ∥(I − Π\overline{R(A^{∗})} )u_{∗} ∥ → 0 as α → 0.

Finally, (3.3) is a consequence of the inequality ∥(g_{α}^{−1} (A_{η}^{∗}A_{η}) − A_{η}^{∗}A_{η})∥ ⩽ sup_{0⩽t⩽a} {1/(g_{α} (t) − t)} ⩽ αβ. □

In the proof of the convergence u_{α} → u_{∗}, we shall also use the following lemma whose proofs can be found in [18, pp. 93 (Lemma 1.2), 99 (Lemma 2.2), 109 (Lemma 3.2) and 114 (inequality 3.60)].

Lemma 3.2. (a) If ∥A − A_{η}∥ ⩽ η, then

(∀p > 0) ∥|A_{η}|^{p} − |A|^{p}∥ ⩽ c(1 + | lnη|)η^{min {1,p}} , c = c(p) ≡ const.

(b) If, in addition, the conditions ∥A_{η}∥^{2} ⩽ a, ∥A∥^{2} ⩽ a, and (2.9) are satisfied, then (b_{1}) ∥A_{η}K_{αη} (|A|^{p} − |A_{η}|^{p})h_{∗}∥ ⩽ γη, γ = γ(α, p) as α → 0, (b_{2}) (∀x ∈ \overline{R(A^{∗})}) ∥K_{αη} x∥ → 0 as α, η → 0, (b_{3}) (∀x ∈ \overline{R(A^{∗})})(∀p_{0} > 1/2) ∥K_{αη} x∥ → 0, \frac{1}{\sqrt{α}} ∥A_{η}K_{αη} x∥ → 0 as α, η → 0, where K_{αη} = I − A_{η}^{∗}A_{η}g_{α} (A_{η}^{∗}A_{η}) and γ_{p} is the constant from (2.9) (b_{4}) ∥A_{η}K_{αη} |A|^{p}h_{∗}∥ ⩽ c_{2}α^{(p+1)/2} ∥h_{∗}∥, c_{2} = c_{2}(p), 0 ⩽ p ⩽ 2p_{0} − 1.

Now, we can prove the main results of the paper. They are related to the convergence of the methods of regularization.

Theorem 3.1. Suppose conditions (1.4) and (2.7)–(2.9) are satisfied. (a_{1}) If Au_{∗} = f and the parameter α in (2.13) is chosen such that α = α(η, δ) → 0 and \frac{η+δ}{α} → 0 as η, δ → 0, then u_{α} → u_{∗} as η, δ → 0. (a_{2}) If Au_{∗} ≠ f and α = α(η, δ) → 0 and \frac{η^{2}}{α^{3}} + \frac{δ}{α} → 0 as η, δ → 0, then u_{α} → u_{∗} as η, δ → 0.

(b) If in addition, condition (2.4) is satisfied, then (b_{1}) in the case of Au_{∗} = f, we have that

∥u_{α} − u_{∗}∥ = O((η + δ)/\sqrt{α} + α^{p/2}) + o(η^{p/(p+2)} ), (3.4)

∥A_{η}(u_{α} − u_{∗})∥ = O(η + δ + α^{\frac{p+1}{2}} ) + \sqrt{α}o(η^{\frac{p}{p+2}} ), 0 ⩽ p ⩽ 2p_{0} − 1, p_{0} > \frac{1}{2}. (3.5)

(b_{2}) in the case of Au_{∗} ≠ f we have that

∥u_{α} − u_{∗}∥ = O( \frac{η}{α} + \frac{δ}{\sqrt{α}} + α^{\frac{p}{2}} ) + o(η^{\frac{p}{p+2}} ), (3.6)

∥A_{η}(u_{α} − u_{∗})∥ = O( \frac{η}{\sqrt{α}} + δ + α^{\frac{p+1}{2}} ) + \sqrt{α} o(η^{\frac{p}{p+2}} ), 0 ⩽ p ⩽ 2p_{0} − 1, p_{0} > \frac{1}{2}. (3.7)

Proof. Let us begin with the variational inequality (2.13) that for u = u_{∗} gives ⟨g_{α}^{−1} (A_{η}^{∗}A_{η})u_{α} − A_{η}^{∗}f_{δ}, u_{∗} − u_{α} ⟩ ⩾ 0. From here, taking into account the inequality ⟨A^{∗}Au_{∗} − A^{∗}f, u_{∗} − u_{α} ⟩ ⩽ 0, and the equality

A_{η}^{∗}f_{δ} = A_{η}^{∗}A_{η}u_{∗} +(A^{∗}f −A^{∗}Au_{∗})+A_{η}^{∗}[(A−A_{η})u_{∗} +(f_{δ}−f)]+(A^{∗} −A_{η}^{∗})(Au_{∗} −f),

96 JAĆIMOVIĆ, KRNIĆ, AND OLEG OBRADOVIĆ

we obtain ⟨g_{α}^{−1} (A_{η}^{∗}A_{η})(u_{α} − u_{∗}), u_{α} − u_{∗}⟩ ⩽ ⟨g_{α}^{−1} (A_{η}^{∗}A_{η})u_{∗}, u_{∗} − u_{α} ⟩ − ⟨A_{η}^{∗}f_{δ}, u_{∗} − u_{α} ⟩ ⩽ ⟨g_{α}^{−1} (A_{η}^{∗}A_{η})u_{∗}, u_{∗} − u_{α} ⟩ − ⟨A_{η}^{∗}A_{η}u_{∗}, u_{∗} − u_{α} ⟩

+ ⟨A_{η}^{∗}[(A_{η} − A)u_{∗} + (f − f_{δ})], u_{∗} − u_{α} ⟩ + ⟨(A_{η}^{∗} − A^{∗})(Au_{∗} − f), u_{∗} − u_{α} ⟩ = ⟨(g_{α}^{−1} (A_{η}^{∗}A_{η}) − A_{η}^{∗}A_{η})u_{∗}, u_{∗} − u_{α} ⟩ + ⟨(A_{η} − A)u_{∗} + (f − f_{δ}), A_{η}(u_{∗} − u_{α})⟩

+ ⟨(A_{η}^{∗} − A^{∗})(Au_{∗} − f), u_{∗} − u_{α} ⟩. From here, using the conditions of the theorem, and the conditions on functions g_{α}, and ε-inequality ⟨a, b⟩ ⩽ \frac{ε}{2}∥a∥^{2} + \frac{1}{(2ε)} ∥b∥^{2}, we obtain

\frac{1}{2}αβ∥u_{α} − u_{∗}∥^{2} + \frac{1}{2}∥A_{η}(u_{α} − u_{∗})∥^{2} ⩽ max\{αβ∥u_{α} − u_{∗}∥^{2}; ∥A_{η}(u_{α} − u_{∗})∥^{2}\}

⩽ ⟨g_{α}^{−1} (A_{η}^{∗}A_{η})(u_{α} − u_{∗}), u_{α} − u_{∗}⟩

⩽ \frac{ε_{1}}{2}∥u_{∗} − u_{α} ∥^{2} + \frac{1}{2ε_{1}}∥g_{α} (A_{η}^{∗}A_{η}) − A_{η}^{∗}A_{η}∥^{2}∥u_{∗}∥^{2}

+ \frac{ε_{2}}{2}∥A_{η}(u_{∗} − u_{α})∥^{2} + \frac{1}{ε_{2}}∥A_{η} − A∥^{2}∥u_{∗}∥^{2} + \frac{ε_{2}}{2}∥A_{η}(u_{∗} − u_{α})∥^{2}

+ \frac{1}{ε_{2}}∥f − f_{δ}∥^{2} + \frac{ε_{3}}{2}∥u_{∗} − u_{α} ∥^{2} + \frac{1}{2ε_{3}}∥Au_{∗} − f∥^{2}∥A_{η}^{∗} − A^{∗}∥^{2}

⩽ \frac{ε_{1}}{2}∥u_{∗} − u_{α} ∥^{2} + \frac{1}{2ε_{1}}(αβ)^{2}∥u_{∗}∥^{2} + ε_{2}∥A_{η}(u_{∗} − u_{α})∥^{2}

+ \frac{1}{ε_{2}}η^{2}∥u_{∗}∥^{2} + \frac{1}{ε_{2}}δ^{2} + \frac{ε_{3}}{2}∥u_{∗} − u_{α} ∥^{2} + \frac{1}{2ε_{3}}∥Au_{∗} − f∥^{2}η^{2}.

For ε_{1} = ε_{3} = \frac{αβ}{2} , ε_{2} = \frac{1}{4}, we have

∥A_{η}(u_{α} − u_{∗})∥^{2} ⩽ 4αβ∥u_{∗}∥^{2} + 4η^{2}∥u_{∗}∥^{2} + 8δ^{2} + \frac{η^{2}}{αβ} ∥Au_{∗} − f∥^{2},

and consequently, for α ∼ η,

∥A_{η}(u_{α} − u_{∗})∥ = O( \sqrt{α} + η + δ + \frac{η}{\sqrt{α}} ) = O(\sqrt{α} + \sqrt{η} + δ).

For ε_{1} = ε_{3} = \frac{αβ}{4} , ε_{2} = \frac{1}{2}, we have

∥u_{α} − u_{∗}∥^{2} ⩽ 8∥u_{∗}∥^{2} + 2\frac{η^{2}}{αβ} ∥u_{∗}∥^{2} + 2\frac{δ^{2}}{αβ} + 8\frac{η^{2}}{α^{2}β^{2}} ∥Au_{∗} − f∥^{2}

It means that (u_{α}) remains bounded as α → 0.

Now, according to the equality u_{∗} = P u_{∗} + (I − P)u_{∗}, where P = Π − \overline{R(A^{∗})} is the (orthogonal) projection onto the subspace \overline{R(A^{∗})}, and bearing in mind that g_{α}^{1/2} (A_{η}^{∗}A_{η}) and g_{α}^{−1/2} (A_{η}^{∗}A_{η}) are self-adjoint operators, the previous inequality can be written in the form

⟨g_{α}^{−1} (A_{η}^{∗}A_{η})(u_{∗} − u_{α}), u_{∗} − u_{α} ⟩ = ∥g_{α}^{−1/2} (A_{η}^{∗}A_{η})(u_{∗} − u_{α})∥^{2}
⩽ ⟨g_{α}^{−1/2} (A_{η}^{∗}A_{η})K_{αη} P u_{∗}, g_{α}^{−1/2} (A_{η}^{∗}A_{η})(u_{∗} − u_{α})⟩
+ ⟨(g_{α}^{−1} (A_{η}^{∗}A_{η}) − A_{η}^{∗}A_{η})(I − P)u_{∗}, u_{∗} − u_{α} ⟩
+ ⟨g_{α}^{1/2} (A_{η}^{∗}A_{η})A_{η}^{∗}[(A_{η} − A)u_{∗} + (f − f_{δ})], g_{α}^{−1/2} (A_{η}^{∗}A_{η})(u_{∗} − u_{α})⟩
+ ⟨g_{α}^{1/2} (A_{η}^{∗}A_{η})(A_{η}^{∗} − A^{∗})(Au_{∗} − f), g_{α}^{−1/2} (A_{η}^{∗}A_{η})(u_{∗} − u_{α})⟩,

where K_{αη} = I − A_{η}^{∗}A_{η}g_{α} (A_{η}^{∗}A_{η}). From here, using again ε-inequality, we get

⟨g_{α}^{−1} (A_{η}^{∗}A_{η})(u_{∗} − u_{α}), u_{∗} − u_{α} ⟩ = ∥g_{α}^{−1/2} (A_{η}^{∗}A_{η})(u_{∗} − u_{α})∥^{2}

⩽ \frac{3}{2}∥g_{α}^{−1/2} (A_{η}^{∗}A_{η})K_{αη} P u_{∗}∥^{2} + ∥g_{α}^{1/2} (A_{η}^{∗}A_{η})A_{η}^{∗}[(A_{η} − A)u_{∗} + (f − f_{δ})]^{2}

+ ∥g_{α}^{1/2} (A_{η}^{∗}A_{η})(A_{η}^{∗} − A^{∗})(Au_{∗} − f)∥^{2} + \frac{1}{2}∥g_{α}^{−1/2} (A_{η}^{∗}A_{η})(u_{∗} − u_{α})∥^{2}

+ ⟨(g_{α}^{−1} (A_{η}^{∗}A_{η}) − A_{η}^{∗}A_{η})(I − P)u_{∗}, u_{∗} − u_{α} ⟩.

or

⟨g_{α}^{−1} (A_{η}^{∗}A_{η})(u_{∗} − u_{α}), u_{∗} − u_{α} ⟩

⩽ 3∥g_{α}^{−1/2} (A_{η}^{∗}A_{η})K_{αη} P u_{∗}∥^{2} + ∥g_{α}^{1/2} (A_{η}^{∗}A_{η})A_{η}^{∗}[(A_{η} − A)u_{∗} + (f − f_{δ})]^{2}

+ {∥g_{α}^{1/2} (A_{η}^{∗}A_{η})(A_{η}^{∗} − A^{∗})(Au_{∗} − f)∥}^{2}

+ 2⟨(g_{α}^{−1} (A_{η}^{∗}A_{η}) − A_{η}^{∗}A_{η})(I − P)u_{∗}, u_{∗} − u_{α} ⟩.

Furthermore, as an immediate consequence of (2.12), we have

(3.8)  ∥g_{α}^{−1/2} (A_{η}^{∗}A_{η})u∥^{2} = ⟨g_{α}^{−1} (A_{η}^{∗}A_{η})u, u⟩ ⩽ (∥A_{η}u∥ + \sqrt{αβ} ∥u∥)^{2}.

Now, according to inequalities (2.10)–(2.12), (3.8), and taking into account conditions (1.4), we obtain

max{∥A_{η}(u_{α} − u_{∗})∥^{2}; αβ∥u_{α} − u_{∗}∥^{2}} (3.9)

⩽ \{∥A_{η}K_{αη} P u_{∗}∥ + \sqrt{αβ} ∥K_{αη} P u_{∗}∥ + η∥u_{∗}∥ + δ + \frac{η}{\sqrt{αβ}} ∥Au_{∗} − f∥\}^{2}

+ 2⟨(g_{α}^{−1} (A_{η}^{∗}A_{η}) − αβI)(I − P)u_{∗}, u_{∗} − u_{α} ⟩

+ 2αβ⟨(I − P)u_{∗}, u_{∗} − u_{α} ⟩− 2⟨A_{η}^{∗}A_{η}(I − P)u_{∗}, u_{∗} − u_{α} ⟩

Because (u_{α}) is bounded, there exist a null-sequences (α_{n}), (η_{n}), and (δ_{n}), and v_{∗} ∈ U, such that u_{α_{n}} converges weakly to v_{∗} as n → +∞. In order to simplify the text, we will assume that u_{α} converges weakly to v_{∗} as α → 0. Then, from (3.9), Lemmas 3.1 and 3.2, and from the conditions of the theorem, bearing in mind the lower-semicontinuity of the norm, it follows that (3.10)

lim_{α→0} ∥A_{η}(u_{∗}−u_{α} ∥ = 0, i.e., Av_{∗} = Au_{∗} and ∥u_{∗}−v_{∗}∥^{2} ⩽ 2⟨(I−Π\overline{R(A^{∗})} )u_{∗}, u_{∗}−v_{∗}⟩.

The equality Av_{∗} = Au_{∗} shows that v_{∗} is also a solution of the problem (1.1) and that v_{∗} − u_{∗} ∈ KerA. Now, inequality (3.10) can be written as ∥u_{∗} − v_{∗}∥^{2} ⩽ 2⟨u_{∗}, u_{∗} − v_{∗}⟩.

Also, as a solution of minimization problem (1.3), u_{∗} satisfies the variational inequality ⟨u_{∗}, u_{∗} −v_{∗}⟩ ⩽ 0. Hence, v_{∗} = u_{∗} and consequently, u_{α} weakly converges to u_{∗}. Finally, if in (3.9) η → 0 and δ → 0, then we get that u_{α} strongly converges to u_{∗}.

Inequality (3.7) remains to be proven. Let us suppose that condition (2.4) is satisfied. Then (I −P)u_{∗} = (I −Π\overline{R(A^{∗})} )u_{∗} = 0 and inequality (3.9) can be written in the form

max\{ \frac{1}{\sqrt{αβ}} ∥A_{η}(u_{α} − u_{∗})∥; ∥u_{α} − u_{∗} ∥\}

⩽ \frac{1}{\sqrt{αβ}} ∥A_{η}K_{αη} |A|^{p}h_{∗}∥ + ∥K_{αη} |A|^{p}h_{∗}∥ + \frac{η∥u_{∗}∥ + δ}{\sqrt{αβ}} + \frac{η∥Au_{∗} − f∥}{αβ} .

Let us estimate \frac{1}{\sqrt{αβ}} ∥A_{η}K_{αη} u_{∗}∥ and ∥K_{αη} |A|^{p}h_{∗}∥. Applying the inequality

(b_{4}) from Lemma 3.2, we obtain the estimate:

\frac{1}{\sqrt{αβ}} ∥A_{η}K_{αη} |A|^{p}h_{∗}∥ ⩽ ( ε_{αp} \frac{η}{\sqrt{αβ}} + \frac{γ_{\frac{p+1}{2}}}{\sqrt{β}} α^{\frac{p}{2}} ) ∥h_{∗}∥, 0 < p ⩽ 2p_{0} − 1.

In a similar way, we can obtain the inequality

∥K_{αη} |A|^{p}h_{∗}∥ ⩽ [c_{p}(1 + | lnη|)η^{min \{1;p\}} + γ_{\frac{p}{2}} α^{\frac{p}{2}} ] ∥h_{∗}∥.

Then from (3.4) we have

max\{ \frac{1}{\sqrt{αβ}} ∥A_{η}(u_{α} − u_{∗})∥; ∥u_{α} − u_{∗}∥\} ⩽ ( ε_{αp} \frac{η}{\sqrt{αβ}} + \frac{γ_{\frac{p+1}{2}}}{\sqrt{β}} α^{\frac{p}{2}} ) ∥h_{∗}∥

+ [c_{p}(1 + lnη)η^{min \{1;p\}} + γ_{\frac{p}{2}} α^{\frac{p}{2}} ] ∥h_{∗}∥ + \frac{η∥u_{∗}∥ + δ}{\sqrt{αβ}} + \frac{η∥Au_{∗} − f∥}{αβ} .

Now, taking into account that \frac{(1+| ln η|)η^{min \{1;p\}}}{η^{p/(p+2)}} → 0 as η → 0, we obtain estimates (3.4)–(3.7). □

To our knowledge, there are no regularization methods based on functions satisfying conditions (2.7)–(2.9) with qualification p_{0} > 1, such that a theorem analogous to Theorem 3.1 holds when condition (2.4) is replaced by the projected source condition (2.6). At the same time, for the standard Tikhonov method of regularization

(3.11)  J_{α} (u) = \frac{1}{2}∥A_{η}u − f_{δ}∥^{2} + \frac{1}{2}α∥u∥^{2}.

generated by the functions g_{T,α} (t) = \frac{1}{t+α} , (that satisfy conditions (2.7)–(2.9) with qualification p_{0} = 1), the estimate analogous to the estimate from Theorem 3.1 is true.

Theorem 3.2. If in (3.11) the condition (2.6) is satisfied, then the following bounds are true:

(i) If Au_{∗} = f then

∥u_{∗} − u_{α} ∥ = O ( \frac{η + δ}{\sqrt{α}} + α^{min \{1/2;p/2\}} ) + o(η^{\frac{p}{p+2}} ), 0 ⩽ p ⩽ 2p_{0} − 1, p_{0} > \frac{1}{2}

∥A_{η}(u_{α} − u_{∗})∥ = O ( η + δ + α^{min \{1;(p+1)/2} ) + \sqrt{α}o(η^{\frac{p}{p+2}} ), 0 ⩽ p ⩽ 2p_{0} − 1, p_{0} > \frac{1}{2}

(ii) If Au_{∗} ≠ f and the parameter of regularization is choosen such that α ⩾ 2η^{2} , then

∥u_{∗} − u_{α} ∥ = O ( \frac{η}{α} + \frac{δ}{\sqrt{α}} + α^{min \{1/2;p/2\}} ) + o(η^{\frac{p}{p+2}} ), 0 ⩽ p ⩽ 2p_{0} − 1, p_{0} > \frac{1}{2}

∥A_{η}(u_{α} − u_{∗})∥ = O ( η + δ + α^{min \{1;(p+1)/2} ) + \sqrt{α}o(η^{\frac{p}{p+2}} ), 0 ⩽ p ⩽ 2p_{0} − 1, p_{0} > \frac{1}{2}

Proof. Here, by u_{α} we denote the unique solution of the problem (3.11). It is well known that u_{α} → u_{∗} as α → 0. Let us derive the estimate of the rate of convergence when condition (2.6) is fulfilled. We will begin with the variational inequality ⟨J′_{α}(u_{α}), u − u_{α} ⟩ ⩾ 0, ∀u ∈ U, that for u = u_{∗} gives ⟨A_{η}^{∗}A_{η}u_{α} + αu_{α} − A_{η}^{∗}f_{δ}, u_{∗} − u_{α} ⟩ ⩾ 0. The above inequality can be written in the form

(3.12) α∥u_{∗} −u_{α} ∥^{2}+∥A_{η}(u_{∗} −u_{α})∥^{2} ⩽ α⟨u_{∗}, u_{∗} −u_{α} ⟩+⟨A_{η}^{∗}A_{η}u_{∗} −A_{η}^{∗}f_{δ}, u_{∗} −u_{α} ⟩.

Taking into account that u_{∗} = Π_{U}(|A|^{p}h_{∗}) and using the ε-inequality and the equality αI = (A_{η}^{∗}A_{η} + αI)K_{αη} , where K_{αη} = I − (A_{η}^{∗}A_{η})(αI + A_{η}^{∗}A_{η})^{−1} , we have the following estimate of the first term in (3.12):

(3.13) α⟨u_{∗}, u_{∗} − u_{α} ⟩ ⩽ α⟨|A|^{p}h_{∗}, u_{∗} − u_{α} ⟩ = ⟨(αI + A_{η}^{∗}A_{η})K_{αη} |A|^{p}h_{∗}, u_{∗} − u_{α} ⟩

= ⟨A_{η}K_{αη} |A|^{p}h_{∗}, A_{η}(u_{∗} − u_{α})⟩ + α⟨K_{αη} |A|^{p}h_{∗}, u_{∗} − u_{α} ⟩

⩽ \frac{ε_{1}}{2}∥A_{η}K_{αη} |A|^{p}h_{∗}∥^{2}+ \frac{1}{2}ε_{1}∥A_{η}(u_{∗}−u_{α})∥^{2}+ \frac{αε_{2}}{2} ∥K_{αη} |A|^{p}h_{∗}∥^{2}+ \frac{α}{2ε_{2}}∥u_{∗}−u_{α} ∥^{2}.

Furthermore, using the inequality ⟨A^{∗}Au_{∗} − A^{∗}f, u_{α} − u_{∗}⟩ ⩾ 0 and the conditions of approximation of A and f by A_{η} and f_{δ}, we have the estimate of the second term in (3.12):

⟨A_{η}^{∗}A_{η}u_{∗} − A_{η}^{∗}f_{δ}, u_{∗} − u_{α} ⟩ ⩽ ⟨A_{η}^{∗}A_{η}u_{∗} − A_{η}^{∗}f_{δ}, u_{∗} − u_{α} ⟩ − ⟨A^{∗}Au_{∗} − A^{∗}f, u_{∗} − u_{α} ⟩

= ⟨A_{η}^{∗}(A_{η} −A)u_{∗}, u_{∗}−u_{α} ⟩+⟨A_{η}^{∗}(f −f_{δ}), u_{∗}−u_{α} ⟩+⟨(A_{η}^{∗} −A^{∗})(Au_{∗}−f)u_{∗}, u_{∗}−u_{α} ⟩. Now, using again ε-inequality, we have the estimates

(3.14) ⟨A_{η}^{∗}(A_{η} − A)u_{∗}, u_{∗} − u_{α} ⟩ + ⟨A_{η}^{∗}(f − f_{δ}), u_{∗} − u_{α} ⟩

= ⟨(A_{η} − A)u_{∗}, A_{η}(u_{∗} − u_{α})⟩ + ⟨f − f_{δ}, A_{η}(u_{∗} − u_{α})⟩

⩽ \frac{ε_{3}}{2}∥(A_{η} − A)u_{∗}∥^{2} + \frac{1}{2ε_{3}}∥A_{η}(u_{∗} − u_{α})∥^{2} + \frac{ε_{4}}{2}∥f − f_{δ}∥^{2} + \frac{1}{2ε_{2}}∥A_{η}(u_{∗} − u_{α})∥^{2}

⩽ \frac{ε_{1}}{2}η^{2}∥u_{∗}∥^{2} + \frac{1}{2ε_{1}}∥A_{η}(u_{∗} − u_{α})∥^{2} + \frac{ε_{2}}{2}δ^{2} + \frac{1}{2ε_{2}}∥A_{η}(u_{∗} − u_{α})∥^{2}

Let us estimate ∥A_{η}K_{αη} |A|^{p}h_{∗}∥^{2} and ∥K_{αη} |A|^{p}h_{∗}∥^{2}. Taking into account Lemma 1.3 and the fact that condition (2.9) in the Tikhonov method of regularization has the form (see [1, 18])

sup_{0⩽t⩽a^{2}} t^{p}|1 − tan_{α} (t)| = sup_{0⩽t⩽a^{2}} t^{p} \frac{α}{t + α} ⩽ γ_{1}α^{p}, p ∈ [0, 1],

while for p > 1, we have sup_{0⩽t⩽a^{2}} t^{p}|1 − tan_{α} (t)| = sup_{0⩽t⩽a^{2}} t^{p−1} t\frac{α}{t+α} ⩽ γ_{2}α. So, we have (see [18, p. 93])

(3.15) ∥A_{η}K_{αη} |A|^{p}h_{∗}∥^{2} ⩽ 2∥A_{η}K_{αη} (|A|^{p} − |A_{η}|^{p})h_{∗}∥^{2} + 2∥A_{η}K_{αη} |A_{η}|^{p}h_{∗}∥^{2}

⩽ 2ε_{αp} η^{2}∥h_{∗}∥^{2} + 2 sup_{0⩽t⩽a} [t^{\frac{p+1}{2}} ( 1 − \frac{t}{t + α} ) ]^{2}∥h_{∗}∥^{2}

⩽ 2(ε_{αp} η^{2} + c_{p}α^{min {p+1;2}} )∥h_{∗}∥^{2}, ε_{αp} → 0 as α → 0.

Furthermore, denoting I − A^{∗}A(αI + A^{∗}A)^{−1} = α(αI + A^{∗}A)^{−1} by K_{α}, we have

(3.16)  ∥K_{αη} |A|^{p}h_{∗}∥^{2} = ∥(K_{αη} − K_{α})|A|^{p}h_{∗} + K_{α} |A|^{p}h_{∗}∥^{2}
⩽ 2∥(K_{αη} − K_{α})|A|^{p}h_{∗}∥^{2} + 2∥K_{α} |A|^{p}h_{∗}∥^{2}.

For the estimate of the first term from the right-hand side of this inequality, we use the equality

∥K_{αη} − K_{α} ∥ = α∥(αI + A_{η}^{∗}A_{η})^{−1} [A_{η}^{∗}(A_{η} − A) + (A_{η}^{∗} − A^{∗})A](αI + A^{∗}A)^{−1} ∥.

and the inequality ∥B(αI + B^{∗}B)^{−1} ∥ = ∥(αI + B^{∗}B)^{−1} B^{∗}∥ ⩽ \frac{1}{2} \frac{1}{\sqrt{α}} for B = A and B = A_{η}. Taking into account the conditions of the theorem, we obtain

(3.17)  ∥(K_{αη} − K_{α})|A|^{p}h_{∗}∥^{2} ⩽ \frac{α^{2}η^{2}∥|A|^{p}h_{∗}∥^{2}}{4α^{3}} = \frac{α^{−1} η^{2}∥|A|^{p}h_{∗}∥}{4} .

Second addition in (3.16) can be estimated by (see (3.15))

(3.18)  ∥K_{α} |A|^{p}h_{∗}∥^{2} ⩽ c′_{p}α^{min {2;p}} ∥h_{∗}∥^{2}.

From (3.16), (3.17), and (3.18), it can be obtained that

(3.19)  ∥K_{αη} |A|^{p}h_{∗}∥^{2} ⩽ \frac{η^{2}}{2α} ∥|A|^{p}h_{∗}∥^{2} + c′α^{min \{2;p\}} ∥h_{∗}∥^{2}.

Therefore, using (3.13) and (3.14), estimate (3.12) can be written as

α∥u_{∗} − u_{α} ∥^{2} + ∥A_{η}(u_{∗} − u_{α})∥^{2} ⩽ \frac{ε_{1}}{2}∥A_{η}K_{αη} |A|^{p}h_{∗}∥^{2} + \frac{1}{2ε_{1}}∥A_{η}(u_{∗} − u_{α})∥^{2}

+ \frac{αε_{2}}{2} ∥K_{αη} |A|^{p}h_{∗}∥^{2} + \frac{α}{2ε_{2}}∥u_{∗} − u_{α} ∥^{2} + ⟨A_{η}^{∗}(A_{η} − A)u_{∗}, u_{∗} − u_{α} ⟩

+ ⟨A_{η}^{∗}(f − f_{δ}), u_{∗} − u_{α} ⟩ + ⟨(A_{η}^{∗} − A^{∗})(Au_{∗} − f)u_{∗}, u_{∗} − u_{α} ⟩. or, using (3.14), (3.14) and (3.19),

(3.20) α∥u_{∗} − u_{α} ∥^{2} + ∥A_{η}(u_{∗} − u_{α})∥^{2} ⩽ \frac{ε_{1}}{2}(ε_{αp} η^{2} + cα^{min \{p+1;2\}} )∥h_{∗}∥^{2}

+ \frac{1}{2ε_{1}}∥A_{η}(u_{∗} − u_{α})∥^{2} + ε_{2}c′α^{min \{3,p+1\}} ∥h_{∗}∥^{2} + \frac{ε_{2}}{2} \frac{|A|^{p}h_{∗}∥^{2}}{2} η^{2}

+ \frac{α}{2ε_{2}}∥u_{∗} − u_{α} ∥^{2} + \frac{ε_{3}}{2}∥u_{∗}∥^{2}η^{2} + \frac{1}{2ε_{3}}∥A_{η}(u_{∗} − u_{α})∥^{2}

+ \frac{ε_{4}}{2}δ^{2} + \frac{1}{2ε_{4}}∥A_{η}(u_{∗} − u_{α})∥^{2} + ⟨Au_{∗} − f, (A_{η} − A)(u_{∗} − u_{α})⟩.

We will distinguish between two possibilities:

(1) Au_{∗} = f. Then using (3.12)–(3.16) and (3.20), for ε_{2} = \frac{1}{2}, ε_{3} = ε_{4} = ε_{1} = 2, we obtain an estimate of the rate of convergence:

∥A_{η}(u_{α} − u_{∗})∥ = O(η + δ + α^{min {1;(p+1)/2}} )

while for ε_{2} = ε_{3} = ε_{4} = 1, ε_{1} = 2, we have

α∥(u_{α} − u_{∗})∥^{2} = O(η^{2} + δ^{2} + α^{min {2;p+1}} ),

i.e., ∥u_{α} − u_{∗}∥ = O( \frac{η+δ}{\sqrt{α}} + α^{min \{1/2;p/2\}} ).

(2) Au_{∗} ≠ f. Then the term ⟨Au_{∗} − f, (A_{η} − A)(u_{∗} − u_{α})⟩ can be estimated by

⟨Au_{∗} − f, (A_{η} − A)(u_{∗} − u_{α})⟩ = ⟨(A_{η}^{∗} − A^{∗})(Au_{∗} − f), u_{∗} − u_{α} ⟩

⩽ ∥Au_{∗} − f∥^{2} \frac{η^{2}}{2α} + \frac{α}{2}∥u_{∗} − u_{α} ∥^{2}.

Now, from (3.20), ε_{1} = ε_{3} = ε_{4} = 2, ε_{2} = 1, we obtain the estimates of the rate of convergence of the method

∥A_{η}(u_{α} − u_{∗})∥ = O( \frac{η}{\sqrt{α}} + δ + α^{min \{1,(p+1)/2\}} ) ,

and (for ε_{2} = ε_{1} = 1, ε_{3} = ε_{4} = 2)

∥u_{α} − u_{∗}∥ = O( \frac{η}{α} + \frac{δ}{\sqrt{α}} + α^{min \{1/2,p/2\}} ) .

□

Acknowledgment. The authors express their gratitude to the referee for his/her suggestions and comments.

References

1. A. B. Bakushinsky, M. Yu. Kokurin, Iterative methods for approximate solutions of inverse problems, Math. Appl. 577, Springer, Dordrecht, 2004.

2. R. I. Bot, B. Hofmann, An extension of the variational inequality approach for nonlinear ill-posed problems, arXiv:0909.5093v1[math.NA], 2010.

3. A. Böttcher, B. Hofmann, U. Tautenhahn, M. Yamamoto, Convergence rates for Tikhonov regularization from different kinds of smoothness conditions, Appl. Anal. 85 (2006), 555–578.

4. M. Burger, S. Osher, Convergence rate of convex variational regularization, Inverse Probl. 20 (2004), 1411–1421.

5. W. H. Engl, M. Hanke, A. Neubauer, Regularization of Inverse Problems, Kluwer, Dordrecht, 1996.

6. K. Frick, O. Scherzer, Regularization of ill-posed linear equations by the non-stationary augmented Lagrangian method J. Integral Equations Appl. 22(2) (2010), 217–257.

7. B. Hofmann, Approximate source conditions in Tikhonov–Phillips regularization and consequences for inverse problems with multiplication operators, Math. Methods Appl. Sci. 29 (2006), 351–371.

8. B. Hofmann, M. Yamamoto, Convergence rates for Tikhonov regularization based on range inclusions, Inverse Probl. 21 (2005), 805–820.

9. M. Jaćimović, I. Krnić, On some classes of regularization methods for minimization problem of quadratic functional on a halfspaces, Hokkaido Math. J. 28 (1999), 57–69.

10. M. Jaćimović, I. Krnić, O. Obradović, Wellposedness and regularization of minimizing sequences in quadratic programming problems in Hilbert space, Proc. Conf. Nonlinear Anal. Optim. Problems, Montenegrin Academy of Sciences and Arts, (2009), 185–206.

11. I. Krnić, M. M. Potapov, Projective sourcewise representability of normal solutions to linear equations on convex sets, Comput. Math. Math. Phys. 41(9) (2001), 1251–1258.

12. I. Krnić, O. Obradović, M. M. Potapov, On the accuracy of regularized solutions to quadratic minimization problems on a halfspace, in case of a normally solvable operator, Yugoslav. J. Oper. Res. 14(1) (2001) 19–26.

13. P. Mathé, B. Hofmann, How general are general source conditions, Inverse Probl. 24 (2008), 015009 (5 pp).

14. P. Mathé, S. V. Pereverzev, Geometry of linear ill-posed problems in variable Hlbert scales, Inverse Probl. 19 (2003), 789–803.

15. M. T. Nair, Linear Operator Equartions. Approximation and Regularization, World Scientific, 2009

16. M. T. Nair, S. V. Pereverzev, U. Tautenhahn, Regualariation in Hilbert space under smooting conditions, Inverse Probl. 21 (2005), 1851–1869.

17. A. Neubauer, Tikhonov-regularization of ill-posed linear operator equations on closed convex sets, J. Approx. Theory 53 (1988), 304–320.

18. G. M. Vainikko, A. Yu. Veretennikov, Iterative Procedures in Ill-Posed Problems, Nauka, Moscow, 1986 (in Russian).

19. F. P. Vasil’ev, Methods of Optimization, Factorial, Moscow, 2003 (in Russian).

Department of Mathematics (Received 27 04 2014) University of Montenegro (Revised 23 09 2014) Podgorica Montenegro

 milojica@jacimovic.me
 dino@rc.pmf.ac.me
 oleg@t-com.me