UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Circumcenter operators in Hilbert spaces Hui, Ouyang 2018

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2018_september_ouyang_hui.pdf [ 2.86MB ]
Metadata
JSON: 24-1.0371095.json
JSON-LD: 24-1.0371095-ld.json
RDF/XML (Pretty): 24-1.0371095-rdf.xml
RDF/JSON: 24-1.0371095-rdf.json
Turtle: 24-1.0371095-turtle.txt
N-Triples: 24-1.0371095-rdf-ntriples.txt
Original Record: 24-1.0371095-source.json
Full Text
24-1.0371095-fulltext.txt
Citation
24-1.0371095.ris

Full Text

Circumcenter Operators in Hilbert SpacesbyHui OuyangB.Sc., Huaihua College, 2012M.Sc., Yunnan University, 2016A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF SCIENCEinTHE COLLEGE OF GRADUATE STUDIES(Mathematics)THE UNIVERSITY OF BRITISH COLUMBIA(Okanagan)August 2018c© Hui Ouyang, 2018The following individuals certify that they have read, and recommend to theCollege of Graduate Studies for acceptance, a thesis/dissertation entitled:Circumcenter Operators in Hilbert Spacessubmitted byHui Ouyangin partial fulfilment of the requirements of the degree ofMaster of Science.Dr. Shawn Wang, Irving K. Barber School of Arts and SciencesSupervisorDr. Heinz Bauschke, Irving K. Barber School of Arts and SciencesCo-supervisorDr. Yves Lucet, Irving K. Barber School of Arts and SciencesSupervisory Committee MemberDr. Md. Jahangir Hossain, School of EngineeringUniversity ExamineriiAbstractThe best approximation problem is of central importance in convexoptimization. It is popular to use the Douglas–Rachford splitting method orthe method of alternating projections to solve this problem.In this thesis, we use a classical concept, circumcenter, in Euclideangeometry to solve the best approximation problem. First, we introduce thenew notion, circumcenter operator. Symmetrical and asymmetrical formu-lae of the circumcenter operator are provided. A sufficient condition of theexistence of the circumcenter is provided. A characterization of the exis-tence of the circumcenter of three distinct points is presented. In addition,we define the new concept: circumcenter mapping induced by operators.When we choose the operators from sets of compositions of reflectors, weobtain the circumcenter mapping induced by reflectors, which is proper,i.e., the value of the circumcenter mapping induced by reflectors is alwaysa unique point in the space. In light of this consequence, we are able todeduce the circumcenter method induced by reflectors. We also considerthe circumcenter operator induced by projectors. Both proper and improperexamples are provided. Moreover, we prove that for some special sets, thecircumcenter methods induced by reflectors converge at least as fast as theMAP, symmetrical MAP or some of their accelerated version to solve thebest approximation problem. We also find some drawbacks of the circum-center mapping induced by reflectors. Finally, numerical experiments areimplemented to compare convergence rates of seven solvers: four circum-center methods induced by reflectors, DRM, MAP and symmetrical MAP.As the plots of performance profiles illustrate, the experimental results areconsistent with our theoretical results in the thesis. Additional comparisonswith the DRM and the circumcenter method induced by reflectors are made.iiiLay SummaryGiven a fixed point and a fixed set, it is always important to find a pointin that fixed set, which is closest to that fixed point. In this thesis, we takeadvantage of a classical concept in Euclidean geometry and introduce a newoperator to solve the problem above. Using facts in convex optimizationand algebra, we prove that our algorithms induced by the new operatorperform better than some of the well-known algorithms to solve the aboveproblem. We implement numerical experiments and use a scientific tool tocompare those experimental data. Our conclusions from the experimentsare consistent with the theoretical results.ivPrefaceThe research work presented in Chapter 5 is based on the paper [11],which will appear in the journal Linear and Nonlinear Analysis (LNA) Vol.4,No.2, 2018.For the co-authored paper, each author contributed equally.More research works related to this thesis will be submitted later byHeinz Bauschke, Hui Ouyang and Xianfu Wang. For the aforementionedresearch works, each author provided equally contribution to the identifica-tion and design of the research program, performance of the research andanalysis of the research data.vTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiChapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 The goal of the thesis . . . . . . . . . . . . . . . . . . . . . . . 11.2 Contributions in this thesis . . . . . . . . . . . . . . . . . . . . 11.3 Setting and notations . . . . . . . . . . . . . . . . . . . . . . . 3Chapter 2: The tool box . . . . . . . . . . . . . . . . . . . . . . . . . 52.1 Hilbert space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Linear independence . . . . . . . . . . . . . . . . . . . . . . . 62.3 Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.3.1 Linear and bounded operators . . . . . . . . . . . . . 82.3.2 Projector . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3.3 Projections onto the Cartesian product of finitely sets 132.3.4 Nonexpansive and firmly nonexpansive operators . . 132.4 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.4.1 Convergence of sequences . . . . . . . . . . . . . . . . 152.4.2 Fejér monotone sequence . . . . . . . . . . . . . . . . 182.4.3 Asymptotically regular . . . . . . . . . . . . . . . . . 19viTABLE OF CONTENTS2.5 Algebra time . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.5.1 Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.5.2 Determinants . . . . . . . . . . . . . . . . . . . . . . . 202.5.3 Gram matrix . . . . . . . . . . . . . . . . . . . . . . . . 222.5.4 Cross product . . . . . . . . . . . . . . . . . . . . . . . 23Chapter 3: Reflection and projection methods . . . . . . . . . . . . 253.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.2 The method of alternating projections (MAP) . . . . . . . . . 253.3 The Douglas–Rachford method (DRM) . . . . . . . . . . . . . 283.4 The accelerated mapping of MAP . . . . . . . . . . . . . . . . 303.5 Circumcentered-reflection method . . . . . . . . . . . . . . . 32Chapter 4: Further auxiliary results . . . . . . . . . . . . . . . . . . 354.1 Span and affine subspaces . . . . . . . . . . . . . . . . . . . . 354.2 Projection and reflection . . . . . . . . . . . . . . . . . . . . . 404.3 Determinant calculation . . . . . . . . . . . . . . . . . . . . . 444.4 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Chapter 5: Circumcenter . . . . . . . . . . . . . . . . . . . . . . . . 505.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505.2 The circumcenter operator . . . . . . . . . . . . . . . . . . . . 505.3 Basic properties of the circumcenter operator . . . . . . . . . 525.4 Explicit formulae for the circumcenter operator . . . . . . . . 535.5 Additional formulae for the circumcenter operator . . . . . . 575.6 Circumcenters of sequences of sets . . . . . . . . . . . . . . . 605.7 The circumcenter of three points . . . . . . . . . . . . . . . . 645.8 The circumcenter in R3 and the cross product . . . . . . . . . 69Chapter 6: Circumcenter method . . . . . . . . . . . . . . . . . . . 716.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716.2 Circumcenter mapping induced by a set of operators . . . . 726.3 Properties of circumcenter operator . . . . . . . . . . . . . . . 776.4 Continuity of circumcenter mapping . . . . . . . . . . . . . . 79Chapter 7: Circumcenter method induced by reflectors . . . . . . 877.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 877.2 Circumcenter method induced by reflectors . . . . . . . . . . 887.3 Explicit formulae of the circumcenter operator . . . . . . . . 917.4 Properties of the circumcenter operator . . . . . . . . . . . . 94viiTABLE OF CONTENTSChapter 8: Applications of circumcenter method induced by re-flectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 998.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 998.2 Properties of the circumcenter operator iteration sequence . 1018.3 Linear convergence of circumcenter methods induced by re-flectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1128.4 Applications to two special classes of sets . . . . . . . . . . . 1148.5 Application to linear subspaces . . . . . . . . . . . . . . . . . 1188.6 Accelerating the Douglas–Rachford method . . . . . . . . . . 1238.7 Circumcenter method in feasibility problems . . . . . . . . . 128Chapter 9: Circumcenter method with projectors . . . . . . . . . . 1309.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1309.2 Circumcenter operator induced by projectors . . . . . . . . . 1319.3 Accelerating the MAP and symmetric MAP . . . . . . . . . . 1369.4 Accelerating the accelerated mapping of MAP . . . . . . . . 142Chapter 10: Drawbacks of circumcenter methods induced by re-flectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14610.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14610.2 Failure to be linear, nonexpansive and self-adjoint . . . . . . 14710.3 Failure to be proper in the inconsistence case . . . . . . . . . 14810.4 Failure to be proper in non-affine examples . . . . . . . . . . 15010.5 Failure to generalize to maximally monotone operators . . . 155Chapter 11: Numerical experiments . . . . . . . . . . . . . . . . . . 15811.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15811.2 Preliminary comparison of convergence of algorithms . . . . 16011.2.1 One instance . . . . . . . . . . . . . . . . . . . . . . . . 16011.2.2 100 instances . . . . . . . . . . . . . . . . . . . . . . . 16211.2.3 Comparison of DRM and circumcenter method in-duced by the first set . . . . . . . . . . . . . . . . . . . 16411.3 Performance profile . . . . . . . . . . . . . . . . . . . . . . . . 16711.4 Performance evaluation . . . . . . . . . . . . . . . . . . . . . 16911.4.1 Comparison of all of the seven algorithms . . . . . . 16911.4.2 Comparisons of circumcenter methods with DRM orMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17211.4.3 Evaluation of circumcenter methods . . . . . . . . . . 177Chapter 12: Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 182viiiTABLE OF CONTENTS12.1 Main results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18212.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188Appendix A: Matlab function files . . . . . . . . . . . . . . . . . . 189Appendix B: Calculation used in Section 10.2 . . . . . . . . . . . . 191Appendix C: Main codes . . . . . . . . . . . . . . . . . . . . . . . . 194Appendix C.1: Codes to generate the Figure 11.3 . . . . . . . 194Appendix C.2: Codes to generate the performance matrices . 200ixList of FiguresFigure 5.1 Continuity of circumcenter operator may fail evenwhen (∀k ≥ 1) CC(S(k)) ∈ H. . . . . . . . . . . . . . 64Figure 5.2 Circumcenter of the four affinely dependent pointsfrom Example 5.7.2. . . . . . . . . . . . . . . . . . . . 66Figure 7.1 ∩mi=1Ui & Fix CCS . . . . . . . . . . . . . . . . . . . . 95Figure 8.1 S = {Id, RU1 , RU2 RU1} in R2 . . . . . . . . . . . . . . 117Figure 8.2 S = {Id, RU1 , RU2} in R2 . . . . . . . . . . . . . . . . 117Figure 9.1 Circumcenters in Example 9.2.4 . . . . . . . . . . . . 133Figure 9.2 CCŜ may be improper . . . . . . . . . . . . . . . . . . 135Figure 9.3 Collinear example of CCŜx = ∅ . . . . . . . . . . . . 136Figure 10.1 Point and line . . . . . . . . . . . . . . . . . . . . . . . 149Figure 10.2 Two parallel affine subspaces . . . . . . . . . . . . . . 150Figure 10.3 Cone and subspace . . . . . . . . . . . . . . . . . . . 151Figure 10.4 Ball and line . . . . . . . . . . . . . . . . . . . . . . . 152Figure 10.5 Line and ball . . . . . . . . . . . . . . . . . . . . . . . 153Figure 10.6 Two balls . . . . . . . . . . . . . . . . . . . . . . . . . 154Figure 10.7 Ball and line . . . . . . . . . . . . . . . . . . . . . . . 154Figure 10.8 Two balls . . . . . . . . . . . . . . . . . . . . . . . . . 155Figure 11.1 CCS3 is the winner . . . . . . . . . . . . . . . . . . . . 161Figure 11.2 CCS4 is the winner . . . . . . . . . . . . . . . . . . . . 161Figure 11.3 Comparison over 100 instances for the seven algorithms163Figure 11.4 Comparison over 100 instances without DRM and CCS1 164Figure 11.5 Comparison DRM and CCS1 with x0 ∈ R10 . . . . . . 165Figure 11.6 Comparison DRM and CCS1 with x0 = PU1 x . . . . . 166Figure 11.7 Comparison DRM and CCS1 with x0 = PU2 x . . . . . 166Figure 11.8 tp,s = the smallest k s.t. ‖a(k)p,s − x‖ ≤ e . . . . . . . . . 170xLIST OF FIGURESFigure 11.9 tp,s = the smallest k such that a(k)p,s passing the test (11.4)170Figure 11.10 tp,s = smallest k s.t. ‖a(k)p,s − x‖ ≤ e . . . . . . . . . . . 171Figure 11.11 tp,s = the smallest k s.t. a(k)p,s passing the test (11.4) . . 172Figure 11.12 tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e . . . . . . . . . . 173Figure 11.13 tp,s = smallest k s.t. a(k)p,s passing the convergence test(11.4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173Figure 11.14 tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e . . . . . . . . . . 174Figure 11.15 tp,s = the smallest k such that a(k)p,s passing the test (11.4)175Figure 11.16 tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e . . . . . . . . . . 176Figure 11.17 tp,s = the smallest k such that a(k)p,s passing the test (11.4)176Figure 11.18 tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e . . . . . . . . . . 177Figure 11.19 tp,s = the smallest k such that a(k)p,s passing the test (11.4)178Figure 11.20 tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e . . . . . . . . . . 179Figure 11.21 tp,s = the smallest k such that a(k)p,s passing the test (11.4)179Figure 11.22 tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e . . . . . . . . . . 180Figure 11.23 tp,s = the smallest k such that a(k)p,s passing the test (11.4)181xiAcknowledgementsMy deepest gratitude goes first and foremost to my supervisors, Profes-sor Heinz Bauschke and Professor Shawn Wang for their brilliant mentorshipand unconditional guidance. They walked me through all the stages of thewriting of this thesis. In the last year, in almost every week, they met withme and gave me countless instructions, advice and comments. Without theirconsistent and illuminating instructions, this thesis could not have reachedits present form.I would also like to thank Dr. Yves Lucet for agreeing to serve on mythesis committee.I would like to express my special gratitude to my teachers in UBC(Okanagan campus): Dr. John Braun, Dr. Qiduan Yang, Dr. Blair Spearman,Dr. Wayne Broughton and Dr. Warren Hare, from whose devoted teachingand enlightening lectures I have benefited a lot.I recognize the support of Unit 5 members and my colleagues in theCOCANA lab at UBC (Okanangan campus), who had helped me duringmy study. In particular, I would like to express my heartfelt gratitude toChayne Planiden, who helped me a lot in my life, my work and my English.I thank UBC (Okanangan campus) for providing me with financial sup-port over the years of my Master’s studies.Last but not least, my thanks would go to my beloved family: my dearparents, my beautiful sister, my respectable brother-in-law and my cuteniece and nephew.xiiDedicationTo my parents and my sister.xiiiChapter 1IntroductionLet m ∈Nr {0} and let U1, U2, . . . , Um be closed convex subsets of thereal Hilbert spaceH. Let x ∈ H. The best approximation problem is to:find x ∈ ∩mi=1Ui such that ‖x− x‖ = minw∈∩mi=1Ui‖w− x‖.The feasibility problem is to:find a point x ∈ ∩mi=1Ui.As we shall mention below, the goal of this thesis is to introduce oper-ators motivated by circumcenters to solve the above best approximationproblem and feasibility problem.1.1 The goal of the thesisBased on a classical Euclidean geometrical concept, circumcenter, inthis thesis, we introduce two new notions: the circumcenter operator andthe circumcenter mapping induced by a set of operators. In particular, wewill prove that when the operators to induce the circumcenter mapping arechosen from sets of compositions of reflectors, the circumcenter mappingis proper, i.e., for every point in the space, the value of the circumcentermapping over that point is a particular circumcenter of some certain points.We also define the circumcenter method induced by reflectors.The goal of this thesis is to use the the circumcenter methods inducedby reflectors to solve the best approximation problem and the feasibilityproblem in real Hilbert space and to explore the related linear convergences.1.2 Contributions in this thesisThis thesis is motivated by the reference [12], which works on an acceler-ated version of the Douglas–Rachford method.11.2. Contributions in this thesisExcept for facts with explicit references, all of the results from Chapter 4to Chapter 11 are new, although some results are mild generalizations of theknown results in [12] or [13].Chapter 2 and Chapter 3 collect some standard materials and basic factsfrom convex analysis and algebra, which are useful in our later proofs. Wedesignate all of the known results as facts with explicit references.Beginning with Chapter 4, all conclusions are new. In Chapter 4, wepresent several auxiliary results to ease the proofs in the later chapters.Our main results start in Chapter 5. We systematically study the circum-centers of a set with finitely many elements. We introduce the circumcenteroperator, which takes the circumcenter of a set with finitely many elementsas its value. We provide several symmetrical and asymmetrical formulae forthe circumcenter operator. We explore the convergence of the circumcentersof sequences of sets. Moreover, we give a characterization of the existenceof the circumcenter of three distinct points.In Chapter 6, when we specifically set the finitely many elements as thevalues of finitely many operators over a point x, we introduce the concept:the circumcenter mapping induced by a set of operators. We define thenotion of proper, which means that the circumcenter exists for every pointin the space. We explore the continuity of the mapping as well.In Chapter 7, we specifically choose the above operators from sets ofcompositions of reflectors, then we obtain the proper circumcenter mappinginduced by reflectors.In Chapter 8, we study the circumcenter method, which is the iterationsequence of the circumcenter mapping induced by reflectors. Many beautifulproperties of the sequence of iterations and some convergence results aboutthe circumcenter method are shown. Linear convergence to solve the bestapproximation problem is obtained for the circumcenter operator inducedby some special set. In addition, the circumcenter mapping induced by acertain set of operators is used to solve the feasibility problem with finitelymany linear subspaces.We consider, in Chapter 9, the circumcenter operator induced by setsof compositions of projectors. Both proper and improper examples arepresented. Notably, we prove that the circumcenter operator induced bysome special sets converges to the solution of best approximation problemat least as fast as the MAP, symmetric MAP or some of their acceleratedversions.Although the circumcenter mapping induced by reflectors performs verywell to solve the best approximation problem and feasibility problem, inChapter 10 we provide some drawbacks of the circumcenter operator.21.3. Setting and notationsNumerical experiments are implemented in Chapter 11. Performanceprofiles are used to compare the convergence rates of seven algorithms:four circumcenter methods, the Douglas–Rachford method, the method ofalternating projections and the symmetric method of alternating projectionsto solve the best approximation problem. Our experimental results areconsistent with the theoretical results proved in this thesis. In addition,some new and interesting results are discovered.1.3 Setting and notationsThroughout this thesis,H is a real Hilbert spacewith inner product 〈·, ·〉 and induced norm ‖·‖. We denote byP(H) the set of all nonempty subsets ofH with finitely many elements.In this thesis, we use the following conventions: N = {0, 1, 2, . . .}; Thezero subspace, {0}, has the basis ∅, that is{0} = span∅.We denote R+ = {λ ∈ R | λ ≥ 0} and R++ = {λ ∈ R | λ > 0}.Let Rn×n be the set of all real (n, n)-matrices and let Cn×n be the set of allcomplex (n, n)-matrices. For every M ∈ Rn×n, and for every j ∈ {1, 2, . . . ,n},we denote the jth column of the matrix M as M∗,j.Let x ∈ H and a real number δ > 0. We denote the open ball asB(x; δ) = {x ∈ H | d(x, x) < δ}. Let C and D be subsets ofH, and let z ∈ H.Then C + D = {x + y | x ∈ C, y ∈ D}, C − D = {x − y | x ∈ C, y ∈ D},z+ C = {z}+ C, C− z = C− {z}, and, for each λ ∈ R, λC = {λx | x ∈ C}.Let C be a subset ofH. The intersection of all the linear subspaces ofHcontaining C, i.e., the smallest linear subspace ofH containing C, is calledthe span of C, and is denoted by span C; its closure is the smallest closedlinear subspace ofH containing C and it is denoted by span C. Likewise, theconvex hull of C is the intersection of all the convex subsets ofH containingC, i.e., the smallest convex subset ofH containing C. It is denoted by conv C.Id stands for the identity operator. The domain of the operator T andthe range of the operator T are denoted as D(T) andR(T) respectively. Let31.3. Setting and notationsX, Y be any two real Hilbert spaces with norms ‖·‖X , and ‖·‖Y respectively.Denote B(X, Y) as the set of bounded linear operators from X to Y. DenoteB(X) as the set of all bounded linear operators from X into X. A fixed pointof a mapping T : X → X of an set X into itself is a point x ∈ X that ismapped onto itself (is “kept fixed” by T), that is, Tx = x, the image Txcoincides with x. The set of fixed points of the operator T is denoted byFix T, i.e.,Fix T = {x ∈ X | Tx = x}.The notation employed in this thesis is standard and largely follows [6].4Chapter 2The tool boxIn this chapter, we collect some background materials on convex analysisand algebra. For all known results, we set them as facts and show explicitreferences.The Gram matrix introduced in Section 2.5.3 plays a critical role in theformulae of our main actor, the circumcenter, in this thesis.2.1 Hilbert spaceDefinition 2.1.1. [22, Definition 1.4-3] A sequence (xk)k∈N in a metric spaceX = (X, d) is said to be Cauchy (or fundamental) if for every e > 0 there isan N = N(e) such that(∀m, k > N) d(xm, xk) < e.The space X is said to be complete if every Cauchy sequence in X converges(that is, has a limit that is an element of X).Definition 2.1.2. [22, Definition 3.1-1] An inner product space is a vector spaceX with an inner product defined on X. A Hilbert space is a complete innerproduct space (complete in the metric defined by the inner product; cf. (2.2)below). Here, an inner product on X is a mapping of X× X into the scalarfield K of X; that is, with every pair of vectors x and y there is associated ascalar that is written〈x, y〉and is called the inner product of x and y, such that for all vectors x, y, z52.2. Linear independenceand scalars α we have(IP1) 〈x + y, z〉 = 〈x, z〉+ 〈y, z〉(IP2) 〈αx, y〉 = α〈x, y〉(IP3) 〈x, y〉 = 〈y, x〉(IP4) 〈x, x〉 ≥ 0(IP5) 〈x, x〉 = 0⇐⇒ x = 0.An inner product on X defines a norm on X given by‖x‖ =√〈x, x〉 (2.1)and a metric on X given byd(x, y) = ‖x− y‖ =√〈x− y, x− y〉. (2.2)Hence, inner product spaces are normed spaces, and Hilbert spaces areBanach spaces.Definition 2.1.3. [16, Definition 4.2] Let S be any nonempty subset of theinner product spaceH. The dual cone (or negative polar) of S is the setS◦ := {x ∈ H | 〈x, y〉 ≤ 0 ∀y ∈ S}.The orthogonal complement of S is the setS⊥ := S◦ ∩ (−S◦) = {x ∈ H | 〈x, y〉 = 0 ∀y ∈ S}.Geometrically, the dual cone S◦ is the set of all vectors in H that make anangle of at least 90 degrees with every vector in S.2.2 Linear independenceWe shall now introduce a concept that will be used over and over againin the next chapters.Definition 2.2.1. [22, Definition 2.1-6] Given a list L of vectors x1, . . . , xr(r ≥ 1) in a vector space X, linear independence and dependence are definedby means of the equationα1x1 + α2x2 + · · ·+ αrxr = 0, (2.3)62.2. Linear independencewhere α1, . . . , αr are scalars. Clearly, (2.3) holds for α1 = α2 = · · · = αr = 0.If this is the only r-tuple of scalars for which (2.3) holds, L are said to belinearly independent. L are said to be linearly dependent ifL are not linearlyindependent, that is, if (2.3) also holds for some r-tuple of scalars, not allzero.Remark 2.2.2. In [22] and many other related literatures, when they talkabout linear independence and dependence of vectors, they represent “alist of vectors” as “a set of vectors” and then they say the set is linearindependent or dependent. In fact, there is a fatal flaw in this expression.Let e1 = (1, 0, 0)ᵀ, e2 = (0, 1, 0)ᵀ, e3 = (0, 0, 1)ᵀ. As we all know,{e1, e1, e2, e3} = {e1, e2, e3}.But clearly, the 4-tuple of vectors e1, e1, e2, e3 are linearly dependent, whilethe 3-tuple of vectors e1, e2, e3 are linearly independent. To avoid problemsfrom the redundance of a set, in this thesis, we only talk about the linearindependence and dependence of a list of vectors.Remark 2.2.3. By Definition 2.2.1, we know the order of the vectors x1, . . . ,xr won’t affect the linear independence or dependence, that is, the order ofthe vectors does not matter when we talk about the linear independence anddependence of the vectors. Moreover, since x1, . . . , xr can be an arbitrary listof vectors, one can rename the vectors arbitrarily as one likes.Definition 2.2.4. [22, Definition 2.1-7] A vector space X is said to be finite-dimensional if there is a positive integer n such that X contains a list of nvectors that are linearly independent, whereas any n + 1 or more vectors ofX are linearly dependent. n is called the dimension of X, written n = dim X.By definition, X = {0} is finite-dimensional and dim X = 0. If X is notfinite-dimensional, it is said to be infinite-dimensional.Definition 2.2.5. [20, Definitions in page 2-3] A list of vectors B in a vectorspace X is a basis for X if(i) B are linearly independent, and(ii) spanB = H.Definition 2.2.6. [6, page 1] S is an affine subspace ifS 6= ∅ and (∀λ ∈ R) S = λS + (1− λ)S.72.3. OperatorsRemark 2.2.7. By a simple inductive argument, we can prove that if S is anaffine subspace, then for every s1, . . . sm ∈ S and for every λ1 · · · λm ∈ Rwith ∑mi=1 λi = 1, we havem∑i=1λisi ∈ S.Fact 2.2.8. [27, page 4] Let S ⊆ H be an affine subspace and let a ∈ H. Thenthe translate of S by a, which is defined to be the setS + a = {x + a | x ∈ S},is another affine subspace.Definition 2.2.9. [27, page 4] An affine subspace S is said to be parallel toan affine subspace M if S = M + a for some a.Fact 2.2.10. [27, Theorem 1.2] Every affine subspace S is parallel to a uniquelinear subspace L. This L is given by(∀y ∈ S) L = S− y = S− S.Definition 2.2.11. [27, page 4] The dimension of an affine subspace is de-fined as the dimension of the linear subspace parallel to it.Definition 2.2.12. [27, page 6] Let x, x1, . . . , xm−1, xm ∈ H. The m+ 1 vectorsx, x1, . . . , xm−1, xm are said to be affinely independent if aff{x, x1, . . . , xm−1,xm} is m-dimensional.2.3 Operators2.3.1 Linear and bounded operatorsDefinition 2.3.1. [22, Definition 2.6-1] Let X and Y be normed spaces. Theoperator T : D(T)→ Y is said to be linear if(i) the domain D(T) of T is a vector space and the range R(T) lies in avector space over the same field;(ii) for all x, y ∈ D(T) and α ∈ R,T(x + y) = T(x) + T(y)T(αx) = αTx.82.3. OperatorsDefinition 2.3.2. [6, page 3] Let X, Y be real vector spaces. Let T : X → Y.T is positively homogeneous if(∀x ∈ X) (∀λ ∈ R++) T(λx) = λTx,and T is affine if(∀x, y ∈ X) (∀λ ∈ R) T(λx + (1− λ)y) = λTx + (1− λ)Ty.Definition 2.3.3. [22, Definition 2.7-1] Let X and Y be normed spaces andT : D(T)→ Y a linear operator, where D(T) ⊆ X. The operator T is said tobe bounded if there is a real number M such that for all x ∈ D(T),‖Tx‖ ≤ M‖x‖. (2.4)In (2.4) the norm on the left is that on Y, and the norm on the right isthat on X. For simplicity, in the whole thesis, we denote both norms by thesame symbol ‖·‖, without danger of confusion.Fact 2.3.4. [22, Lemma 2.7-2] Recall thatB(X, Y) consists of all of the boundedlinear operators from X to Y. Then B(X, Y) with the norm‖T‖ = supx∈D(T)x 6=0‖Tx‖‖x‖ = supx∈D(T)‖x‖=1‖Tx‖ = supx∈D(T)‖x‖≤1‖Tx‖,is a normed space. Here we use the convention that when D(T) = {0},‖T‖ = 0.2.3.2 ProjectorDefinition 2.3.5. [16, Definition 2.1] Let K be a nonempty subset of theinner product space X and let x ∈ X. An element y0 ∈ K is called a bestapproximation, or nearest point, to x from K if‖x− y0‖ = d(x, K),where d(x, K) := infy∈K‖x− y‖. The number d(x, K) is called the distancefrom x to K.The (possibly empty) set of all best approximations from x to K is de-noted by PK(x). ThusPK(x) := {y ∈ K | ‖x− y‖ = d(x, K)}.92.3. OperatorsThis defines a mapping PK from X into the subsets of K called the metricprojection onto K (Other names for metric projection include projector, nearestpoint mapping and best approximation operator).If each x ∈ X has at least (respectively exactly) one best approximationin K, then K is called a proximinal (respectively Chebyshev) set. Thus, K isproximinal (respectively Chebyshev) if and only if PK(x) 6= ∅ (respectivelyPK(x) is a singleton) for each x ∈ X.Fact 2.3.6. [16, Theorem 3.1] Let K be a proximinal subset of the innerproduct space X. Then K is closed.Fact 2.3.7. [16, Theorem 3.5] Every nonempty, closed, convex subset of aHilbert space is a Chebyshev set.Fact 2.3.8. [16, Theorem 5.5] Let K be a convex Chebyshev set inH. Then:(i) PK is idempotent:(∀x ∈ H) PK(PKx) = PK(x).Briefly, P2K = PK.(ii) PK is firmly nonexpansive: for every x, y ∈ H,‖PK(x)− PK(y)‖2 + ‖(Id−PK)(x)− (Id−PK)(y)‖2 ≤ ‖x− y‖2.(iii) PK is monotone:(∀x, y ∈ H) 〈x− y, PK(x)− PK(y)〉 ≥ 0.(iv) PK is nonexpansive:(∀x, y ∈ H) ‖PK(x)− PK(y)‖ ≤ ‖x− y‖.(v) PK is uniformly continuous.Definition 2.3.9. [2, Definition 3.2] Let S be a nonempty, closed and convexset inH. The reflector associated with S is,RS = 2PS − Id .Fact 2.3.10. [16, Theorem 5.8] Let M be a Chebyshev linear subspace of theinner product space X (e.g., a closed linear subspace of a Hilbert space).Then:102.3. Operators(i) M⊥ is a Chebyshev linear subspace.(ii) x = PMx + PM⊥x for each x ∈ H. Briefly, Id = PM + PM⊥ . Moreover,this representation is unique in the sense that if x = y + z, wherey ∈ M and z ∈ M⊥, then y = PMx and z = PM⊥x.(iii) ‖x‖2 = ‖PM(x)‖2 + ‖PM⊥(x)‖2 for all x. Hence, ‖x‖2 = d(x, M)2 +d(x, M⊥)2.(iv) M⊥ = {x ∈ X | PM(x) = 0} and M = {x ∈ X | PM⊥(x) = 0} = {x ∈X | PM(x) = x}.(v) ‖PM(x)‖ ≤ ‖x‖ for all x ∈ X; ‖PM(x)‖ = ‖x‖ if and only if x ∈ M.(vi) M⊥⊥ = M.Fact 2.3.11. [16, Theorem 5.13] Let M be a Chebyshev linear subspace ofthe inner product space X (e.g., a closed linear subspace of a Hilbert space).Then:(i) PM is a bounded linear operator and ‖PM‖ = 1 (unless M = {0}, inwhich case ‖PM‖ = 0).(ii) PM is idempotent: P2M = PM.(iii) PM is self-adjoint, that is(∀x, y ∈ X) 〈PMx, y〉 = 〈x, PMy〉.(iv) For every x ∈ X,〈PMx, x〉 = ‖PMx‖2.(v) PM is monotone, that is,(∀x ∈ X) 〈PMx, x〉 ≥ 0.Fact 2.3.12. [16, Theorem 4.9] Let M be a linear subspace in the innerproduct space X, x ∈ X, and y0 ∈ M. Then y0 = PM(x) if and only ifx− y0 ∈ M⊥; that is,〈x− y0, y〉 = 0 for all y ∈ M.112.3. OperatorsFact 2.3.13. [16, Theorem 9.26] Let V be an affine set in the inner productspace X. Thus, V = M + v, where M is a linear subspace and v is any givenelement of V. Let x ∈ X and y0 ∈ V. Then the following statements areequivalent:(i) y0 = PV(x);(ii) x− y0 ∈ M⊥;(iii) 〈x− y0, y− v〉 = 0 for all y ∈ V.Moreover,PV(x + e) = PV(x) for all x ∈ X, e ∈ M⊥.Fact 2.3.14. [16, Lemma 9.2] Let M and N be Chebyshev linear subspacesof the inner product space X. Then the following statements are equivalent:(i) PM and PN “commute”; i.e., PMPN = PN PM;(ii) PN(M) ⊆ M;(iii) PM(N) ⊆ N;(iv) PMPN = PM∩N ;(v) PMPN is the orthogonal projection onto a Chebyshev linear subspace.In particular, if M ⊆ N or N ⊆ M, then PM and PN commute.Fact 2.3.15. [8, Lemma 3.4(4)] Let T be a nonexpansive linear operator onH. Then(x ∈ H) (k ∈N) PFix T(Tkx) = PFix Tx.Fact 2.3.16. [6, Proposition 29.1] Let C be a nonempty, closed, convex subsetof the Hilbert spaceH and let x ∈ H. Then the following hold:(i) Set D = z + C, where z ∈ H. Then PDx = z + PC(x− z).(ii) Set D = ρC, where ρ ∈ Rr {0}. Then PDx = ρPC(ρ−1x).(iii) Set D = −C. Then PDx = −PC(−x).Fact 2.3.17. [6, Proposition 29.15] Suppose that {ei}i∈I is a countable or-thonormal subset of the Hilbert spaceH and set C = span{ei}i∈I . Then(∀x ∈ H) PCx =∑i∈I〈x, ei〉ei.In particular, if u ∈ H and ‖u‖ = 1, then the projection of x ∈ H onto theline U = Ru is given by PUx = 〈x, u〉u.122.3. Operators2.3.3 Projections onto the Cartesian product of finitely setsIn this subsection, we consider the subsets of and projection onHN forsome N ≥ 1. Let I = {1, . . . , N}. Let (∀i ∈ I) Ci be a nonempty closedconvex subset ofH. Define two subsets ofHN :C := ΠNi=1Ci and D := {(x)i∈I ∈ HN | x ∈ H},which are both closed and convex (in fact, D is a linear subspace).Let x ∈ H. Clearly,x ∈ ∩mi=1Ci ⇐⇒ (x, x, . . . , x) ∈ C ∩D.Fact 2.3.18. [14, Remark 3.4] Let x = (xi)i∈I . Then(i) PCx = ΠNi=1PCi xi.(ii) PDx = ΠNi=1(1N ∑Ni=1 xi).Definition 2.3.19. [4, Definition 5.6] Denote C := ∩mi=1Ci. We say that theN-tuple of closed convex sets (C1, . . . , CN) is linearly regular if∃κ>0∀x∈H d(x, C) ≤ κ max{d(x, Cj) : j ∈ {1, . . . , N}}.Again, if this holds only on bounded sets, i.e.,∀H⊇S bounded∃κS>0∀x∈H d(x, C) ≤ κS max{d(x, Cj) : j ∈ {1, . . . , N}}.then we say that (C1, . . . , CN) is boundedly linearly regular.Fact 2.3.20. [4, Lemma 5.18] Assume each set Ci is a closed linear subspace.Then the following conditions are equivalent:(i) C⊥1 + · · ·+ C⊥N is closed.(ii) D + C is closed.(iii) (D, C) is (boundedly) (linearly) regular.2.3.4 Nonexpansive and firmly nonexpansive operatorsDefinition 2.3.21. [6, Definition 4.1] Let D be a nonempty subset ofH andlet T : D → H. Then T is(i) firmly nonexpansive if(∀x, y ∈ D) ‖Tx− Ty‖2 + ‖(Id−T)x− (Id−T)y‖2 ≤ ‖x− y‖2;132.3. Operators(ii) nonexpansive if it is Lipschitz continuous with constant 1, i.e.,(∀x, y ∈ D) ‖Tx− Ty‖ ≤ ‖x− y‖;(iii) strictly nonexpansive if(∀x, y ∈ D) x 6= y⇒ ‖Tx− Ty‖ < ‖x− y‖;(iv) firmly quasinonexpansive if(∀x ∈ D) (∀y ∈ Fix(T)) ‖Tx− y‖2 + ‖Tx− x‖2 ≤ ‖x− y‖2;(v) quasinonexpansive if(∀x ∈ D) (∀y ∈ Fix(T)) ‖Tx− y‖ ≤ ‖x− y‖;(vi) and strictly quasinonexpansive if(∀x ∈ Dr {Fix(T)}) (∀y ∈ Fix(T)) ‖Tx− y‖ < ‖x− y‖.Remark 2.3.22. [6, page 70] Concerning Definition 2.3.21, by definition wehave the implications:(i)⇒ (ii)⇒ (v)(i)⇒ (iv)⇒ (vi)⇒ (v)(iii)⇒ (vi).Definition 2.3.23. [6, Definition 4.33] Let D be a nonempty subset of H,let T : D → H be nonexpansive, and let α ∈]0, 1[. Then T is averaged withconstant α, α-averaged, if there exists a nonexpansive operator R : D → Hsuch that T = (1− α) Id+αR.Fact 2.3.24. [6, Proposition 4.4] Let D be a nonempty subset of H and letT : D → H. Then the following are equivalent:(i) T is firmly nonexpansive.(ii) Id−T is firmly nonexpansive.(iii) 2T − Id is nonexpansive.(iv) (∀x ∈ D) (∀y ∈ D) ‖Tx− Ty‖2 ≤ 〈x− y, Tx− Ty〉.142.4. Sequences(v) (∀x ∈ D) (∀y ∈ D) 0 ≤ 〈Tx− Ty, (Id− T)x− (Id− T)y〉.(vi) (∀x ∈ D) (∀y ∈ D) ‖Tx− Ty‖ ≤ ‖α(x− y) + (1− α)(Tx− Ty)‖.Fact 2.3.25. [6, Remark 4.34 (iii)] Let D be a nonempty subset of H, letT : D → H. Then T is firmly nonexpansive if and only if it is 12 -averaged.Fact 2.3.26. [6, Proposition 4.42] Let D be a nonempty subset ofH, let (Ti)i∈Ibe a finite family of nonexpansive operators from D toH, let (ωi)i∈I be realnumbers in ]0, 1] such that ∑i∈I ωi = 1, and let (αi)i∈I be real numbers in]0, 1[ such that, for every i ∈ I, Ti is αi-averaged, and set α = ∑i∈I ωiαi. Then∑i∈I ωiTi is α-averaged.Fact 2.3.27. [6, Proposition 4.47] Let D be a nonempty subset ofH, let (Ti)i∈Ibe a finite family of quasinonexpansive operators from D to H such that∩i∈I Fix Ti 6= ∅, and let (ωi)i∈I be strictly positive real numbers such that∑i∈I ωi = 1. Then Fix∑i∈I ωiTi = ∩i∈I Fix Ti.Fact 2.3.28. [6, Proposition 4.51] Let D be a nonempty subset of H, let mbe a strictly positive integer, set I = {1, . . . , m}, let (Ti)i∈I be a family ofaveraged nonexpansive operators from D to D such that ∩i∈I Fix Ti 6= ∅,and set T = T1 · · · Tm. Then Fix T = ∩i∈I Fix TiFact 2.3.29. [6, Proposition 4.42, Corollary 4.48] Let D be a nonempty subsetof H, let (Ti)i∈I be a finite family of firmly quasinonexpansive operators(firmly nonexpasive operators) from D to H, let (ωi)i∈I be real numbersin ]0, 1] such that ∑i∈I ωi = 1. Then ∑i∈I ωiTi is firmly quasinonexpansiveoperator (firmly nonexpasive operator).2.4 Sequences2.4.1 Convergence of sequencesDefinition 2.4.1. [22, Definition 4.8-1] A sequence (xk)k∈N in a normedspace X is said to be convergent (strongly convergent or convergent in thenorm) if there is an x ∈ X such thatlimk→∞‖xk − x‖ = 0.This is writtenlimk→∞xk = x152.4. Sequencesor simplyxk → x.The x is called the limit of (xk)k∈N, and we say that (xk)k∈N converges to x.Definition 2.4.2. [22, Definition 4.8-2] A sequence (xk)k∈N in a normedspace X is said to be weakly convergent if there is an x ∈ X such that forevery bounded linear functional f on X,limk→∞f (xk) = f (x).This is writtenxk ⇀ x.The element x is called the weak limit of (xk)k∈N, and we say that (xk)k∈Nconverges weakly to x.Fact 2.4.3. [22, Theorem 4.8-4] Let (xk)k∈N be a sequence in a normed spaceX. Then:(i) Strong convergence implies weak convergence with the same limit.(ii) The converse of (i) is not generally true.(iii) If dim X < ∞, then weak convergence implies strong convergence.Definition 2.4.4. [6, page 8] Let (xk)k∈N ⊆ H.(i) (xk)k∈N has a norm cluster if (xk)k∈N possesses a subsequence thatstrongly converges to a point x ∈ H.(ii) (xk)k∈N has a weak cluster if (xk)k∈N possesses a subsequence thatweakly converges to a point x ∈ H.Definition 2.4.5. [9, Definition 2.3] Let (xk)k∈N be a sequence inH and letx ∈ H. (xk)k∈N converges µ-linearly to x if µ ∈ [0, 1[ and there exists M ≥ 0such that(∀k ∈N) ‖xk − x‖ ≤ Mµk. (2.5)(xk)k∈N converges linearly to x if there exists µ ∈ [0, 1[ and M ≥ 0 such that(2.5) holds.Recall that we denote the set of all bounded linear operators fromH intoH as B(H).162.4. SequencesDefinition 2.4.6. [9, Example 2.4] Let (Tk)k∈N be a sequence in B(H), andlet T∞ ∈ B(H). Then one says:(i) (Tk)k∈N converges or converges uniformly to T∞ in B(H) iflimk→∞‖Tk − T∞‖ = 0,that is, Tk converges to T∞ in B(H).(ii) (Tk)k∈N converges pointwise to T∞ iflimk→∞Tkx = T∞x ∀x ∈ H.Remark 2.4.7. It is easy to see that the convergence of a sequence of boundedlinear operators implies pointwise convergence; however, the converse isnot true (see, e.g. [22, Example 4.9-2])Definition 2.4.8. [3, Definition 2.1] Let A ∈ Cn×n. We say A is convergentto A∞ ∈ Cn×n if and only if‖Ak − A∞‖ → 0 as k→ ∞.We say A is linearly convergent to A∞ with rate µ ∈ [0, 1[ if there are someM, N > 0 such that(∀k ≥ N) ‖Ak − A∞‖ ≤ Mµk.Then µ is called a convergence rate of T. When the infimum of all theconvergence rates is also a convergence rate, we say this minimum is the theoptimal convergence rate.Fact 2.4.9. [9, Lemma 2.6] Let Y be a real Banach space, let (Ln)n∈N be asequence in B(Y), let L∞ ∈ B(Y), and let µ ∈]0, 1[. Then(∀y ∈ Y) Lny→ L∞y µ-linearly (in Y)⇐⇒ Ln → L∞ µ-linearly (in B(Y)).Fact 2.4.10. [9, Corollary 2.8] Suppose that X is finite-dimensional, letL : X → X be linear, and let L∞ : X → X be such that Lk → L∞ pointwise.Then Lk → L∞ linearly.172.4. Sequences2.4.2 Fejér monotone sequenceDefinition 2.4.11. [6, Definition 5.1] Let C be a nonempty subset ofH andlet (xk)k∈N be a sequence inH. Then (xk)k∈N is Fejér monotone with respectto C if(∀x ∈ C) (∀k ∈N) ‖xk+1 − x‖ ≤ ‖xk − x‖.Fact 2.4.12. [6, Example 5.3] Let D be a nonempty subset ofH, let T : D →D be a quasinonexpansive operator such that Fix T 6= ∅, and let x0 ∈ D. Setxk+1 = Txk. Then (xk)k∈N is Fejér monotone with respect to Fix T.Fact 2.4.13. [6, Proposition 5.9] Let C be a closed affine subspace ofH andlet (xk)k∈N be a sequence in H. Suppose that (xk)k∈N is Fejér monotonewith respect to C. Then the following hold:(i) (∀k ∈N) PCxk = PCx0.(ii) Suppose that every weak sequence cluster point of (xk)k∈N belongs toC. Then xk ⇀ PCx0.Fact 2.4.14. [6, Proposition 5.16] Let α ∈]0, 1[, let T : H → H be an α-averaged operator such that Fix T 6= ∅, let (λk)k∈N be a sequence in[0, 1α]such that ∑k∈N λk(1− αλk) = +∞, and let x0 ∈ H. Set(∀k ∈N) xk+1 = xk + λk(Txk − xk).Then the following hold:(i) (xk)k∈N is Fejér monotone with respect to Fix T.(ii) (Txk − xk)k∈N converges strongly to 0.(iii) (xk)k∈N converges weakly to a point in Fix T.Fact 2.4.15. [4, Theorem 2.16] Suppose C is a closed, convex, nonempty setinH and the sequence (xk)k∈N is Fejér monotone with respect to C. Then(i) (xk)k∈N is bounded and d(xk+1, C) ≤ d(xk, C).(ii) (xk)k∈N has at most one weak cluster point in C. Consequently,(xk)k∈N converges weakly to some point in C if and only if all weakcluster points of (xk)k∈N lie in C.(iii) If the interior of C is nonempty, then (xk)k∈N converges in norm.182.5. Algebra time(iv) The sequence (PCxk)k∈N converges in norm.(v) The following are equivalent:(a) (xk)k∈N converges in norm to some point in C.(b) (xk)k∈N has norm cluster points, all lying in C.(c) (xk)k∈N has norm cluster points, one lying in C.(d) d(xk, C)→ 0.(e) xk − PCxk → 0.Moreover, if (xk)k∈N converges to some x ∈ C, then ‖xk − x‖ ≤2d(xk, C) for all k ≥ 0.(vi) If there is some constant α > 0 such that αd2(xk, C) ≤ d2(xk, C) −d2(xk+1, C) for every k, then (xk)k∈N converges linearly to some pointx in C. More precisely,(∀k ≥ 0) ‖xk − x‖ ≤ 2(1− α) k2 d(x0, C).2.4.3 Asymptotically regularDefinition 2.4.16. Given an operator T : H → H. Then T is asymptoticallyregular if for each x ∈ H,limk→∞Tkx− Tk+1x = 0.Fact 2.4.17. [9, Fact 2.1] Let T : X → X. If T is firmly nonexpansive andFix T 6= ∅, then T is asymptotically regular.Fact 2.4.18. [9, Theorem 3.3] Let T : X → X be an affine and nonexpansiveoperator with Fix T 6= ∅. Then the following are equivalent:(I) Tk → PFix T pointwise.(II) T is asymptotically regular.Fact 2.4.19. [9, Corollary 3.6] Suppose X is finite-dimensional. Let T : X →X be an affine and nonexpansive operator with Fix T 6= ∅. Suppose T isasymptotically regular. Then Tn → PFix T pointwise linearly.2.5 Algebra timeIn this section, we will review some background of algebra.192.5. Algebra time2.5.1 MatrixDefinition 2.5.1. [25, page 466] A minor determinant (or simply a minor) ofAm×n is defined to be the determinant of any k× k submatrix of A, wherek ∈Nr {0} and k ≤ min{m, n}.Definition 2.5.2. [25, page 115] For a given square matrix An×n, the matrixBn×n that satisfies the conditionsAB = In and BA = Inis called the inverse of A and is denoted by B = A−1. Not all square matricesare invertible–the zero matrix is a trivial example, but there are also manynonzero matrices that are not invertible. An invertible matrix is said to benonsingular, and a square matrix with no inverse is called a singular matrix.Fact 2.5.3. [25, page 115] Notice that matrix inversion is defined for squarematrices only–the condition AA−1 = A−1A rules out inverses of nonsquarematrices.Definition 2.5.4. [25, page 477] The cofactor of An×n associated with the(i, j)-position is defined asA˚ij = (−1)i+j Mij,where Mij is the n− 1× n− 1 minor obtained by deleting the ith row andjth column of A. The matrix of cofactors is denoted by A˚.Fact 2.5.5. [25, page 479] The adjugate of An×n is defined to be adj(A) = A˚ᵀ,the transpose of the matrix of cofactors — some older texts call this theadjoint matrix. If A is nonsingular, thenA−1 =A˚ᵀdet(A)=adj(A)det A.2.5.2 DeterminantsDefinition 2.5.6. [25, page 460] A permutation p = (p1, p2, . . . , pn) of thenumbers (1, 2, . . . , n) is simply any rearrangement.Fact 2.5.7. [25, page 460] The parity of a permutation is unique– i.e., if apermutation p can be restored to natural order by an even (odd) number ofinterchanges, then every other sequence of interchanges that restores p tonatural order must also be even (odd).202.5. Algebra timeDefinition 2.5.8. [25, page 461] The sign of a permutation p is defined to bethe numberσ(p) =+1 if p can be restored to natural order by an even numberof interchanges,−1 if p can be restored to natural order by an odd numberof interchanges.Definition 2.5.9. [25, page 461] For an n× n matrix A = [aij], the determi-nant of A is defined to be the scalardet(A) =∑pσ(p)a1p1 a2p2 · · · anpn , (2.6)where the sum is taken over the n! permutations p = (p1, p2, . . . , pn) of(1, 2, . . . , n). Observe that each term a1p1 a2p2 · · · anpn in (2.6) contains exactlyone entry from each row and each column of A. The determinant of A canbe denoted by det(A) or |A|, whichever is more convenient.Remark 2.5.10. The determinant of a nonsquare matrix is not defined.Fact 2.5.11. [25, page 463] For every n× n matrices A, we havedet(Aᵀ) = det(A).Fact 2.5.11 insures that it is not necessary to distinguish between rowsand columns when discussing properties of determinants, so theoremsconcerning determinants that involve row manipulations will remain truewhen the word “row” is replaced by “column”. Hence, in the following factabout how elementary row and column operations alter the determinant ofa matrix, it suffices to limit the discussion to elementary row operations.Fact 2.5.12. [25, page 463] Let B be the matrix obtained from An×n by oneof the three elementary row operations:Type I: Interchange rows i and j.Type II: Multiply row i by α 6= 0.Type III: Add α times row i to row j.The value of det(B) is as follows:det(B) = −det(A)for Type I operations.det(B) = αdet(A)for Type II operations.det(B) = det(A)for Type III operations.212.5. Algebra timeFact 2.5.13. [25, page 478] Given an n× n matrix A = [aij] with n ≥ 2.det(A) = ai1 A˚i1 + ai2A˚i2 + · · ·+ ain A˚in (about row i)det(A) = a1j A˚1j + a2j A˚2j + · · ·+ anj A˚nj (about column j).Fact 2.5.14. [23, Theorem 2.1.4] Let A be an n× n matrix.(i) If A has a row or column consisting entirely of zeros, then det(A) = 0.(ii) If A has two identical tows or two identical columns, then det(A) = 0.Fact 2.5.15. [25, page 465] An×n is nonsingular if and only if det(A) 6= 0 or,equivalently, An×n is singular if and only if det(A) = 0.Fact 2.5.16. (Cramer’s rule) [25, page 476] If A ∈ Rn×n is invertible andAx = b, then for every i ∈ {1, . . . , n}, we havexi =det(Ai)det(A),where Ai = [A∗,1| · · · |A∗,i−1|b|A∗,i+1| · · · |A∗,n]. That is, Ai is identical to Aexcept that column A∗,i has been replaced by b.2.5.3 Gram matrixDefinition 2.5.17. Let a1, . . . , am ∈ H. Then the matrixG(a1, . . . , am) :=‖a1‖2 〈a1, a2〉 · · · 〈a1, am〉〈a2, a1〉 ‖a2‖2 · · · 〈a2, am〉.........〈am, a1〉 〈am, a2〉 · · · ‖am‖2is called the the Gram matrix of a1, . . . , am.Fact 2.5.18. [16, Lemma 7.5] Let a1, . . . , am ∈ H. Then the Gram matrixG(a1, . . . , am) :=‖a1‖2 〈a1, a2〉 · · · 〈a1, am〉〈a2, a1〉 ‖a2‖2 · · · 〈a2, am〉.........〈am, a1〉 〈am, a2〉 · · · ‖am‖2is invertible if and only if a1, . . . , am are linearly independent.222.5. Algebra timeRemark 2.5.19. Let x, y, z be affinely independent vectors in R3. Set a =y− x and b = z− x. Then, by the following Lemma 4.1.4 and Fact 2.5.18,‖a‖2‖b‖2 − 〈a, b〉2 6= 0 and ‖a‖ 6= 0, ‖b‖ 6= 0.Recall Rn×n is the set of all real (n, n)-matrices. In the n2-dimensionalvector space Rn×n, we can define many kinds of norms. The following ‖·‖1is one of them.Fact 2.5.20. [21, page 291] The l1 norm defined for A = (aij)1≤i≤n1≤j≤n∈ Rn×nby‖A‖1 =n∑i,j=1|aij|is a norm on Rn×n.Hence, we can analyze the topological properties of the spaces Rn×n.Fact 2.5.21. [29, page 16] Let S = {A ∈ Rn×n | A is invertible}. Then themapping S→ S : A 7→ A−1 is continuous.2.5.4 Cross productDefinition 2.5.22. [1, page 483] Let x = (x1, x2, x3) and y = (y1, y2, y3) betwo vectors in R3. The cross product x× y (in that order) isx× y = (x2y3 − x3y2, x3y1 − x1y3, x1y2 − x2y1).The following properties are easily deduced from this definition.Fact 2.5.23. [1, Theorem 13.12] and [15, Theorem 17.12] Let x, y, z be in R3.Then the following hold.(i) The cross product defined in Definition 2.5.22 is a bilinear function,that is, for every α, β ∈ R,(αx + βy)× z = α(x× z) + β(y× z) andx× (αy + βz) = α(x× y) + β(x× z).(ii) x× y ∈ (span{x, y})⊥, that is(∀α ∈ R) (∀β ∈ R) 〈x× y, αx + βy〉 = 0.(iii) We have(x× y)× z = 〈x, z〉y− 〈y, z〉x and x× (y× z) = 〈x, z〉y− 〈x, y〉z.232.5. Algebra time(iv) (Lagrange’s identity) ‖x× y‖2 = ‖x‖2‖y‖2 − 〈x, y〉2.Definition 2.5.24. [1, page 458] Let x and y be two vectors inRn, with y 6= 0and n ≥ 1. Then the angle θ between x and y is defined byθ = arccos〈x, y〉‖x‖‖y‖ ,where arccos : [−1, 1]→ [0,pi].Remark 2.5.25. Let x and y be two vectors in Rn, with y 6= 0 and n ≥ 1.Then〈x, y〉 = ‖x‖‖y‖ cos θ,where θ is the angle between x and y.Fact 2.5.26. [1, page 485] Let x, y be two vectors in R3. Let θ be the anglebetween x and y. Then ‖x× y‖ = ‖x‖‖y‖ sin θ= area of the parallelogramdetermined by x and y.Fact 2.5.27. [24, Theorem I] Suppose that n ≥ 3, and a cross product isdefined which assigns to any two vectors v, w ∈ Rn a vector v× w ∈ Rnsuch that the following three properties hold.(i) v× w is a bilinear function of v and w.(ii) The vector v× w is perpendicular to both v and w.(iii) ‖v× w‖2 = ‖v‖2‖w‖2 − 〈v, w〉2.Then n = 3 or 7.24Chapter 3Reflection and projectionmethods3.1 OverviewSince the circumcenter operator defined in Definition 5.2.2 is neitherlinear nor nonexpansive (see counterexamples in Chapter 10), thus it isdifficult to analyze the convergence of the circumcenter method induced byreflectors defined in Section 8.1. In this thesis, motivated by [12] and [13], wetake advantage of the well-known algorithms, DRM, MAP, symmetric MAPand the accelerated MAP and symmetric MAP defined in [8, Definition 3.1],as bridges to prove the linear convergence of circumcenter methods. Hence,in this chapter, we recall some facts about those well-known algorithms.Those facts are necessary in our later proofs. As in the last two chapters, allknown results are designated as facts with explicit references.As we mentioned before, this thesis is motivated by [12]. In fact, [13] is ageneralization of [12] by the same authors. In Section 3.5, we collect all ofthe results of [12] and [13] that we will use and set them as facts. We explainclearly the use of those facts (see Section 3.5).3.2 The method of alternating projections (MAP)Definition 3.2.1. [16, Definition 9.4] The Friedrichs angle between twolinear subspaces U and V is the angle α(U, V) between 0 and pi2 whosecosine, c(U, V) = cos α(U, V), is defined by the expressionc(U, V)= sup{〈u, v〉 | u ∈ U ∩ (U ∩V)⊥, v ∈ V ∩ (U ∩V)⊥, ‖u‖ ≤ 1, ‖v‖ ≤ 1}.Fact 3.2.2. [16, Lemma 9.5] Let U and V be closed linear subspaces of theHilbert spaceH. Then:(i) 0 ≤ c(U, V) ≤ 1.253.2. The method of alternating projections (MAP)(ii) c(U, V) = c(V, U).(iii) c(U, V) = c(U ∩ (U ∩V)⊥, V ∩ (U ∩V)⊥).(iv) c(U, V) = sup{〈u, v〉 | u ∈ U, v ∈ V ∩ (U ∩V)⊥, ‖u‖ ≤ 1, ‖v‖ ≤ 1}.(v) c(U, V) = sup{〈u, v〉 | u ∈ U ∩ (U ∩V)⊥, v ∈ V, ‖u‖ ≤ 1, ‖v‖ ≤ 1}.(vi) |〈u, v〉| ≤ c(U, V)‖u‖‖v‖ whenever u ∈ U, v ∈ V, and at least one ofu or v is in (U ∩V)⊥.(vii) c(U, V) = ‖PV PU− PU∩V‖ = ‖PV PU P(U∩V)⊥‖ = ‖PU∩(U∩V)⊥PV∩(U∩V)⊥‖.(viii) c(U, V) = 0 if and only if PU and PV commute. (In this case, PV PU =PU∩V .) In particular, c(U, V) = 0 if U ⊆ V or V ⊆ U.Fact 3.2.3. [16, Theorem 9.31] Let U and V be closed linear subspaces of theHilbert spaceH. For each k ∈N,‖(PV PU)k − PU∩V‖ = c(U, V)2k−1.Fact 3.2.4. [16, Theorem 9.35] Let U and V be closed linear subspaces of theHilbert spaceH. Then the following statements are equivalent.(i) c(U, V) < 1;(ii) U ∩ (U ∩V)⊥ +V ∩ (U ∩V)⊥ is closed;(iii) U +V is closed;(iv) U⊥ +V⊥ is closed.Lemma 3.2.5. Let U and V be closed linear subspaces of the Hilbert spaceH. Then the following statements are equivalent.(i) U +V is closed.(ii) ‖PV PU P(U∩V)⊥‖ < 1.Proof. The equivalency follows immediately from Fact 3.2.2 (vii) and Fact 3.2.4.Definition 3.2.6. [5, Definition 3.7.5] Let L1, . . . , LN be closed linear sub-spaces of H and L := ∩Ni=1Li. Define the angle β := β(L1, . . . , LN) ∈ [0, pi2 ]of the N tuple (L1, . . . , LN) bycos β = ‖PLN · · · PL1 PL⊥‖.263.2. The method of alternating projections (MAP)Fact 3.2.7. [5, Remark 3.7.6] The classical definition of the angle α(L1, L2)between two linear subspaces in Definition 3.2.1 was given by Friedrichsin 1937. Deutsch showed that ‖PL2 PL1 P(L2∩L1)⊥‖ is equal to cos α(L1, L2) (see[16, Lemma 9.5]); therefore, β(L1, L2) = α(L1, L2) and the Definition 3.2.6 ofthe angle is consistent with Friedrichs’ original one.Fact 3.2.8. [5, Proposition 3.7.7] Let L1, . . . , LN be closed linear subspacesof H. The angle of the N-tuple (L1, . . . , LN) is positive if and only if thesum L⊥1 + · · ·+ L⊥N is closed. In particular, this holds whenever one of thefollowing conditions is satisfied:(i) Some Li ∩ L⊥ is finite-dimensional.(ii) Some Li is finite-dimensional.(iii) H is finite-dimensional.(iv) All Li, except possibly one, are finite-codimensional.(v) Each Li is a hyperplane.(vi) Each angle β(Li, L1 ∩ · · · ∩ LN) is positive.Corollary 3.2.9. Let L1, . . . , LN be closed linear subspaces of H and L :=∩Ni=1Li. Then ‖PLN · · · PL1 PL⊥‖ ∈ [0, 1[ if and only if L⊥1 + · · ·+ L⊥N is closed.Proof. The result is from Definition 3.2.6 and Fact 3.2.8.Fact 3.2.10. [8, Lemma 2.4] Let U1, . . . , Um be closed linear subspaces ofH,let U := ∩mi=1Ui and let T := PUm PUm−1 · · · PU1 . Then T is nonexpansive andFix T = Fix T∗ = Fix(TT∗) = Fix(T∗T) = U.Fact 3.2.11. [8, Lemma 3.14] Let T : H → H be linear and nonexpansive onH. Denote M = Fix T. Then(i) (∀k ∈N) ck(T) := ‖(TPM⊥)k‖ = ‖Tk − PM‖. In particular,(∀k ∈N) (∀x ∈ H) ‖Tkx− PMx‖ ≤ ck(T)‖x− PMx‖. (3.1)and ck(T) is the smallest constant independent of x for which (3.1) isvalid.(ii) (∀y ∈ M⊥) ‖Tky‖ ≤ ck(T)‖y‖.(iii) (∀k ∈N) ‖(TPM⊥)k‖ ≤ ‖TPM⊥‖k.273.3. The Douglas–Rachford method (DRM)(iv) ‖T∗TPM⊥‖ ≤ ‖TPM⊥‖2 and ‖T∗TPM⊥‖ = ‖TPM⊥‖2 if Fix(T∗T) =Fix T. In particular, if T = PMk PMk−1 · · · PM1 , then ‖T∗TPM⊥‖ = ‖TPM⊥‖2.Fact 3.2.12. [8, Theorem 3.16] Let T be a nonexpansive linear operation onH and M := Fix T. Then(∀x ∈ H) (∀k ∈N) ‖Tkx− PMx‖ ≤ ‖TPM⊥‖k‖x− PMx‖.Corollary 3.2.13. Let U1, . . . , Um be closed linear subspaces of H, let U :=∩mi=1Ui and let T := PUm PUm−1 · · · PU1 . Then(∀x ∈ H) (∀k ∈N) ‖(T∗T)kx− PUx‖ ≤ ‖TPM⊥‖2k‖x− PUx‖.Proof. The required result follows directly from Fact 3.2.12, Fact 3.2.11(iv)and Fact 3.2.10.3.3 The Douglas–Rachford method (DRM)The Douglas–Rachford splitting algorithm was originally proposed in1956 to solve a system of linear equations arising from the discretizationof a partial differential equation. When specialized to two normal coneoperators, it yields an algorithm for finding a point in the intersection oftwo convex sets. Below we conclude some results of DRM to solve bestapproximation problem.Definition 3.3.1. [2, page 2] Let U and V be closed, convex subsets of Hsuch that U ∩V 6= ∅. The Douglas–Rachford splitting operator isTV,U := PV(2PU − Id) + Id−PUBy easy algebraic calculation, we get thatTV,U = PV(2PU − Id) + Id−PU = Id+RV RU2 .Lemma 3.3.2. Let U and V be closed convex subsets ofH such that U ∩V 6=∅. Then the Douglas–Rachford operator TU,V is firmly nonexpansive.Proof. By the following Lemma 4.2.2, we know RU , RV are nonexpansive.So RV RU is nonexpansive. Hence by Definition 2.3.23 and Fact 2.3.25, weknow TU,V = Id+RV RU2 is firmly nonexpansive.283.3. The Douglas–Rachford method (DRM)To simplify the notation, without other statement, in the whole section,we assume U and V are closed linear subspaces ofH. We denote T := TV,Uthe Douglas–Rachford splitting operator associated with the U and V.Fact 3.3.3. [2, Proposition 3.6] Let n ∈N. Then the following hold:(i) Fix T = Fix T∗ = Fix T∗T = (U ∩V)⊕ (U⊥ ∩V⊥).(ii) PFix T = PU∩V + PU⊥∩V⊥ .Fact 3.3.4. [2, Theorem 4.1] Let n ∈ Nr {0} and let x ∈ H. Denote thec(U, V) defined in Definition 3.2.1 by cF. Then‖Tn − PFix T‖ = cnF, (3.2a)‖(TT∗)n − PFix T‖ = c2nF , (3.2b)and‖Tnx− PFix Tx‖ ≤ cnF‖x− PFix Tx‖ ≤ cnF‖x‖. (3.3)Lemma 3.3.5. Let U, V be two closed, linear subspaces in H. Let x ∈ H.ThenPU∩V(x) = PFix T(x)⇐⇒ x ∈ span(U ∪V)⇐⇒ x ∈ U +V.Proof. Clearly,PU∩V(x) = PFix T(x)⇐⇒ PU⊥∩V⊥x = 0 (by Fact 3.3.3(ii))⇐⇒ P(span(U∪V))⊥x = 0 (by Lemma 4.1.9)⇐⇒ x ∈ ((span(U ∪V))⊥)⊥ (by Fact 2.3.10(iv))⇐⇒ x ∈ span(U ∪V) (by Fact 2.3.10(vi))⇐⇒ x ∈ U +V (by Lemma 4.1.8).Therefore the required result is true.Corollary 3.3.6. Let U and V be closed, convex subsets of H such thatU ∩V 6= ∅ with U + V being closed. Let x ∈ H. Then the DRM iterationsequence (Tk(x))k∈N linearly converges to PU∩V x if and only if x ∈ U +V.Proof. The result follows easily from Fact 3.3.4, Fact 3.2.4 and Lemma 3.3.5.293.4. The accelerated mapping of MAPLemma 3.3.7. Let x ∈ H. Let W be a closed linear subspace ofH such thatU ∩V ⊆W ⊆ U +V.ThenPFix TPW x = PU∩V PW x = PU∩V x.Proof. Since PW x ∈W ⊆ U +V, thus, by Lemma 3.3.5,PFix TPW x = PU∩V PW x.On the other hand, by assumption, U ∩V ⊆ W. Hence, by Fact 2.3.14, wegetPU∩V PW x = PW PU∩V x = PU∩V x.In conclusion, we obtainPFix TPW x = PU∩V PW x = PU∩V x.3.4 The accelerated mapping of MAPIn the whole section, we assumeU1, . . . , Un are closed linear subspaces in the real Hilbert spaceH.Definition 3.4.1. [8, Definition 3.1] Let T be a nonexpansive linear operatoronH. The accelerated mapping AT of T is defined onH byAT(x) := txTx + (1− tx)x,wheretx = tx,T :={ 〈x,x−Tx〉‖x−Tx‖2 if Tx 6= x,1 if Tx = x.Fact 3.4.2. [8, Theorem 3.7] For each x ∈ H and y ∈ H, we have(∀t ∈ R) ‖AT(x)− y‖2 = ‖tTx + (1− t)x− y‖2 − (t− tx)2‖Tx− x‖2.Moreover,(∀x ∈ H) AT(x) = Paff{x,Tx}(PFix Tx).303.4. The accelerated mapping of MAPFact 3.4.3. [8, Theorem 3.20] Let T be linear, nonexpansive, self-adjoint andmonotone operator onH. Then(∀x ∈ H) (∀k ∈N) ‖Ak−1T (T(x))− PFix Tx‖ ≤ ‖Tkx− PFix Tx‖.In other words, the accelerated algorithm converges at least as fast as itsunaccelerated counterpart.Remark 3.4.4. The authors in [8] used the notion nonnegative. By definition,since T is linear, T is nonnegative is equivalent to T is monotone. In thewhole thesis, we shall use monotone in replace of nonnegative.Fact 3.4.5. [8, Lemma 3.26] Let T be a linear, nonexpansive and self-adjointoperator onH. Denote M := Fix T. Letc1 := inf{〈Tx, x〉 | x ∈ M⊥, ‖x‖ = 1}, (3.4)andc2 := sup{〈Tx, x〉 | x ∈ M⊥, ‖x‖ = 1}, (3.5)where both c1 and c2 are defined to be 0 if M⊥ = {0}, i.e., if M = H. Thenmax{c2,−c1} = c(T) := ‖TPM⊥‖.Moreover, if T is also monotone, thenc2 = c(T).Fact 3.4.6. [8, Lemma 3.27] Let T be a linear, nonexpansive and self-adjointoperator onH. Let c1 and c2 be defined as in (3.4) and (3.5). Then(∀y ∈ M⊥) ‖AT(y)‖ ≤ c2 − c12− c1 − c2 ‖y‖.In particular,(∀y ∈ M⊥) (∀k ∈N) ‖AkT(y)‖ ≤( c2 − c12− c1 − c2)k‖y‖.Lemma 3.4.7. Let T be a linear, nonexpansive, self-adjoint and monotoneoperator on H. Let c1 and c2 be defined as in (3.4) and (3.5). Denote M :=Fix T and c(T) := ‖TPM⊥‖. Thenc2 − c12− c1 − c2 =c(T)− c12− c1 − c(T) ≤c(T)2− c(T) . (3.6)313.5. Circumcentered-reflection methodProof. The proof can be found in the proof of [8, Theorem 3.29].Fact 3.4.8. [8, Theorem 3.29] Let T be a linear, nonexpansive, self-adjointand monotone operator onH. Let M = Fix T. Then(∀x ∈ H) (∀k ∈N) ‖Ak−1T (Tx)− PMx‖ ≤c(T)k(2− c(T))k−1 ‖x− PMx‖.3.5 Circumcentered-reflection methodAs we mentioned before, this thesis is motivated by [12]. In fact, [13]generalizes [12] from two linear subspaces to any finite numer of linearsubspaces. The two papers are written by the same authors. The ideas ofsome proofs in this thesis are from the above two references. In fact, someresults in this thesis are mild generalizations of the results in the above tworeferences.In the whole section, let U, V be two linear subspaces ofRn. Denote T =Id+RV RU2 which is the Douglas–Rachford splitting operator. In [12, Section1], the authors defined the Circumcentered–Douglas–Rachford method (C–DRM): from a point x ∈ Rn, the next iterate is the circumcenter of thetriangle of vertices x, RUx and RV RUx, denoted byCTx := circumcenter{x, RUx, RV RUx}.The authors defined CT from the Douglas–Rachford operator T andthen they studied the convergence rate of CT. Actually, their CT is our CCSdefined in Definition 6.2.1 to come with S = {Id, RU , RV RU}.In [12], the authors presented the fact below and although they namedit as Proposition, they didn’t show any proof and said the results are from[16, Theorem 5.8]. In the following Proposition 4.2.5, using [6, Corollary3.22], we extend the following fact from linear subspaces to affine subspaces.In turn, Proposition 4.2.5 plays an essential role in the proofs of one of ourmain theorem Theorem 7.2.5 in Chapter 7.Fact 3.5.1. [12, Proposition 1] Let S be a given linear subspace in Rn andx ∈ Rn arbitrary but fixed. Then for all s ∈ S we have:(i) 〈x− PSx, s〉 = 0;(ii) ‖x− PSx‖2 = ‖x− s‖2 − ‖s− PSx‖2;(iii) ‖x− s‖ = ‖RSx− s‖;323.5. Circumcentered-reflection method(iv) the projection and reflection mappings PS and RS are linear.The following three facts were generalized to our Proposition 8.2.6,which plays an important role to prove the linear convergence of our cir-cumcenter method induced by reflectors defined in Section 8.1.Fact 3.5.2. [12, Lemma 1] Let x ∈ Rn. Then we have PU∩V x = PU∩V RUx =PU∩V RV RUx and d(x, U ∩V) = d(RUx, U ∩V) = d(RV RUx, U ∩V). More-over, PU∩V x = PU∩V(Tx) and (∀k ∈N) PU∩V x = PU∩V(Tkx).Fact 3.5.3. [12, Lemma 3] Let x ∈ Rn and k ∈N. Then,PU∩V(CkTx) = PU∩V(CTx) = PU∩V(x).Fact 3.5.4. [12, Lemma 4] Let x ∈ Rn, we haved(CTx, U ∩V)2 = d(Tx, U ∩V)2 − ‖Tx− CTx‖2.In particular,d(CTx, U ∩V) = ‖CTx− PU∩V(x)‖ ≤ ‖Tx− PU∩V(x)‖ = d(Tx, U ∩V).In the following fact, the authors characterized CTx as the projectionof any point w ∈ U ∩V onto the affine subspace aff{x, RUx, RV RUx}. Theidea of the proof of Theorem 7.2.5 is from the following fact. In fact, ourTheorem 7.2.5 is the foundation to study the circumcenter method inducedby reflectors.Fact 3.5.5. [12, Lemma 2] Let x ∈ Rn and Wx := {x, RUx, RV RUx}. Then,(∀w ∈ U ∩V) PWx w = CTx.In particular, PWx(PU∩V x) = CTx.Using the idea from the proof of the following fact, we prove the muchmore general result Proposition 8.6.1. In view of Proposition 8.6.1, we getCorollary 8.6.3 which extends [12, Theorem 1] from Rn to the general realHilbert spaceH. In addition, another application of our Proposition 8.6.1 isProposition 8.6.6 which proves three of our circumcenter methods inducedby reflectors converge linearly to PU1∩U2 x with linear rate at least c2F ∈ [0, 1[,where cF is given below.Fact 3.5.6. [12, Theorem 1] Let x ∈ Rn. Then, the three C–DRM sequences:(CkT(PUx))k∈N, (CkT(PV x))k∈N and (CkT(PU+V x))k∈N, converge linearly toPU∩V x. Moreover, their rate of convergence is at least the cosine of theFriedrichs angle cF ∈ [0, 1[ between U and V.333.5. Circumcentered-reflection methodLet (Ui)1≤i≤m be a family of finitely many affine subspaces in Rn. In[13, Section 1], the authors defined the circumcentered–reflection method(CRM): at a given point x ∈ Rn, CRM generates a next iterate C(x) ∈ Rnwith the following two properties:(a) C(x) belongs to the affine subspaceWx := aff{x, RU1 x, RU2 RU1 x, . . . , RUm · · · RU2 RU1 x};(b) C(x) is equidistant to the points x, RU1 x, RU2 RU1 x, . . . , RUm · · · RU2 RU1 x.In [13], in order to prove the linear convergence of the CRM, the authorsintroduced the following convenient operator A : Rn → Rn, withA(x) =1mm∑i=1Ai(x),where A1 = 12 (Id+PU1) and (∀i ∈ {2, . . . , m}) Ai = 12 (Id+PUi RUi−1 · · · RU1).Moreover, they gave the following properties of the operator A. In fact, us-ing the operator A, we get the Proposition 8.5.6. The operator TS definedin the Proposition 8.5.6 below is the above operator A. In addition, us-ing the idea of the proof in [13, Lemma 2.1], we obtain Lemma 8.5.7 andLemma 8.5.8. In the light of the above two lemmas, we get the Proposi-tion 8.5.9, Proposition 8.5.10 and Corollary 8.5.11. The Proposition 4.4.2below gets idea of its proof from the following fact.Fact 3.5.7. [13, Lemma 2.1(i) to (iv)] Let A : Rn → Rn be as above. Then:(i) A is linear and 12 -averaged, i.e., firmly nonexpansive;(ii) Fix A = ∩mi=1Ui;(iii) For all x ∈ Rn, A(x) ∈Wx;(iv) There exists rA ∈ [0, 1[ such that for all x ∈ Rn it holds that ‖A(x)−P∩mi=1Ui x‖ ≤ rA‖x− P∩mi=1Ui x‖.The Proposition 8.3.3 below is a mild generalization of the fact below. Inturn, our Proposition 8.5.9 and Proposition 8.5.10 below are two corollary ofthe Proposition 8.3.3.Fact 3.5.8. [13, Theorem 3.3] Let x ∈ Rn be given. Then the CRM se-quence (Ckx)k∈N converges to P∩mi=1Ui x with a linear rate rA ∈ [0, 1[, givenin Fact 3.5.7(iv).34Chapter 4Further auxiliary resultsOur new results start in this chapter. In this chapter, we present someauxiliary results to facilitate our proofs in the later chapters.The effects of the main results in this chapter are summarized as follows.• Proposition 4.3.3 plays an important role to show the symmetry of theformulae of the circumcenter operator.• To prove Corollary 4.3.4, we need Lemma 4.1.7. In turn, Corollary 4.3.4is necessary to explore the continuity of the circumcenter operator.• Proposition 4.2.5 mildly extends [12, Proposition 1] from linear sub-space to affine subspace and from Rn toH. It plays an important rolewhen we prove the beautiful properties of the circumcenter mappinginduced by reflectors that is one of our main actors in this thesis.• The proof of Proposition 4.4.2(i) is a mild generalization of [13, Lemma2.1(iv)]. In fact, Proposition 4.4.2(i) plays a critical role in Proposi-tion 8.3.3, which in turn is a generalization of [13, Theorem 3.3] that isthe main result in [13].4.1 Span and affine subspacesLemma 4.1.1. Let {x, x1, . . . , xm} ⊆ H. Thenspan{x1 − x, . . . , xm − x}={λ1(x1 − x) + · · ·+ λm(xm − x) | λ1, . . . ,λm ∈ R}.Proof. Let A :={λ1(x1 − x) + · · · + λm(xm − x) | λ1, . . . ,λm ∈ R}. Bydefinition, {x1 − x, . . . , xm − x} ⊆ A and A is a linear subspace. Hence, forevery linear subspace L containing {x1 − x, . . . , xm − x}, we have A ⊆ L.354.1. Span and affine subspacesTherefore, A is the smallest linear subspace containing {x1 − x, . . . , xm − x},which meansspan{x1 − x, . . . , xm − x}={λ1(x1 − x) + · · ·+ λm(xm − x) | λ1, . . . ,λm ∈ R}.Similarly, we can prove the lemma below.Lemma 4.1.2. Let {x1, . . . , xm} ⊆ H. Then for every i0 ∈ {1, . . . , m}aff{x1, . . . , xm}={λ1x1 + · · ·+ λm−1xm−1 + λmxm∣∣∣ λ1, . . . ,λm ∈ R and m∑i=1λi = 1}=xi0 +{m−1∑j=1j 6=i0λj(xj − xi0)∣∣∣∣∣ λj ∈ R, j ∈ {1, . . . , m− 1}}=xi0 + span{x1 − xi0 , . . . , xi0−1 − xi0 , xi0+1 − xi0 , . . . , xm − xi0}.Proof. LetB ={λ1x1 + · · ·+ λmxm∣∣∣ λ1, . . . ,λm ∈ R and m∑i=1λi = 1}.Let b ∈ B, i.e., there are λ1, . . . ,λm ∈ Rwith ∑mi=1 λi = 1 such thatb = λ1x1 + · · ·+ λmxm.By Remark 2.2.7, we get b ∈ aff{x1, . . . , xm}. Hence B ⊆ aff{x1, . . . , xm}. Inaddition, for every M being an affine subspace containing {x1, . . . , xm}, bythe definition of affine subspace, we know that B ⊆ M. Therefore, B is thesmallest affine subspace containing {x, x1, . . . , xm}, which means that,aff{x1, . . . , xm} = B={λ1x1 + · · ·+ λm−1xm−1 + λmxm∣∣∣ λ1, . . . ,λm ∈ R and m∑i=1λi = 1}.364.1. Span and affine subspacesMoreover, let i0 ∈ {1, . . . , m} be arbitrary but fixed. Since ∑mi=1 λi = 1implies that λi0 = 1−∑m−1j=1j 6=i0λj, thus{λ1x1 + · · ·+ λm−1xm−1 + λmxm∣∣∣ λ1, . . . ,λm ∈ R and m∑i=1λi = 1}={xi0 + λ1(x1 − xi0) + · · ·+ λi0−1(xi0−1 − xi0) + λi0+1(xi0+1 − xi0)+ · · ·+ λm(xm − xi0)∣∣∣ λ1, . . . ,λi0−1,λi0+1, . . . ,λm ∈ R}=xi0 + span{x1 − xi0 , . . . , xi0−1 − xi0 , xi0+1 − xi0 , . . . , xm − xi0}.Therefore, we are done.Corollary 4.1.3. Let {x, x1, . . . , xm} ⊆ H. Thenaff{x, x1, . . . , xm} = x + span{x1 − x, . . . , xm − x}.Proof. The required result follows directly from Lemma 4.1.2.Lemma 4.1.4. Let x, x1, . . . , xm−1, xm ∈ H. Then the following statementsare equivalent:(i) the m + 1 vectors x, x1, . . . , xm are affinely independent;(ii) the m vectors x1 − x, . . . , xm − x are linearly independent.Proof. The result follows easily from the Definition 2.2.12, Corollary 4.1.3and Definition 2.2.4.Proposition 4.1.5. Let S = {x1, x2, . . . , xm} ⊆ H. Let xi1 , . . . , xit be elementsof S, and set K = {x1, xi1 , . . . , xit}. Thenaff(K) = aff(S) and x1, xi1 , . . . , xit are affinely independent.⇐⇒ xi1 − x1, . . . , xit − x1 is a basis of span{x2 − x1, . . . , xm − x1}Proof. Indeed,aff(K) = aff(S) and x1, xi1 , . . . , xit are affinely independent⇐⇒{x1 + span{xi1 − x1, . . . , xit − x1} = x1 + span{x2 − x1, . . . , xm − x1}x1, xi1 , . . . , xit are affinely independent.(4.1)374.1. Span and affine subspacesOn the other hand,xi1 − x1, . . . , xit − x1 is a basis of span{x2 − x1, . . . , xm − x1}⇐⇒{xi1 − x1, . . . , xit − x1 are linearly independent,span{xi1 − x1, . . . , xit − x1} = span{x2 − x1, . . . , xm − x1}(4.2)By Lemma 4.1.4, we get that (4.1)⇔ (4.2). Hence, the proof is done.Lemma 4.1.6. Let {x1, . . . , xm} ⊆ H and let p ∈ H. Assume x1, . . . , xm areaffinely independent. Assume further that p ∈ aff{x1, . . . , xm}. Then thereexists a unique(α1 · · · αm)ᵀ ∈ Rm with ∑mi=1 αi = 1 such thatp = α1x1 + · · ·+ αmxm.Proof. By Lemma 4.1.2, we know that there exists(α1 · · · αm)ᵀ ∈ Rmwith ∑mi=1 αi = 1 such that p = α1x1 + · · ·+ αmxm.Assume there exists(β1 · · · βm)ᵀ ∈ Rm with ∑mi=1 βi = 1 such thatp = β1x1 + · · ·+ βmxm. Now since ∑mi=1 αi = 1,p = α1x1 + α2x2 + · · ·+ αmxm= (1−m∑i=2αi)x1 + α2x2 + · · ·+ αmxm,which yields thatp− x1 = −m∑i=2αix1 + α2x2 + · · ·+ αmxm= α2(x2 − x1) + · · ·+ αm(xm − x1) (4.3)Similarly, we can also obtainp− x1 = β2(x2 − x1) + · · ·+ βm(xm − x1). (4.4)Using Lemma 4.1.4 and the assumption, we know that the m− 1 vectorsx2 − x1, . . . , xm − x1 are linearly independent. Hence, (4.3) and (4.4) tell usthat(∀i ∈ {2, . . . , m}) αi = βi,which also implies thatα1 = (1−m∑i=2αi) = (1−m∑i=2βi) = β1.Altogether, we obtain(α1 · · · αm)ᵀ=(β1 · · · βm)ᵀ. Therefore theuniqueness is true.384.1. Span and affine subspacesThe following lemma is necessary in the proof of Corollary 4.3.4 below.Lemma 4.1.7. LetO ={(x1, . . . , xm) ∈ Hm∣∣∣ x1, . . . , xm are affinely independent}.Then O is open.Proof. Assume to the contrary that there exist (x1, . . . , xm) ∈ O such that forevery k ∈Nr {0}, there exist (x(k)1 , . . . , x(k)m ) ∈ B((x1, . . . , xm); 1k)such thatx(k)1 , . . . , x(k)m are affinely dependent.By Lemma 4.1.4 for every k, there exists b(k) = (β(k)1 , β(k)2 , . . . , β(k)m−1) ∈Rm−1r {0} such thatβ(k)1 (x(k)2 − x(k)1 ) + · · ·+ β(k)m−1(x(k)m − x(k)1 ) = 0. (4.5)Without loss of generality we assume(∀k ∈Nr {0}) ‖b(k)‖2 =m−1∑i=1(β(k)i )2 = 1, (4.6)and there exists b¯ = (β1, . . . , βm−1) ∈ Rm−1 such thatlimk→∞(β(k)1 , . . . , β(k)m−1) = limk→∞b(k) = b¯ = (β1, . . . , βm−1).Let k go to infinity in (4.6), we get‖b¯‖2 = β21 + · · ·+ β2m−1 = 1,which yields that (β1, . . . , βm−1) 6= 0.Let k go to infinity in (4.5), we obtainβ1(x2 − x1) + · · ·+ βm−1(xm − x1) = 0,which means that x2− x1, . . . , xm− x1 are linearly dependent. By Lemma 4.1.4it contradicts with the assumption that x1, . . . , xm are affinely independent.Hence, O is indeed an open set.Lemma 4.1.8. Let U, V be two linear subspaces inH. Thenspan(U1 ∪U2) = U1 +U2.394.2. Projection and reflectionProof. Let x ∈ H.x ∈ span(U1 ∪U2)⇐⇒ (∃u1 ∈ U1) (∃u2 ∈ U2) (∃a, b ∈ R) x = au1 + bu2⇐⇒ x ∈ U1 +U2.Lemma 4.1.9. Let U, V be two closed linear subspace inH. ThenU⊥ ∩V⊥ = (span(U ∪V))⊥.Proof. Let w ∈ H. Thenw ∈ U⊥ ∩V⊥ ⇐⇒ (∀x ∈ U ∪V) 〈w, x〉 = 0⇐⇒ (∀u ∈ U) (∀v ∈ V) (∀a, b ∈ R) 〈w, au + bv〉 = 0⇐⇒ (∀y ∈ span(U ∪V)) 〈w, y〉 = 0⇐⇒ (∀y ∈ span(U ∪V)) 〈w, y〉 = 0⇐⇒ w ∈ (span(U ∪V))⊥.Hence, the proof is done.4.2 Projection and reflectionLemma 4.2.1. Let U be a nonempty, closed, convex subset inH. ThenRU RU = Id .Proof. Now by Definition 2.3.9,RU RU =(2PU − Id)(2PU − Id)=4PU − 2PU − 2PU + Id (by Fact 2.3.8(i))= IdLemma 4.2.2. Let C be a nonempty, closed, convex subset of H. Then thereflector RC = 2PC − Id is nonexpansive.Proof. The required result follows easily from (i)⇔ (iii) in Fact 2.3.24 andthe Fact 2.3.8(ii).404.2. Projection and reflectionThe following two results are necessary for the proofs in Section 5.2.Proposition 4.2.3. Let p, x, y ∈ H. Denote U := aff{x, y}. Then the follow-ing statements are equivalent:(i) ‖p− x‖ = ‖p− y‖;(ii) 〈p− x, y− x〉 = 12‖y− x‖2;(iii) PU(p) =x+y2 ;(iv) p ∈ x+y2 + (U −U)⊥.Proof. It is clear that‖p− x‖ = ‖p− y‖ ⇐⇒ ‖p− x‖2 = ‖(p− x) + (x− y)‖2⇐⇒ ‖p− x‖2 = ‖p− x‖2 + 2〈p− x, x− y〉+ ‖x− y‖2⇐⇒ 〈p− x, y− x〉 = 12‖y− x‖2.Hence, we get (i)⇔ (ii).Notice x+y2 ∈ U and U = x+ span{y− x}. Using (i)⇔ (iii) in Fact 2.3.13,we obtain thatx + y2= PU(p)⇐⇒(∀u ∈ U)〈p− x + y2, u− x〉= 0⇐⇒(∀α ∈ R)〈p− x + y2,(x + α(y− x))− x〉 = 0⇐⇒〈p− x + y2, y− x〉= 0⇐⇒〈p− (x− x− y2), y− x〉= 0⇐⇒〈p− x, y− x〉+〈 x− y2, y− x〉= 0⇐⇒〈p− x, y− x〉 = 12‖y− x‖2,which imply that (iii)⇔ (ii).On the other hand, by (i)⇔ (ii) in Fact 2.3.13 and by Fact 2.2.10,x + y2= PU(p)⇐⇒p− x + y2 ∈ (U −U)⊥⇐⇒p ∈ x + y2+ (U −U)⊥,which yields that (iii)⇔ (iv).In conclusion, we obtain (i)⇔ (ii)⇔ (iii)⇔ (iv).414.2. Projection and reflectionCorollary 4.2.4. Let x1, . . . , xm be inH. Let p ∈ H. Then‖p− x1‖ = · · · = ‖p− xm−1‖ = ‖p− xm‖⇐⇒〈p− x1, x2 − x1〉 = 12‖x2 − x1‖2...〈p− x1, xm−1 − x1〉 = 12‖xm−1 − x1‖2〈p− x1, xm − x1〉 = 12‖xm − x1‖2.Proof. Set I = {2, . . . , m− 1, m}, and let i ∈ I. In Proposition 4.2.3, substitutex = x1 and y = xi and use (i)⇔ (ii). Then we get ‖p− x1‖ = ‖p− xi‖ ⇐⇒〈p− x1, xi − x1〉 = 12‖xi − x1‖2. Hence,(∀i ∈ I) ‖p− x1‖ = ‖p− xi‖ ⇐⇒ 〈p− x1, xi − x1〉 = 12‖xi − x1‖2.Therefore, the proof is complete.The proposition below plays an essential role in the proofs of one of ourmain theorems Theorem 7.2.5 in Chapter 7.Proposition 4.2.5. Let S be a closed affine subspace of the real Hilbert spaceH. Let x ∈ H be arbitrary but fixed. Then for all s ∈ S we have:(i) Let p be inH. Thenp = PSx ⇐⇒ p ∈ S and (∀u ∈ S)(∀v ∈ S) 〈x− p, u− v〉 = 0;(ii) ‖x− PSx‖2 + ‖s− PSx‖2 = ‖x− s‖2;(iii) ‖x− s‖ = ‖RSx− s‖;(iv) The projector PS and the reflector RS are affine operators.Proof. (i): follows from [6, Corollary 3.22 (i)] or from Fact 2.3.13.(iv): PS is an affine operator, which follows from [6, Corollary 3.22 (ii)].Since RS = 2PS − Id, thus it is clear that RS is affine.(ii): Indeed,‖x− s‖2 = ‖x− PSx− (s− PSx)‖2= ‖x− PSx‖2 − 2〈x− PSx, s− PSx〉+ ‖s− PSx‖2= ‖x− PSx‖2 + ‖s− PSx‖2. (by (i))424.2. Projection and reflection(iii): RS = 2PS − Id implies‖RSx− PSx‖ = ‖PSx− x‖ (4.7)Using (iv), we getPS(RSx) = PS(2PSx− x) = 2PSx− PSx = PSx (4.8)Combining (4.7), (4.8) and the above (ii), we get‖x− PSx‖2 + ‖s− PSx‖2 = ‖RSx− PSx‖2 + ‖s− PSx‖2= ‖RSx− PS(RSx)‖2 + ‖s− PS(RSx)‖2= ‖RSx− s‖2On the other hand, by (ii), ‖x− PSx‖2 + ‖s− PSx‖2 = ‖x− s‖2. Therefore‖x− s‖ = ‖RSx− s‖.In order to prove the second explicit formula of the circumcenter map-ping induced by reflectors in Proposition 7.3.1, the following lemma isnecessary.Lemma 4.2.6. Let x, x1, . . . , xn ∈ H, and x1 − x, . . . , xn − x are linearly inde-pendent. Denote M := aff{x, x1, . . . , xn}. Then for every y ∈ H,PM(y) = x +n∑i=1〈y− x, ei〉ei,where(∀i ∈ {1, . . . , n})ei =xi − x−∑i−1j=1〈xi−x,xj−x〉〈xj−x,xj−x〉 (xj − x)∥∥∥xi − x−∑i−1j=1 〈xi−x,xj−x〉〈xj−x,xj−x〉 (xj − x)∥∥∥ .Proof. Since x1− x, . . . , xn− x are linearly independent, by the Gram-Schmidtorthogonalization process [28, Theorem 9.10], let(∀i ∈ {1, . . . , n}) ei =xi−x−∑i−1j=1〈xi−x,xj−x〉〈xj−x,xj−x〉 (xj−x)∥∥∥xi−x−∑i−1j=1 〈xi−x,xj−x〉〈xj−x,xj−x〉 (xj−x)∥∥∥ , then e1, . . . , en are orthonormal sequence. More-overspan{e1, . . . , en} = span{x1 − x, . . . , xn − x} := L.Since, by Lemma 4.1.2, M = x + L, thus by Fact 2.3.16(i), we knowPM(y) = x + PL(y− x).434.3. Determinant calculationAccording to Fact 2.3.17, we get that for every z ∈ H,PL(z) =n∑i=1〈z, ei〉ei,where (∀i ∈ {1, . . . , n}) ei =xi−x−∑i−1j=1〈xi−x,xj−x〉〈xj−x,xj−x〉 (xj−x)∥∥∥xi−x−∑i−1j=1 〈xi−x,xj−x〉〈xj−x,xj−x〉 (xj−x)∥∥∥ . Hence, for everyy ∈ H,PM(y) = x +n∑i=1〈y− x, ei〉ei,where (∀i ∈ {1, . . . , n}) ei =xi−x−∑i−1j=1〈xi−x,xj−x〉〈xj−x,xj−x〉 (xj−x)∥∥∥xi−x−∑i−1j=1 〈xi−x,xj−x〉〈xj−x,xj−x〉 (xj−x)∥∥∥ .4.3 Determinant calculationLemma 4.3.1. Let A = [aij] = [A∗,1| · · · |A∗,j| · · · |A∗,n] be an n× n matrix,where (j ∈ {1, . . . , n}) A∗,j is the jth column of A. Let b =(b1 · · · bn)ᵀ bean n× 1 column vector. Let j ∈ {1, . . . , n}, adding b to the jth column of A,we get B = [A∗,1| · · · |A∗,j−1|A∗,j + b|A∗,j+1| · · · |A∗,n]. Thendet(B) = det(A) + det([A∗,1| · · · |A∗,j−1|b|A∗,j+1| · · · |A∗,n]).Proof. Let j ∈ {1, . . . , n}. By definition of B, we know that the jth column is(b1 + a1j b2 + a2j · · · bn + anj)ᵀ. By Definition 2.5.4, we know that(∀i ∈ {1, . . . , n}) B˚ij = A˚ij.Using Fact 2.5.13, we knowdet(B) = (a1j + b1)B˚1j + (a2j + b2)B˚2j + · · ·+ (anj + bn)B˚nj= (a1j + b1)A˚1j + (a2j + b2)A˚2j + · · ·+ (anj + bn)A˚nj=n∑i=1aij A˚ij +n∑i=1bi A˚ij= det(A) + det([A∗,1| · · · |A∗,j−1|b|A∗,j+1| · · · |A∗,n]).The last equality follows from the Fact 2.5.13 again.444.3. Determinant calculationLemma 4.3.2. Let A = [aij] be an n× n matrix. Let b =(b1 · · · bn)ᵀ be acolumn vector. DenoteA˜ = [A∗,1 + b| · · · |A∗,j + b| · · · |A∗,n + b],which is the result of adding b to each column of the matrix A. Thendet(A˜) = det(A) +n∑j=1det Aj,where Aj = [A∗,1| · · · |A∗,j−1|b|A∗,j+1| · · · |A∗,n]. That is, Aj is identical to Aexcept that column A∗,j has been replaced by b.Proof. By Lemma 4.3.1,det(A˜) =det([b|A∗,2 + b| · · · |A∗,j + b| · · · |A∗,n + b])+ det([A∗,1|A∗,2 + b| · · · |A∗,j + b| · · · |A∗,n + b]) (4.9)Combining Fact 2.5.14 (ii) and Lemma 4.3.1, we getdet([b|A∗,2 + b| · · · |A∗,j + b| · · · |A∗,n + b])=det([b|A∗,2| · · · |A∗,j| · · · |A∗,n]) = det(A1). (4.10)Using the trick above for the columns 2 to n of the matrix[A∗,1|A∗,2 + b| · · · |A∗,j + b| · · · |A∗,n + b],we get thatdet([A∗,1|A∗,2 + b| · · · |A∗,j + b| · · · |A∗,n + b])=det(A) +n∑j=2det(Aj) (4.11)Combining (4.9), (4.10) with (4.11), we obtain thatdet(A˜) = det(A) +n∑j=1det(Aj).Proposition 4.3.3. Let x1, . . . , xm ∈ H. Then for every k ∈ {2, . . . , m},det(G(x2 − x1, . . . , xm − x1))=det(G(x1 − xk, . . . , xk−1 − xk, xk+1 − xk, . . . , xm − xk))(4.12)454.3. Determinant calculationProof. By Definition 2.5.17, G(x1 − xk, . . . , xk−1 − xk, xk+1 − xk, . . . , xm − xk)is〈x1 − xk, x1 − xk〉 · · · 〈x1 − xk, xk−1 − xk〉 〈x1 − xk, xk+1 − xk〉 · · · 〈x1 − xk, xm − xk〉〈x2 − xk, x1 − xk〉 · · · 〈x2 − xk, xk−1 − xk〉 〈x2 − xk, xk+1 − xk〉 · · · 〈x2 − xk, xm − xk〉... · · · ... ... · · · ...〈xk−1 − xk, x1 − xk〉 · · · 〈xk−1 − xk, xk−1 − xk〉 〈xk−1 − xk, xk+1 − xk〉 · · · 〈xk−1 − xk, xm − xk〉〈xk+1 − xk, x1 − xk〉 · · · 〈xk+1 − xk, xk−1 − xk〉 〈xk+1 − xk, xk+1 − xk〉 · · · 〈xk+1 − xk, xm − xk〉... · · · ... ... · · · ...〈xm − xk, x1 − xk〉 · · · 〈xm − xk, xk−1 − xk〉 〈xm − xk, xk+1 − xk〉 · · · 〈xm − xk, xm − xk〉.(4.13)In (4.13), we perform the following elementary row and column operations:for every i ∈ {2, 3, . . . , m− 1}, subtract the 1st row from the ith row, andthen subtract the 1st column from the ith column. Then multiply 1st rowand 1st column by −1. It follows that the determinant of (4.13) equals thedeterminant of〈xk − x1, xk − x1〉 · · · 〈xk − x1, xk−1 − x1〉 〈xk − x1, xk+1 − x1〉 · · · 〈xk − x1, xm − x1〉〈x2 − x1, xk − x1〉 · · · 〈x2 − x1, xk−1 − x1〉 〈x2 − x1, xk+1 − x1〉 · · · 〈x2 − x1, xm − x1〉... · · · ... ... · · · ...〈xk−1 − x1, xk − x1〉 · · · 〈xk−1 − x1, xk−1 − x1〉 〈xk−1 − x1, xk+1 − x1〉 · · · 〈xk−1 − x1, xm − x1〉〈xk+1 − x1, xk − x1〉 · · · 〈xk+1 − x1, xk−1 − x1〉 〈xk+1 − x1, xk+1 − x1〉 · · · 〈xk+1 − x1, xm − x1〉... · · · ... ... · · · ...〈xm − x1, xk − x1〉 · · · 〈xm − x1, xk−1 − x1〉 〈xm − x1, xk+1 − x1〉 · · · 〈xm − x1, xm − x1〉.(4.14)In (4.14), we interchange ith row and (i+ 1)th successively for i = 1, . . . , k− 2.In addition, we interchange jth column and (j + 1)th column successivelyfor j = 1, . . . , k − 2. Then the resulting matrix is G(x2 − x1, . . . , xm − x1).Because the number of interchange operations we performed is even, thedeterminant is unchanged. Therefore, we obtaindet(G(x1 − xk, . . . , xk−1 − xk, xk+1 − xk, . . . , xm − xk))=det(G(x2 − x1, . . . , xm − x1))as claimed.Corollary 4.3.4. Let {x1, . . . , xm} ⊆ H with x1, . . . , xm being affinely inde-pendent. Let((x(k)1 , . . . , x(k)m ))k∈N ⊆ Hm such thatlimk→∞(x(k)1 , . . . , x(k)m ) = (x1, . . . , xm).ThenG(x2 − x1, . . . , xm − x1)−1 = limk→∞G(x(k)2 − x(k)1 , . . . , x(k)m − x(k)1 )−1.464.4. ConvergenceProof. By Lemma 4.1.7, we know there exists K ∈N such that(∀k ≥ K) x(k)1 , . . . , x(k)m are affinely independent.Using Lemma 4.1.4, we knowx2 − x1, . . . , xm − x1 are linearly independent,and(∀k ≥ K) x(k)2 − x(k)1 , . . . , x(k)m − x(k)1 are linearly independent.Hence, Fact 2.5.18 tells us that G(x2 − x1, . . . , xm − x1)−1 and (∀k ≥ K)G(x(k)2 − x(k)1 , . . . , x(k)m − x(k)1 )−1 exist. Therefore, the required result followsdirectly from Fact 2.5.21.4.4 ConvergenceLemma 4.4.1. LetH be a real Hilbert space and U be a closed linear subspaceinH. Let T ∈ B(H), that is, T is a bounded linear operator fromH intoH.Then if U = {0}, ‖TPU‖ = 0. Otherwise‖TPU‖ = supy∈U‖y‖≤1‖Ty‖ = supy∈U‖y‖=1‖Ty‖.Proof. If U = {0}, then by Fact 2.3.4, ‖TPU‖ = sup x∈Rn‖x‖≤1‖TPUx‖ = 0.Now assume U 6= {0}. Since D(TPU) = H, by Fact 2.3.4 and easyalgebraic calculation,‖TPU‖ = supx∈H‖x‖≤1‖TPUx‖ y=PU x= supy∈U‖y‖≤1‖Ty‖. (4.15)Moreover, since U 6= {0},supy∈U‖y‖=1‖Ty‖ ≤ supy∈U‖y‖≤1‖Ty‖1‖y‖≥1≤ supy∈U‖y‖≤1,y 6=0‖Ty‖‖y‖ ≤ supy∈Uy 6=0‖T y‖y‖‖ ≤ supy∈U‖y‖=1‖Ty‖.(4.16)474.4. ConvergenceCombining (4.15) and (4.16), we obtain‖TPU‖ = supy∈U‖y‖≤1‖Ty‖ = supy∈U‖y‖=1‖Ty‖.Proposition 4.4.2. AssumeH = Rn. Let T : Rn → Rn be linear and firmlynonexpansive, i.e.(∀x ∈ Rn) (∀x ∈ Rn) ‖Tx− Ty‖2 + ‖(Id−T)x− (Id−T)y‖2 ≤ ‖x− y‖2.(4.17)Then the following statements hold.(i) ‖TP(Fix T)⊥‖ < 1.(ii) For every x ∈ Rn, Tnx converges linearly to PFix Tx with linear rate‖TP(Fix T)⊥‖.(iii) (Tn)n∈N converges ‖TP(Fix T)⊥‖-linearly to PFix T in B(Rn).Proof. If (Fix T)⊥ = {0}, then Fix T = Rn. So T = PFix T = Id, and‖TP(Fix T)⊥‖ = 0. Hence, the required results in (i), (ii) and (iii) aboveare trivial.Assume (Fix T)⊥ 6= {0}. Because T : Rn → Rn is firmly nonexpansive,by Remark 2.3.22, T is nonexpansive, and so T is continuous. Combiningwith the linearity of T, we know Fix T is a closed linear subspace of Rn, sois the (Fix T)⊥.(i): Substitute the closed linear subspace U in Lemma 4.4.1 by our(Fix T)⊥ to get‖TP(Fix T)⊥‖ = supy∈(Fix T)⊥‖y‖=1‖Ty‖. (4.18)Using Fact 2.3.8(ii), we know P(Fix T)⊥ is firmly nonexpansive. So ‖TP(Fix T)⊥‖ ≤1. Assume ‖TP(Fix T)⊥‖ = 1. By (4.18), it means that there exist (yk)k∈N ⊆(Fix T)⊥ with (∀k ∈N) ‖yk‖ = 1 such that1 = ‖TP(Fix T)⊥‖ = supy∈(Fix T)⊥‖y‖=1‖Ty‖ = limk→∞‖Tyk‖. (4.19)484.4. ConvergenceBy Bolzano-Weierstrass Theorem, without loss of generality, we can saythat there is a point y¯ ∈ (Fix T)⊥ with ‖y¯‖ = 1 such that limk→∞ yk = y¯(if necessary, we can take a subsequence of (yk)k∈N). For every x ∈ Rn,substituting y = PFix Tx in (4.17), we get,‖Tx− PFix Tx‖2 + ‖x− Tx‖2 ≤ ‖x− PFix Tx‖2,which implies that(∀x 6∈ Fix T) ‖Tx− PFix Tx‖ < ‖x− PFix Tx‖. (4.20)Now y¯ ∈ (Fix T)⊥ and ‖y¯‖ = 1, so y¯ 6∈ Fix T. By Fact 2.3.10(iv), y¯ ∈ (Fix T)⊥implies that PFix T(y¯) = 0, thus substituting x = y¯ in (4.20), we obtain,‖Ty¯‖ = ‖Ty¯− PFix T y¯‖ < ‖y¯− PFix T y¯‖ = ‖y¯‖ = 1.But on the other hand, since T is continuous,‖Ty¯‖ = ‖T limk→∞yk‖ = limk→∞‖Tyk‖ (4.19)= 1.It is a contradiction. Therefore, ‖TP(Fix T)⊥‖ < 1(ii): The desired result follows from Fact 3.2.12 and (i) above.(iii): The last result follows from the above (ii) and Fact 2.4.9.Corollary 4.4.3. Assume H = Rn. Let (i ∈ I) Ti : Rn → Rn be a finitefamily of linear and firmly nonexpansive operators such that ∩i∈I Fix Ti 6= ∅,and let (ωi)i∈I be strictly positive real numbers such that ∑i∈I ωi = 1. letT = ∑i∈I ωiTi. Then ‖TP(Fix T)⊥‖ < 1.In consequence, for every x ∈ H, Tkx converges linearly to PFix Tx withlinear rate ‖TP(Fix T)⊥‖. Moreover, (Tk)k∈N converges ‖TP(Fix T)⊥‖-linearlyto PFix T in B(H).Proof. The corollary follows directly from Fact 2.3.29 and Proposition 4.4.2.49Chapter 5Circumcenter5.1 OverviewRecall thatH is a real Hilbert space, andP(H) is the set of all nonempty subsets ofH with finitely many elements.Throughout this chapter, we assumeS = {x1, x2, . . . , xm} ∈ P(H).The goal of this chapter is to provide a systematic study of the circum-center of S, i.e., of the (unique if it exists) point in the affine hull of S that isequidistant to all points in S. We summarize our main results in this chapteras follows:• We define the circumcenter operator in Definition 5.2.2 and provideexplicit formulae (symmetric and nonsymmetric) for the circumcenteroperator in Sections 5.4, 5.5 and 5.7.• Based on the definition, we provide two basic properties of the circum-center in Section 5.3.• We explore the behaviour of the circumcenter operator when se-quences of sets are considered (see Section 5.6).• A characterization of the existence of circumcenter of three pairwisedistinct points is given in Theorem 5.7.1.• The circumcenter and circumradius in R3 are expressed by using thecross product in Theorem 5.8.1.5.2 The circumcenter operatorBefore we are able to define the main actor in this chapter, the circum-center operator, we shall require the following proposition.505.2. The circumcenter operatorProposition 5.2.1. Recall S = {x1, x2, . . . , xm}, where m ∈ Nr {0} andx1, x2, . . . , xm are inH. Then there is at most one point p ∈ H satisfying thefollowing two conditions:(i) p ∈ aff(S), and(ii) {‖p − s‖ | s ∈ S} is a singleton: ‖p − x1‖ = ‖p − x2‖ = · · · =‖p− xm‖.Proof. Assume both of p, q satisfy conditions (i) and (ii).By assumption and Lemma 4.1.2, p, q ∈ aff(S) = aff{x1, . . . , xm} =x1 + span{x2 − x1, . . . , xm − x1}. Thus p− q ∈ span{x2 − x1, . . . , xm − x1},and so there exist α1, . . . , αm−1 ∈ R such that p− q = ∑m−1i=1 αi(xi+1 − x1).Using the Corollary 4.2.4 and using the condition (ii) satisfied by both p andq, we observe that for every i ∈ I = {2, . . . , m}, we have〈p− x1, xi − x1〉 = 12‖xi − x1‖2 and〈q− x1, xi − x1〉 = 12‖xi − x1‖2.Subtracting the above equalities, we get(∀i ∈ I) 〈p− q, xi − x1〉 = 0.Multiplying αi on both sides of the corresponding ith equality and thensumming up the m− 1 equalities, we get0 =〈p− q,m−1∑i=1αi(xi+1 − x1)〉= 〈p− q, p− q〉 = ‖p− q‖2.Hence, p = q, which implies that if such point satisfying conditions (i) and(ii) exists, then it must be unique.We are now in a position to define the circumcenter operator.Definition 5.2.2. (circumcenter operator) Define the circumcenter operatorCC : P(H)→ H∪ {∅} asS 7→{p, if p ∈ aff(S) and {‖p− s‖ | s ∈ S} is a singleton;∅, otherwise.The circumradius operator isCR : P(H)→ R : S 7→{‖CC(S)− s‖, if CC(S) ∈ H and s ∈ S,+∞, if CC(S) = ∅.515.3. Basic properties of the circumcenter operatorIn particular, when CC(S) ∈ H, that is, CC(S) 6= ∅, we say that the circum-center of S exists and we call CC(S) the circumcenter of S and CR(S) thecircumradius of S.Note that in the Proposition 5.2.1 above, we have already proved thatfor every S ∈ P(H), there is at most one point p ∈ aff(S) such that {‖p−s‖ | s ∈ S} is a singleton, so the notions are well-defined. Hence, we obtainthe following alternative expression of the circumcenter operator:Remark 5.2.3. Let S ∈ P(H). Then the CC(S) is either∅ or the unique pointp ∈ H such that(i) p ∈ aff(S) and,(ii) {‖p− s‖ | s ∈ S} is a singleton.Corollary 5.2.4. Denote the cardinality of the set S as l. Suppose l ≤ 2, sayS = {x, y}. ThenCC(S) =x + y2.Proof. Assume l = 1, that is, x = x1 = · · · = xm. In particular, y = x. Thenby Definition 5.2.2, CC(S) = x. Since x = x+y2 , thus CC(S) =x+y2 .Assume l = 2. Since S = {x, y}, by Definition 5.2.2 again, we getCC(S) = x+y2 . Hence, the proof is done.5.3 Basic properties of the circumcenter operatorIn this section, based on the Definition 5.2.2, we deduce some fundamen-tal properties of the circumcenter operator. Recall thatm ∈Nr {0}, x1, . . . , xm are points inH, and S = {x1, . . . , xm}.Proposition 5.3.1. (scalar multiples) Let λ ∈ Rr {0}. Then CC(λS) =λCC(S).Proof. Let p ∈ H. By Definition 5.2.2,p = CC(S)⇐⇒{p ∈ aff(S){‖p− s‖ | s ∈ S} is a singleton⇐⇒{λp ∈ aff(λS){‖λp− λs‖ | λs ∈ λS} is a singleton⇐⇒ λp = CC(λS),525.4. Explicit formulae for the circumcenter operatorand the result follows.The example below illustrates why we had to exclude the case λ = 0 inProposition 5.3.1.Example 5.3.2. Suppose thatH = R and that S = {0,−1, 1}. ThenCC(0 · S) = {0} 6= ∅ = 0 · CC(S).Proposition 5.3.3. translations Let y ∈ H. Then CC(S + y) = CC(S) + y.Proof. Let p ∈ H. By Lemma 4.1.2,p ∈ aff{x1, . . . , xm}⇐⇒(∃ λ1, . . . ,λm ∈ Rwithm∑i=1λi = 1) p =m∑i=1λixi⇐⇒(∃ λ1, . . . ,λm ∈ Rwithm∑i=1λi = 1) p + y =m∑i=1λi(xi + y)⇐⇒p + y ∈ aff{x1 + y, . . . , xm + y},that isp ∈ aff(S)⇐⇒ p + y ∈ aff(S + y). (5.1)By (5.1) and Remark 5.2.3, we havep = CC(S) ∈ H ⇐⇒{p ∈ aff(S){‖p− s‖ | s ∈ S} is a singleton⇐⇒{p + y ∈ aff(S + y){‖(p + y)− (s + y)‖ | s + y ∈ S + y} is a singleton⇐⇒ p + y = CC(S + y) ∈ H.Moreover, because ∅ = ∅+ y, the proof is complete.5.4 Explicit formulae for the circumcenter operatorWe continue to assume thatm ∈Nr {0}, x1, . . . , xm are points inH, and S = {x1, . . . , xm}.535.4. Explicit formulae for the circumcenter operatorIf S is a singleton, say S = {x1}, then, by Definition 5.2.2, we clearly haveCC(S) = x1. So in this section, to deduce the formula of CC(S), we alwaysassume thatm ≥ 2.We are ready for an explicit formula for the circumcenter operator.Theorem 5.4.1. Suppose that x1, . . . , xm are affinely independent. ThenCC(S) ∈ H, which means that CC(S) is the unique point satisfying thefollowing two conditions:(i) CC(S) ∈ aff(S), and(ii) {‖CC(S)− s‖ | s ∈ S} is a singleton.Moreover,CC(S)=x1 +12(x2 − x1, . . . , xm − x1)G(x2 − x1, . . . , xm − x1)−1‖x2 − x1‖2...‖xm − x1‖2 ,where G(x2 − x1, . . . , xm−1 − x1, xm − x1) is the Gram matrix that is definedin Definition 2.5.17:G(x2 − x1, . . . , xm−1 − x1, xm − x1)=‖x2 − x1‖2 〈x2 − x1, x3 − x1〉 · · · 〈x2 − x1, xm − x1〉.........〈xm−1 − x1, x2 − x1〉 〈xm−1 − x1, x3 − x1〉 · · · 〈xm−1 − x1, xm − x1〉〈xm − x1, x2 − x1〉 〈xm − x1, x3 − x1〉 · · · ‖xm − x1‖2 .Proof. By assumption and Lemma 4.1.4, we get that x2 − x1, . . . , xm − x1are linearly independent. Then by Fact 2.5.18, the Gram matrix G(x2 − x1,x3 − x1, . . . , xm − x1) is invertible. Setα1α2...αm−1 = 12 G(x2 − x1, x3 − x1, . . . , xm − x1)−1‖x2 − x1‖2‖x3 − x1‖2...‖xm − x1‖2 ,andp = x1 + α1(x2 − x1) + α2(x3 − x1) + · · ·+ αm−1(xm − x1).545.4. Explicit formulae for the circumcenter operatorBy the definition of G(x2 − x1, x3 − x1, . . . , xm − x1) and by the definitionsof(α1 α2 · · · αm−1)ᵀ and p, we obtain the equivalencesG(x2 − x1, x3 − x1, . . . , xm − x1)α1α2...αm−1 = 12‖x2 − x1‖2‖x3 − x1‖2...‖xm − x1‖2⇐⇒〈α1(x2 − x1) + · · ·+ αm−1(xm − x1), x2 − x1〉 = 12‖x2 − x1‖2...〈α1(x2 − x1) + · · ·+ αm−1(xm − x1), xm − x1〉 = 12‖xm − x1‖2⇐⇒〈p− x1, x2 − x1〉 = 12‖x2 − x1‖2...〈p− x1, xm − x1〉 = 12‖xm − x1‖2.Hence, by Corollary 4.2.4, we know that p satisfies condition (ii). In addition,it is clear that p = x1 + α1(x2 − x1) + α2(x3 − x1) + · · ·+ αm−1(xm − x1) ∈x1 + span{x2 − x1, . . . , xm − x1} = aff(S), which is just the condition (i).Hence, the point satisfying conditions (i) and (ii) exists.Moreover, by Proposition 5.2.1, if the point exists, then it must be unique.Proposition 5.4.2. Suppose that CC(S) ∈ H, and let K ⊆ S such thataff(K) = aff(S). ThenCC(S) = CC(K).Proof. By assumption, CC(S) ∈ H, that is:(i) CC(S) ∈ aff(S), and(ii) {‖CC(S)− s‖ | s ∈ S} is a singleton.Because K ⊆ S, we get {‖CC(S)− s‖ | s ∈ K} is a singleton, by (ii). Sinceaff(K) = aff(S), by (i), the point CC(S) satisfy(I) CC(S) ∈ aff(K), and(II) {‖CC(S)− u‖ | u ∈ K} is a singleton.Replacing S in Proposition 5.2.1 by K and combining with Definition 5.2.2,we know CC(K) = CC(S).555.4. Explicit formulae for the circumcenter operatorCorollary 5.4.3. Suppose that CC(S) ∈ H. Let xi1 , . . . , xit be elements of Ssuch that x1, xi1 , . . . , xit are affinely independent, and set K = {x1, xi1 , . . . ,xit}. Furthermore, assume that aff(K) = aff(S). ThenCC(S) = CC(K)=x1 +12(xi1 − x1, . . . , xit − x1)G(xi1 − x1, . . . , xit − x1)−1‖xi1 − x1‖2...‖xit − x1‖2 .Proof. By Theorem 5.4.1, x1, xi1 , . . . , xit are affinely independent implies thatCC(K) 6= ∅, andCC(K)=x1 +12(xi1 − x1, . . . , xit − x1)G(xi1 − x1, . . . , xit − x1)−1‖xi1 − x1‖2...‖xit − x1‖2 .Then the desired result follows from Proposition 5.4.2.By Proposition 4.1.5, in all numerical experiments in the following Chap-ter 11, we always choose a basis of span{x1 − x, . . . , xm − x} to act as therole of K in the Corollary 5.4.3 to give an explicit formula of the CC(S).Definition 5.4.4. Let S1, S2 ∈ P(H). We say S2 is a maximally pairwisedistinct subset of S1, if(i) S2 ⊆ S1;(ii) elements in S2 are pairwise distinct;(iii) for every x ∈ S1, there is y ∈ S2 such that x = y.For example, in R, {2, 0, 1, 8} is the maximally pairwise distinct subsetof {2, 0, 1, 8, 2, 0, 1, 8}.Corollary 5.4.5. Let K be the maximally pairwise distinct subset of S. ThenCC(S) = CC(K).Proof. The required result directly follows from the Definition 5.2.2.565.5. Additional formulae for the circumcenter operator5.5 Additional formulae for the circumcenteroperatorUpholding the assumptions of Section 5.4, we assume additionally thatx1, . . . , xm are affinely independent.By Theorem 5.4.1, CC(S) ∈ H. Letk ∈ {2, 3, . . . , m} be arbitrary but fixed.By Theorem 5.4.1 again, we know thatCC(S) = x1 + α1(x2 − x1) + α2(x3 − x1) + · · ·+ αm−1(xm − x1)=(1−∑m−1i=1 αi)x1 + α1x2 + · · ·+ αm−1xm, (5.2)whereα1α2...αm−1 = 12 G(x2 − x1, x3 − x1, . . . , xm − x1)−1‖x2 − x1‖2‖x3 − x1‖2...‖xm − x1‖2 . (5.3)By the symmetry of the positions of the points x1, . . . , xk, . . . , xm in S inDefinition 5.2.2 and by Proposition 5.2.1, we also getCC(S) =xk + β1(x1 − xk) + · · ·+ βk−1(xk−1 − xk) + βk(xk+1 − xk)+ · · ·+ βm−1(xm − xk)=β1x1 + · · ·+ βk−1xk−1 + (1−∑m−1i=1 βi)xk + βkxk+1 + · · ·+ βm−1xm,(5.4)where(β1 β2 · · · βm−1)ᵀ=12G(x1 − xk, . . . , xk−1 − xk, xk+1 − xk, . . . , xm − xk)−1‖x1 − xk‖2...‖xk−1 − xk‖2‖xk+1 − xk‖2...‖xm − xk‖2.(5.5)575.5. Additional formulae for the circumcenter operatorProposition 5.5.1. The following equalities hold:(1−∑m−1i=1 αi)= β1, (coefficient of x1) (5.6)αk−1 =(1−∑m−1i=1 βi), (coefficient of xk) (5.7)(∀i ∈ {2, . . . , k− 1}) αi−1 = βi and (∀j ∈ {k, k + 1, . . . , m− 1}) αj = β j.(5.8)Proof. Recall that at the beginning of this section we assumed x1, . . . , xm areaffinely independent. Combining the equations (5.2), (5.4) and Lemma 4.1.6,we get the required results.To simplify the statements, we use the following abbreviations.A = G(x2 − x1, . . . , xk − x1, . . . , xm − x1),B = G(x1 − xk, . . . , xk−1 − xk, xk+1 − xk, . . . , xm − xk),and the determinant of matrix A (by Proposition 4.3.3, it is also the determi-nant of matrix B) is denoted by:δ = det(A).We denote the two column vectors a, b respectively by:a =(‖x2 − x1‖2 · · · ‖xk − x1‖2 · · · ‖xm − x1‖2)ᵀ ,b =(‖x1 − xk‖2 · · · ‖xk−1 − xk‖2 ‖xk+1 − xk‖2 · · · ‖xm − xk‖2)ᵀ .Recall that for every M ∈ Rn×n, and for every j ∈ {1, 2, . . . , n},we denote the jth column of the matrix M as M∗,j.In turn, for every i ∈ {1, . . . , m− 1},Ai = [A∗,1| · · · |A∗,i−1|a|A∗,i+1| · · · |A∗,m−1],andBi = [B∗,1| · · · |B∗,i−1|b|B∗,i+1| · · · |B∗,m−1].That is, Ai is identical to A except that column A∗,i has been replaced by aand Bi is identical to B except that column B∗,i has been replaced by b.Lemma 5.5.2. The following statements hold:585.5. Additional formulae for the circumcenter operator(i)(α1 · · · αm−1)ᵀ defined in (5.3) is the unique solution of the nonsingularsystem Ay = 12 a where y is the unknown variable. In consequence, forevery i ∈ {1, . . . , m− 1},αi =det(Ai)2δ;(ii)(β1 · · · βm−1)ᵀ defined in (5.5) is the unique solution of the nonsingularsystem By = 12 b where y is the unknown variable. In consequence, forevery i ∈ {1, . . . , m− 1},βi =det(Bi)2δ.Proof. By assumption, x1, . . . , xm are affinely independent, and by Proposi-tion 4.3.3, we know det(B) = det(A) = δ 6= 0.(i): By definition of(α1 · · · αm−1)ᵀ,(α1 · · · αm−1)ᵀ=12A−1a.Clearly, we know it is the unique solution of the nonsingular system Ay =12 a. Hence, the desired result follows directly from the Fact 2.5.16, the CramerRule.(ii): Using the same method of proof of (i), we can prove (ii) .Using Theorem 5.4.1, Lemma 5.5.2 and the equalities (5.6), (5.7) and (5.8),we readily obtain the following result.Corollary 5.5.3. Suppose that x1, . . . , xm are affinely independent. ThenCC(S) =(1−∑m−1i=1 αi)x1 + α1x2 + · · ·+ αm−1xm,where (∀i ∈ {1, . . . , m− 1}) αi = 12δ det(Ai). Moreover,1−m−1∑i=1αi =12δdet(B1), αk−1 = 1−m−1∑i=112δdet(Bi),(∀i ∈ {2, . . . , k− 1}) αi−1 = 12δ det(Bi), and(∀j ∈ {k, k + 1, . . . , m− 1}) αj = 12δ det(Bj).595.6. Circumcenters of sequences of sets5.6 Circumcenters of sequences of setsWe uphold the assumption thatm ∈Nr {0}, x1, . . . , xm are points inH, and S = {x1, . . . , xm}.In this section, we explore the convergence of the circumcenter operatorover a sequence of sets.Theorem 5.6.1. Suppose that CC(S) ∈ H. Then the following hold:(i) Set t = dim(span{x2 − x1, . . . , xm − x1}), and let S˜ = {x1, xi1 , . . . ,xit} ⊆ S such that xi1 − x1, . . . , xit − x1 is a basis of span{x2 − x1,. . . , xm − x1}. Furthermore, let((x(k)1 , x(k)i1, . . . , x(k)it ))k≥1⊆ Ht+1 withlimk→∞(x(k)1 , x(k)i1, . . . , x(k)it ) = (x1, xi1 , . . . , xit), and set (∀k ≥ 1) S˜(k) ={x(k)1 , x(k)i1 , . . . , x(k)it }. Then there exist N ∈N such that for every k ≥ N,CC(S˜(k)) ∈ H andlimk→∞CC(S˜(k)) = CC(S˜) = CC(S).(ii) Suppose that x1, . . . , xm are affinely independent. Assume that((x(k)1 ,. . . , x(k)m ))k≥1⊆ Hm satisfy limk→∞(x(k)1 , . . . , x(k)m ) = (x1, . . . , xm). Set(∀k ≥ 1) S(k) = {x(k)1 , . . . , x(k)m }. Thenlimk→∞CC(S(k)) = CC(S).Proof. (i): Let l be the cardinality of the set S. Assume first that l = 1. Thent = 0, and S = S˜ = {x1}. Let (x(k)1 )k≥1 ⊆ H satisfy limk→∞ x(k)1 = x1. ByDefinition 5.2.2, we know CC({x(k)1 }) = x(k)1 and CC({x1}) = x1. Hence,limk→∞CC(S˜(k)) = limk→∞x(k)1 = x1 = CC(S).Now assume that l ≥ 2. By Corollary 5.4.3 and Proposition 4.1.5, we obtainCC(S) = CC(S˜)=x1 +12(xi1 − x1, . . . , xit − x1)G(xi1 − x1, . . . , xit − x1)−1‖xi1 − x1‖2...‖xit − x1‖2 .(5.9)605.6. Circumcenters of sequences of setsUsing the assumptions and the Lemma 4.1.7, we know that there existsN ∈N such that(∀k ≥ N) x(k)1 , x(k)i1 , . . . , x(k)it are affinely independent.By Theorem 5.4.1, we know (k ≥ N) CC(S˜(k)) ∈ H. Moreover, for everyk ≥ N,CC(S˜(k)) = x(k)1 +12(x(k)i1 − x(k)1 , . . . , x(k)it − x(k)1 )G(x(k)i1− x(k)1 , . . . , x(k)it − x(k)1 )−1‖x(k)i1 − x(k)1 ‖2...‖x(k)it − x(k)1 ‖2.(5.10)Comparing (5.9) with (5.10) and using Corollary 4.3.4, we obtainlimk→∞CC(S˜(k)) = CC(S˜) = CC(S).(ii): Let x1, . . . , xm ∈ H be affinely independent. Then t = m− 1 andS˜ = S. Substitute the S˜ and S˜(k) in part (i) by our S and S(k) respectively.Then we obtainlimk→∞CC(S(k)) = CC(S)and the proof is complete.Corollary 5.6.2. The mappingΨ : Hm → H∪ {∅} : (x1, . . . , xm) 7→ CC({x1, . . . , xm}).is continuous at every point (x1, . . . , xm) ∈ Hm where x1, . . . , xm are affinelyindependent.Proof. This follows directly from Theorem 5.6.1(ii).Let us record the doubleton case explicitly.Proposition 5.6.3. Suppose that m = 2. Let((x(k)1 , x(k)2 ))k≥1 ⊆ H2 satisfylimk→∞(x(k)1 , x(k)2 ) = (x1, x2). Thenlimk→∞CC({x(k)1 , x(k)2 }) = CC({x1, x2}).615.6. Circumcenters of sequences of setsProof. Indeed, we deduce from Corollary 5.2.4 thatlimk→∞CC({x(k)1 , x(k)2 }) = limk→∞x(k)1 + x(k)22=x1 + x22= CC({x1, x2})and the result follows.The following example illustrates that the assumption that “m = 2” inProposition 5.6.3 cannot be replaced by “the cardinality of S is 2”.Example 5.6.4. Suppose that H = R2, that m = 3, and that S = {x1, x2,x3} with x1 = (−1, 0), x2 = x3 = (1, 0). Then there exists ((x(k)1 , x(k)2 ,x(k)3 ))k≥1 ⊆ H3 with limk→∞(x(k)1 , x(k)2 , x(k)3 ) = (x1, x2, x3) such thatlimk→∞CC({x(k)1 , x(k)2 , x(k)3 }) 6= CC(S).Proof. For every k ≥ 1, let (x(k)1 , x(k)2 , x(k)3 ) =((−1, 0), (1, 0), (1 + 1k , 0))∈H3. Then limk→∞(x(k)1 , x(k)2 , x(k)3 ) = (x1, x2, x3). By Definition 5.2.2, weknow that (∀k ≥ 1), CC(S(k)) = ∅, since there is no point in R2 which isequdistant to all of the three points. On the other hand, by Definition 5.2.2again, we know CC(S) = (0, 0) ∈ H. Hence, limk→∞ CC({x(k)1 , x(k)2 , x(k)3 }) =∅ 6= (0, 0) = CC(S).The following question now naturally arises:Question 5.6.5. Suppose that CC({x1, x2, x3}) ∈ H, and let((x(k)1 , x(k)2 ,x(k)3 ))k≥1 ⊆ H3 be such that limk→∞(x(k)1 , x(k)2 , x(k)3 ) = (x1, x2, x3). Is it truethat the implication(∀k ≥ 1) CC({x(k)1 , x(k)2 , x(k)3 }) ∈ H=⇒ limk→∞CC({x(k)1 , x(k)2 , x(k)3 }) = CC({x1, x2, x3})holds?When x1, x2, x3 are affinely independent, then Theorem 5.6.1(ii) gives usan affirmative answer. However, the answer is negative if x1, x2, x3 are notassumed to be affinely independent.Example 5.6.6. Suppose that H = R2 and S = {x1, x2, x3} with x1 = (−2,0), x2 = x3 = (2, 0). Then there exists a sequence((x(k)1 , x(k)2 , x(k)3 ))k≥1 ⊆ H3such that625.6. Circumcenters of sequences of sets(i) limk→∞(x(k)1 , x(k)2 , x(k)3 ) = (x1, x2, x3),(ii) (∀k ≥ 1) CC({x(k)1 , x(k)2 , x(k)3 }) ∈ R2, and(iii) limk→∞ CC({x(k)1 , x(k)2 , x(k)3 }) 6= CC(S).Proof. By Definition 5.2.2, we know that CC(S) = (0, 0) ∈ H. Set(∀k ≥ 1) S(k) = {x(k)1 , x(k)2 , x(k)3 } ={(−2, 0), (2, 0), (2− 1k , 14k)}.(i): In this case,limk→∞(x(k)1 , x(k)2 , x(k)3 ) = limk→∞((−2, 0), (2, 0), (2− 1k , 14k))=((−2, 0), (2, 0), (2, 0))=(x1, x2, x3).(ii): It is clear that for every k ≥ 1, the vectors (−2, 0), (2, 0), (2− 1k , 14k ) arenot collinear, that is, (−2, 0), (2, 0), (2− 1k , 14k ) are affinely independent. ByTheorem 5.4.1, we see that(∀k ≥ 1) CC({x(k)1 , x(k)2 , x(k)3 }) ∈ R2.(iii): Let k ≥ 1. By definition of CC(S(k)) and (ii), we deduce that CC(S(k)) =(p(k)1 , p(k)2 ) ∈ R2 and that‖CC(S(k))− x(k)1 ‖ = ‖CC(S(k))− x(k)2 ‖ = ‖CC(S(k))− x(k)3 ‖.Because CC(S(k)) must be in the intersection of the perpendicular bisectorof x(k)1 = (−2, 0), x(k)2 = (2, 0) and the perpendicular bisector of x(k)2 = (2,0), x(k)3 = (2− 1k , 14k ), we obtainp(k)1 = 0 and p(k)2 = 4(p(k)1 −2+ 2− 1k2)+ 18k ;thus,CC(S(k)) = (p(k)1 , p(k)2 ) =(0,−8+ 2k + 18k). (5.11)(Alternatively, we can use the formula in Theorem 5.4.1 to get (5.11)). There-fore,limk→∞CC(S(k)) = limk→∞(0,−8+ 2k + 18k)= (0,−8) 6= (0, 0) = CC(S),635.7. The circumcenter of three pointsand the proof is complete.As the picture below shows, (∀k ≥ 1) x(k)3 = (2− 1k , 14k ) converges tox3 = (2, 0) along the purple line L = {(x, y) ∈ R2 | y = − 14 (x − 2)}. Infact, CC(S(k)) is just the intersection point between the two lines M1 andM2, where M1 is the perpendicular bisector between the points x1 and x2,and M2 is the perpendicular bisector between the points x(k)3 and x2.Figure 5.1: Continuity of circumcenter operator may fail even when (∀k ≥ 1) CC(S(k)) ∈ H.5.7 The circumcenter of three pointsIn this section, we study the circumcenter of a set containing three points.We will give a characterization of the existence of circumcenter of threepairwise distinct points. In addition, we shall provide asymmetric andsymmetric formulae.Theorem 5.7.1. Suppose that S = {x, y, z} ∈ P(H) and that l = 3 is thecardinality of S. Then x, y, z are affinely independent if and only if CC(S) ∈H.Proof. If S is affinely independent, then CC(S) ∈ H by Theorem 5.4.1.645.7. The circumcenter of three pointsTo prove the converse implication, suppose that CC(S) ∈ H, i.e.,(i) CC(S) ∈ aff{x, y, z} , and(ii) ‖CC(S)− x‖ = ‖CC(S)− y‖ = ‖CC(S)− z‖.We argue by contradiction and thus assume that the elements of S are affinelydependent:dim(span{S− x}) = dim(span{y− x, z− x}) ≤ 1.Note that y− x 6= 0 and z− x 6= 0. SetU = x + span{y− x, z− x} = x + span{y− x} = x + span{z− x}.Combining with Lemma 4.1.2, we getU = aff{x, y, z} = aff{x, y} = aff{x, z}. (5.12)By definition of CC(S), we haveCC(S) ∈ aff{x, y} (5.12)= U and ‖CC(S)− x‖ = ‖CC(S)− y‖, (5.13)andCC(S) ∈ aff{x, z} (5.12)= U and ‖CC(S)− x‖ = ‖CC(S)− z‖. (5.14)Now using (i)⇔ (iii) in Proposition 4.2.3 and using (5.13), we getCC(S) = PU(CC(S))=x + y2.Similarly, using (i)⇔ (iii) in Proposition 4.2.3 and using (5.14), we can alsogetCC(S) = PU(CC(S))=x + z2.Therefore,x + y2= CC(S) =x + z2=⇒ y = z,which contradicts the assumption that l = 3. The proof is complete.655.7. The circumcenter of three pointsIn contrast, when the cardinality of S is 4, thenCC(S) ∈ H 6⇒ elements of S are affinely independentas the following example demonstrates. Thus, the characterization of theexistence of circumcenter in Theorem 5.7.1 is generally not true when weconsider l ≥ 3 pairwise distinct points.Example 5.7.2. Suppose that H = R2, that m = 4, and S = {x1, x2, x3, x4},where x1 = (0, 0), x2 = (4, 0), x3 = (0, 4), and x4 = (4, 4) (see Figure 5.2).Then x1, x2, x3, x4 are pairwise distinct and affinely dependent, yet CC(S) =(2, 2).Figure 5.2: Circumcenter of the four affinely dependent points from Example 5.7.2.In the Theorem 5.4.1 above, where we presented formula for CC(S), wegave special importance to the first point x1 in S. We now provide somelonger yet symmetric formulae for CC(S).Remark 5.7.3. Suppose that S = {x, y, z} and that l = 3 is the cardinality ofS. Assume furthermore that CC(S) ∈ H, i.e., there is an unique point CC(S)satisfying(i) CC(S) ∈ aff{x, y, z} and(ii) ‖CC(S)− x‖ = ‖CC(S)− y‖ = ‖CC(S)− z‖.665.7. The circumcenter of three pointsBy Theorem 5.7.1, the points x, y, z must be affinely independent. FromTheorem 5.4.1 we obtainCC(S) = x +12(y− x, z− x)( ‖y− x‖2 〈y− x, z− x〉〈z− x, y− x〉 ‖z− x‖2)−1 (‖y− x‖2‖z− x‖2)= x +(‖y− x‖2‖z− x‖2 − ‖z− x‖2〈y− x, z− x〉)(y− x)2(‖y− x‖2‖z− x‖2 − 〈y− x, z− x〉2)+(‖y− x‖2‖z− x‖2 − ‖y− x‖2〈y− x, z− x〉)(z− x)2(‖y− x‖2‖z− x‖2 − 〈y− x, z− x〉2)=1K1(‖y− z‖2〈x− z, x− y〉x + ‖x− z‖2〈y− z, y− x〉y+ ‖x− y‖2〈z− x, z− y〉z),where K1 = 2(‖y− x‖2‖z− x‖2 − 〈y− x, z− x〉2).Similarly,CC(S) =1K2(‖y− z‖2〈x− z, x− y〉x + ‖x− z‖2〈y− z, y− x〉y+ ‖x− y‖2〈z− x, z− y〉z),where K2 = 2(‖x− y‖2‖z− y‖2 − 〈x− y, z− y〉2) andCC(S) =1K3(‖y− z‖2〈x− z, x− y〉x + ‖x− z‖2〈y− z, y− x〉y+ ‖x− y‖2〈z− x, z− y〉z),where K3 = 2(‖x− z‖2‖y− z‖2 − 〈x− z, y− z〉2).In view of Proposition 5.2.1 (the uniqueness of the circumcenter), we nowaverage the three formulae from above to obtain the following symmetricformula for p:CC(S) =1K(‖y− z‖2〈x− z, x− y〉x + ‖x− z‖2〈y− z, y− x〉y+ ‖x− y‖2〈z− x, z− y〉z),whereK =16( 1‖y− x‖2‖z− x‖2 − 〈y− x, z− x〉2 +1‖x− y‖2‖z− y‖2 − 〈x− y, z− y〉2+1‖x− z‖2‖y− z‖2 − 〈x− z, y− z〉2).In fact, Proposition 4.3.3 yields K1 = K2 = K3.675.7. The circumcenter of three pointsWe now summarize the above discussion so far in the following twopleasing main results.Theorem 5.7.4. (nonsymmetric formula for the circumcenter) Supposethat S = {x, y, z} and denote the cardinality of S by l. Then exactly one ofthe following cases occurs:(i) l = 1 and CC(S) = x.(ii) l = 2, say S = {u, v}, where u, v ∈ S and u 6= v, and CC(S) = u+v2 .(iii) l = 3 and exactly one of the following two cases occurs:(a) x, y, z are affinely independent; equivalently, ‖y− x‖‖z− x‖ >〈y − x, z − x〉, and consider K := 2(‖y − x‖2‖z − x‖2 − 〈y − x,z− x〉2),CC(S) =1K(‖y− z‖2〈x− z, x− y〉x + ‖x− z‖2〈y− z, y− x〉y+ ‖x− y‖2〈z− x, z− y〉z).(b) x, y, z are affinely dependent; equivalently, ‖y − x‖‖z − x‖ =〈y− x, z− x〉, and CC(S) = ∅.Theorem 5.7.5. (symmetric formula of the circumcenter) Suppose thatS = {x, y, z} and denote the cardinality of S by l. Then exactly one of thefollowing cases occurs:(i) l = 1 and CC(S) = x = y = z = x+y+z3 .(ii) l = 2 and CC(S) = ‖x−y‖z+‖x−z‖y+‖y−z‖x‖x−y‖+‖x−z‖+‖y−z‖ .(iii) l = 3, consider K = 16( 1‖y−x‖2‖z−x‖2−〈y−x,z−x〉2 +1‖x−y‖2‖z−y‖2−〈x−y,z−y〉2+ 1‖x−z‖2‖y−z‖2−〈x−z,y−z〉2), and exactly one of the following two casesoccurs:(a) K ∈ ]0,+∞[ andCC(S) =1K(‖y− z‖2〈x− z, x− y〉x + ‖x− z‖2〈y− z, y− x〉y+ ‖x− y‖2〈z− x, z− y〉z).(b) K is not defined (because of a zero denominator) and CC(S) = ∅.685.8. The circumcenter in R3 and the cross product5.8 The circumcenter in R3 and the cross productWe conclude this section by expressing the circumcenter and circumra-dius in R3 by using the cross product.Theorem 5.8.1. Suppose thatH = R3, that x, y, z are affinely independent,and that S = {x, y, z}. Set a = y− x, and b = z− x and let the angle betweena and b, defined in Definition 2.5.24, be θ. Then(i) CC(S) = x + (‖a‖2b−‖b‖2a)×(a×b)2‖a×b‖2 .(ii) [15, 1.54] CR(S) = ‖a‖‖b‖‖a−b‖2‖a×b‖ =‖a−b‖2 sin θ .Proof. (i): Using the formula of circumcenter in Theorem 5.4.1, we haveCC(S) = x + 12(y− x z− x) ( ‖y− x‖2 〈y− x, z− x〉〈z− x, y− x〉 ‖z− x‖2)−1 (‖y− x‖2‖z− x‖2)= x +12(a b) (‖a‖2 〈a, b〉〈b, a〉 ‖b‖2)−1 (‖a‖2‖b‖2)= x +12(‖a‖2‖b‖2 − 〈a, b〉2)(a b) ( ‖b‖2 −〈a, b〉−〈b, a〉 ‖a‖2)(‖a‖2‖b‖2)= x +12(‖a‖2‖b‖2 − 〈a, b〉2)(a b) (‖a‖2‖b‖2 − ‖b‖2〈a, b〉‖a‖2‖b‖2 − ‖a‖2〈a, b〉)= x +(‖a‖2‖b‖2 − ‖b‖2〈a, b〉)a + (‖a‖2‖b‖2 − ‖a‖2〈a, b〉)b2(‖a‖2‖b‖2 − 〈a, b〉2)= x +〈‖a‖2b− ‖b‖2a, b〉a− 〈‖a‖2b− ‖b‖2a, a〉b2(‖a‖2‖b‖2 − 〈a, b〉2) .Using the Fact 2.5.23 (iii) and (iv), we getCC(S) = x +(‖a‖2b− ‖b‖2a)× (a× b)2‖a× b‖2 .(ii): By Definition 5.2.2, we haveCR(S) = ‖CC(S)− x‖ =∥∥∥ (‖a‖2b− ‖b‖2a)× (a× b)2‖a× b‖2∥∥∥. (5.15)695.8. The circumcenter in R3 and the cross productUsing Fact 2.5.23(iv) and Fact 2.5.23(ii), we obtain∥∥∥(‖a‖2b− ‖b‖2a)× (a× b)∥∥∥=(∥∥∥‖a‖2b− ‖b‖2a∥∥∥2‖a× b‖2 − 〈‖a‖2b− ‖b‖2a, a× b〉2) 12=∥∥∥‖a‖2b− ‖b‖2a∥∥∥‖a× b‖. (5.16)In addition, by Remark 2.5.19, since ‖a‖ 6= 0, ‖b‖ 6= 0, thus∥∥∥‖a‖2b− ‖b‖2a∥∥∥ = ‖a‖‖b‖∥∥∥∥‖a‖‖b‖b− ‖b‖‖a‖ a∥∥∥∥. (5.17)Now ∥∥∥∥‖a‖‖b‖b− ‖b‖‖a‖ a∥∥∥∥2 = ∥∥∥∥‖a‖‖b‖b∥∥∥∥2 − 2〈‖a‖‖b‖b, ‖b‖‖a‖ a〉+∥∥∥∥‖b‖‖a‖ a∥∥∥∥2= ‖a‖2 − 2〈b, a〉+ ‖b‖2= ‖a− b‖2. (5.18)Upon combining (5.16), (5.17) and (5.18), we obtain∥∥∥(‖a‖2b− ‖b‖2a)× (a× b)∥∥∥ = ‖a‖‖b‖‖a− b‖‖a× b‖.Hence, (5.15) yieldsCR(S) =12‖a× b‖2∥∥∥(‖a‖2b− ‖b‖2a)× (a× b)∥∥∥=12‖a× b‖2 ‖a‖‖b‖‖a− b‖‖a× b‖=‖a‖‖b‖‖a− b‖2‖a× b‖ .By Fact 2.5.26, we know ‖a× b‖ = ‖a‖‖b‖ sin θ. Thus, we obtainCR(S) =‖a‖‖b‖‖a− b‖2‖a× b‖ =‖a− b‖2 sin θand the proof is complete.Remark 5.8.2. In view of Fact 2.5.27 and our proof of Theorem 5.8.1, wecannot generalize the latter result to a general Hilbert spaceH— unless thedimension ofH is either 3 or 7.70Chapter 6Circumcenter methodUnless otherwise specified, the standing assumptions in all of the re-maining chapters are as follows. Let T1, . . . , Tm−1, Tm be operators from thereal Hilbert space H to H, with m ≥ 1. Denote by S the nonempty set offinitely many operators onH. SetS = {T1, . . . , Tm−1, Tm}, (6.1)and for every x ∈ H,S(x) = {T1x, . . . , Tm−1x, Tmx}. (6.2)Recall that we use the convention that{0} = span∅.6.1 OverviewIn this chapter, we focus on the circumcenter operator defined in Defini-tion 5.2.2 with S being specifically the set-valued operator S(x) defined in(6.2). We call the specific circumcenter operator as circumcenter mappinginduced by S and denote it as CCS . In particular, we call CCS proper if(∀x ∈ H) CCSx ∈ H. Our main results in this chapter are the following.• We provide two sufficient conditions for the CCS to be proper (seeTheorem 6.2.7 and Theorem 6.2.8). When the S consists of three oper-ators, we provide a sufficient and necessary condition for CCS to beproper (see Theorem 6.2.10).• Using formulae of circumcenter operator in Chapter 5, we obtain anexplicit formula for the proper CCS in Corollary 6.2.11.• We explore the continuity of the circumcenter mapping (see Propo-sition 6.4.3 and Remark 6.4.10). We summarize the conditions under716.2. Circumcenter mapping induced by a set of operatorswhich the proper circumcenter mapping, CCS , is continuous at x (seeProposition 6.4.5). Moreover, several examples are provided to illus-trate the tightness of our assumptions (see Examples 6.4.4, 6.4.6, 6.4.7and 6.4.8).6.2 Circumcenter mapping induced by a set ofoperatorsRecall that S was given by (6.1). Then for every x ∈ H, we get theunique set S(x) defined in (6.2). Now S(x) ∈ P(H), so by Definition 5.2.2,we have either CC(S(x)) = ∅ or CC(S(x)) is the circumcenter of the setS(x). Therefore, for every set of finitely many operators, we can define amapping induced by this set, which is the idea of the following definition.Definition 6.2.1. Define the circumcenter mapping CCS : H → H ∪ {∅}induced by S as:(∀x ∈ H) CCSx = CC(S(x)),that is, for every x ∈ H, if the circumcenter of the set S(x) defined inDefinition 5.2.2 does not exist, then CCSx = ∅. Otherwise, CCSx is theunique point satisfying the two conditions below:(i) CCSx ∈ aff(S(x)) = aff{T1(x), . . . , Tm−1(x), Tm(x)}, and(ii){‖CCSx− Ti(x)‖∣∣∣ i ∈ {1, . . . , m− 1, m}} is a singleton, that is,‖CCSx− T1(x)‖ = · · · = ‖CCSx− Tm−1(x)‖ = ‖CCSx− Tm(x)‖.In particular, if for every x ∈ H, CCSx ∈ H, then we say the circumcentermapping CCS induced by S , is proper. Otherwise, we call the CCS improper.Here we use the letter S to stand for “the set of operators on H”. Todistinguish with the S stands for “the set of points inH” in the Chapter 5,we use S but not S again.Proposition 6.2.2. Assume m = 2 and S = {T1, T2}. Then CCS is proper.Moreover,(∀x ∈ H) CCSx = T1x + T2x2 .726.2. Circumcenter mapping induced by a set of operatorsProof. Using Definition 6.2.1 and Corollary 5.2.4, we getCCSx = CC(S(x)) = CC({T1x, T2x}) = T1x + T2x2 .Proposition 6.2.3. Let x ∈ H. Assume T1x, . . . , Tmx are affinely indepen-dent. Then CCSx ∈ H; more precisely,CCSx = T1x + α1(T2x− T1x) + · · ·+ αm−1(Tmx− T1x),where(α1 · · · αm−1)ᵀ=12G(T2x− T1x, . . . , Tmx− T1x)−1‖T2x− T1x‖2...‖Tmx− T1x‖2 ,and G(x1 − x, . . . , xm − x) is the Gram matrix of T2x− T1x, . . . , Tmx− T1x.Proof. The required result is clear from Theorem 5.4.1 and Definition 6.2.1.Example 6.2.4. Assume m = 2, and T1 = Id, T2 = 2 Id, that is S = {Id, 2 Id}.By Proposition 6.2.2,(∀x ∈ H) CCSx = x + 2x2 =32x.So in this case, the circumcenter mapping CCS induced by S is proper.Example 6.2.5. Assume H = R and T1 = Id, T2 = 2 Id, T3 = 3 Id, that isS = {Id, 2 Id, 3 Id}. In this case, for every x ∈ Rr {0}, by Theorem 5.7.1,there exists no point CCSx ∈ R = aff{x, 2x, 3x} such that‖CCSx− x‖ = ‖CCSx− 2x‖ = ‖CCSx− 3x‖.Hence, in this case CCS is improper.As the above example shows, in order to let the mapping CCS be proper,the set of finitely many operators, S , must be carefully chosen. In otherwords, the operators must satisfy some special conditions. Below, we shallshow some sufficient conditions for CCS to be proper, and we will show alot of proper CCS ’s in the chapters to come.For every x ∈ H, denote the cardinality of the set S(x) as lx. If lx = 1,by definition of CCS , CCSx = x, which is a trivial case. Assume lx ≥ 2. By736.2. Circumcenter mapping induced by a set of operatorsCorollary 5.4.3, we know that when the CCS is proper, the formula of CCSxis determined by the basis of the subspace span{T2x− T1x, . . . , Tm−1x− T1x,Tmx− T1x}. Because for different x ∈ H, we may get different bases of thesubspace span{T2x− T1x, . . . , Tm−1x− T1x, Tmx− T1x}. In other words, thebasis of {T2x− T1x, . . . , Tm−1x− T1x, Tmx− T1x} is determined by x whenS is fixed. To unify and simplify the notations, we have the definition below.Definition 6.2.6. Denote I = {1, . . . , m} as the index set. For every x ∈ H,letlx be the cardinality of the set S(x) = {T1x, T2x, . . . , Tmx}and letdx = dim(span{T2x− T1x, . . . , Tmx− T1x}).Let Ix ⊆ I r {1} (may not be unique) such that the list of vectors(∀j ∈ Ix) Tj(x)− T1(x) is a basis of span{T2x− T1x, . . . , Tmx− T1x}.When lx = 1, then span{T2x− T1x, . . . , Tmx− T1x} = {0}. Thus,dx = dim(span{T2x− T1x, . . . , Tmx− T1x} = dim({0}) = 0,and Ix = ∅.1When lx 6= 1, to simplify the statements in the later proofs, we denoteIx = {i1, i2, . . . , idx},with i1 < i2 < · · · < idx .Using the Definition 6.2.1 and Theorem 5.4.1, we obtain the followingtwo sufficient conditions for CCS being proper. The first one will play acritical role in the next chapters.Theorem 6.2.7. Suppose for every x ∈ H, there exists a point p(x) ∈ Hsuch that(i) p(x) ∈ aff{T1x, . . . , Tm−1x, Tmx}, and(ii) ‖p(x)− T1x‖ = · · · = ‖p(x)− Tm−1x‖ = ‖p(x)− Tmx‖.Then CCS is proper. Moreover,(∀x ∈ H) CCSx = p(x).1Remember that we use the convention that {0} = span∅.746.2. Circumcenter mapping induced by a set of operatorsProof. The result follows directly from Definition 6.2.1.Theorem 6.2.8. For every x ∈ H, let S˜(x) = {T1x, Ti1 x, . . . , Ti(tx−1)x, Titx x} ⊆S(x) be the maximally pairwise distinct subset defined in Definition 5.4.4.Suppose for every x ∈ H, T1x, Ti1 x, . . . , Ti(tx−1)x, Titx x are affinely indepen-dent inH, then CCS is proper.Proof. The result follows clearly from Theorem 5.4.1 and Corollary 5.4.5.One of the converse results of the Theorem 6.2.8 is that: Assume CCS isproper. Then the elements in the maximally pairwise distinct subset of S(x)are affinely independent.Unfortunately, the example below shows that the above converse resultis not true.Example 6.2.9. Let U be a linear subspace ofH with {0} 6= U & H. Denotealso 0 as the zero operator such that (∀x ∈ H) 0(x) = 0. Let S = {Id, PU ,PU⊥ , 0}. Then the following statements hold.(i) CCS is proper.(ii) (∀x ∈ Hr (U ∪U⊥)) x, PUx, PU⊥x, 0(x) are pairwise distinct.(iii) (∀x ∈ H) Id x, PUx, PU⊥x, 0(x) are affinely dependent.Proof. (i): Let x ∈ H. By Proposition 4.2.5, 〈 x2 − PU x2 , 0− PU x2 〉 = 0, 〈 x2 − PU x2 ,PUx− PU x2 〉 = 0, thus‖ x2− PU x2‖2 + ‖PU x2‖2 = ‖ x2‖2‖ x2− PU x2‖2 + ‖PUx− PU x2‖2 = ‖ x2− PUx‖2,which implies that ‖ x2‖ = ‖ x2 − PUx‖, since the left-hand side of the abovetwo equalities are the same. Similarly, by Proposition 4.2.5 again, replaceU in the above equalities by U⊥, we can get ‖ x2‖ = ‖ x2 − PU⊥x‖. Hence,we obtain that the point x2 ∈ span{x, PUx, PU⊥x} = aff{x, PUx, PU⊥x, 0(x)}satisfy‖ x2‖ = ‖ x2− 0(x)‖ = ‖ x2− x‖ = ‖ x2− PUx‖ = ‖ x2 − PU⊥x‖,which means that (∀x ∈ H) CCSx = x2 , that is CCS is proper.756.2. Circumcenter mapping induced by a set of operators(ii): In fact,x = PUx ⇐⇒ x ∈ U;x = PU⊥x ⇐⇒ x ∈ U⊥;U ∩U⊥ = {0}.In addition,PUx = PU⊥x =⇒ PUx = PU⊥x = 0 =⇒ x = PUx + PU⊥x = 0 ∈ U ∪U⊥.Hence, for every x ∈ Hr (U ∪U⊥), we knowx, PUx, PU⊥x, 0(x) are pairwise distinct.(iii): Now for every x ∈ H,0(x)− PUx = 0− PUx = −PUx = −(x− PU⊥x) = PU⊥x− x=⇒(0(x)− x)− (PUx− x) = PU⊥x− x,which yields that PU⊥x − x ∈ span{0(x)− x, PUx − x}, which in turn im-plies that PUx− x, PU⊥x− x, 0(x)− x are linearly dependent. By Lemma 4.1.4,we get that Id x, PUx, PU⊥x, 0(x) are affinely dependent.Although the example above shows that the converse of the Theo-rem 6.2.8 fails, when we only consider the set S with three operators, wehave the following beautiful theorem.Theorem 6.2.10. Let S = {T1, T2, T3}. Then CCS is proper if and only iffor every x ∈ H with the cardinality of the S(x) being 3, T1x, T2x, T3x areaffinely independent inH.Proof. “=⇒” Assume CCS is proper, i.e., for every x ∈ H, CCSx ∈ H. ByTheorem 5.7.1, for every x ∈ H with the cardinality of the S(x) being 3,CCSx ∈ H ⇐⇒ T1x, T2x, T3x are affinely independent inH. (6.3)“⇐=” Assume that for every x ∈ H with the cardinality of the S(x)being 3, the set T1x, T2x, T3x are affinely independent in H. Let x ∈ H.Denote lx as the cardinality of S(x). By (6.3), if lx = 3, then CCSx ∈ H.Assume lx ≤ 2, by Corollary 5.2.4, CCSx ∈ H. Hence, for every x ∈ H,CCSx ∈ H, which means that CCS is proper.766.3. Properties of circumcenter operatorThe Corollary below shows an explicit formula for the proper CCS .Corollary 6.2.11. Assume CCS is proper. Uphoading the notations in Defi-nition 6.2.6, for every x ∈ H, we assume thatTi1 x− T1x, . . . , Tidx x− T1x is a basis of span{T2x− T1x, . . . , Tmx− T1x}.Denote S˜ := {T1, Ti1 , . . . , Tidx }. Then 2CCSx = CCS˜x = T1x +dx∑j=1αij(x)(Tij x− T1x) (6.4)where αi1(x)...αidx (x) = 12 G(Ti1 x− T1x, . . . , Tidx x− T1x)−1 ‖Ti1 x− T1x‖2...‖Tidx x− T1x‖2 .Proof. The desired result follows directly from Corollary 5.4.3 and Proposi-tion 4.1.5.6.3 Properties of circumcenter operatorIn this section, we shall show some fundamental properties of the map-ping CCS .Proposition 6.3.1. ∩mi=1 Fix Ti ⊆ Fix CCS .Proof. Let x ∈ ∩mi=1 Fix Ti. Then(∀i ∈ {1, . . . , m− 1, m}) Tix = x, (6.5)which yields that aff{T1x, . . . , Tm−1x, Tmx} = aff{x} = {x}. In addition, by(6.5),‖x− T1x‖ = · · · = ‖x− Tm−1x‖ = ‖x− Tmx‖ = 0.Therefore, by definition, we obtain that CCSx = x, which means that x ∈Fix CCS . Hence, ∩mi=1 Fix Ti ⊆ Fix CCS .2Note that by Definition 6.2.6, if lx = 1, then dx = 0 and so CCS x = T1x.776.3. Properties of circumcenter operatorThe following two propositions are inherited from Proposition 5.3.1 andProposition 5.3.3 respectively.Proposition 6.3.2. Assume that T1, . . . , Tm−1, Tm are homogeneous, that is(∀x ∈ H) (∀λ ∈ R) (∀i ∈ {1, . . . , m}) Ti(λx) = λTi(x). (6.6)Then(∀x ∈ H) (∀λ ∈ Rr {0}) CCS (λx) = λCCSx.Proof. Remember that S = {T1, . . . , Tm−1, Tm} and that for every x ∈ H, wedenoteS(x) = {T1x, . . . , Tmx}.Hence, for every x ∈ H and for every λ ∈ R, by (6.6), we knowλS(x) = {λT1x, . . . ,λTmx} (6.6)= {T1(λx), . . . , Tm(λx)} = S(λx). (6.7)Now by Definition 6.2.1 and Proposition 5.3.1, for every λ ∈ Rr {0},CCS (λx) = CC(S(λx)) (6.7)= CC(λS(x)) = λCC(S(x)) = λCCSx,where the first and the last equalities follow from Definition 6.2.1 and thelast second equality follows from Proposition 5.3.1. Hence, the proof iscomplete.Proposition 6.3.3. Assume that T1, T2, . . . , Tm−1, Tm are additive, that is(∀x, y ∈ H) (∀i ∈ {1, . . . , m− 1, m}) Ti(x + y) = Tix + Tiy. (6.8)Then(∀x ∈ H) (∀u ∈ ∩mi=1 Fix Ti) CCS (x + u) = CCSx + u.Proof. Let x ∈ H and u ∈ ∩mi=1 Fix Ti. By (6.8), we know that,S(x + u) ={T1(x + u), . . . , Tm(x + u)}(6.8)= {T1(x) + u, . . . , Tm(x) + u} = S(x) + u. (6.9)Now,CCS (x + u) =CC(S(x + u)) (by Definition 6.2.1)(6.9)= CC(S(x) + u)=CC(S(x)) + u (by Proposition 5.3.3)=CCSx + u. (by Definition 6.2.1)Hence, the proof is done.786.4. Continuity of circumcenter mapping6.4 Continuity of circumcenter mappingProposition 6.4.1. Assume m = 2, which means that S = {T1, T2}. Assumefurther T1, T2 are continuous operator. Let x ∈ H and (x(k))k∈N ⊆ H withlimk→∞ x(k) = x. Thenlimk→∞CCS (x(k)) = CCSx.Proof. Since limk→∞ x(k) = x and T1, T2 are continuous, thuslimk→∞(T1(x(k)), T2(x(k))) = (T1x, T2x).Using Proposition 6.2.2, we getlimk→∞CCS (x(k)) = limk→∞T1(x(k)) + T2(x(k))2=T1x + T2x2= CCSx.The proof is done.Example 6.4.2. Assume m = 2. Let U1 and U2 be two closed convex sets inH such that U1 ∩U2 6= ∅. Assume S = {Id, RU2 RU1}. By Proposition 6.4.1,CCS is continuous. In fact, by Proposition 6.2.2,CCSx =x + RU2 RU1 x2.Hence, CCS is the Dougls–Rachford operator associated with U1 and U2. Infact, by Lemma 3.3.2, CCS is firmly nonexpansive.Proposition 6.4.3. Assume the elements of S = {T1, . . . , Tm−1, Tm} are con-tinuous operators. Let x ∈ H satisfying CCSx ∈ H.(i) By the notations in Definition 6.2.6, let S˜x = {T1, Ti1 , . . . , Tidx } ⊆ Ssuch that3 Ti1 x− T1x, . . . , Tidx x− T1x is a basis of span{T2x− T1x, . . . ,Tmx− T1x} . Then for every (x(k))k∈N ⊆ H satisfying limk→∞ x(k) = x,there exists N ∈ N such that for every k ≥ N, CCS˜x(x(k)) ∈ H.Moreoverlimk→∞CCS˜x(x(k)) = CCS˜x x = CCSx. (6.10)(ii) Suppose T1x, . . . , Tm−1x, Tmx are affinely independent, then CCS iscontinuous at x.796.4. Continuity of circumcenter mappingProof. Let (x(k))k∈N ⊆ H satisfying limk→∞ x(k) = x. NowS = {T1, . . . , Tm−1, Tm},S(x) = {T1(x), . . . , Tm−1x, Tmx},S˜x = {T1, Ti1 , . . . , Tidx },S˜x(x) = {T1x, Ti1 x, . . . , Tidx x},S˜x(x(k)) = {T1x(k), Ti1 x(k), . . . , Tidx x(k)}.By Definition 6.2.1, CCSx ∈ H means that CC(S(x)) ∈ H. By assumptions,Ti1 x− T1x, . . . , Tidx x− T1x is a basis of span{T2x− T1x, . . . , Tmx− T1x}.Hence, we can substitute the S, S˜ and S˜(k) in Theorem 5.6.1 by the aboveS(x), S˜x(x) and S˜x(x(k)) respectively. Then by Theorem 5.6.1 and Defini-tion 6.2.1, we get the corresponding two results below:(I) There exists N ∈N such that for every k ≥ N, we have CC(S˜x(x(k))) ∈H andlimk→∞CC(S˜x(x(k))) = CC(S˜x(x)) = CC(S(x))⇐⇒ limk→∞CCS˜x(x(k)) = CCS˜x x = CCSx.(II) In particular, when T1x, . . . , Tm−1x, Tmx are affinely independent, wehave thatlimk→∞CC(S(x(k))) = CC(S(x))⇐⇒ limk→∞CCS (x(k)) = CCSx.Clearly, the above two items yield our (i) and (ii) respectively.The example below shows that when m = 3 but elements of S(x) are notaffinely independent, the consequence about continuity in Proposition 6.4.3may not be true.Example 6.4.4. Assume H = R. Let S = {Id, 2 Id, 3 Id}. Let x = 0.By definition CCS(x) = 0 ∈ H. Take (∀k ∈ Nr {0}) x(k) = 1k . Thenlimk→∞ x(k) = x and (∀k ∈Nr {0}) CCS(x(k)) = ∅. Hence,limk→∞CCS˜ (x(k)) 6= CCS(x).3When dx = 0, that is, S(x) is a singleton, then S˜x = {T1}, since we use the conventionthat ∅ the basis of the zero subspaces.806.4. Continuity of circumcenter mappingThe proposition below summarize the conditions under which theproper circumcenter mapping, CCS , is continuous at x.Proposition 6.4.5. Assume the elements of S = {T1, . . . , Tm−1, Tm} are con-tinuous operators. Assume further that CCS is proper. Then for everyx ∈ H,(i) If T1x, . . . , Tm−1x, Tmx are affinely independent, then CCS is continu-ous at x.(ii) Assume T1x, . . . , Tm−1x, Tmx are affinely dependent. Assume furtherthat m ≤ 2. Then CCS is continuous at xProof. (i) follows from Proposition 6.4.3.(ii) is a direct result of Proposition 5.6.3.The examples below show that when T1x, . . . , Tm−1x, Tmx are affinelydependent and m ≥ 3, the continuity of the circumcenter mapping at x is alittle bit complicated.Example 6.4.6. Let U be a linear subspace in the real Hilbert spaceH, with{0} 6= U & H. Let S = {Id, RU , RU⊥}. Then CCS is continuous on H. Butthere exists x ∈ H such that elements of S(x) are affinely dependent.Proof. Since Id = PU + PU⊥ and RU = 2PU − Id, thusRU + RU⊥2=(2PU − Id) + (2PU⊥ − Id)2=12(2PU − Id+2(Id−PU)− Id)= 0.Let x ∈ H. Then 0 = RU x+RU⊥ x2 ∈ aff{x, RUx, RU⊥x}. In addition, clearly0 ∈ U ∩U⊥. In Proposition 4.2.5(iii), substitute S = U, and let the points = 0. We get ‖x‖ = ‖RUx‖. Similarly, In Proposition 4.2.5(iii), substituteS = U⊥ and let the point s = 0 . We get ‖x‖ = ‖RU⊥x‖. Hence, we have(i) 0 ∈ aff{x, RUx, RU⊥x} and(ii) ‖0− x‖ = ‖0− RUx‖ = ‖0− RU⊥x‖,which means that (∀x ∈ H) CCS(x) = 0. Hence, it is clear that CCS iscontinuous onH.But for every x ∈ U (respectively x ∈ U⊥), RUx = x (respectivelyRU⊥x = x), which implies that x, RUx, RU⊥x, which is x, x, RU⊥x (respec-tively x, RUx, x) are not affinely independent.816.4. Continuity of circumcenter mappingExample 6.4.7. Assume H = R2 and m = 3, i.e., S = {T1, T2, T3}. Assumefurther that for every (x, y) ∈ R2,T1(x, y) = (x, y);T2(x, y) = (−x, y);T3(x, y) = (x,−14 (x− 2)).Then CCS is proper and is continuous onH.Proof. Given (x, y) ∈ R2. Now by Lemma 4.1.4,T1(x, y), T2(x, y), T3(x, y) are affinely independent⇐⇒T2(x, y)− T1(x, y), T3(x, y)− T1(x, y) are linearly independent⇐⇒(−2x, 0),(0,−14(x− 2)− y)are linearly independent⇐⇒det(A) 6= 0, where A =(−2x 00 − 14 (x− 2)− y)⇐⇒2x(− 14(x− 2)− y)6= 0.Hence, by Proposition 6.2.3, when 2x(− 14 (x− 2)− y)6= 0, CCS (x, y) ∈ H.Now, assume 2x(− 14 (x− 2)− y)= 0. If x = 0, we get S(x, y) = {(0,y), (0, y), (0, 12 )}. Then by Theorem 5.7.4, CCS (x, y) = (0, 12 (y + 12 )) ∈ H. Inthe other case, when x 6= 0, but − 14 (x − 2)− y = 0, then y = − 14 (x − 2).Hence, S(x, y) ={(x,− 14 (x− 2)), (−x,− 14 (x− 2)), (x,− 14 (x− 2))}. Again,by Theorem 5.7.4, CCS (x, y) = (0,− 14 (x − 2))) ∈ H. Therefore, we haveproved thatCCS is proper.On the other hand, let (x, y) ∈ R2. We have already shown that whenx = 0,CCS (x, y) = (0,12(y +12)). (6.11)When y = − 14 (x− 2),CCS (x, y) = (0,−14 (x− 2))). (6.12)826.4. Continuity of circumcenter mappingAssume x 6= 0 and y 6= − 14 (x− 2). Denote CCS (x, y) = (p1, p2). Now bydefinition, {‖CCS (x, y)− T1(x, y)‖ = ‖CCS (x, y)− T2(x, y)‖‖CCS (x, y)− T1(x, y)‖ = ‖CCS (x, y)− T3(x, y)‖⇐⇒{‖CCS (x, y)− (x, y)‖ = ‖CCS (x, y)− (−x, y)‖‖CCS (x, y)− (x, y)‖ = ‖CCS (x, y)− (x,− 14 (x− 2))‖⇐⇒{(p1 − x)2 + (p2 − y)2 = (p1 + x)2 + (p2 − y)2(p1 − x)2 + (p2 − y)2 = (p1 − x)2 + (p2 + 14 (x− 2))2⇐⇒{(p1 − x)2 = (p1 + x)2(p2 − y)2 = (p2 + 14 (x− 2))2⇐⇒{4p1x = 012 p2(x− 2+ 4y) = 116 (4y + (x− 2))(4y− (x− 2))x 6=0y 6=− 14 (x−2)⇐⇒{p1 = 0p2 =y− 14 (x−2)2 .Therefore, combining the above results with (6.11) and (6.12), we obtain that(∀(x, y) ∈ R2) CCS (x, y) = (0, y− 14 (x− 2)2 ).Therefore, CCS is continuous on R2.The example below shows that even if the elements of S are continuous,CCS is proper ; CCS is continuous.The Example 5.6.6 plays an important role in the proof of the followingexample.Example 6.4.8. Assume H = R2 and m = 3, that is, S = {T1, T2, T3}.Assume further that for every (x, y) ∈ R2,T1(x, y) = (2, 0);T2(x, y) = (−2, 0);T3(x, y) = (x,−14 (x− 2)).Then although CCS is proper, CCS is not continuous at the point (2, 0).836.4. Continuity of circumcenter mappingProof. Let (x, y) ∈ R2. Now by Lemma 4.1.4,T1(x, y), T2(x, y), T3(x, y) are affinely independent⇐⇒T2(x, y)− T1(x, y), T3(x, y)− T1(x, y) are linearly independent⇐⇒(−4, 0),(x− 2,−14(x− 2))are linearly independent⇐⇒det(A) 6= 0, where A =(−4 x− 20 − 14 (x− 2))⇐⇒x− 2 6= 0.Hence, by Proposition 6.2.3, when x − 2 6= 0, CCS (x, y) ∈ H. Actually,when x− 2 = 0, that is x = 2, then for every y ∈ R,T1(2, y) = (2, 0), T2(2, y) = (−2, 0), T3(2, y) = (2,−14 (2− 2)) = (2, 0).By Proposition 6.2.2, we know that CCS (x, y) = (0, 0) ∈ H. Hence,CCS is proper.In addition, let (x, y¯) = (2, 0), and (∀k ≥ 1) (x(k), y(k)) = (2− 1k , 0). Bythe analysis above, we knowCCS (x, y¯) = (0, 0). (6.13)On the other hand, since S(x(k), y(k)) ={T1(x(k), y(k)), T2(x(k), y(k)), T3(x(k),y(k))}={(2, 0), (−2, 0), (2− 1k , 14k )}, and since, by Definition 6.2.1, CCS (x(k),y(k)) = CC(S(x(k), y(k))), thus (5.11) in Example 5.6.6 tells us thatCCS (x(k), y(k)) = (0,−8+ 2k +18k). (6.14)Hence,limk→∞CCS (x(k), y(k)) = (0,−8) 6= (0, 0) = CCS (x, y¯),which means that the proper circumcenter mapping CCS is not continuousat the point (2, 0).Remark 6.4.9. In Example 6.4.8,S(x) = {T1(2, 0), T2(2, 0), T3(2, 0)} = {(2, 0), (−2, 0), (2, 0)}.846.4. Continuity of circumcenter mappingClearly, (2, 0), (−2, 0), (2, 0) are affinely dependent. But the list of vectorsconsisting the maximally pairwise distinct subset of S(x), (2, 0), (−2, 0),are affinely independent. Therefore, we know that in Proposition 6.4.5(i),we cannot replace the condition “If T1x, T2x, . . . , Tm−1x, Tmx are affinely in-dependent” by “If the list of vectors consisting of the maximally pairwisedistinct subset of S(x) = {T1x, T2x, . . . , Tm−1x, Tmx} are affinely indepen-dent”.Remark 6.4.10. Claim: The (6.10) in Proposition 6.4.3(i), i.e.,limk→∞CCS˜x(x(k)) = CCS˜x x = CCSx,can not be replaced bylimk→∞CCS˜x(k)(x(k)) = CCS˜x x = CCSx.We use the Example 6.4.8 as a counterexample to prove the above claim.Recall that for every x ∈ H,S˜x = {T1, Ti1 , . . . , Tidx } ⊆ S (6.15)such thatTi1 x− T1x, . . . , Tidx x− T1x is a basis of span{T2x− T1x, . . . , Tmx− T1x}.BecauseS(x, y¯) = {T1(x, y¯), T2(x, y¯), T3(x, y¯)} = {(2, 0), (−2, 0), (2, 0)},replacing the x in (6.15) by (x¯, y¯), we get S˜(x,y¯) = {T1, T2}. SinceS(x(k), y(k)) ={T1(x(k), y(k)), T2(x(k), y(k)), T3(x(k), y(k))}={(2, 0), (−2, 0), (2− 1k,14k)},replacing the x in (6.15) by (x(k), y(k)), we getS˜(x(k),y(k)) = {T1, T2, T3} = S . (6.16)856.4. Continuity of circumcenter mappingNowlimk→∞CCS˜(x(k) ,y(k))(x(k), y(k))(6.16)= limk→∞CCS (x(k), y(k))(6.14)= limk→∞(0,−8+ 2k+18k)= (0,−8)6= (0, 0) (6.13)= CCS (x, y¯) = CCS˜(x,y¯)(x, y¯),the last equality is from Proposition 6.4.3(i). Therefore, the above claim istrue.86Chapter 7Circumcenter method inducedby reflectors7.1 OverviewUnless otherwise specified, we have the following assumptions. Letm ∈Nr {0}, andlet U1, . . . , Um be closed affine subspaces in the real Hilbert spaceH,with ∩mi=1Ui 6= ∅. For every i ∈ {1, . . . , m}, let RUi = 2PUi − Id be thereflector associated with Ui. LetΩ ={RUir · · · RUi2 RUi1∣∣∣ r ∈N, and i1, . . . , ir ∈ {1, . . . , m}}. (7.1)We use the empty product convention that0∏j=1RUij = Id .So when r = 0 in (7.1), Id ∈ Ω. Hence, in fact, Ω is the set consist of theidentity operator, Id, and all of the compositions of (∀i ∈ {1, . . . , m}) RUi .Throughout this chapter, we assume specifically thatS is a subset of the Ω defined in (7.1) with{Id} & S .Recall that for every x ∈ H,S(x) = {Tx | T ∈ S}.Note that for every T in S , there exists r ∈N and i1, . . . , ir ∈ {1, . . . , m}such that T = RUir · · · RUi2 RUi1 . Therefore, from now on we always assumeRUir · · · RUi1 is the representative element of the set S .877.2. Circumcenter method induced by reflectorsIn this chapter we focus on the circumcenter mapping CCS defined inDefinition 6.2.1 with S specifically satisfying that {Id} & S ⊆ Ω. The CT in[12] and the CRM operator in [13] are two special examples of our CCS inthis chapter. Our main results in this chapter are summarized as follows:• We prove that(∀x ∈ H) CCSx = Paff(S(x))(P∩mi=1Ui x), (7.2)which mean that CCSx is the closest point to P∩mi=1Ui x among the pointsin the affine subspace aff(S(x)). This result plays an important role inthe linear convergence of the circumcenter method. In fact, our proofis a mild generalization of the [12, Lemma 2].• Based on (7.2), we show another new formula of CCS (see Proposi-tion 7.3.1).• We provide some useful new properties of CCS in Section 7.4.7.2 Circumcenter method induced by reflectorsExample 7.2.1. Assume that m = 3 and S = {Id, RU1 , RU2 , RU3 , RU1 RU2 RU1 ,RU1 RU3}. ThenS(x) ={x, RU1 x, RU2 x, RU3 x, RU1 RU2 RU1 x, RU1 RU3 x},aff(S(x)) = aff{x, RU1 x, RU2 x, RU3 x, RU1 RU2 RU1 x, RU1 RU3 x}.Remark 7.2.2. Note that since Id ∈ S , thus x must be in aff(S(x)).To facilitate the proofs later, we first provide the simple but useful lemmabelow. Remember that every element in S can be represented in the formRUir · · · RUi1 .Lemma 7.2.3. Let x ∈ H. Then for every RUir · · · RUi1 ∈ S ,(∀u ∈ ∩mi=1Ui) ‖x− u‖ = ‖RUir RUir−1 · · · RUi2 RUi1 x− u‖.Proof. Let u ∈ ∩mi=1Ui. Because U1, . . . , Um are closed affine subspaces andu ∈ ∩mi=1Ui ⊆ ∩rj=1Uij ∈ H, by Proposition 4.2.5(iii), we have‖x− u‖ = ‖RUi1 x− u‖ (by u ∈ Ui1)‖RUi1 x− u‖ = ‖RUi2 RUi1 x− u‖ (by u ∈ Ui2)· · ·‖RUir−1 · · · RUi2 RUi1 x− u‖ = ‖RUir RUir−1 · · · RUi2 RUi1 x− u‖ (by u ∈ Uir),887.2. Circumcenter method induced by reflectorswhich yields‖x− u‖ = ‖RUir RUir−1 · · · RUi2 RUi1 x− u‖.Theorem 7.2.4. Let x ∈ H. Then for every u ∈ ∩mi=1Ui,(i) Paff(S(x))(u) ∈ aff(S(x)), and(ii) for every RUir · · · RUi2 RUi1 ∈ S ,‖Paff(S(x))(u)− x‖ = ‖Paff(S(x))(u)− RUir · · · RUi2 RUi1 x‖.Proof. Let u ∈ ∩mi=1Ui. Because aff(S(x)) is the translation of a finite-dimensional linear subspace, aff(S(x)) is a closed affine subspace. ByFact 2.3.7 and Definition 2.3.5, we know Paff(S(x))(u) is well-defined. Denotep = Paff(S(x))(u).Clearly, p = Paff(S(x))(u) ∈ aff(S(x)), i.e., (i) is true.In addition, take an arbitrary but fixed element RUir · · · RUi2 RUi1 in S .Since Id, RUir · · · RUi2 RUi1 ∈ S , we know x, RUir · · · RUi2 RUi1 x ∈ S(x) ⊆aff(S(x)). Substituting S = aff(S(x)), x = u and s = x in Proposi-tion 4.2.5(ii), we get‖u− p‖2 + ‖x− p‖2 = ‖u− x‖2. (7.3)Again, substituting S = aff(S(x)), x = u and s = RUir · · · RUi2 RUi1 x inProposition 4.2.5(ii), we get‖u− p‖2 + ‖RUir · · · RUi2 RUi1 x− p‖2 = ‖u− RUir · · · RUi2 RUi1 x‖2 (7.4)On the other hand, by Lemma 7.2.3, we get‖x− u‖ = ‖RUir · · · RUi2 RUi1 x− u‖.Combining the equality above with (7.3) and (7.4), we get‖p− x‖ = ‖p− RUir · · · RUi2 RUi1 x‖.Since RUir · · · RUi2 RUi1 ∈ S is chosen arbitrarily, thus (ii) holds as well.Therefore, the proof is done.897.2. Circumcenter method induced by reflectorsCombining Theorem 6.2.7 with Theorem 7.2.4, we have the theorembelow, which plays an essential role in this chapter.Theorem 7.2.5. Recall {Id} & S ⊆ Ω. The following assertions hold.(i) the circumcenter mapping CCS : H → H induced by S is proper,i.e., for every x ∈ H, CCSx is the unique point satisfying the twoconditions below:(a) CCSx ∈ aff(S(x)), and(b) (∀RUik · · · RUi1 ∈ S) ‖CCSx− x‖ = ‖CCSx− RUik · · · RUi1 x‖.(ii)(∀x ∈ H) (∀u ∈ ∩mi=1Ui) CCSx = Paff(S(x))(u).(iii) (∀x ∈ H) CCSx = Paff(S(x))(P∩mi=1Ui x).Proof. (i): By Theorem 6.2.7 and Theorem 7.2.4, we know the CCS inducedby S is proper.(ii) and (iii): Since ∩mi=1Ui is a closed affine subspace, by Fact 2.3.7 andDefinition 2.3.5, P∩mi=1Ui x ∈ ∩mi=1Ui is well-defined. Hence, the last two itemsare also from Theorem 7.2.4 and Theorem 6.2.7.Later, we will frequently use some operators TS ∈ aff(S) as a bridge toprove the properties of CCS . In other words, among the possible infinitelymany elements in aff(S), some TS ∈ aff(S) have the properties that willbenefit our proofs of the related properties of CCS .Example 7.2.6. Assume m = 2 and S = {Id, RU1 , RU2 RU1}. Then for everyx ∈ H, aff(S(x)) = aff{x, RU1 x, RU2 RU1 x} and CCSx is the unique pointsatisfying(i) CCSx ∈ aff(S(x));(ii) ‖CCSx− x‖ = ‖CCSx− RU1 x‖ = ‖CCSx− RU2 RU1 x‖.Now TS =Id+RU2 RU12 ∈ conv(S(x)) ⊆ aff(S(x)). Here TS is the Douglas–Rachford splitting operator TU1,U2 defined in Definition 3.3.1.Example 7.2.7. Assume m = 2 and S = {Id, RU2 RU1}. By Example 6.4.2,CCS =Id+RU2 RU12.Like Example 7.2.6, TS =Id+RU2 RU12 ∈ conv(S(x)) ⊆ aff(S(x)).907.3. Explicit formulae of the circumcenter operatorExample 7.2.8. Assume S = {Id, RU1 , RU2 , . . . , RUm}. Then for every x ∈ H,aff(S(x)) = aff{x, RU1 x, . . . , RUm−1 x, RUm x} and CCSx is the unique pointsatisfying(i) CCSx ∈ aff(S(x));(ii) ‖CCSx− x‖ = ‖CCSx−RU1 x‖ = · · · = ‖CCSx−RUm−1 x‖ = ‖CCSx−RUm x‖.Now for each (∀i ∈ {1, . . . , m}) λi > 0 with ∑mi=1 λi = 1, TS = ∑mi=1 λiRUi ∈conv(S(x)) ⊆ aff(S(x)).Example 7.2.9. Assume S = {Id, RU2 RU1 , RU3 RU2 , . . . , RUm RUm−1 , RU1 RUm}.Then for every x ∈ H,aff(S(x)) = aff{x, RU2 RU1 x, RU3 RU2 x, . . . , RUm RUm−1 x, RU1 RUm x}and CCSx is the unique point satisfying(i) CCSx ∈ aff(S(x));(ii) ‖CCSx− x‖ = ‖CCSx− RU2 RU1 x‖ = · · · = ‖CCSx− RUm RUm−1 x‖ =‖CCSx− RU1 RUm x‖.In this case, for any (∀i ∈ {1, . . . , m}) λi > 0 with ∑mi=1 λi = 1,m∑i=1λiTUi ,Ui+1 = TS =m∑i=1λiId + RUi+1 RUi2∈ conv(S(x)) ⊆ aff(S(x)),where Um+1 = U1.Remark 7.2.10. In [12], the T , CT are the TS , CCS respectively in Exam-ple 7.2.6. But actually in Example 7.2.7, for the same TS , we have differentCCS . In fact, here the CCS is only determined by the set of operators S butnot by the TS . Actually, for the same CCS , we have possibly infinitely manychoices of TS .7.3 Explicit formulae of the circumcenter operatorIn Corollary 6.2.11, we have already obtained an explicit formula of theproper CCS . Below, combining the Theorem 7.2.5(iii) and the Lemma 4.2.6,we get an alternative formula of the CCS .917.3. Explicit formulae of the circumcenter operatorProposition 7.3.1. Recall {Id} & S ⊆ Ω. Let x ∈ H. Denote the dimensionof span(S(x) − x) as n. Let {x, T1x, T2x, . . . , Tnx} ⊆ S(x) such that 4T1x− x, T2x− x, . . . , Tnx− x are linearly independent, and aff{x, T1x, T2x,. . . , Tnx} = affS(x). ThenCCSx = x +n∑i=1〈P∩mi=1Ui x− x, ei〉ei,where (i ∈ {1, . . . , n}) ei =Tix−x−∑i−1j=1〈Ti x−x,Tj x−x〉〈Tj x−x,Tj x−x〉 (Tjx−x)‖Tix−x−∑i−1j=1〈Ti x−x,Tj x−x〉〈Tj x−x,Tj x−x〉 (Tjx−x)‖.Proof. By Theorem 7.2.5(iii),CCSx = Paff(S(x))(P∩mi=1Ui x).By Lemma 4.1.2, we know thataff(S(x)) = aff{x, T1x, · · · , Tnx} = x + span{T1x− x, . . . , Tnx− x}.Substitute x1 = T1x, . . . , xn = Tnx in Lemma 4.2.6. Now the affine subspaceM in Lemma 4.2.6 is our aff(S(x)) above. Hence, we obtain the desiredresult.Below we assume n = 2. Using Corollary 6.2.11 and Proposition 7.3.1,we show two closed formulae of the CCS .Proposition 7.3.2. Assume S = {Id, T1, T2} ⊆ Ω. Let x ∈ H. Denote thecardinality of the set {x, T1x, T2x} as lx. Then there are precisely the casesbelow.(1) When lx = 1, CCSx = x.(2) When lx = 2,CCSx ={x+T2x2 , if T1x− x = 0,x+T1x2 , if T1x− x 6= 0.(3) When lx = 3,CCSx = x + α1(x)(T1x− x) + α2(x)(T2x− x)= x + 〈P∩mi=1Ui x− x, e1〉e1 + 〈P∩mi=1Ui x− x, e2〉e2,4if n = 0, that is, S(x) = {x}. Then CCS x = x.927.3. Explicit formulae of the circumcenter operatorwhere α1(x) =‖T1x−x‖2‖T2x−x‖2−‖T2x−x‖2〈T2x−x,T1x−x〉2(‖T1x−x‖2‖T2x−x‖2−〈T2x−x,T1x−x〉2) ,α2(x) =‖T1x−x‖2‖T2x−x‖2−‖T1x−x‖2〈T2x−x,T1x−x〉2(‖T1x−x‖2‖T2x−x‖2−〈T2x−x,T1x−x〉2) , e1 =T1x−x‖T1x−x‖ , and e2 =T2x−x− 〈T2x−x,T1x−x〉〈T1x−x,T1x−x〉 (T1x−x)‖T2x−x− 〈T2x−x,T1x−x〉〈T1x−x,T1x−x〉 (T1x−x)‖.Proof. By Theorem 7.2.5, CCSx ∈ H. Hence, by Theorem 5.7.4, when lx = 3,the two vectors T1x− x, T2x− x must be linearly independent. The requiredresults follow directly from Remark 5.7.3 and Proposition 7.3.1.Now let’s see a particular example of the Proposition 7.3.2.Example 7.3.3. Let U, V be two closed affine subspaces in H. AssumeS = {Id, RU , RV RU}. Hence, for every x ∈ H, CCSx is the unique pointsatisfying(i) CCSx ∈ aff(S(x)) = aff{x, RUx, RV RUx}, and(ii) ‖CCSx− x‖ = ‖CCSx− RUx‖ = ‖CCSx− RV RUx‖.Denote lx as the cardinality of the set {x, RUx, RV RUx}. By Proposition 7.3.2,we have precisely the cases below.(1) When lx = 1, CCSx = x.(2) When lx = 2,CCSx ={x+RV RU x2 , if RUx− x = 0,x+RU x2 , if RUx− x 6= 0.(3) When lx = 3,CCSx = x + α1(x)(RUx− x) + α2(x)(RV RUx− x)= x + 〈PU∩V x− x, e1(x)〉e1(x) + 〈PU∩V x− x, e2(x)〉e2(x),where α1(x) =‖RU x−x‖2‖RV RU x−x‖2−‖RV RU x−x‖2〈RV RU x−x,RU x−x〉2(‖RU x−x‖2‖RV RU x−x‖2−〈RV RU x−x,RU x−x〉2) ,α2(x) =‖RU x−x‖2‖RV RU x−x‖2−‖RU x−x‖2〈RV RU x−x,RU x−x〉2(‖RU x−x‖2‖RV RU x−x‖2−〈RV RU x−x,RU x−x〉2) , e1(x) =RU x−x‖RU x−x‖ ,and e2(x) =RV RU x−x− 〈RV RU x−x,RU x−x〉〈RU x−x,RU x−x〉 (RU x−x)‖RV RU x−x− 〈RV RU x−x,RU x−x〉〈RU x−x,RU x−x〉 (RU x−x)‖.Remark 7.3.4. Actually, the CCS in this Example 7.3.3 is the main actor CTin [12].937.4. Properties of the circumcenter operator7.4 Properties of the circumcenter operatorIn this section, we shall show some fundamental properties of the opera-tor CCS , which will play an important role in Chapter 8.Proposition 7.4.1. ∩mi=1Ui ⊆ Fix CCS .Proof. Let x ∈ ∩mi=1Ui. By Definition 2.3.5, we have(∀i ∈ {1, . . . , m}) x = Id x = PUi x = RUi x. (7.5)Since every element of S is in the form of RUir · · · RUi1 and since, by (7.5),RUir · · · RUi1 x = x, thusaff(S(x)) = aff{RUir · · · RUi1 x | RUir · · · RUi1 ∈ S} = {x}.By definition, we get CCSx = x. Hence, ∩mi=1Ui ⊆ Fix CCS .The following examples show that the converse inclusion in Proposi-tion 7.4.1 is generally not true.Example 7.4.2. Assume ∩mi=1Ui 6= H and S = {Id}. Then ∩mi=1Ui &Fix CCS .Proof. Let x ∈ H. Clearly, aff(S(x)) = {x}. By Theorem 7.2.5, CCSx = x.Hence, Fix CCS = H.Example 7.4.3. Assume H = R2 and m = 2 and Assume S = {Id, RU1}.Assume U1 is the x-axis and U2 is the y-axis. Then U1 ∩U2 & Fix CCS .Proof. By Remark 7.2.2, since x ∈ aff(Sx), thus by Theorem 7.2.5,x = CCSx ⇐⇒ ‖x− x‖ = ‖x− RU1x‖⇐⇒ x = RU1 x⇐⇒ x ∈ U1.Hence, U1 ∩U2 = {(0, 0)} & U1 = Fix CCS .Example 7.4.4. Assume H = R2 and m = 3. As the picture below shows,U1 is x-axis, U2 = {(x, y) ∈ R2 | y = x} and U3 is the y-axis. AssumeS = {Id, RU3 RU2 RU1}. Then ∩3i=1Ui & Fix CCS .947.4. Properties of the circumcenter operatorProof. Let x ∈ U2. Thenx = RU3 RU2 RU1 x,by Definition 6.2.1, which yields that x = CCSx. Hence, U2 ⊆ Fix CCS .Therefore∩3i=1Ui = {(0, 0)} & U2 ⊆ Fix CCS .Figure 7.1: ∩mi=1Ui & Fix CCSLemma 7.4.5. Let TS ∈ affS . Let x ∈ H. Then for every u ∈ ∩mi=1Ui, thefollowing assertions hold:(i) ‖x− CCSx‖2 + ‖CCSx− u‖2 = ‖x− u‖2;(ii) ‖TSx− CCSx‖2 + ‖CCSx− u‖2 = ‖TSx− u‖2.Proof. Let u ∈ ∩mi=1Ui. Using Theorem 7.2.5, we getCCSx = Paff(S(x))(u). (7.6)957.4. Properties of the circumcenter operator(i): By Remark 7.2.2, we know thatx ∈ aff(S(x)).Applying Proposition 4.2.5(ii), i.e., the Pythagorean theorem, we get‖u− Paff(S(x))(u)‖2 + ‖Paff(S(x))(u)− x‖2 = ‖x− u‖2(7.6)⇐⇒‖x− CCSx‖2 + ‖CCSx− u‖2 = ‖x− u‖2.(ii): Because TS ∈ aff(S), by definition of S(x), we have TSx ∈ aff(S(x)).The same proof as above shows that‖TSx− CCSx‖2 + ‖CCSx− u‖2 = ‖TSx− u‖2.So the proof is done.Corollary 7.4.6. Assume additionally U1, . . . , Um are linear subspaces. Then(i) CCS is homogeneous, that is(∀x ∈ H) (∀λ ∈ R) CCS (λx) = λCCSx;(ii)(∀x ∈ H) (∀u ∈ ∩mi=1Ui) CCS (x + u) = CCS (x) + u.Proof. Because U1, . . . , Um are closed linear subspaces. By Fact 2.3.11(i), forevery i ∈ {1, . . . , m}, PUi is linear, hence so is RUi = 2PUi − Id, which yieldsthat for every x, y ∈ H, for every α, β ∈ R,(∀RUir · · · RUi1 ∈ S) RUir · · · RUi1 (αx + βy) = αRUir · · · RUi1 x + βRUir · · · RUi1 y.Note that by Theorem 7.2.5(i), CCS is proper. By Proposition 7.4.1,0 ∈ ∩mi=1Ui ⊆ Fix CCS . Hence,(∀x ∈ H) CCS (0x) = 0 = 0CCSx.Therefore, (i) and (ii) are directly from Proposition 6.3.2 and Proposi-tion 6.3.3 respectively.Lemma 7.4.7. Assume additionally U1, . . . , Um are linear subspaces. Letx ∈ H and RUir · · · RUi1 ∈ S . Then RUir · · · RUi1 x− x ∈ (∩mi=1Ui)⊥, that is,(∀u ∈ ∩mi=1Ui) 〈RUir · · · RUi1 x− x, u〉 = 0.967.4. Properties of the circumcenter operatorProof. Let u ∈ ∩mi=1Ui. Take an arbitrary j ∈ {1, 2, . . . , m}. Let x ∈ H andy ∈ ∩mi=1Ui. Now by Fact 2.3.10(ii)〈RUj(x)− x, u〉 = 〈2(PUj − Id)x, u〉 = 〈−2PU⊥j x, u〉 = 0.Actually, we proved(∀z ∈ H) (∀t ∈ {1, 2, . . . , m}) 〈RUt(z)− z, u〉 = 0. (7.7)Recall ∏0j=1 RUij = Id. So we haveRUir RUir−1 · · · RUi1 (x)− x =r−1∑j=0(RUij+1 RUij · · · RUi1 (x)− RUij · · · RUi1 (x)).(7.8)Hence, 〈RUir RUir−1 · · · RUi1 (x)− x, u〉(7.8)=〈 r−1∑j=0(RUij+1 RUij · · · RUi1 (x)− RUij · · · RUi1 (x)), u〉=r−1∑j=0〈RUij+1(RUij · · · RUi1 (x))− RUij · · · RUi1 (x), u〉(7.7)= 0.Hence, the proof is done.Proposition 7.4.8. Assume additionally U1, . . . , Um are linear subspaces. Letx ∈ H. Then CCSx− x ∈ (∩mi=1Ui)⊥, that is,(∀u ∈ ∩mi=1Ui) 〈CCSx− x, u〉 = 0.Proof. By Theorem 7.2.5(i), we know that CCS is proper. Hence, by Corol-lary 6.2.11 and Id ∈ S , there exist n ∈ N and α1, . . . , αn ∈ R and T1, . . . ,Tn ∈ S such thatCCSx = x +n∑j=1αj(Tjx− x). (7.9)977.4. Properties of the circumcenter operatorLet u ∈ ∩mi=1Ui. Apply Lemma 7.4.7. Since {T1, . . . , Tn} ⊆ S ,n∑j=1αj〈Tjx− x, u〉 = 0.Therefore,〈CCSx− x, u〉 (7.9)=n∑j=1αj〈Tjx− x, u〉 = 0.The proof is complete.98Chapter 8Applications of circumcentermethod induced by reflectors8.1 OverviewThis chapter is devoted to study the iteration sequence of the circumcen-ter mapping induced by reflectors. Therefore, we revisit the assumptions inChapter 7. Let m ∈Nr {0}. We assume thatU1, . . . , Um are closed affine subspaces in the real Hilbert spaceH,with ∩mi=1Ui 6= ∅. LetΩ ={RUir · · · RUi2 RUi1∣∣∣ r ∈N, and i1, . . . , ir ∈ {1, . . . , m}}.SupposeS is a subset of the Ω with{Id} & S ,and for every x ∈ H,S(x) = {Tx | T ∈ S}.As we did in Chapter 7, since for every element T in S , there exists r ∈N andi1, . . . , ir ∈ {1, . . . , m} such that T = RUir · · · RUi2 RUi1 , we always assumethatRUir · · · RUi1 is the representative element of the set S .Let x ∈ H. We get the circumcenter method induced by S :x0 = x, and xk = CCS (xk−1) = CCkSx, where k = 1, 2, . . . .The iteration sequence, (CCkSx)k∈N is our central character in this chapter.Because S consists of compositions of reflectors, we say that the circumcentermethod is the circumcenter method induced by reflectors.The main results of this chapter are summarized as follows.998.1. Overview• Except generalizing all of the properties of the sequence (CkT)k∈N in [12,Lemmas 1, 3 and 4] for our (CCkSx)k∈N (see Proposition 8.2.6), we alsoprove the Fejér monotonicity of (CCkSx)k∈N (see Proposition 8.2.1(i)).The Fejér monotonicity of (CCkSx)k∈N brings us some convergenceresults (see Theorem 8.2.14). Our proof of Proposition 8.2.6(i) uses theFejér monotonicity, which is different from the proof of [12, Lemmas3]. The proof of Theorem 8.2.14 uses [4, Theorem 2.16].• We provide counterexamples such that for some x ∈ H, limk→∞ CCkSx =PFix CCS x 6∈ ∩mi=1Ui (see Proposition 8.2.15).• Motivated by the effect of DRM in the proof of the linear convergenceof (CkTx)k∈N in [12] , we provide the powerful Theorem 8.3.2, which isnecessary for us to prove the linear convergence of (CCkSx)k∈N.• As we mentioned in the overview of Chapter 4, we proved Propo-sition 8.3.3, which is a generalization of [13, Lemma 3.2]. Based onProposition 8.3.3 we show several linear convergence results of thesequence (CCkSx)k∈N (see Proposition 8.5.6 and Proposition 8.5.9).• Under the additional condition that U1, . . . , Um are linear subspaces,we present some applications of the previous Section 7.4 and somenew properties of the sequence (CCkSx)k∈N (see Section 8.5).• Using the idea of the proof of [12, Theorem 1], we prove the muchmore general result Proposition 8.6.1. In view of Proposition 8.6.1, weget Corollary 8.6.3, which extends [12, Theorem 1] from Rn to the realHilbert spaceH. In addition, as an application of our Proposition 8.6.1,the Proposition 8.6.6 considers {Id, RU2 RU1 , RU2 RU1 RU2 RU1} ⊆ S andproves that the three sequences (CCkS (PU1 x))k∈N, (CCkS (PU2 x))k∈Nand (CCkS (PU1+U2 x))k∈N converge to PU1∩U2 x with a linear rate at leastthe square of the sharp linear rate of DRM.• Using the operator A defined in [13, Lemma 2.1], which is an essentialtool in the proof of their main result [13, Theorem 3.3], we obtain theProposition 8.5.6. The operator TS defined in the Proposition 8.5.6below is the operator A defined in [13, Lemma 2.1]. In addition, usingthe idea of the proof in [13, Lemma 2.1], we obtain Lemma 8.5.7 andLemma 8.5.8. In the light of the above two lemmas, we provide theProposition 8.5.9, Proposition 8.5.10 and Corollary 8.5.11.1008.2. Properties of the circumcenter operator iteration sequence• We use the circumcenter method induced by reflectors to solve thefeasibility problem of finitely many closed linear subspaces. (seeProposition 8.7.1)8.2 Properties of the circumcenter operator iterationsequenceProposition 8.2.1. Let x ∈ H. Then for every u ∈ ∩mi=1Ui and for everyk ∈N,‖CCkSx− CCk+1S x‖2 = ‖CCkSx− u‖2 − ‖CCk+1S x− u‖2. (8.1)In particular,(i) (CCkSx)k∈N is a Fejér monotone sequence with respect to ∩mi=1Ui.(ii) CCS is asymptotically regular, that is for every y ∈ H,limk→∞CCkSy− CCk+1S y = 0.Proof. For every k ∈N, substituting x by CCkSx in Lemma 7.4.5, we get(∀u ∈ ∩mi=1Ui) ‖CCkSx− CCk+1S x‖2 + ‖CCk+1S x− u‖2 = ‖CCkSx− u‖2.Hence, (8.1) is proved.(i): By (8.1), it is clear that(∀u ∈ ∩mi=1Ui) (∀k ∈N) ‖CCk+1S x− u‖ ≤ ‖CCkSx− u‖. (8.2)By Definition 2.4.11, (CCkSx)k∈N is a Fejér monotone sequence with respectto ∩mi=1Ui.(ii): Let u ∈ ∩mi=1Ui. By (8.2), we knowLu := limk→+∞‖CCkSx− u‖exists. Summing over k from 0 to infinity in both sides of (8.1), we obtain∞∑k=0‖CCkSx− CCk+1S x‖2 = ‖x− u‖2 − L2u < +∞.So limk→+∞‖CCkSx− CCk+1S x‖ = 0, which yields limk→+∞ CCkSx− CCk+1S x= 0. By Definition 2.4.16, CCS is asymptotically regular.1018.2. Properties of the circumcenter operator iteration sequenceExample 8.2.2. Consider the Example 7.2.6. In this case, m = 2 and S ={Id, RU1 , RU2 RU1}. By Proposition 8.2.1, (CCkSx)k∈N is a Fejér monotonesequence with respect to U1 ∩U2.Example 8.2.3. Consider the Example 7.2.8. Now, S = {Id, RU1 , RU2 , . . . ,RUm}. By Proposition 8.2.1, (CCkSx)k∈N is a Fejér monotone sequence withrespect to ∩mi=1Ui.Lemma 8.2.4. Let x ∈ aff(∪mi=1Ui). Then the following statements hold.(i) affS(x) ⊆ aff(∪mi=1Ui).(ii) (CCkSx)k∈N ⊆ aff(∪mi=1Ui).Proof. (i): Let RUir · · · RUi1 be an arbitrary but fixed element in S . If r = 0,RUir · · · RUi1 x = x ∈ aff(∪mi=1Ui). Assume r ≥ 1. Since i1 ∈ {1, . . . , m},PUi1 x ∈ aff(∪mi=1Ui). SoRUi1 x = 2PUi1 x− x ∈ aff(∪mi=1Ui).Assume for some k ∈ {1, . . . , r− 1},RUik · · · RUi1 x ∈ aff(∪mi=1Ui).Now since ik+1 ∈ {1, . . . , m}, thus PUik+1 (RUik · · · RUi1 x) ∈ aff(∪mi=1Ui).Hence,RUik+1 RUik · · · RUi1 x = 2PUik+1 (RUik · · · RUi1 x)− RUik · · · RUi1 x ∈ aff(∪mi=1Ui).Hence, we have inductively proved RUir · · · RUi1 x ∈ aff(∪mi=1Ui).Since RUir · · · RUi1 x ∈ S(x) is chosen arbitrarily, thusS(x) ⊆ aff(∪mi=1Ui),which yieldsaffS(x) ⊆ aff(∪mi=1Ui).(ii): By Theorem 7.2.5(i) and by the result obtained from the above (i),CCSx ∈ affS(x) ⊆ aff(∪mi=1Ui).Because x ∈ aff(∪mi=1Ui) is chosen arbitrarily, in fact, we proved(∀y ∈ aff(∪mi=1Ui)) CCSy ∈ aff(∪mi=1Ui). (8.3)1028.2. Properties of the circumcenter operator iteration sequenceAssume for some k ∈N,CCkSx ∈ aff(∪mi=1Ui).Then we can substitute y in (8.3) by CCkSx. Hence, we getCCk+1S (x) = CCS (CCkSx)(8.3)∈ aff(∪mi=1Ui).Therefore, the proof is done.Using the same logic in the proof of Lemma 8.2.4, we can get the lemmabelow.Lemma 8.2.5. Let x ∈ span(∪mi=1Ui). Then the following assertions aresatisfied.(i) affS(x) ⊆ span(∪mi=1Ui) = ∑mi=1 Ui.(ii) (CCkSx)k∈N ⊆ span(∪mi=1Ui) = ∑mi=1 Ui.Proof. Using a similar proof in Lemma 4.1.8 and by a simple inductiveargument, we obtainspan(∪mi=1Ui) =m∑i=1Ui.Since the remaining proof is similar with the proof in Lemma 8.2.4, we omitit.Below, we show some properties of CCS , TS ∈ aff(S), RUir · · · RUi1 ∈ S .Recall that we use RUir · · · RUi1 to be the representative element in S .Proposition 8.2.6. Let TS ∈ aff(S) and RUir · · · RUi1 ∈ S . Let x ∈ H. Then(i) (∀k ∈N) P∩mi=1Ui(CCkSx) = P∩mi=1Ui(x).(ii) P∩mi=1Ui(RUir · · · RUi2 RUi1 x) = P∩mi=1Ui(x) andd(x,∩mi=1Ui) = d(RUir · · · RUi2 RUi1 x,∩mi=1Ui).(iii) (∀k ∈N) P∩mi=1Ui(TkS (x)) = P∩mi=1Ui(x).(iv) ‖CCSx− P∩mi=1Ui x‖2 = ‖TSx− P∩mi=1Ui x‖2 − ‖TSx− CCSx‖2.(v) ‖P∩mi=1Ui(CCSx)−CCSx‖2 = ‖P∩mi=1Ui(TSx)−TSx‖2−‖CCSx−TSx‖2.1038.2. Properties of the circumcenter operator iteration sequence(vi) d(CCSx,∩mi=1Ui) = ‖CCSx − P∩mi=1Ui(x)‖ ≤ ‖TSx − P∩mi=1Ui(x)‖ =d(TSx,∩mi=1Ui).Proof. (i): Combining Fact 2.4.13 with Proposition 8.2.1(i), we obtain (i).(ii): By Lemma 7.2.3, we get(∀u ∈ ∩mi=1Ui) ‖x− u‖ = ‖RUir RUir−1 · · · RUi2 RUi1 x− u‖ (8.4)Denote x = P∩mi=1Ui x, xR = P∩mi=1Ui RUir · · · RUi1 (x). Now using x, xR ∈∩mi=1Ui and (8.4), we have‖x− x‖ = ‖RUir · · · RUi1 x− x‖, (8.5)‖x− xR‖ = ‖RUir · · · RUi1 x− xR‖. (8.6)Therefore,‖x− x‖ ≤ ‖x− xR‖ (by x = P∩mi=1Ui x, xR ∈ ∩mi=1Ui)(8.6)= ‖RUir · · · RUi1 x− xR‖≤ ‖RUir · · · RUi1 x− x‖ (by xR = P∩mi=1Ui RUir · · · RUi1 (x), x ∈ ∩mi=1Ui)(8.5)= ‖x− x‖,so ‖x − x‖ = ‖x − xR‖ = ‖RUir · · · RUi1 x − xR‖ = ‖RUir · · · RUi1 x − x‖.Hence,d(x,∩mi=1Ui) = ‖x− x‖= ‖x− xR‖= ‖RUir · · · RUi1 x− x‖= ‖RUir · · · RUi1 x− xR‖= d(RUir · · · RUi1 x,∩mi=1Ui).Moreover, because ∩mi=1Ui is nonempty, closed and convex, by Fact 2.3.7and Definition 2.3.5, both P∩mi=1Ui x and P∩mi=1Ui RUir · · · RUi1 (x) are singletons.Hence, x = xR. Therefore,P∩mi=1Ui x = x = xR = P∩mi=1Ui(RUir · · · RUi2 RUi1 x).In fact, we have proved that(∀T ∈ S) P∩mi=1Ui(Tx) = P∩mi=1Ui(x). (8.7)1048.2. Properties of the circumcenter operator iteration sequence(iii): Since TS ∈ aff(S),TS =t∑i=1αiTifor some T1, . . . , Tt ∈ S and (∀i ∈ {1, 2, . . . , t}) αi ∈ [0, 1] with ∑ti=1 αi = 1.Because P∩mi=1Ui is affine, which follows from Proposition 4.2.5(iv), we haveP∩mi=1Ui(TSx) = P∩mi=1Ui(t∑i=1αiTi(x))=t∑i=1αiP∩mi=1Ui(Ti(x))(8.7)=t∑i=1αiP∩mi=1Ui(x) = P∩mi=1Ui(x).Actually, we have proved(∀y ∈ H) P∩mi=1Ui(TS (y)) = P∩mi=1Ui(y). (8.8)Assume for some k ∈N, we haveP∩mi=1Ui(TkS (x)) = P∩mi=1Ui(x). (8.9)NowP∩mi=1Ui(Tk+1S (x)) = P∩mi=1Ui(TS (TkS (x)))(8.8)= P∩mi=1Ui(TkS (x))(8.9)= P∩mi=1Ui(x).Hence, we have inductively proved(∀k ∈N) P∩mi=1Ui(TkS (x)) = P∩mi=1Ui(x)(iv): Substitute u in Lemma 7.4.5(ii) by P∩mi=1Ui x. The desired resultfollows.(v): Using (iv), we get‖P∩mi=1Ui x− TSx‖2 = ‖P∩mi=1Ui x− CCSx‖2 + ‖TSx− CCSx‖2(iii)⇐⇒‖P∩mi=1Ui(TSx)− TSx‖2 = ‖P∩mi=1Ui x− CCSx‖2 + ‖TSx− CCSx‖2(i)⇐⇒‖P∩mi=1Ui(TSx)− TSx‖2 = ‖P∩mi=1Ui(CCSx)− CCSx‖2 + ‖CCSx− TSx‖2.1058.2. Properties of the circumcenter operator iteration sequence(vi): Nowd(CCSx,∩mi=1Ui) = ‖CCSx− P∩mi=1Ui(CCSx)‖(i)= ‖CCSx− P∩mi=1Ui x‖(iv)≤ ‖TSx− P∩mi=1Ui x‖(iii)= ‖TSx− P∩mi=1Ui(TSx)‖= d(TSx,∩mi=1Ui).The proof is done.Corollary 8.2.7. Let x ∈ H. Then‖CCSx− P∩mi=1Ui x‖ = d(aff(S(x)),∩mi=1Ui).Proof. By Theorem 7.2.5, (∀u ∈ ∩mi=1Ui) CCSx = Paff(S(x))(u), which impliesthat(∀u ∈ ∩mi=1Ui) ‖CCSx− u‖ = d(aff(S(x)), u). (8.10)Now taking infimum over all u in ∩mi=1Ui in (8.10), we getminu∈∩mi=1Ui‖CCSx− u‖ = minu∈∩mi=1Uid(aff(S(x)), u) (8.11)‖CCSx− P∩mi=1Ui(x)‖ = ‖CCSx− P∩mi=1Ui(CCSx)‖ (by Proposition 8.2.6(i))= d(CCSx,∩mi=1Ui)= minu∈∩mi=1Ui‖CCSx− u‖(8.11)= minu∈∩mi=1Uid(aff(S(x)), u)= d(aff(S(x)),∩mi=1Ui).Proposition 8.2.8. Let x ∈ H. Then the following statements are equivalent:(i) CCSx ∈ ∩mi=1Ui;(ii) CCSx = P∩mi=1Ui x;1068.2. Properties of the circumcenter operator iteration sequence(iii) (∀k ≥ 1) CCkSx = P∩mi=1Ui x.Proof. (i)⇒ (ii). If CCSx ∈ ∩mi=1Ui, then CCSx = P∩mi=1Ui(CCSx) = P∩mi=1Ui xusing Proposition 8.2.6(i).(ii)⇒ (iii). Assume CCSx = P∩mi=1Ui x. By Proposition 7.4.1, P∩mi=1Ui x ∈∩mi=1Ui ⊆ Fix CCS . Hence,(∀k ≥ 2) CCkSx = CCk−1S (CCSx) = CCk−1S (P∩mi=1Ui x) = P∩mi=1Ui x.(iii)⇒ (i). Take k = 1.Corollary 8.2.9. Let x ∈ H. Assume limk→∞ CCkSx 6= P∩mi=1Ui x. Then(∀k ∈N) CCkSx 6∈ ∩mi=1Ui. (8.12)Proof. We argue by contradiction and thus assume there exists n ∈N suchthat CCnSx ∈ ∩mi=1Ui. If n = 0, then, by Proposition 7.4.1, (∀k ∈N) CCkSx =x = P∩mi=1Ui x, which contradicts the assumption, limk→∞ CCkSx 6= P∩mi=1Ui x.Assume n ≥ 1. Apply x in Proposition 8.2.8 for (i)⇔ (ii) to CCn−1S (x). Weget CCnS (x) = P∩mi=1Ui(CCn−1S (x)) = P∩mi=1Ui x, by Proposition 8.2.6(i). UsingProposition 7.4.1 again, we get(∀k ≥ n) CCkSx = CCnS (x) = P∩mi=1Ui x,which yields that limk→∞ CCkSx = P∩mi=1Ui x. It contradicts with the assump-tion again.Corollary 8.2.10. Recall that {Id} & S ⊆ Ω. The following assertions hold.(i) P∩mi=1Ui CCS = P∩mi=1Ui = CCSP∩mi=1Ui .(ii) Assume, in addition, U1, . . . , Um are linear subspaces inH. ThenCCSP(∩mi=1Ui)⊥ = CCS − P∩mi=1Ui = P(∩mi=1Ui)⊥CCS .Proof. (i): Let x ∈ H. Use Proposition 8.2.6(i) and take k = 1. We getP∩mi=1Ui(CCSx) = P∩mi=1Ui x. (8.13)By Proposition 7.4.1, P∩mi=1Ui x ∈ Fix CCS . Hence,CCS (P∩mi=1Ui x) = P∩mi=1Ui x. (8.14)1078.2. Properties of the circumcenter operator iteration sequenceCombining (8.13) with (8.14), we obtain(∀x ∈ H) P∩mi=1Ui(CCSx) = P∩mi=1Ui x = CCS (P∩mi=1Ui x). (8.15)Hence, P∩mi=1Ui CCS = P∩mi=1Ui = P∩mi=1Ui CCS .(ii): Assume U1, . . . , Um are linear subspaces in H. Let x ∈ H. ByFact 2.3.10(ii), we get P(∩mi=1Ui)⊥ = Id−P∩mi=1Ui . Take u = −P∩mi=1Ui x in Corol-lary 7.4.6(ii). We get CCS (x− P∩mi=1Ui x) = CCSx− P∩mi=1Ui x. Hence,CCS (P(∩mi=1Ui)⊥x) = CCS (x− P∩mi=1Ui x) = CCSx− P∩mi=1Ui x. (8.16)On the other hand,P(∩mi=1Ui)⊥(CCSx) = CCSx− P∩mi=1Ui CCSx(8.13)= CCSx− P∩mi=1Ui x. (8.17)Thus, (8.16) and (8.17) yieldCCSP(∩mi=1Ui)⊥ = CCS − P∩mi=1Ui = P(∩mi=1Ui)⊥CCS .The following example tells us that in Corollary 8.2.10(ii), the condition“U1, . . . , Um are linear subspaces inH” is indeed necessary.Example 8.2.11. Assume H = R2 and U1 := {(x, y) ∈ R2 | y = 1} andU2 := {(x, y) ∈ R2 | y = x + 1}. Let x ∈ R2. Assume S = {Id, RU1 ,RU2 RU1} or S = {Id, RU1 , RU2}. ThenCCSP(U1∩U2)⊥x 6= CCSx− PU1∩U2 x = P(U1∩U2)⊥CCSx.Proof. Since U1 ∩ U2 = {(0, 1)}, (∀y ∈ R2) PU1∩U2 y = (0, 1). Using thefollowing Example 8.4.4, we get(∀y ∈ R2) CCSy = PU1∩U2 y = (0, 1). (8.18)Substituting y in (8.18) by P(U1∩U2)⊥x, we getCCSP(U1∩U2)⊥x = (0, 1) 6= (0, 0).On the other hand, by Definition 2.1.3,(U1 ∩U2)⊥ = {(x, y) ∈ R2 | 〈(x, y), (0, 1)〉 = 0} = {(x, y) ∈ R2 | y = 0}.1088.2. Properties of the circumcenter operator iteration sequenceSoP(U1∩U2)⊥CCSx(8.18)= P(U1∩U2)⊥(0, 1) = (0, 0).Moreover,CCSx− PU1∩U2 x(8.18)= (0, 1)− (0, 1) = (0, 0).The proof is done.Lemma 8.2.12. Assume m = 2. Assume U1, U2 are closed linear subspaces.Let w ∈ U1 +U2. Denote T := TU2,U1 defined in Definition 3.3.1. Then(∀k ∈N) PU1∩U2(w) = PU1∩U2(CCkSw) = PFix T(CCkSw).Proof. Using Lemma 8.2.5(ii), we get(CCkSw)k∈N ⊆ U1 +U2.Combining Lemma 3.3.5 and Proposition 8.2.6(i), we get(∀k ∈N) PFix Tw = PU1∩U2(w) = PU1∩U2(CCkSw) = PFix T(CCkSw).Corollary 8.2.13. Assume m = 2. Assume U1, U2 are closed linear subspaces.Let x ∈ H. Let W be a closed linear subspace ofH such thatU1 ∩U2 ⊆W ⊆ U1 +U2.Denote T := TU2,U1 defined in Definition 3.3.1 and w := PW x. Then(∀k ∈N) PU1∩U2 x = PFix Tw = PU1∩U2 w = PU1∩U2(CCkSw) = PFix T(CCkSw).Proof. Because W ⊆ U1 +U2, w = PW x ∈W ⊆ U1 +U2. Then Lemma 3.3.7implies thatPU1∩U2 x = PFix Tw = PU1∩U2 w. (8.19)Using w ∈ U1 +U2 again and applying Lemma 8.2.12, we get the desiredresult.Using the Proposition 8.2.1 and Proposition 8.2.6(i), we can get theconvergence properties of CCS below.1098.2. Properties of the circumcenter operator iteration sequenceTheorem 8.2.14. Let x ∈ H. Then the following hold:(i) If (CCkSx)k∈N has a norm cluster point in ∩mi=1Ui, then (CCkSx)k∈Nconverges in norm to P∩mi=1Ui(x).(ii) (CCkSx)k∈N is bounded and(∀k ∈N) d(CCk+1S (x),∩mi=1Ui) ≤ d(CCkSx,∩mi=1Ui).(iii) (CCkSx)k∈N has at most one weak cluster point in ∩mi=1Ui. Conse-quently, (CCkSx)k∈N converges weakly to some point in ∩mi=1Ui if andonly if all weak cluster points of (CCkSx)k∈N lie in ∩mi=1Ui.(iv) The following are equivalent.(a) (CCkSx)k∈N converges in norm to P∩mi=1Ui(x).(b) (CCkSx)k∈N converges in norm to some point in ∩mi=1Ui.(c) (CCkSx)k∈N has norm cluster points, all lying in ∩mi=1Ui.(d) (CCkSx)k∈N has norm cluster points, one lying in ∩mi=1Ui.Proof. (i): Assume x ∈ ∩mi=1Ui is a norm cluster point of (CCkSx)k∈N, that isthere exists a subsequence (CCk jS (x))j∈N of (CCkSx)k∈N such that limj→∞ CCk jS x= x. Now for every j ∈N,‖CCk jS x− P∩mi=1Ui(x)‖ = ‖CCk jS x− P∩mi=1Ui(CCk jS x)‖ (by Proposition 8.2.6(i))≤ ‖CCk jS x− x‖. (since x ∈ ∩mi=1Ui)So0 ≤ limj→∞‖CCk jS x− P∩mi=1Ui(x)‖ ≤ limj→∞‖CCk jS x− x‖ = 0.Hence, limj→+∞ CCk jS x = P∩mi=1Ui(x).Because P∩mi=1Ui(x) ∈ ∩mi=1Ui, we can substitute u in (8.2) of Proposi-tion 8.2.1 by P∩mi=1Ui(x), then we knowlimk→+∞‖CCkSx− P∩mi=1Ui(x)‖ exists.Hence,limk→+∞‖CCkSx− P∩mi=1Ui(x)‖ = limj→+∞‖CCk jS x− P∩mi=1Ui(x)‖ = 0,from which follows that (CCkSx)k∈N converges in norm to P∩mi=1Ui(x).(ii), (iii) and (iv) follow from Fact 2.4.15 and our (i) above.1108.2. Properties of the circumcenter operator iteration sequenceThe results below show that there exist S with {Id} & S ⊆ Ω and x ∈ Hsuch that limk→∞ CCkSx 6∈ ∩mi=1Ui, which implies that (CCkSx)k∈N has nonorm cluster point in ∩mi=1Ui.Proposition 8.2.15. Assume m = 2 and U1, U2 are two closed linear sub-spaces in H such that U1 + U2 is closed. Assume S = {Id, RU2 RU1}. Letx ∈(Hr (U1 +U2)). Thenlimk→∞CCkSx = PFix CCS x 6∈ U1 ∩U2.Proof. By definition of S and by Proposition 6.2.2,CCS = TU2,U1 ,where the TU2,U1 is the Douglas–Rachford operator defined in Definition 3.3.1.By assumptions, Fact 3.2.4 and Fact 3.3.4 imply that (CCkSx)k∈N convergeslinearly to PFix CCS x. Solimk→∞CCkSx = PFix CCS x. (8.20)Since x 6∈ U1 +U2 = U1 +U2, Lemma 3.3.5 yield thatPFix CCS x 6= PU1∩U2 x. (8.21)Assume to the contrary PFix CCS x ∈ U1 ∩U2. By Theorem 8.2.14(iv) and(8.20), we get PFix CCS x = PU1∩U2 x, which contradicts (8.21).Therefore, limk→∞ CCkSx = PFix CCS x 6∈ U1 ∩U2.Remark 8.2.16. Under the assumptions in Proposition 8.2.15, we knowlimk→∞ CCkSx = PFix CCS x 6∈ U1 ∩U2. Then clearly limk→∞ CCkSx 6= P∩mi=1Ui x.Hence, by Corollary 8.2.9, we get(∀k ∈N) CCkSx 6∈ ∩mi=1Ui.Example 8.2.17. Assume H = R3. Assume m = 2, U1 = {(x, y, z) ∈R3 | x ∈ R, y = 0, z = 0} and U2 = {(x, y, z) ∈ R3 | x = 0, y ∈ R, z = 0}.Assume S = {Id, RU2 RU1}. Let x = (x1, x2, x3) ∈ R3 with x3 6= 0. Thenlimk→∞CCkS (x) 6∈ ∩mi=1Ui.1118.3. Linear convergence of circumcenter methods induced by reflectorsProof. Clearly, PU1 x = (x1, 0, 0), RU1 x = 2PU1 x− x = (x1,−x2,−x3), PU2 RU1 x= (0,−x2, 0), RU2 RU1 x = 2PU2 RU1 x−RU1 x = 2(0,−x2, 0)− (x1,−x2,−x3) =(−x1,−x2, x3). So, by Proposition 6.2.2,x(1) := CCSx =Id x + RU2 RU1 x2= (0, 0, x3).Now PU1 x(1) = (0, 0, 0), RU1 x(1) = 2PU1 x(1)− x(1) = (0, 0,−x3), PU2 RU1 x(1) =(0, 0, 0), RU2 RU1 x(1) = 2PU2 RU1 x(1) − RU1 x(1) = 2(0, 0, 0)− (0, 0,−x3) = (0,0, x3). Hence,CC2Sx =Id x(1) + RU2 RU1 x(1)2= (0, 0, x3).Then it is easy to see that(∀k ∈Nr {0}) CCkSx = (0, 0, x3).Because x3 6= 0, and U1 ∩U2 = {(0, 0, 0)}, we obtain thatlimk→∞CCkSx = (0, 0, x3) 6∈ U1 ∩U2.Remark 8.2.18. In Example 8.2.17, clearlyU1 +U2 = U1 +U2 = R2 × {0}.Hence, the assumption x = (x1, x2, x3) ∈ R3 with x3 6= 0 is x 6∈ U1 + U2.Therefore, actually Example 8.2.17 is a corollary of Proposition 8.2.15.8.3 Linear convergence of circumcenter methodsinduced by reflectorsTo facilitate the proof later, first we provide the lemma below.Lemma 8.3.1. Assume there exists γ ∈ [0, 1[ such that(∀x ∈ H) ‖CCSx− P∩mi=1Ui(x)‖ ≤ γ‖x− P∩mi=1Ui(x)‖. (8.22)Then(∀x ∈ H) (∀k ∈N) ‖CCkSx− P∩mi=1Ui(x)‖ ≤ γk‖x− P∩mi=1Ui(x)‖.1128.3. Linear convergence of circumcenter methods induced by reflectorsProof. For k = 0, the result is trivial.Assume for some k ∈Nwe have(∀x ∈ H) ‖CCkSx− P∩mi=1Ui(x)‖ ≤ γk‖x− P∩mi=1Ui(x)‖. (8.23)Let x ∈ H. Now‖CCk+1S x− P∩mi=1Ui(x)‖= ‖CCS (CCkSx)− P∩mi=1Ui(CCkSx)‖ (by Proposition 8.2.6(i))(8.22)≤ γ‖CCkSx− P∩mi=1Ui(CCkSx)‖= γ‖CCkSx− P∩mi=1Ui(x)‖ (by Proposition 8.2.6(i))(8.23)≤ γk+1‖x− P∩mi=1Ui(x)‖Hence, we get the desired result inductively.The powerful result below will play an essential role to prove the linearconvergence of the circumcenter method induced by reflectors.Theorem 8.3.2. Assume there exist TS ∈ aff(S) and γ ∈ [0, 1[, such that(∀x ∈ H) ‖TSx− P∩mi=1Ui(x)‖ ≤ γ‖x− P∩mi=1Ui(x)‖. (8.24)Then(∀x ∈ H) (∀k ∈N) ‖CCkSx− P∩mi=1Ui(x)‖ ≤ γk‖x− P∩mi=1Ui(x)‖.Consequently, CCkSx linearly converges to P∩mi=1Ui(x) with a linear rate γ.Proof. Using Proposition 8.2.6(iv) and the assumption, we have(∀x ∈ H) ‖CCSx− P∩mi=1Ui(x)‖ ≤ ‖TSx− P∩mi=1Ui(x)‖(8.24)≤ γ‖x− P∩mi=1Ui(x)‖.So(∀x ∈ H) ‖CCSx− P∩mi=1Ui(x)‖ ≤ γ‖x− P∩mi=1Ui(x)‖.Hence, the required result follows directly from Lemma 8.3.1.Proposition 8.3.3. Let H = Rn. Assume there exists TS ∈ aff(S) suchthat TS is linear and firmly nonexpansive and Fix TS = ∩mi=1Ui. Then forevery x ∈ H, (CCkSx)k∈N converges linearly to P∩mi=1Ui x with a linear rate‖TSP(∩mi=1Ui)⊥‖ ∈ [0, 1[.1138.4. Applications to two special classes of setsProof. Since Fix TS = ∩mi=1Ui, thusTSP∩mi=1Ui = P∩mi=1Ui . (8.25)Combining Fix TS = ∩mi=1Ui with Proposition 4.4.2, we knowγ := ‖TSP(∩mi=1Ui)⊥‖ = ‖TSP(Fix TS )⊥‖ < 1. (8.26)Now for every x ∈ Rn,‖TSx− P∩mi=1Ui(x)‖(8.25)= ‖TSx− TSP∩mi=1Ui(x)‖= ‖TS (x− P∩mi=1Ui(x))‖ (TS linear)= ‖TSP(∩mi=1Ui)⊥(x)‖ (by Fact 2.3.10(ii))= ‖TSP(∩mi=1Ui)⊥P(∩mi=1Ui)⊥(x)‖ (by Fact 2.3.11(ii))≤ ‖TSP(∩mi=1Ui)⊥‖‖P(∩mi=1Ui)⊥(x)‖= ‖TSP(∩mi=1Ui)⊥‖‖x− P∩mi=1Ui(x)‖ (by Fact 2.3.10(ii))= γ‖x− P∩mi=1Ui(x)‖Hence, the desired result follows directly from Theorem 8.3.2.In Section 8.5, we shall see some particular applications of the Proposi-tion 8.3.3.8.4 Applications to two special classes of setsIn the whole section, we assume either{Id, RU1 , RU2 , . . . , RUm} ⊆ S (8.27)or{Id, RU1 , RU2 RU1 , RU3 RU2 RU1 , . . . , RUm RUm−1 · · · RU2 RU1} ⊆ S . (8.28)Under the special assumption, we find more interesting properties of theCCS . In the remaining sections in the chapter, we mainly consider the twospecial classes of S above.Although Examples 7.4.2, 7.4.3 and 7.4.4 tell us that ∩mi=1Ui = Fix CCS isnot always true, the proposition below says that for the special cases in thissection, ∩mi=1Ui = Fix CCS always hold.1148.4. Applications to two special classes of setsProposition 8.4.1. ∩mi=1Ui = Fix CCS .Proof. Assume {Id, RU1 , RU2 , . . . , RUm} ⊆ S . Now, since x ∈ affS(x), byTheorem 7.2.5,CCSx = x ⇐⇒ ‖x− x‖ = ‖x− RU1 x‖ = · · · = ‖x− RUm x‖⇐⇒ x = RU1 x = · · · = RUm x. (8.29)Hence,x ∈ Fix CCS ⇐⇒ CCSx = x(8.29)⇐⇒ (∀i ∈ {1, 2, . . . , m}) x = RUi x⇐⇒ (∀i ∈ {1, 2, . . . , m}) x = PUi x (by RUi x = 2PUi x− x)⇐⇒ x ∈ ∩mi=1Ui.Similarly, when {Id, RU1 , RU2 RU1 , . . . , RUm RUm−1 · · · RU2 RU1} ⊆ S , we havex ∈ Fix CCS ⇐⇒ x = CCSx⇐⇒ x = RU1 x = RU2 RU1 x = · · · = RUm RUm−1 · · · RU2 RU1 x⇐⇒ (∀i ∈ {1, 2, . . . , m}) x = PUi x⇐⇒ x ∈ ∩mi=1Ui.Therefore, in both cases, we obtain Fix CCS = ∩mi=1Ui.From Proposition 8.4.1, we get the corollary below.Corollary 8.4.2. Assume S = {Id, RU1 , RU2 RU1 , . . . , RUm RUm−1 · · · RU2 RU1}or S = {Id, RU1 , RU2 , . . . , RUm} , then ∩mi=1Ui = Fix CCS .Proposition 8.4.3. CCS is firmly quasinonexpansive, that is,(∀x ∈ H) (∀u ∈ Fix CCS ) ‖CCSx− u‖2 + ‖CCSx− x‖2 ≤ ‖x− u‖2.Proof. Apply (8.1) in Proposition 8.2.1 to k = 0. We know(∀x ∈ H) (∀u ∈ ∩mi=1Ui) ‖x− CCSx‖2 = ‖x− u‖2 − ‖CCSx− u‖2.Hence, by Definition 2.3.21(iv), Proposition 8.4.1 yields that CCS is firmlyquasinonexpansive.The example below says that for some x ∈ R2, the best approximatePU1∩U2 x is our CCSx, that is, in some cases, we only need one iteration ofthe circumcenter method to find the best approximate PU1∩U2 x.1158.4. Applications to two special classes of setsExample 8.4.4. Assume H = R2. Assume U1, U2 are closed affine spacesin R2, with U1 ∩ U2 6= ∅, assume S = {Id, RU1 , RU2 RU1} (respectivelyS = {Id, RU1 , RU2}). Let x ∈ R2. ThenPU1∩U2 x = CCSx,provided that x, RU1 x, RU2 RU1 x are pairwise distinct (respectively x, RU1 x,RU2 x are pairwise distinct).Proof. Let x ∈ R2. Assume U1 = U2. Then RU1 x = RU2 x. By Proposi-tion 6.2.2, in both of the S ,CCSx =Id x + RU1 x2= PU1 x = PU1∩U2 x.If U1 = R2 (or U2 = R2), then RU1 = Id (RU2 = Id). Hence,CCSx =Id x + RU2 x2= PU2 x = PU1∩U2 x.(CCSx =Id x + RU1 x2= PU1 x = PU1∩U2 x.)Suppose U1 6= U2, U1 6= R2 and U2 6= R2. SinceH = R2 and U1 ∩U2 6=∅, thus U1 ∩U2 is a singleton. Hence,U1 ∩U2 = {PU1∩U2 x}. (8.30)Now consider S = {Id, RU1 , RU2 RU1}. By Theorem 7.2.5,‖CCSx− x‖ = ‖CCSx− RU1 x‖ = ‖CCSx− RU2 RU1 x‖.InR2, since x, RU1 x, RU2 RU1 x are pairwise distinct, ‖CCSx− x‖ = ‖CCSx−RU1 x‖ implies that CCSx ∈ U1 and ‖CCSx− RU1 x‖ = ‖CCSx− RU2 RU1 x‖implies that CCSx ∈ U2. Hence, CCSx ∈ U1 ∩U2. Combining with (8.30),we obtainCCSx = PU1∩U2 x.Because the proof of the case when S = {Id, RU1 , RU2} is similar, we omit ithere. Therefore, we are done.The figure below is consistent with the result in Example 8.4.4. (Notethat the following pictures illustrate convergence in one step while theDouglas–Rachford method will spiral slowly towards the solution.)1168.4. Applications to two special classes of setsFigure 8.1: S = {Id, RU1 , RU2 RU1} in R2Figure 8.2: S = {Id, RU1 , RU2} in R2With the condition that (∀i ∈ {1, . . . , m}) Ui is a linear subspace, weshall deduce more pleasing properties for the two special classes of S above.1178.5. Application to linear subspaces8.5 Application to linear subspacesIn the whole section, we assume additionally thatU1, . . . , Um are linear subspaces. (8.31)Hence, by Fact 2.3.11(i), (∀i ∈ {1, . . . , m}) PUi is linear. Since by Defini-tion 2.3.9 (∀i ∈ {1, . . . , m}) RUi = 2PUi − Id, thus (∀i ∈ {1, . . . , m}) RUi islinear.Proposition 8.5.1. Recall {Id} & S ⊆ Ω. The following assertions hold.(i) Let T : H → H satisfy (∀x ∈ H) Tx ∈ affS(x). Then(∀x ∈ H) ‖CCSx‖2 + ‖CCSx− Tx‖2 = ‖Tx‖2.(ii) Let TS ∈ affS . Then(∀x ∈ H) ‖CCSx‖2 + ‖CCSx− TSx‖2 = ‖TSx‖2.(iii) (x ∈ H) ‖CCSx‖ ≤ ‖x‖.Proof. (i): By Theorem 7.2.5(ii), we get(∀x ∈ H) (∀u ∈ ∩mi=1Ui) CCSx = Paff(S(x))(u).Because (∀x ∈ H) Tx ∈ affS(x). Using Proposition 4.2.5(ii), we get(∀x ∈ H) (∀u ∈ ∩mi=1Ui) ‖u− CCSx‖2 + ‖CCSx− Tx‖2 = ‖u− Tx‖2.(8.32)Because U1, . . . , Um are linear subspaces, we know 0 ∈ ∩mi=1Ui. Take u = 0in (8.32) to obtain(∀x ∈ H) ‖CCSx‖2 + ‖CCSx− Tx‖2 = ‖Tx‖2.(ii): Since TS ∈ affS ,(∀x ∈ H) TSx ∈ affS(x).Substitute T in (i) by TS to obtain the required result.(iii): Since Id ∈ affS , the desired result follows from (ii) with TS =Id.1188.5. Application to linear subspacesLemma 8.5.2. Let x ∈ (∩mi=1Ui)⊥. Then(∀k ∈N) CCkSx ∈ (∩mi=1Ui)⊥.Proof. Because U1, . . . , Um are linear subspaces, using Corollary 8.2.10(ii),we getCCSP(∩mi=1Ui)⊥ = P(∩mi=1Ui)⊥CCS . (8.33)Since x ∈ (∩mi=1Ui)⊥,CCSx = CCSP(∩mi=1Ui)⊥x(8.33)= P(∩mi=1Ui)⊥CCSx ∈ (∩mi=1Ui)⊥,which implies that(∀y ∈ (∩mi=1Ui)⊥) CCSy ∈ (∩mi=1Ui)⊥. (8.34)Now assume that for some k ∈N,CCkSx ∈ (∩mi=1Ui)⊥. (8.35)Using (8.34) and (8.35), we getCCk+1S x = CCS (CCkSx) ∈ (∩mi=1Ui)⊥.Therefore, we have inductively proved the result.Lemma 8.5.3. Let u ∈ ∩mi=1Ui. Then(∀x ∈ H) (∀k ∈N) CCkS (x + u) = CCkSx + uProof. If k = 0, the result is trivial. By Corollary 7.4.6(ii), we get(∀x ∈ H) CCS (x + u) = CCSx + u (8.36)Assume for some k ∈Nwe haveCCkS (x + u) = CCkS (x) + u. (8.37)NowCCk+1S (x + u) = CCS (CCkS (x + u))(8.37)= CCS (CCkS (x) + u)(8.36)= CCk+1S (x + u)Therefore, we have inductively proved the desired result.1198.5. Application to linear subspacesProposition 8.5.4. Let x ∈ H. Then(∀k ∈N) CCkSx− x ∈ (∩mi=1Ui)⊥,that is,(∀k ∈N) (∀u ∈ ∩mi=1Ui) 〈CCkSx− x, u〉 = 0. (8.38)Proof. When k = 0, (8.38) is trivial. By Proposition 7.4.8,(∀z ∈ H) (∀u ∈ ∩mi=1Ui) 〈CCSz− z, u〉 = 0. (8.39)Then for every k ∈Nr {0}, and for every u ∈ ∩mi=1Ui,〈CCkSx− x, u〉 =〈 k−1∑i=0(CCi+1S (x)− CCiS (x)), u〉=〈 k−1∑i=0(CCS (CCiS (x))− CCiS (x)), u〉=k−1∑i=0〈CCS (CCiS (x))− CCiS (x), u〉(8.39)= 0.Hence, we are done.Remark 8.5.5. Under the condition that U1, . . . , Um are linear subspaces,using Proposition 8.5.4, we can get(∀x ∈ H) (∀k ∈N) P∩mi=1Ui(x)− P∩mi=1Ui(CCkSx) = P∩mi=1Ui(x− CCkSx) = 0.Hence, we can also prove Proposition 8.2.6(i).The operator TS defined in the Proposition 8.5.6 below is the operator Adefined in [13, Lemma 2.1].Proposition 8.5.6. AssumeH = Rn. Assume{Id, RU1 , RU2 RU1 , . . . , RUm RUm−1 · · · RU2 RU1} ⊆ S .Define TS : Rn → Rn byTS (x) =1mm∑i=1Ti(x),where T1 = 12 (Id+PU1) and (∀i ∈ {2, . . . , m}) Ti = 12 (Id+PUi RUi−1 · · · RU1).Then (CCkSx)k∈N converges to P∩mi=1Ui x with a linear rate ‖TSP(∩mi=1Ui)⊥‖ ∈[0, 1[.1208.5. Application to linear subspacesProof. NowT1 =12(Id+PU1) =12(Id+Id+RU12) =34Id+14RU1∈ aff{Id, RU1 , RU2 RU1 , . . . , RUm RUm−1 · · · RU2 RU1},and for every i ∈ {2, . . . , m},Ti =12(Id+PUi RUi−1 · · · RU1)=12(Id+(RUi + Id2)RUi−1 · · · RU1)=12Id+14RUi RUi−1 · · · RU1 +14RUi−1 · · · RU1∈ aff{Id, RU1 , RU2 RU1 , . . . , RUm RUm−1 · · · RU2 RU1},which yield thatTS =1mm∑i=1Ti(x) ∈ aff{Id, RU1 , RU2 RU1 , . . . , RUm RUm−1 · · · RU2 RU1} ⊆ aff(S).Using [13, Lemma 2.1 (i)], we know the TS is linear and firmly nonexpansive,and by [13, Lemma 2.1 (ii)], Fix TS = ∩mi=1Ui. Hence, the required resultfollows readily from Proposition 8.3.3.The idea of the proof in the following two lemmas are obtained from [13,Lemma 2.1].Lemma 8.5.7. Assume H = Rn and {Id, RU1 , . . . , RUm−1 , RUm} ⊆ S . Definethe operator TS : Rn → Rn by(∀x ∈ Rn) TS (x) = 1mm∑i=1PUi x.Then the following statements hold.(i) TS ∈ aff(S).(ii) TS is linear and firmly nonexpansive.(iii) Fix TS = ∩mi=1Ui.1218.5. Application to linear subspacesProof. (i): Now (∀i ∈ {1, . . . , m}), PUi =Id+RUi2 , soTS =1mm∑i=1PUi =1mm∑i=1Id+RUi2∈ aff{Id, RU1 , . . . , RUm−1 , RUm} ⊆ aff(S).(ii): Let i ∈ {1, . . . , m}. Using Fact 2.3.8(ii) and Fact 2.3.25, we know PUiis firmly nonexpansive and it is 12 -averaged. Using Fact 2.3.26 and Fact 2.3.25again, we know TS is firmly nonexpansive. By Fact 2.3.11(i), TS is linear.(iii): Since (∀i ∈ {1, . . . , m}) Fix PUi = Ui. The required result is obtainedfrom Fact 2.3.8(ii), Remark 2.3.22 and Fact 2.3.27.Lemma 8.5.8. Assume H = Rn and {Id, RU1 , . . . , RUm−1 , RUm} ⊆ S . Definethe operator TS : Rn → Rn by(∀x ∈ Rn) TS (x) = 1mm∑i=1Tix,where (∀i ∈ {1, 2, . . . , m}) Ti = 12 (Id+PUi). Then(i) TS ∈ aff(S).(ii) TS is linear and firmly nonexpansive.(iii) Fix TS = ∩mi=1Ui.Proof. (i): Now for every i ∈ {1, . . . , m}, Ti = 12 (Id+PUi) = 12 (Id+Id+RUi2 ) =34 Id+14 RUi . Hence,TS =1mm∑i=1Ti =1mm∑i=1(34Id+14RUi) ∈ aff{Id, RU1 , RU2 , . . . , RUm} ⊆ aff(S).(ii): Let i ∈ {1, . . . , m}. Using Fact 2.3.25, we know the 12 -average opera-tor Ti = 12 (Id+PUi) is firmly nonexpansive. Using Fact 2.3.26 and Fact 2.3.25,we know TS is firmly nonexpansive. By Fact 2.3.11(i), TS is linear.(iii): Since Ti is firmly nonexpansive and since clearly (∀i ∈ {1, . . . ,m}) Fix Ti = Ui, the desired result is directly from Remark 2.3.22 andFact 2.3.27.Proposition 8.5.9. Assume H = Rn and {Id, RU1 , . . . , RUm−1 , RUm} ⊆ S .Then for every x ∈ H, (CCkSx)k∈N converges linearly to P∩mi=1Ui x with alinear rate ‖( 1m ∑mi=1 PUi)P(∩mi=1Ui)⊥‖.1228.6. Accelerating the Douglas–Rachford methodProof. The result is from Lemma 8.5.7 and Proposition 8.3.3.Proposition 8.5.10. Assume H = Rn and {Id, RU1 , RU2 , . . . , RUm} ⊆ S . De-note TS = 1m ∑mi=1 Tix where (∀i ∈ {1, 2, . . . , m}) Ti = 12 (Id+PUi). Letx ∈ Rn. Then (CCkSx)k∈N linearly converges to P∩mi=1Ui x with a linear rate‖TSP(∩mi=1Ui)⊥‖.Proof. The result comes from Lemma 8.5.8 and Proposition 8.3.3.Clearly, {Id, RU1 , RU2 , . . . , RUm} ⊆ {Id, RU1 , RU2 , . . . , RUm}. So we cantake S = {Id, RU1 , RU2 , . . . , RUm} in Proposition 8.5.9 and Proposition 8.5.10,then we get the corollary below.Corollary 8.5.11. AssumeH = Rn and S = {Id, RU1 , RU2 , . . . , RUm}. DenoteTS = 1m ∑mi=1 Tix where (∀i ∈ {1, 2, . . . , m}) Ti = 12 (Id+PUi). Setγ = min{‖( 1mm∑i=1PUi)P(∩mi=1Ui)⊥‖, ‖TSP(∩mi=1Ui)⊥‖}.Let x ∈ Rn. Then (CCkSx)k∈N converges to P∩mi=1Ui x with a linear rate γ.8.6 Accelerating the Douglas–Rachford methodUsing our Corollary 8.2.13, Proposition 8.2.6(iv), Fact 3.2.4 and Fact 3.3.4and using the similar idea in the proof of [12, Theorem 2.10], we can get thefollowing more general result.Proposition 8.6.1. Assume m = 2 and U1, U2 are closed linear subspace. LetW be a closed linear subspace ofH such thatU ∩V ⊆W ⊆ U1 +U2.Let x ∈ H. Denote T := TU2,U1 defined in Definition 3.3.1 and w := PW x.Assume there exists γ ∈Nr {0} such that Tγ ∈ affS . Then(∀k ∈N) ‖CCkSw− PU1∩U2 x‖ ≤ (cF)γk‖w− PU1∩U2 x‖Proof. If k = 0, then the result is trivial. It suffices to prove the case wherek ∈Nr {0}. Using Corollary 8.2.13, we get(∀n ∈N) PU1∩U2 x = PFix Tw = PU1∩U2 w = PU1∩U2(CCnSw) = PFix T(CCnSw).(8.40)1238.6. Accelerating the Douglas–Rachford methodSince Tγ ∈ affS , thus by Proposition 8.2.6(iv),(∀y ∈ H) ‖CCS (y)− PU1∩U2 y‖ ≤ ‖Tγ(y)− PU1∩U2 y‖. (8.41)Using Fact 3.3.4, we get(∀y ∈ H) (∀k ∈Nr {0}) ‖Tky− PFix Ty‖ ≤ ckF‖y− PFix Ty‖. (8.42)Hence,‖CCS (w)− PU1∩U2 x‖(8.40)= ‖CCS (w)− PU1∩U2 w‖(8.41)≤ ‖Tγ(w)− PU1∩U2 w‖(8.40)= ‖Tγ(w)− PFix Tw‖(8.42)≤ cγF‖w− PFix Tw‖(8.40)= cγF‖w− PU1∩U2 x‖.Assume for some k ≥ 1 we have‖CCkS (w)− PU1∩U2 x‖ ≤ (cF)γk‖w− PU1∩U2 x‖. (8.43)Now‖CCk+1S (w)− PU1∩U2 x‖(8.40)= ‖CCS (CCkS (w))− PU1∩U2(CCkSw)‖(8.41)≤ ‖Tγ(CCkS (w))− PU1∩U2(CCkSw)‖(8.40)= ‖Tγ(CCkS (w))− PFix T(CCkSw)‖(8.42)≤ cγF‖CCkS (w)− PFix T(CCkSw)‖(8.40)= cγF‖CCkS (w)− PU1∩U2 x‖(8.43)≤ cγF(cF)γk‖w− PU1∩U2 x‖= (cF)γ(k+1)‖w− PU1∩U2 x‖.Therefore, we have inductively proved the required result.Proposition 8.6.2. Let H be an real Hilbert space. Assume m = 2. As-sume U1, U2 are two closed linear subspaces with U1 + U2 being closed.Assume further that {Id, RU2 RU1} ⊆ S . Let x ∈ H. Let cF be the co-sine of the Friedrichs angle between U1 and U2. Then the three sequences(CCkS (PU1 x))k∈N, (CCkS (PU2 x))k∈N and (CCkS (PU1+U2 x))k∈N converge lin-early to PU1∩U2 x. Moreover, their rates of convergence are at least cF ∈ [0, 1[.1248.6. Accelerating the Douglas–Rachford methodProof. Because U1 ∩U2 ⊆ U1 ⊆ U1 + U2, U1 ∩U2 ⊆ U2 ⊆ U1 + U2 andU1 ∩U2 ⊆ U1 +U2 = U1 +U2, we are able to apply W in Corollary 8.2.13to any one of the U1, U2 or U1 +U2. Denote w = PW x.Since {Id, RU2 RU1} ⊆ S ,T =Id+RU2 RU12∈ aff{Id, RU2 RU1} ⊆ affS .Substitute γ = 1 in Proposition 8.6.1. Then we get(∀k ∈Nr {0}) ‖CCkSw− PU1∩U2 x‖ ≤ ckF‖w− PU1∩U2 x‖.Because U1 +U2 is closed, by Fact 3.2.4, we konw cF ∈ [0, 1[. The proof iscomplete.The following corollary extends the [12, Theorem 2.10] from Rn to thereal Hilbert space.Corollary 8.6.3. Assume U1, U2 are two closed linear subspaces with U1 +U2 being closed. Assume S = {Id, RU1 , RU2 RU1}. Let x ∈ H. Let cF be thecosine of the Friedrichs angle between U1 and U2. Then the three sequences(CCkS (PU1 x))k∈N, (CCkS (PU2 x))k∈N and (CCkS (PU1+U2 x))k∈N converge lin-early to PU1∩U2 x. Moreover, their rates of convergence are at least cF ∈ [0, 1[.Proof. The result follows directly from Proposition 8.6.2.To make S satisfy m = 2 and {Id, RU2 RU1} ⊆ S , clearly we have morechoices except for S = {Id, RU1 , RU2 RU1} in Corollary 8.6.3. In fact, usingProposition 8.6.2, we also get the following corollary.Corollary 8.6.4. Assume U1, U2 are two closed linear subspaces with U1 +U2 being closed. Assume S = {Id, RU2 , RU2 RU1} (or S = {Id, RU1 RU2 ,RU2 RU1} or S = {Id, RU1 , RU2 , RU2 RU1}). Let x ∈ H. Let cF be the co-sine of the Friedrichs angle between U1 and U2. Then the three sequences(CCkS (PU1 x))k∈N, (CCkS (PU2 x))k∈N and (CCkS (PU1+U2 x))k∈N converge lin-early to PU1∩U2 x. Moreover, their rates of convergence are at least cF ∈ [0, 1[.In order to further compare the convergence rate of the circumcentermethod induced by reflectors and the Douglas–Rachford method, we needthe lemma below.Lemma 8.6.5. Assume S = {Id, RU2 RU1 , RU2 RU1 RU2 RU1}. Let T := TU2,U1 =Id+RU2 RU12 be the Douglas–Rachford operator defined in Definition 3.3.1.Then the following statements hold.1258.6. Accelerating the Douglas–Rachford method(i) aff{Id, T, T2} = affS .(ii) Let Ŝ := {Id, T, T2}. Then the mapping CCŜ : H → H defined byDefinition 6.2.1 is proper, that is, (∀x ∈ H), CCŜx ∈ H.Proof. (i): By Lemma 4.1.2, it suffices to proveaff{Id, T, T2} − Id = aff(S)− Id⇐⇒ span{T − Id, T2 − Id} = span{RU2 RU1 − Id, RU2 RU1 RU2 RU1 − Id}.Now,T − Id = RU2 RU1 + Id2− Id = RU2 RU1 − Id2, (8.44)andT2 − Id = T2 − T + T − Id = (T − Id)T + (T − Id)=RU2 RU1 − Id2(RU2 RU1 + Id2)+RU2 RU1 − Id2=14(RU2 RU1 RU2 RU1 − Id) +12(RU2 RU1 − Id), (8.45)that is(T − Id T2 − Id) = (RU2 RU1 − Id RU2 RU1 RU2 RU1 − Id) ( 12 120 14).(8.46)Denote A :=( 12120 14). Then det(A) = 18 6= 0. Therefore (8.46) and somealgebraic calculations yield (i).(ii): Let x ∈ H. Denote the cardinality of {x, RU2 RU1 x, RU2 RU1 RU2 RU1 x}as l. If l = 1, thenx = RU2 RU1 x = RU2 RU1 RU2 RU1 x(8.44)(8.45)=====⇒ Tx = x = T2x =⇒ CCŜ (x) = x.Assume l ≥ 2. If T(x) − x, T2x − x are linearly independent, by Theo-rem 6.2.7, we know CCŜx ∈ H.Assume T(x) − x, T2x − x are linearly dependent. If Tx − x = 0, byCorollary 5.2.4, CCŜx =x+T2x2 . Now suppose Tx − x 6= 0. By (8.44),RU2 RU1 x 6= x. By (8.46), in this case, RU2 RU1 x − x, RU2 RU1 RU2 RU1 x − xare linearly dependent, since det(A) 6= 0. Applying Theorem 7.2.5, we1268.6. Accelerating the Douglas–Rachford methodknow CCSx ∈ H. Hence, Theorem 5.7.1 tells us the cardinality of theset {x, RU2 RU1 x, RU2 RU1 RU2 RU1 x} must be equal to or less than 2. There-fore either RU2 RU1 RU2 RU1 x = RU2 RU1 x or RU2 RU1 RU2 RU1 x = x. SupposeRU2 RU1 RU2 RU1 x = RU2 RU1 x. Multiply both sides by RU2 , RU1 respectively,by Lemma 4.2.1, we getRU2 RU1 x = x,which contradicts the assumption RU2 RU1 x 6= x. Suppose RU2 RU1 RU2 RU1 x= x, by (8.44) and (8.45), which implies,Tx = T2x.Then by Definition 6.2.1, we obtain CCŜx =x+Tx2 ∈ H.In conclusion,(∀x ∈ H) CCŜx ∈ H.To conclude this section, using the same idea in the proof of Proposi-tion 8.6.2, we get the proposition below.Now let {Id, RU2 RU1 , RU2 RU1 RU2 RU1} ⊆ S instead of {Id, RU1 , RU2 RU1}⊆ S in Corollary 8.6.3. This time our convergence rate is at least c2F ratherthan at least cF.Proposition 8.6.6. Assume U1, U2 are two closed linear subspaces in H,with U1 + U2 being closed. Let cF be the cosine of the Friedrichs anglebetween U1 and U2. Let T =RU2 RU1+Id2 be the Douglas–Rachford operatorassociated with U1, U2. Assume {Id, RU2 RU1 , RU2 RU1 RU2 RU1} ⊆ S . Letx ∈ H. Then the three sequences (CCkS (PU1 x))k∈N, (CCkS (PU2 x))k∈N and(CCkS (PU1+U2 x))k∈N converge linearly to PU1∩U2 x. Moreover, their rates ofconvergence are at least c2F.Proof. Since {Id, RU2 RU1 , RU2 RU1 RU2 RU1} ⊆ S , by Lemma 8.6.5(i), we getT2 ∈ affS .The remainder of the proof is similar with the proof in Proposition 8.6.2. Theonly difference is that this time we substitute γ = 2 but not γ = 1.1278.7. Circumcenter method in feasibility problemsBy Fact 2.2.10, for every affine subspace M, there is a unique linearsubspace M − M that is parallel to M. The translation won’t change thecloseness of subspaces. Hence, for almost all of the results about U1, U2being linear subspaces in this chapter, we have the corresponding results forU1, U2 being affinely subspaces. For notational simplicity, we only considerthe cases where U1, U2 are two closed linear subspace in H, with U1 +U2being closed.8.7 Circumcenter method in feasibility problemsIn the results above in this chapter, we used the circumcenter methodinduced by reflectors to solve the best approximation problem.The below proposition shows that we can also use the circumcentermethod induced by reflectors to solve the feasibility problem of finitelymany closed linear subspaces.Proposition 8.7.1. Assume U1, . . . , Um are closed linear subspaces inH, with∩mi=1Ui 6= ∅ and U⊥1 + · · ·+U⊥m being closed. Denote I := {1, . . . , m} andDenote U := Πi∈IUi, D := {(x, . . . , x) ∈ Hm | x ∈ H}. Assume S = {Id,RD, RURD} (or S = {Id, RU, RURD}). Let x ∈ H and denote x = (x, . . . ,x) ∈ Hm ∩D. Then there exists x ∈ ∩mi=1Ui such that PU∩Dx = (x, . . . , x)and (CCkSx)k∈N converges to PU∩Dx linearly.Proof. By Fact 2.3.18, we know that for every z = (z1, . . . , zm) ∈ Hm,PD(z) = (1mm∑i=1zi, . . . ,1mm∑i=1zi),andPU(z) = (PUi(zi))i∈I .Hence, we can calculate RD and RU easily.Now U1, . . . , Um are closed linear subspaces implies that U is closedlinear subspace. It is clear that D is a closed linear subspace. BecauseU⊥1 + · · ·+U⊥m is closed, by Fact 2.3.20, we get U + D is closed. Then usingCorollary 8.6.4, we know there exists cF ∈ [0, 1[ such that(∀k ∈N) ‖CCkSx− PU∩Dx‖ = ‖CCkSPDx− PU∩Dx‖ ≤ ckF‖PDx− PU∩Dx‖,1288.7. Circumcenter method in feasibility problemswhich implies that (CCkSx)k∈N linearly converges to PU∩Dx := (x, . . . , x) forsome x ∈ H. It is clear thatPU∩Dx = (x, . . . , x) ∈ U ∩D⇐⇒ x ∈ ∩mi=1Ui.Hence, the proof is done.129Chapter 9Circumcenter method withprojectors9.1 OverviewRecall that for every nonempty set S consisting of operators fromH toH, and for every x ∈ H,S(x) = {Tx | T ∈ S}.In Definition 6.2.1, we defined the circumcenter mapping CCS : H →H∪ {∅} induced by S as:(∀x ∈ H) CCSx = CC(S(x)),where CC(S(x)) is the circumcenter of the set S(x) defined in Defini-tion 5.2.2, that is, for every x ∈ H, either CCSx = ∅, or CCSx ∈ H isthe unique point satisfying the two conditions below:(i) CCSx ∈ aff(S(x)),(ii){‖CCSx− T(x)‖∣∣∣ T ∈ S} is a singleton.Recall that CCS is proper, if(∀x ∈ H) CCSx ∈ H.In this chapter, we revisit the assumptions of the last two chapters:m ∈Nr {0} andU1, . . . , Um are closed affine subspaces in the real Hilbert spaceH,with ∩mi=1Ui 6= ∅. LetΩ ={RUir · · · RUi2 RUi1∣∣∣ r ∈N, and i1, . . . , ir ∈ {1, . . . , m}}. (9.1)1309.2. Circumcenter operator induced by projectorsIn particular, denoteΘ ={PUir · · · PUi2 PUi1∣∣∣ r ∈N, and i1, . . . , ir ∈ {1, . . . , m}}. (9.2)Recall that we use the empty product convention that ∏0j=1 RUij = Id and∏0j=1 PUij = Id, hence Id ∈ Ω and Id ∈ Θ. From now on,S is a subset of the Ω with{Id} & S ,andŜ is a subset of the affΘ with{Id} & Ŝ .We summarize the main results in this chapter as follows:• In Section 9.2, we show both proper and improper examples aboutCCŜ . We prove that if Ŝ = {Id, PU1 , . . . , PUm−1 , PUm}, then CCŜ isproper (see Corollary 9.2.3).• We use the circumcenter method induced by reflectors to acceleratethe MAP and symmetric MAP for finitely many linear subspaces inH (see Theorem 9.3.5 and Corollary 9.3.6). We point out that theLemma 9.3.4 plays an essential role in the above accelerations. More-over, the Lemma 9.3.4 is itself an interesting result.• Consider the T := PU1 · · · PUn−1 PUn PUn−1 · · · PU1 . We prove that thesequence (CCkS (Tx))k∈N converges to P∩mi=1Ui x at least as fast as the se-quence (AkT(Tx))k∈N, where AT is defined in [8] (see Proposition 9.4.2).In [8], (AkT(Tx))k∈N was proved to be an accelerating of the symmetricMAP.9.2 Circumcenter operator induced by projectorsFirst we see some proper CCŜ ’s.Proposition 9.2.1. Let α ∈ R. AssumeŜ = {Id, (1− α) Id+αPU1 , . . . , (1− α) Id+αPUm}, (9.3)andS = {Id, RU1 , . . . , RUm}. (9.4)Then CCŜ is proper. Moreover,(∀x ∈ H) CCŜx =α2CCSx + (1− α2 )x ∈ H.1319.2. Circumcenter operator induced by projectorsProof. If α = 0, then Ŝ = {Id}, by Definition 6.2.1,(∀x ∈ H) CCŜx = x = 0CCSx + (1− 0)x ∈ H.Now assume α 6= 0. Let x ∈ H. For every i ∈ {1, . . . , m}, since RUi =2PUi − Id, thus(1− α)x + αPUi x = x + α(PUi x− x) = x +α2(RUi x− x). (9.5)Hence,CCŜx= CC(Ŝ(x)) (by Definition 6.2.1)= CC({x, (1− α)x + αPU1 x, . . . , (1− α)x + αPUm x})(by (9.3))= CC({0,α2(RU1 x− x), . . . ,α2(RUm x− x)}+ x)(by (9.5))= CC({0,α2(RU1 x− x), . . . ,α2(RUm x− x)})+ x (by Proposition 5.3.3)=α2CC({0, RU1 x− x, . . . , RUm x− x}) + x (by Proposition 5.3.1)=α2CC({x, RU1 x, . . . , RUm x} − x) + x=α2CC({x, RU1 x, . . . , RUm x})−α2x + x (by Proposition 5.3.3)=α2CC(S(x)) + (1− α2)x (by (9.4))=α2CCSx + (1− α2 )x ∈ H (by Definition 6.2.1 and Theorem 7.2.5)Therefore the proof is done.Remark 9.2.2. In Proposition 9.2.1, for α ∈ Rr {0},aff(S(x)) = aff{x, RU1(x), . . . , RUm(x)}= x + span{RU1(x)− x, . . . , RUm(x)− x}= x + span{2(PU1(x)− x), . . . , 2(PUm(x)− x)}= x + span{α(PU1(x)− x), . . . , α(PUm(x)− x)}= aff{x, (1− α)x + αPU1(x), . . . , (1− α)x + αPUm(x)}= aff(Ŝ(x)).In particular, take α = 1 in Proposition 9.2.1, we can get the corollarybelow.1329.2. Circumcenter operator induced by projectorsCorollary 9.2.3. Assume Ŝ = {Id, PU1 , . . . , PUm−1 , PUm}. Then CCŜ is proper,that is for every x ∈ H, there exists unique CCŜx ∈ H satisfying(i) CCŜ (x) ∈ Ŝ(x) = aff{x, PU1(x), . . . , PUm−1(x), PUm(x)}(ii) ‖CCŜ (x)− x‖ = ‖CCŜ (x)− PU1(x)‖ = · · · = ‖CCŜ (x)− PUm−1(x)‖ =‖CCŜ (x)− PUm(x)‖.The following example is consistent with the result obtained in Proposi-tion 9.2.1.Example 9.2.4. Assume H = R2, U1 is the x-axis and U2 is the line {(x,y) ∈ R2 | y = x}. Assume further that S = {Id, RU1 , RU2} and Ŝ = {Id, PU1 ,PU2}. Then for x = (2.48, 0.99), CCSx = (0, 0) and CCŜ . Hence,CCŜx =12CCSx +12x.Figure 9.1: Circumcenters in Example 9.2.4Proposition 9.2.5. Assume U2 is a linear subspace. Assume Ŝ = {Id, PU1 ,PU2 PU1}. Then CCŜ is proper.Proof. Let x ∈ H. Denote the cardinality of the set {x, PU1 x, PU2 PU1 x} by l.If l ≤ 2, by Definition 6.2.1, CCŜx ∈ H.Now assume l = 3. If x, PU1 x, PU2 PU1 x are affinely independent, byTheorem 5.7.1, CCŜx ∈ H. Suppose x, PU1 x, PU2 PU1 x are affinely dependent.Note l = 3 imply that PU1 x− x 6= 0 and there exists α 6= 1 such thatPU2 PU1 x− x = α(PU1 x− x) (9.6)1339.2. Circumcenter operator induced by projectorsBecause U2 is linear, PU2 is linear. Multiplying both sides of (9.6) by PU2 , wegetPU2 PU2 PU1 x− PU2 x = α(PU2 PU1 x− PU2 x)=⇒ PU2 PU1 x− PU2 x = α(PU2 PU1 x− PU2 x) (by Fact 2.3.8(i))=⇒ (1− α)PU2 PU1 x = (1− α)PU2 x=⇒ PU2 PU1 x = PU2 x. (α 6= 1)Hence,CCŜx = CC(Ŝ(x)) = CC({x, PU1 x, PU2 PU1 x}) = CC({x, PU1 x, PU2 x}) ∈ H.The last inclusion follows from Corollary 9.2.3 with m = 2. Therefore,(∀x ∈ H) CCŜx ∈ H.Proposition 9.2.6. Assume U2 is a linear subspace. Assume Ŝ = {Id, PU2 ,PU2 PU1}. Then CCŜ is proper.Proof. Let x ∈ H. Denote the cardinality of the set {x, PU2 x, PU2 PU1 x} by l.Similarly with the proof in Proposition 9.2.5, it suffices to prove the casewhen l = 3 and there exists α 6= 1 such thatPU2 PU1 x− x = α(PU2 x− x). (9.7)As Proposition 9.2.5, we multiply both sides of (9.7) by PU2 . ThenPU2 PU1 x− x = α(PU2 x− x)=⇒ PU2 PU2 PU1 x− x = α(PU2 PU2 x− PU2 x)=⇒ PU2 PU1 x− PU2 x = α(PU2 x− PU2 x) = 0 (by Fact 2.3.8(i))=⇒PU2 PU1 x = PU2 x.which contradicts l = 3. In conclusion,(∀x ∈ H) CCŜx ∈ H.The examples below show that there exists S consisting of compositionsof projectors such that CCS is improper.1349.2. Circumcenter operator induced by projectorsExample 9.2.7. Assume H = R2 and m = 2. Assume U1 is the x-axis andU2 is the line {(x, y) ∈ R2 | y = x}. Assume further that Ŝ = {Id, PU1 , PU2 ,PU2 PU1}. Take x = (4, 2). Let K = {Id, PU1 , PU2}. Clearly, PU2 PU1 x − x ∈R2 = span{PU1 x − x, PU2 x − x}, which imply that aff(K(x)) = aff(Ŝ(x)).By Proposition 5.4.2, if only CCŜx ∈ H, then CCŜx = CCKx. As the picturebelow shows,‖CCKx− x‖ = ‖CCKx− PU1 x‖ = ‖CCKx− PU2 x‖ 6= ‖CCKx− PU2 PU1 x‖Therefore, we know CCŜx = ∅.Figure 9.2: CCŜ may be improperExample 9.2.8. Assume H = R2, m = 2, U1 is the x-axis and U2 is the liney = 2x. Assume further that Ŝ = {Id, PU2 PU1 , PU2 PU1 PU2 PU1}. Take x = (2,4) ∈ U2. As the following picture illustrates, x, PU2 PU1 x, and PU2 PU1 PU2 PU1 xare all located on the line U2. They are pairwise distinct and collinear, byTheorem 5.7.1, CCŜx = ∅.1359.3. Accelerating the MAP and symmetric MAPFigure 9.3: Collinear example of CCŜ x = ∅9.3 Accelerating the MAP and symmetric MAPIn this section, we assume additionally thatU1, . . . , Um are closed linear subspaces in the real Hilbert spaceH.Recall thatΩ ={RUir · · · RUi2 RUi1∣∣∣ r ∈N, and i1, . . . , ir ∈ {1, . . . , m}}.In the whole section, we denoteΨ ={RUir · · · RUi2 RUi1∣∣∣ r, i1, i2, . . . , ir ∈ {0, 1, . . . , m} and i1 < · · · < ir},and we assumeΨ ⊆ S ⊆ Ω. (9.8)In this section, we will deduce some linear convergence results of theCCS . It turns out that for some special S , the CCS does not have worseconvergence rate than that of MAP or the symmetric MAP. Moreover, weshall show that for some special S , the CCS does not even have slowerconvergence rate than that of the accelerated symmetric Map defined in [8].1369.3. Accelerating the MAP and symmetric MAPRemark 9.3.1. We claim that there are exactly 2m possible combinations forthe indices of the reflectors making up the elements of the set Ψ.In fact, for every r ∈ I := {0, 1, . . . , m}, the r-combination of the set I isa subset of r distinct items of I.5 In addition, the number of r-combinationsof I equals to the binomial coefficient (mr ). Moreover, by Binomial Theorem,2m = (1+ 1)m =m∑r=0(mr).Therefore, the claim is true.Actually with consideration of duplication, there are at most 2m pairwisedistinct elements in Ψ. (For instance, if U1 = U2, then RU2 RU1 = Id.)For example, when m = 1, Ψ = {Id, RU1}. When m = 2,Ψ = {Id, RU1 , RU2 , RU2 RU1}.When m = 3,Ψ = {Id, RU1 , RU2 , RU3 , RU2 RU1 , RU3 RU1 , RU3 RU2 , RU3 RU2 RU1}.When m = 4,Ψ ={Id, RU1 , RU2 , RU3 , RU4 , RU2 RU1 , RU3 RU1 , RU4 RU1 , RU3 RU2 , RU4 RU2 ,RU4 RU3 ,RU3 RU2 RU1 , RU4 RU2 RU1 , RU4 RU3 RU1 , RU4 RU3 RU2 , RU4 RU3 RU2 RU1}.First, let’s see two examples where m = 2 to get some feeling aboutour main result Theorem 9.3.5 below. Actually the two examples are alsocorollaries of Theorem 9.3.5 below.Example 9.3.2. Assume m = 2 and S = {Id, RU1 , RU2 , RU2 RU1}. Assumefurther that U1 +U2 is closed. Denote γ := ‖PU2 PU1 P(U1∩U2)⊥‖. Then γ ∈ [0,1[ and (CCkSx)k∈N converges to PU1∩U2 x with a linear rate γ.Proof. By assumption and Lemma 3.2.5, we know γ ∈ [0, 1[. Denote TS :=PU2 PU1 . Using Fact 3.2.10, we get that Fix TS = Fix PU2 PU1 = U1 ∩U2. ApplyFact 3.2.12 with T replaced by TS to obtain that(∀x ∈ H) (∀k ∈N) ‖(TS )kx− PU1∩U2 x‖ ≤ γk‖x− PU1∩U2 x‖. (9.9)5With the empty product convention that ∏0j=1 RUij = Id, we know the 0-combination ofthe set I is the Id.1379.3. Accelerating the MAP and symmetric MAPMoreover, sinceTS = PU2 PU1=12(RU2 + Id)12(RU1 + Id)=122(RU2 RU1 + RU2 + RU1 + Id) ∈ aff(S),combining Theorem 8.3.2 with (9.9), we obtain(∀x ∈ H) (∀k ∈N) ‖CCkSx− PU1∩U2(x)‖ ≤ γk‖x− PU1∩U2(x)‖.Example 9.3.3. Assume m = 2 and S = {Id, RU1 , RU2 , RU1 RU2 , RU2 RU1 ,RU1 RU2 RU1}. Assume further that U1 +U2 is closed. Denoteγ := ‖PU2 PU1 P(U1∩U2)⊥‖.Then γ ∈ [0, 1[ and (CCkSx)k∈N converges to PU1∩U2 x with a linear rate γ2.Proof. Denote TS = PU1 PU2 PU1 . By Fact 2.3.8(i), we know (PU2 PU1)∗PU2 PU1 =PU1 PU2 PU1 = TS . Similarly with the proof in Example 9.3.2, we know γ ∈ [0,1[ and Fix TS = U1 ∩U2. Apply Corollary 3.2.13 with T replaced by PU2 PU1to get(∀x ∈ H) (∀k ∈N) ‖TkSx− PU1∩U2 x‖ ≤γ2k‖x− PU1∩U2 x‖. (9.10)Because123(RU1 + Id)(RU2 + Id)(RU1 + Id)=123(RU1 RU2 RU1 + RU1 RU2 + RU1 RU1 + RU1 + RU2 RU1 + RU2 + RU1 + Id)=123(RU1 RU2 RU1 + RU1 RU2 + RU2 RU1 + 2RU1 + RU2 + 2 Id) (Lemma 4.2.1)∈ aff{Id, RU1 , RU2 , RU1 RU2 , RU2 RU1 , RU1 RU2 RU1} = affS ,we obtainTS = PU1 PU2 PU1 =123(RU1 + Id)(RU2 + Id)(RU1 + Id) ∈ affS .Hence, combining Theorem 8.3.2 and (9.10), we obtain(∀x ∈ H) (∀k ∈N) ‖CCkSx− P∩mi=1Ui(x)‖ ≤ γ2k‖x− P∩mi=1Ui(x)‖.1389.3. Accelerating the MAP and symmetric MAPIn order to prove the more general result, which is the main result in thissection, we need the lemma below.Lemma 9.3.4. Recall that m ∈ Nr {0}, U1, . . . , Um are closed linear sub-spaces inH and Ψ ⊆ S ⊆ Ω. Then:(i)RUm RUm−1 · · · RU1= 2mPUm PUm−1 · · · PU1 −(Id+m−1∑k=1∑i1,...,ik−1,ik∈{1,2,...,m}i1<···<ik−1<ikRUik RUik−1 · · · RUi1).(ii)PUm PUm−1 · · · PU1 =12m( m∑k=0∑i1,...,ik−1,ik∈{1,2,...,m}i1<···<ik−1<ikRUik RUik−1 · · · RUi1).(9.11)(iii) PUm PUm−1 · · · PU1 ∈ convΨ ⊆ affS .Proof. Because when k = m, the only one possibility for the RUik RUik−1 · · · RUi1such that i1, . . . , ik−1, ik ∈ {1, 2, . . . , m} and i1 < · · · < ik is RUm RUm−1 · · · RU1 .Hence, clearly (i)⇔ (ii). It suffices to prove (i) and (iii).(i): Suppose m = 1. By definition, RU1 = 2PU1 − Id. Assume the result istrue for some m ∈N, with 1 ≤ m, that isRUm RUm−1 · · · RU1= 2mPUm PUm−1 · · · PU1 −(Id+m−1∑k=1∑i1,...,ik∈{1,2,...,m}i1<···<ik−1<ikRUik RUik−1 · · · RUi1)(9.12)1399.3. Accelerating the MAP and symmetric MAPNowRUm+1 RUm RUm−1 · · · RU1(9.12)= RUm+1(2mPUm · · · PU1 −(Id+m−1∑k=1∑i1,...,ik∈{1,2,...,m}i1<···<ik−1<ikRUik RUik−1 · · · RUi1))= 2mRUm+1 PUm PUm−1 · · · PU1−(RUm+1 +m−1∑k=1∑i1,...,ik∈{1,2,...,m}i1<···<ik−1<ikRUm+1 RUik RUik−1 · · · RUi1)= 2m(2PUm+1 − Id)PUm PUm−1 · · · PU1−(RUm+1 +m−1∑k=1∑i1,...,ik∈{1,2,...,m}i1<···<ik−1<ikRUm+1 RUik RUik−1 · · · RUi1)= 2m+1PUm+1 PUm PUm−1 · · · PU1 − 2mPUm PUm−1 · · · PU1−(RUm+1 +m−1∑k=1∑i1,...,ik∈{1,2,...,m}i1<···<ik−1<ikRUm+1 RUik RUik−1 · · · RUi1)(9.12)= 2m+1PUm+1 PUm PUm−1 · · · PU1− RUm RUm−1 · · · RU1 −(Id+m−1∑k=1∑i1,...,ik∈{1,2,...,m}i1<···<ik−1<ikRUik RUik−1 · · · RUi1)−(RUm+1 +m−1∑k=1∑i1,...,ik∈{1,2,...,m}i1<···<ik−1<ikRUm+1 RUik RUik−1 · · · RUi1)= 2m+1PUm+1 PUm · · · PU1 −(Id+m∑k=1∑i1,...,ik∈{1,2,...,m+1}i1<···<ik−1<ikRUik RUik−1 · · · RUi1),which is (9.12) with m being replaced by m + 1. Therefore, (i) is true.(iii): By Remark 9.3.1, we know there are exactly 2m items in the bigbracket in right–hand side of (9.11), since the items in the big bracket inright-hand side of (9.11) are all of the items in the set Ψ. Hence,PUm PUm−1 · · · PU11409.3. Accelerating the MAP and symmetric MAP=12m( m∑k=0∑i1,...,ik−1,ik∈{1,2,...,m}i1<···<ik−1<ikRUik RUik−1 · · · RUi1)∈ convΨ ⊆ affS .Therefore, the proof is done.Now we are ready to use the circumcenter method induced by reflec-tors to accelerate the method of alternating projections and the symmetricmethod of alternating projections.Theorem 9.3.5. Recall U1, . . . , Um are closed linear subspaces ofH and Ψ ⊆S ⊆ Ω. Denote U := ∩mi=1Ui, and γ := ‖PUm PUm−1 · · · PU1 PU⊥‖. Assumem ≥ 2 and U⊥1 + · · ·+U⊥m is closed. Then γ ∈ [0, 1[ and(∀x ∈ H) (∀k ∈N) ‖CCkSx− P∩mi=1Ui x‖ ≤ γk‖x− P∩mi=1Ui x‖.Hence, (CCkSx)k∈N converges to P∩mi=1Ui x with a linear rate at least γ.Proof. Let TS = PUm PUm−1 · · · PU1 . By Fact 3.2.10, we know Fix TS = ∩mi=1Ui =U. By assumption and Corollary 3.2.9, we getγ = ‖TSPU⊥‖ ∈ [0, 1[. (9.13)Applying Fact 3.2.12 with T replaced by TS , we get(∀x ∈ H) (∀k ∈N) ‖TkSx− PUx‖ ≤ γk‖x− PUx‖. (9.14)Since by Lemma 9.3.4, we know TS ∈ aff(S), thus by Theorem 8.3.2, (9.13)and (9.14) yield the required results.Corollary 9.3.6. Let n ∈Nr {0}. Let V1, . . . , Vn be closed linear subspacesin the real Hilbert spaceH, with V := ∩ni=1Vi 6= ∅ and V⊥1 + · · ·+V⊥n beingclosed. Denote γ := ‖PVn PVn−1 · · · PV1 PV⊥‖. Assume m = 2n− 1 and assume(∀i ∈ {1, . . . , n}) Ui = Vi, (∀j ∈ {1, . . . , n− 1}) Un+j = Vn−j. (9.15)Recall Ψ ⊆ S ⊆ Ω for the fixed m = 2n− 1. Then γ ∈ [0, 1[ and(∀x ∈ H) (∀k ∈N) ‖CCkSx− P∩ni=1Vi x‖ ≤ γ2k‖x− P∩ni=1Vi x‖,that is (CCkSx)k∈N converges to P∩ni=1Vi x with a linear rate at least γ2.1419.4. Accelerating the accelerated mapping of MAPProof. By (9.15), we know ∩ni=1Vi = ∩mi=1Ui,PUm PUm−1 · · · PU1 = PV1 · · · PVn−1 PVn PVn−1 · · · PV1 ,andV⊥1 + · · ·+V⊥n = U⊥1 + · · ·+U⊥n−1 +U⊥n +U⊥n+1 + · · ·+U⊥m .Denote ρ := ‖PUm PUm−1 · · · PU1 PU⊥‖. Since V⊥1 + · · ·+ V⊥n is closed, Theo-rem 9.3.5 implies(∀x ∈ H) (∀k ∈N) ‖CCkSx− P∩mi=1Ui x‖ ≤ ρk‖x− P∩mi=1Ui x‖. (9.16)Denote T := PVn PVn−1 · · · PV1 . Then Fact 3.2.11(iv) and Corollary 3.2.9yieldγ = ‖TPV⊥‖ = ‖T∗TPV⊥‖12(9.15)= ρ12 ∈ [0, 1[. (9.17)Since ∩ni=1Vi = ∩mi=1Ui, (9.16) and (9.17) yield(∀x ∈ H) (∀k ∈N) ‖CCkSx− P∩ni=1Vi x‖ ≤ γ2k‖x− P∩ni=1Vi x‖.The proof is complete.The theorems above show that for some particular S , the sequence(CCkSx)k∈N converges to P∩mi=1Ui at least as fast as the MAP or the symmetricMAP.9.4 Accelerating the accelerated mapping of MAPLemma 9.4.1. Assume U1, . . . , Un are closed linear subspaces of H withU := ∩ni=1Ui where n ∈ Nr {0}. Let T = PU1 · · · PUn−1 PUn PUn−1 · · · PU1 .Let AT defined in Definition 3.4.1 be the accelerated mapping of T. Thefollowing statements hold:(i) T is linear, nonexpansive, self-adjoint and monotone operator onH.(ii) Fix T = ∩ni=1Ui.(iii) If U⊥1 + · · ·+U⊥n is closed, then ‖TP(Fix T)⊥‖ ∈ [0, 1[.(iv) (∀x ∈ H) AT(x) = Paff{x,Tx}(P∩ni=1Ui x).1429.4. Accelerating the accelerated mapping of MAP(v) For every x ∈ H, y ∈ ∩ni=1Ui and for every k ∈N,AkT(x + y) = AkT(x) + yProof. (i) was proved in [8, page 3454]. (ii) is from Fact 3.2.10. (iii) followsfrom Corollary 3.2.9 and (ii). (iv) comes from Fact 3.4.2 and the above (ii).Using (ii), we know (v) is directly from [8, Lemma 3.8(3)].Proposition 9.4.2. Assume m = 2n− 1 for some n ∈Nr {0}. Assume U1,. . . , Un are closed linear subspaces of H with U⊥1 + · · ·+U⊥n being closed.Assume (∀i ∈ {1, . . . , n− 1}) Un+i = Un−i and Ψ ⊆ S ⊆ Ω for the fixedm. Denote T := PU1 · · · PUn−1 PUn PUn−1 · · · PU1 and c(T) := ‖TP(Fix T)⊥‖. LetAT defined in Definition 3.4.1 be the accelerated mapping of T. Then 0 ≤c(T)2−c(T) ≤ c(T) < 1 and(∀x ∈ H)(∀k ∈N) ‖CCkS (Tx)− P∩ni=1Ui x‖ ≤c(T)k+1(2− c(T))k ‖x− P∩ni=1Uix‖.(9.18)Proof. Clearly, c(T) is the ρ in Corollary 9.3.6 and we proved ρ ∈ [0, 1[ inCorollary 9.3.6, so c(T) ∈ [0, 1[ and 2− c(T) ∈]1, 2]. Hence, 0 ≤ c(T)2−c(T) ≤c(T) < 1.By Lemma 9.3.4, T ∈ affS . Thus by Definition 3.4.1, we know(∀x ∈ H) ATx ∈ aff{x, Tx} ⊆ affS(x).Hence, substitute the T in Proposition 8.5.1(i) by AT to get(∀x ∈ H) ‖CCSx‖ ≤ ‖ATx‖. (9.19)Using Fact 2.3.14, we getTP∩ni=1Ui = P∩ni=1Ui T = P∩ni=1Ui . (9.20)Thus, for every x ∈ H,Tx− P∩ni=1Ui x(9.20)= Tx− P∩ni=1Ui Tx= (Id−P∩ni=1Ui)Tx= P(∩ni=1Ui)⊥Tx, (by Fact 2.3.10(ii))1439.4. Accelerating the accelerated mapping of MAPwhich implies that,(∀x ∈ H) Tx− P∩ni=1Ui x ∈ (∩ni=1Ui)⊥. (9.21)Combine Lemma 8.5.2 and (9.21) to get(∀k ∈N) CCkS (Tx− P∩ni=1Ui x) ∈ (∩ni=1Ui)⊥. (9.22)By Lemma 9.4.1(i), we know T is a linear, nonexpansive, self-adjoint andmonotone operator onH. Thus, by Fact 3.4.6 and Lemma 3.4.7, we get(∀k ∈N)∥∥∥AT(CCkS (Tx− P∩ni=1Ui x))∥∥∥ ≤ c(T)2− c(T)‖CCkS (Tx− P∩ni=1Ui x)‖(9.23)Below we are going to prove (9.18) by induction. Applying Fact 3.2.12 toT = PU1 · · · PUn−1 PUn PUn−1 · · · PU1 and k = 1, we get(∀x ∈ H) ‖Tx− P∩mi=1Ui x‖ ≤ c(T)‖x− P∩mi=1Ui x‖,which mean that for k = 0, (9.18) is true.Suppose for some k ∈N, we have(∀x ∈ H) ‖CCkS (Tx)− P∩ni=1Ui x‖ ≤c(T)k+1(2− c(T))k ‖x− P∩ni=1Uix‖. (9.24)Let x ∈ H. Now,‖CCk+1S (Tx)− P∩ni=1Ui x‖= ‖CCk+1S (Tx− P∩ni=1Ui x)‖ (by Lemma 8.5.3)=∥∥∥CCS(CCkS (Tx− P∩ni=1Ui x))∥∥∥(9.19)≤∥∥∥AT(CCkS (Tx− P∩ni=1Ui x))∥∥∥(9.23)≤ c(T)2− c(T)‖CCkS (Tx− P∩ni=1Ui x)‖=c(T)2− c(T)‖CCkS (Tx)− P∩ni=1Ui x‖ (by Lemma 8.5.3)(9.24)≤ c(T)k+2(2− c(T))k+1 ‖x− P∩ni=1Uix‖.Therefore, we have inductively proved the required result.1449.4. Accelerating the accelerated mapping of MAPRemark 9.4.3. Now we revisit the assumptions of Proposition 9.4.2. ByLemma 9.4.1(iv), we know(∀x ∈ H) AT(x) = Paff{x,Tx}(P∩ni=1Ui x). (9.25)On the other hand, using Theorem 7.2.5, we get(∀x ∈ H) CCSx = Paff(S(x))(P∩mi=1Ui x).In the proof of Proposition 9.4.2, we have already showed that for everyx ∈ H, aff{x, Tx} ⊆ aff(S(x)), thus in some sense the CCS can be viewedas more aggressive than the AT to converge to the point P∩ni=1Ui x.145Chapter 10Drawbacks of circumcentermethods induced by reflectors10.1 OverviewIn Theorem 7.2.5, we showed that the circumcenter mapping inducedby reflectors, CCS , is proper. In Sections 8.4 and 8.5 for some special Swhich include the cases S = {Id, RU1 , RU2 RU1} and S = {Id, RU1 , RU2},we proved that the sequence (CCkSx)k∈N converges linearly to the solu-tion P∩mi=1Ui x of the related best approximate problem. In particular, forS = {Id, RU1 , RU2 RU1}, the sequences (CCkSPU1 x)k∈N, (CCkSPU2 x)k∈N and(CCkSPU1+U2 x)k∈N are accelerations of the Douglas–Rachford method. In ad-dition, in Sections 8.4 to 8.6, for some special cases of S , we found many beau-tiful properties of the mapping CCS . In particular, S = {Id, RU1 , RU2 RU1}and S = {Id, RU1 , RU2} are included in those cases.Let U1 and U2 be two closed convex sets inH. In the remainder part ofthis chapter, unless mentioned otherwise, we assumeS = {Id, RU1 , RU2 RU1} or S = {Id, RU1 , RU2}. (10.1)This chapter is devoted to exploring disadvantages of the circumcentermapping. Our main discoveries are summarized below.• In Section 10.2, we use two counterexamples to show that CCS is notlinear, nonexpansive or self-adjoint.• In Section 10.3, we show that generally CCS is improper in the incon-sistent case.• In Section 10.4, we prove that generally CCS fails to be proper in non-affine examples. In [10, Section 3.3], the authors showed the potentialof C–DRM to find a point in the intersection of non-affine convex sets.The counterexamples provided in this section show the importance ofthe choice of the initial points.14610.2. Failure to be linear, nonexpansive and self-adjoint• In Section 10.5, we observe that for the circumcenter method inducedby reflectors, the natural extension of the theory from normal coneoperators to maximally monotone operators fails.10.2 Failure to be linear, nonexpansive andself-adjointThe following two examples show that for S = {Id, RU1 , RU2 RU1} orS = {Id, RU1 , RU2}, CCS may fail to be linear, nonexpansive or self-adjoint.Example 10.2.1. LetA =(7 1 5 7), B =(10 5 5 109 8 10 9).Define the two linear subspaces asU1 = {x ∈ R4 | Ax = 0},U2 = {x ∈ R4 | Bx = 0}.Let S = {Id, RU1 , RU2 RU1}. Then CCS is not linear. In addition, CCS isneither nonexpansive nor self-adjoint.Proof. Let x =(4 5 3 8)ᵀ, y = (9 10 6 6)ᵀ ∈ R4. According to theMatlab calculation, (for details see the Appendix ), we getCCSx ≈−2.59761.73561.32621.4024 , CCSy ≈0.72958.80480.3965−2.2705 , CCS (x + y) ≈−2.268111.83972.5827−1.2681 .Hence, CCSx + CCSy 6= CCS (x + y), and so CCS is not additive and so it isnot linear.In addition, ‖x − y‖ ≈ 7.9373 and ‖CCSx − CCSy‖ ≈ 8.6832. Hence,‖CCSx− CCSy‖ > ‖x− y‖. Therefore, the CCS is not nonexpansive.Since 〈CCSx, y〉 ≈ 10.3487 and 〈x, CCSy〉 ≈ 29.9673, 〈CCSx, y〉 6= 〈x,CCSy〉 and so CCS is not self-adjoint.Example 10.2.2. LetA =(5 5 4 8), B =(7 10 2 78 10 2 1),14710.3. Failure to be proper in the inconsistence caseDefine the two linear subspaces asU1 = {u ∈ R4 | Au = 0},U2 = {v ∈ R4 | Bv = 0}.Let S = {Id, RU1 , RU2}. Then CCS is not linear. Moreover, it is neithernonexpansive nor sef-adjoint.Proof. Let x =(6 6 9 5)ᵀ, y = (4 7 8 6)ᵀ ∈ R4. Then according tothe Matlab calculation, (for details see the Appendix ),CCSx ≈0.69690.56504.9265−3.2519 , CCSy ≈−1.15332.02663.6855−2.3886 , CCS (x + y) ≈−0.38042.73258.5635−5.7518 .Hence, CCSx + CCSy 6= CCS (x + y), and so CCS is not additive and soit is not linear.In addition, since ‖x − y‖ ≈ 2.6458 and ‖CCSx − CCSy‖ ≈ 2.8009,‖CCSx− CCSy‖ > ‖x− y‖. Therefore, the CCS is not nonexpansive.Since 〈CCSx, y〉 ≈ 26.6430 and 〈x, CCSy〉 ≈ 26.4667, thus 〈CCSx, y〉 6=〈x, CCSy〉 and so CCS is not self-adjoint.10.3 Failure to be proper in the inconsistence caseLet U and V be two nonempty, closed, convex (possibly non-intersecting)setsH. The best approximation pair relative to (U, V) is(a, b) ∈ (U, V) such that ‖a− b‖ = inf ‖U −V‖.In [7], the authors used the Douglas–Rachford operator T = RV RU+Id2 to findthe best approximation pair relative to (U, V).Fact 10.3.1. [7, Fact 2.3(i), Proposition 2.5 and Theorem 3.13] Let U and V betwo nonempty, closed, convex (possibly non-intersecting) sets inH. Supposethe best approximation pairs relative to (U, V) exist. Denote T := RV RU+Id2 .Let x0 ∈ H and set xn = Tnx0, for all n ∈N. Then the weak cluster pointsof ((PV RUxn, PUxn))n∈N and((PV PUxn, PUxn))n∈Nare best approximation pairs relative to (U, V).14810.3. Failure to be proper in the inconsistence caseThe following examples show that even if U1, U2 are affine subspaces,when U1 ∩ U2 = ∅, the CCS may not be proper where S = {Id, RU1 ,RU2 RU1} or S = {Id, RU1 , RU2}. (Notice that in Example 10.3.2, the U1is even a compact set.) Hence, we can not directly generalize Fact 10.3.1 bythe circumcenter method induced by reflectors.Example 10.3.2. Assume H = R2. Let U1 = {(2, 0)} and U2 be the y-axis.Recall S = {Id, RU1 , RU2 RU1} or S = {Id, RU1 , RU2}. Then CCS is improper.Proof. Take x = (3, 0). As the Figure 10.1 shows, the pairwise distinct vectorsx, RU1 x, RU2 x are affinely dependent and the pairwise distinct vectors x,RU1 x, RU2 RU1 x are also affinely dependent. Hence, by Theorem 5.7.1, weknow CCSx doesn’t exist. Therefore CCS is improper.Figure 10.1: Point and lineExample 10.3.3. Assume H = R2. Let U1 be y-axis and U2 be the linex = 2. Recall S = {Id, RU1 , RU2 RU1} or S = {Id, RU1 , RU2}. Then CCSx isimproper.Proof. Take x = (3, 0). As the Figure 10.2 shows, the pairwise distinct vectorsx, RU1 x, RU2 x are affinely dependent and the pairwise distinct vectors x,RU1 x, RU2 RU1 x are also affinely dependent. Hence, by Theorem 5.7.1, weknow CCSx doesn’t exist. Therefore CCS is improper.14910.4. Failure to be proper in non-affine examplesFigure 10.2: Two parallel affine subspaces10.4 Failure to be proper in non-affine examplesOne of the charming aspects of Douglas–Rachford method is that itcan be used for general convex sets. Recall S = {Id, RU1 , RU2 RU1} or S ={Id, RU1 , RU2}. In this section, we provide more counterexamples aboutCCS being improper where at least one of U1 and U2 is non-affine set andU1 ∩U2 6= ∅.Example 10.4.1. Assume H = R2. Assume U1 is the convex cone R2+ andU2 is the line {(x, y) ∈ R2 | x = 2}. Recall S = {Id, RU1 , RU2 RU1} orS = {Id, RU1 , RU2}. Then CCS is improper.Proof. Clearly, U2 is parallel with the left bound L = {(x, y) ∈ R2 | x = 0}of U1. As the following picture shows, take x = (−1, 0). Actually, in ourcase, RU1 x = RLx. Then the vectors x, RU1 x, RU2 x, RU2 RU1 x are collinear. ByTheorem 5.7.1, we know CCSx doesn’t exist. Therefore CCS is improper.15010.4. Failure to be proper in non-affine examplesFigure 10.3: Cone and subspaceExample 10.4.2. Assume H = R2. Assume U1 := {(x, y) ∈ R2 | x2 + (y−1)2 ≤ 1} and U2 is the x-axis. Recall S = {Id, RU1 , RU2 RU1} or S = {Id, RU1 ,RU2}. Then CCS is improper.Proof. As the Figure 10.4 shows, take x = (0, 2.21). Then x, RU1 x, RU2 RU1 xare affinely dependent and x, RU1 x, RU2 x are affinely dependent. So byTheorem 5.7.1, CCSx = ∅. Hence, CCS is improper.15110.4. Failure to be proper in non-affine examplesFigure 10.4: Ball and lineExample 10.4.3. Assume H = R2. Assume U1 is the x-axis and U2 = {(x,y) ∈ R2 | x2 + (y − 1)2 ≤ 1}. Assume S = {Id, RU1 , RU2}. Then CCS isimproper.Proof. As the Figure 10.5 shows, take x = (0, 2.18). Then x, RU1 x, RU2 xare affinely dependent. So by Theorem 5.7.1, CCSx = ∅. Hence, CCS isimproper.15210.4. Failure to be proper in non-affine examplesFigure 10.5: Line and ballExample 10.4.4. Assume H = R2. Assume U1 = {(x, y) ∈ R2 | (x + 2)2 +y2 ≤ 4} and U2 = {(x, y) ∈ R2 | (x− 2)2 + y2 ≤ 4}. Recall S = {Id, RU1 ,RU2 RU1} or S = {Id, RU1 , RU2}. Then CCS is improper.Proof. As the Figure 10.6 shows, take x = (−5, 0) on the x-axis, Then x, RU1 x,RU2 RU1 x are affinely dependent and x, RU1 x, RU2 x are affinely dependent.Hence, by Theorem 5.7.1, CCSx doesn’t exist.15310.4. Failure to be proper in non-affine examplesFigure 10.6: Two ballsConsider U1 = {(x, y) ∈ R2 | (x + 2)2 + y2 = 4} and U2 = {(x, y) ∈R2 | (x − 2)2 + y2 = 4}. Suppose S = {Id, RU1 , RU2 RU1} or S = {Id,RU1 , RU2}. Take x = (−5, 0). Similarly with Example 10.4.4, x, RU1 x, RU2 x,RU2 RU1 x are collinear. Hence, by Theorem 6.2.7, CCSx = ∅. Therefore CCSis improper.Recall S = {Id, RU1 , RU2 RU1} or S = {Id, RU1 , RU2}. Consider the caseswhere U1 ∩U2 is not a singleton. The following pictures show two morecounterexamples where the CCS is improper in this case.Figure 10.7: Ball and line15410.5. Failure to generalize to maximally monotone operatorsFigure 10.8: Two ballsRemark 10.4.5. Clearly, when we replace the balls in the above figures bythe corresponding circles, we will get the same results.10.5 Failure to generalize to maximally monotoneoperatorsLet U1, U2 be linear subspaces. Set S = {Id, RU1 , RU2 RU1}. By Theo-rem 7.2.5, for every x ∈ H, CCSx is the unique point satisfying the followingtwo properties:(i) CCSx ∈ aff{x, RU1 x, RU2 RU1 x};(ii) ‖CCSx− x‖ = ‖CCSx− RU1 x‖ = ‖CCSx− RU2 RU1 x‖.Moreover, again by Theorem 7.2.5,(∀x ∈ H) Paff(S(x))(PU1∩U2 x) = CCSx. (10.2)In fact, the excellent performance of the circumcenter mapping in linearconvergence mainly comes from the equation (10.2).In order to show a counterexample where the definition of CCS is failedto be directly generalized to maximally monotone theory, we need thedefinition and facts below.Definition 10.5.1. [6, Definition 23.1] Let A : H → 2H and let γ ∈ R++. Theresolvent of A isJA = (Id+A)−1.15510.5. Failure to generalize to maximally monotone operatorsFact 10.5.2. [6, Corollary 23.11] Let A : H → 2H be maximally monotoneand let γ ∈ R++. Then the following hold:(i) JγA : H → H and Id−JγA : H → H are firmly nonexpansive andmaximally monotone.(ii) The reflected resolventRγA : H → H : x 7→ 2JγAx− x.is nonexpansive.Fact 10.5.3. [6, Corollary 20.28] Let A : H → H be monotone and continu-ous. Then A is maximally monotone.Fact 10.5.4. [6, Corollary 20.26] Let C be a nonempty closed convex subsetofH. Then NC is maximally monotone.Fact 10.5.5. [6, Corollary 23.4] Let C be a nonempty closed convex subset ofH. ThenJNC = (Id+NC)−1 = PC.By Fact 10.5.5, RU1 = 2PU1 − Id = 2JNU1 − Id and RU2 = 2PU2 − Id =2JNU2 − Id. In the following example, we replace the two maximally mono-tone operators NU1 , NU2 in the set S = {Id, 2JNU1 − Id, 2JNU2 − Id} by othertwo special maximally monotone operators, 2 Id and 3 Id. We shall showthat the new CCS is no longer proper.Example 10.5.6. Let A = 2 Id, and B = 3 Id. Then A and B are maximallymonotone operator, by Fact 10.5.3. (Moreover A, B are linear relation.) Bydefinitions, we getJA = (A + Id)−1 = (3 Id)−1 =13Id; RA = 2JA − Id = 23 Id− Id = −13Id;JB = (B + Id)−1 = (4 Id)−1 =14Id; RB = 2JB − Id = 12 Id− Id = −12Id .Denote S1 = {Id, RA, RB} and S2 = {Id, RA, RBRA}. Thenaff(S1(x)) = aff{x, RAx, RBx} = x + span{RAx− x, RBx− x}= x + span{− 43x,−32x}= x +{− α43x− β32x∣∣∣ α, β ∈ R} = {αx | α ∈ R};15610.5. Failure to generalize to maximally monotone operatorsaff(S2(x)) = aff{x, RAx, RBRAx} = x + span{RAx− x, RBRAx− x}= x + span{− 43x,−56x}= x +{− α43x− β56x∣∣∣ α, β ∈ R} = {αx | α ∈ R}.Let x ∈ Hr {0}. Now(∃y ∈ aff(S1(x)))‖y− x‖ = ‖y− RAx‖ = ‖y− RBx‖⇐⇒(∃α ∈ R) ‖αx− x‖ = ‖αx + 13x‖ = ‖αx + 12x‖But clearly there exists no α ∈ R satisfying|α− 1| =∣∣∣α+ 13∣∣∣ = ∣∣∣α+ 12∣∣∣.Hence,(∀x ∈ H) CCS1 x = ∅.Similarly, there is no point y ∈ aff(S2(x)), such that‖y− x‖ = ‖y− RAx‖ = ‖y− RBRAx‖,which implies that(∀x ∈ Hr {0}) CCS2 x = ∅.From the example above, we know that the most direct way to generalizethe definition of CCS by maximally monotone theory doesn’t work. But itmay possible that there are other ways to generalize the circumcenter opera-tor by maximally monotone theory to satisfy the closest distance propertyof CCS i.e., the equality (10.2).157Chapter 11Numerical experiments11.1 OverviewThroughout this chapter, we assumeH = R10 and U1, U2 are two linearsubspaces in R10. LetS1 = {Id, RU1 , RU2},S2 = {Id, RU1 , RU2 RU1},S3 = {Id, RU1 , RU2 , RU2 RU1},S4 = {Id, RU1 , RU2 , RU2 RU1 , RU1 RU2 RU1}.Notice thatCCS2 is the CT in [12]and hence, it is also the CRM operator C in [13] when m = 2.The goal of this chapter is to compare the convergence rates of the sevenalgorithms: the four circumcenter methods induced by S1, S2, S3, S4 re-spectively, the Douglas–Rachford method (DRM), the method of alternatingprojections (MAP) and the symmetric method of alternating projections(SymMAP) to solve the best approximation problem.All of the calculations are implemented by Matlab. Part of the maincodes are attached in the appendix. We use the performance profiles intro-duced by Dolan and Moré [18] to evaluate and compare the performance ofthe seven algorithms. (Note that interesting phenomena may occur whencomparing more than two algorithms [19].) In Section 11.3, we use twodifferent performance measures:the number of iterations to attain the tolerance e = 10−4,andthe number of iterations to satisfy the convergence test in 50 iterations.(Note that other performance measures are possible, e.g., computing time.This will likely have an impact on the comparisons of the algorithms.) After15811.1. Overviewobtaining the related performance matrices T and (T(k))1≤k≤7, we use theperf.m file in [17] and the perf_profile.m file in [26] respectively to generatethe plots of performance profile.The numerical experiments not only illustrate the theoretical resultsshowed in the previous chapters, but also show some new discoveries. Ourmain findings are summarized as follows:• Denote TU1,U2 as the Douglas–Rachford operator. In all of our ex-amples, we find that: (see Figures 11.1 and 11.2, Section 11.2.3, andFigures 11.12 and 11.13)∗ Let x ∈ R10. Set the initial point x0 = PU1 x or x0 = PU2 x. Then(∀k ∈N) ‖CCkS1 x0 − PU1∩U2 x0‖ = ‖TkU1,U2 x0 − PU1∩U2 x0‖.∗ For every x0 ∈ R10, and for every k ∈N,‖CCkS1 x0 − PU1∩U2 x0‖ ≤ ‖TkU1,U2 x0 − PU1∩U2 x0‖.In our future work, we shall try to prove algebraically the aboveresults.• Compared with the circumcenter method induced by S1, MAP andSymMAP have much higher probability to be the optimal solver (seeFigure 11.14 and 11.15).• Compared with the MAP and symmetric MAP, the circumcentermethod induced by S4 is always the optimal solver (see Figure 11.16and 11.17).• Using the perf.m in [17], we observe thatCCS3 is better than CCS2 which is better than CCS4 .Using the perf_profile.m in [26], we obtain thatCCS3 is the optimal solver.The comparison between CCS2 and CCS4 is a little bit complicated.(See Figures 11.20, 11.21, 11.22 and 11.23.)15911.2. Preliminary comparison of convergence of algorithmsWe conclude this overview section with some comments on notations.In this chapter, to facilitate the statements later, for each problem p and foreach solver (or algorithm) s,we denote the kth iteration of solver s to solve problem p by a(k)p,s .For every i ∈ {1, 2, 3, 4}, in all of our figures, for notational simplicity,we denote the circumcenter method induced by Si as CCSi .Let x0 be the common initial point of the algorithms. We denotex = PU1∩U2 x0,which is the solution of the best approximation problem.11.2 Preliminary comparison of convergence ofalgorithmsRecall that U1, U2 are two linear subspaces in R10 and x ∈ R10. So0 ∈ U1 ∩U2 6= ∅. By the theory results shown in the last chapters, exceptDRM sequence all of the rest six iteration sequences will linearly converge toPU1∩U2 x (see, Fact 3.2.12, Corollary 3.2.13, Proposition 8.5.6, Corollary 8.5.11,Example 9.3.2 and Example 9.3.3). By Corollary 3.3.6, the DRM iterationsequence (TkU1,U2(x))k∈N converge to PU1∩U2(x) if and only if x ∈ U1 +U2,since in our caseH = R10 which implies that the U1 +U2 is always closed.In this section, for all of the experiments with considering DRM, to let DRMiteration sequence converge to PU1∩U2(x) and for the sake of fairness, weset the common initial point x0 as PU1 x or PU2 x. For all of the experimentswithout considering DRM, the common initial point x0 = x ∈ R10.11.2.1 One instanceIn each of the following two experiments, we randomly generate twolinear subspaces U1, U2 in R10 and x ∈ R10. We perform 100 iterations forall of the seven algorithms. Recall that a(k)p,s is the kth iterate of the solver sthat we monitor to solve problem p and that x = PU1∩U2 x0. We measure andplot the difference of ‖a(k)p,s − x‖ for all 1 ≤ k ≤ 100. Then we get the twopictures below.16011.2. Preliminary comparison of convergence of algorithms10 20 30 40 50 60 70 80 90 100100 iterations00.20.40.60.811.21.4Difference with the target projectionComparison of convergence rates over one instanceDRMMAPSymMAPCCS1CCS2CCS3CCS4Figure 11.1: CCS3 is the winner10 20 30 40 50 60 70 80 90 100100 iterations00.20.40.60.811.21.41.61.8Difference with the target projectionComparison of convergence rates over one instanceDRMMAPSymMAPCCS1CCS2CCS3CCS4Figure 11.2: CCS4 is the winnerWe find that the iteration sequences from DRM overlap the sequence16111.2. Preliminary comparison of convergence of algorithmsfrom circumcenter method induced by S1. The circumcenter method in-duced by S2, S3 and S4 perform better than DRM, MAP and symmetricMAP.11.2.2 100 instancesBecause for different instances, the comparison results of the sevenalgorithms may be different (say, in Figure 11.1, CCS3 is the fastest, whilein Figure 11.2, CCS4 is the fastest), in this section, we randomly generate10 pairs of linear subspaces U1 and U2 in R10 and 10 initial points x0. Thisresulted in a total of 100 problems for each algorithm in every experiment.For every experiment, we compare the algorithms from two aspects:compare the related distances for fixed iterations and compare the numberof iterations to attain a certain requirements about the distances. Moreover,we consider two kinds of distance:6(i) the true error, given by ‖a(k)p,s − x‖ < e; and(ii) the gap distance, given by ‖PU1(a(k)p,s )− PU2(a(k)p,s )‖ < e.Therefore, for every experiment, we get four subplots.To be specific, we perform 100 iterations and then calculate and plot thedistances‖a(100)p,s − PU1∩U2(x)‖ and ‖PU1(a(100)p,s )− PU2(a(100)p,s )‖to get the two subplots in the first row. Meanwhile, we record and plot thesmallest number of iterations m such that‖a(m)p,s − PU1∩U2(x)‖ < 10−2 and m ≤ 1000 (11.1)in the first subplot of the second row. We record and plot the smallestnumber of iterations m such that‖PU1(a(m)p,s )− PU2(a(m)p,s )‖ < 10−2 and m ≤ 1000 (11.2)in the second subplot of the second row. Here in order to avoid spendingtoo much time in the experiment, we set 1000 as the upper limit of iterationnumber. Because in our case our algorithms always converge very fast, 10006The consideration of gap distance is motivated by [12]. Although the true error can beviewed as the best way to assure that one is sufficiently close to the solution point and in ourcases x = PU1∩U2 (x) is easily available, this is not always the case in application.16211.2. Preliminary comparison of convergence of algorithmsis very large for us. In order to see the differences of the convergence ratesof our algorithms, iteration numbers larger than 1000 has no difference with1000.Below we compare the seven algorithms together. The Matlab codes forthe Figure 11.3 is attached in Section 12.20 20 40 60 80 100100 instances (x0=PU1x)0246810121416|| a(100)s,p - PU1  U2x ||Differences of iteration sequences and solutionDRMMAPSymMAPCCS1CCS2CCS3CCS40 20 40 60 80 100100 instances (x0=PU1x)00.511.522.5|| PU1(a(100)s,p)-PU2(a(100)s,p) ||Differences of sequences' projections onto U 1  and U2DRMMAPSymMAPCCS1CCS2CCS3CCS40 20 40 60 80 100100 instances (x0=PU1x)02004006008001000Requied iteration numbers under error bound 0.01Stop Criteria: True Error (  = 0.01)DRMMAPSymMAPCCS1CCS2CCS3CCS40 20 40 60 80 100100 instances (x0=PU1x)0100200300400500600700Requied iteration numbers under error bound 0.01Stop Criteria: Gap Distace (  = 0.01)DRMMAPSymMAPCCS1CCS2CCS3CCS4Figure 11.3: Comparison over 100 instances for the seven algorithmsNote that in the above pictures sometimes the points overlap with eachother. We see that iterations of DRM always overlap with the correspondingiterations of CCS1 , and the two sequences are the slowest ones to convergeto PU1∩U2 x0.In the following experiment, we exclude the DRM and CCS1 , to get amore exact picture. Now our initial point is arbitrarily chosen from R10.16311.2. Preliminary comparison of convergence of algorithms0 20 40 60 80 100100 instances024681012|| a(100)s,p - PU1  U2x ||Differences of iteration sequences and solutionMAPSymMAPCCS2CCS3CCS40 20 40 60 80 100100 instances00.511.5|| PU1(a(100)s,p)-PU2(a(100)s,p) ||Differences of sequences' projections onto U 1  and U2MAPSymMAPCCS2CCS3CCS40 20 40 60 80 100100 instances0100200300400500Requied iteration numbers under error bound 0.01 Stop Criteria: True Error (  = 0.01)MAPSymMAPCCS2CCS3CCS40 20 40 60 80 100100 instances050100150200250300350Requied iteration numbers under error bound 0.01 Stop Criteria: Gap Distace (  = 0.01)MAPSymMAPCCS2CCS3CCS4Figure 11.4: Comparison over 100 instances without DRM and CCS1From the Figure 11.4, it is clear that the convergence rates of (∀i ∈ {2, 3,4}) (CCkSi x)k∈N are much better than the iteration sequences from MAP andsymmetric MAP. But the differences of the (CCkSi x)k∈N where k = 2, 3, 4 isnot clear in the two subplots in the first row. To analyze the benchmarkingdata better, we shall compare algorithms by performance profiles in thefollowing Section 11.3.11.2.3 Comparison of DRM and circumcenter method inducedby the first setIn this subsection, we carefully compare the DRM and the circumcentermethod induced by S1. We find that if the initial point x0 ∈ R10, the CCS116411.2. Preliminary comparison of convergence of algorithmsperforms better than DRM in every instance.0 20 40 60 80 100100 instances (x0   R10 )050100150|| a(100)s,p - PU1  U2x ||Differences of iteration sequences and solutionDRMCCS10 20 40 60 80 100100 instances (x0   R10 )024681012|| PU1(a(100)s,p)-PU2(a(100)s,p) ||Differences of sequences' projections onto U 1  and U2DRMCCS10 20 40 60 80 100100 instances (x0   R10 )020040060080010001200Requied iteration numbers under error bound 0.01 Stop Criteria: True Error (  = 0.01)DRMCCS10 20 40 60 80 100100 instances (x0   R10 )020040060080010001200Requied iteration numbers under error bound 0.01 Stop Criteria: Gap Distace (  = 0.01)DRMCCS1Figure 11.5: Comparison DRM and CCS1 with x0 ∈ R10But the following two figures show that if x0 ∈ U1 ∪U2, then the bench-marking data from the circumcenter method induced by S1 overlap withthat from the DRM in every example.16511.2. Preliminary comparison of convergence of algorithms0 20 40 60 80 100100 instances (x0=PU1x)01020304050|| a(100)s,p - PU1  U2x ||Differences of iteration sequences and solutionDRMCCS10 20 40 60 80 100100 instances (x0=PU1x)012345|| PU1(a(100)s,p)-PU2(a(100)s,p) ||Differences of sequences' projections onto U 1  and U2DRMCCS10 20 40 60 80 100100 instances (x0=PU1x)020040060080010001200Requied iteration numbers under error bound 0.01 Stop Criteria: True Error (  = 0.01)DRMCCS10 20 40 60 80 100100 instances (x0=PU1x)020040060080010001200Requied iteration numbers under error bound 0.01 Stop Criteria: Gap Distace (  = 0.01)DRMCCS1Figure 11.6: Comparison DRM and CCS1 with x0 = PU1 x0 20 40 60 80 100100 instances (x0=PU2x)0102030405060|| a(100)s,p - PU1  U2x ||Differences of iteration sequences and solutionDRMCCS10 20 40 60 80 100100 instances (x0=PU2x)012345|| PU1(a(100)s,p)-PU2(a(100)s,p) ||Differences of sequences' projections onto U 1  and U2DRMCCS10 20 40 60 80 100100 instances (x0=PU2x)020040060080010001200Requied iteration numbers under error bound 0.01Stop Criteria: True Error (  = 0.01)DRMCCS10 20 40 60 80 100100 instances (x0=PU2x)020040060080010001200Requied iteration numbers under error bound 0.01Stop Criteria: Gap Distace (  = 0.01)DRMCCS1Figure 11.7: Comparison DRM and CCS1 with x0 = PU2 x16611.3. Performance profileTherefore, we guess that(∀x ∈ U1 ∪U2) (∀k ∈N) ‖CCkS1 x− PU1∩U2‖ = ‖TkU1,U2 x− PU1∩U2‖.and(∀x ∈ H) (∀k ∈N) ‖CCkS1 x− PU1∩U2‖ ≤ ‖TkU1,U2 x− PU1∩U2‖.To prove algebraically this conjecture is part of our future work.11.3 Performance profilePerformance profiles have proved to be an important tool for bench-marking optimization solvers. Dolan and Moré in [18] define a benchmarkin terms of a set P of benchmark problems, a set S of optimization solvers,and a performance matrix T. Once the performance matrix T (respectively(T(i))1≤i≤7) is obtained, the perf.m file in [17] (respectively the perf_profile.min [26]) can be used to generate the plot of performance profiles.First, we review some background of the performance profile. Theremainder of this section is borrowed from [18, Section 2].Benchmark results are generated by running a solver s ∈ S on a set P ofproblems and recording information of interest such as CPU time, numberof function evaluations, or iteration counts for algorithms.Assume now we consider the set of solvers S on a test set P and we havens solvers and np problems. In this thesis, we shall consider two performancemeasures. The first performance measure is that for each problem p andsolver s, we define eithertp,s = the smallest k such that ‖a(k)p,s − x‖ ≤ 10−4. (11.3)In the second performance measure, denote dp = mins∈Sol‖a(100)p,s − x¯‖. Weconsider additionally the convergence test(∀k ∈ {1, . . . , 50}) ‖a(k)p,s − x¯‖ ≤ dp + τ(‖a(1)p,s − x¯‖ − dp), (11.4)where τ is the parameter of gate. The performance measure is defined astp,s = the smallest k such that a(k)p,s passing the test (11.4). (11.5)The performance ratio is defined byrp,s =tp,smin{tp,s | s ∈ S} ,16711.3. Performance profilewhich is the function of the performance on problem p by solver s versusthe best performance by any solver on this problem.The performance profile function ρs is defined asρs(τ) : R→ [0, 1] : τ 7→ 1np size {p ∈ P : rp,s ≤ τ},then ρs(τ) is the probability for solver s ∈ S that a performance ratio rp,sis within a factor τ ∈ R of the best possible ratio. The function ρs is the(cumulative) distribution function for the performance ratio.The term performance profile is used for the distribution function of aperformance metric. The plot of performance profile reveals all of the majorperformance characteristics. In particular, if the set of problems P is suitablylarge and representative of problems that are likely to occur in applications,then solvers with large probability ρs(τ) are to be preferred.The value ρs(1) is the probability that the solver will win over the rest ofthe solvers. Thus, if we are interested only in the number of wins, we needonly to compare the values of ρs(1) for all of the solvers.We assume that a parameter rM ≥ rp,s, for all p, s is chosen, and rp,s = rMif and only if solver s does not solve problem p. As a result of this convention,ρs(rM) = 1, andρ∗s ≡ limτ→(rM)−ρs(τ)is the probability that the solver solves a problem. Thus, if we are interestedonly in solvers with a high probability of success, then we need to comparethe values of ρ∗s for all solvers and choose the solvers with the largest value.The value of ρ∗s can be readily seen in a performance profile because ρsflatlines for large values of τ; that is, ρs(τ) = ρ∗s for τ ∈ [rs, rM[ for somers < rM. By the definition of perf.m in [17], since the ρ∗s of every solver willmust be 1, we can not get the probability of problems solved by a certainsolver. This is the reason why we additionally consider the convergence test(11.4) and use the perf_profile.m in [26] to generate the performance plot.In [18, Section 4] even extending τ to 100, the authors fail to capture thecomplete performance data fo the two softwares they considered. So theydisplayed a log scale of the performance profiles, that is they plotτ 7→ 1npsize {p ∈ P : log2(rp,s) ≤ τ}.Therefore, in the next section we will mainly show the log2 scaled plots.16811.4. Performance evaluation11.4 Performance evaluationIn all of the experiments in this section, we randomly generate 10 pairs oflinear subspaces U1, U2 and 10 initial points x inR10, so we get 100 instancesof problems. We use the tolerance e = 10−4. Recall that we denote the kthiteration of solver s to solve problem p by a(k)p,s . Recall that our performancemeasure will be eithertp,s = the smallest k such that ‖a(k)p,s − x‖ ≤ 10−4.ortp,s = the smallest k such that a(k)p,s passing the test (11.4).11.4.1 Comparison of all of the seven algorithmsSince the performance profile avoid the need to discard solver failuresfrom the performance data, there is no need to set the initial point x ∈U1 +U2 to assure iteration sequence from DRM to converge to x = PU1∩U2 x.In the following two experiments, we compare all of the seven solvers:DRM, MAP, SymMAP and the circumcenter methods induced by S1, S2, S3and S4. First, we calculate the related performance matrix T = (tp,s)100×7(respectively the (T(i))1≤i≤7). Thereafter, we use the perf.m file in [17] (re-spectively the perf_profile.m in [26]) to generate the following performanceprofile plots. (The main Matlab codes to generate the performance matrixT = (tp,s)100×7 are attached in Section 12.2.) As the caption shows, Fig-ure 11.8 is from the performance measure (11.3) and Figure 11.9 comes fromthe performance measure (11.5).16911.4. Performance evaluation0 0.5 1 1.5 2 2.5 3 3.5 400.10.20.30.40.50.60.70.80.91P(r p,s   : 1  s  ns)Log2  Scaled Performance Profile (  = 10-4 )DRMMAPSymMAPCCS1CCS2CCS3CCS4Figure 11.8: tp,s = the smallest k s.t. ‖a(k)p,s − x‖ ≤ e1 2 4 8 16 32ratio of distance 00.10.20.30.40.50.60.70.80.91Portion of problem solved p ()Log2  Scaled Performance Profile for 50 iterarions (  =10%)DRMMAPSymMAPCCS1CCS2CCS3CCS4Figure 11.9: tp,s = the smallest k such that a(k)p,s passing the test (11.4)Comparing the values of ρs(1) for all solvers in the above four figures,17011.4. Performance evaluationwe know that the probability that CCS3 is the winner on a given problemis over 0.6 in both of the two performance measures. In addition, it is clearthat CCS2 , CCS3 , CCS4 are always faster than the other four algorithms.From both pictures in Figure 11.8, CCS3 dominates all other solvers,which indicates that for every τ ≥ 1, CCS3 solves more problems within afactor of τ of the best solver. By the second figure in Figure 11.8, if we choosebeing within a factor of 14 of the best solver as the scope of our interest, thenall of the solvers except the DRM suffice. But these results do not imply thatthe circumcenter method induced by CCS3 is faster on every problem thanthe circumcenter method induced by CCS2 , CCS4 .From Figure 11.9, comparing the values of ρ∗s for all solvers, we get thatall of the solvers except DRM have high probability to solve the problemsunder the convergence test of (11.4). CCS4 dominates the other solvers.One of our future work is to give clear explanation for the differentwinners from the two performance profile generating files.Therefore, in the next section, we will consider on the circumcentermethod induced by S2, S3 and S4 and we will only analyze the log2 scaledperformance profile. Below we show the log2 scaled performance profilefor the two performance measures (11.3) and (11.5) without considering theDRM.0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 500.10.20.30.40.50.60.70.80.91P(r p,s   : 1  s  ns)Log2  Scaled Performance Profile (  = 10-4 )MAPSymMAPCCS1CCS2CCS3CCS4Figure 11.10: tp,s = smallest k s.t. ‖a(k)p,s − x‖ ≤ e17111.4. Performance evaluation1 2 4 8 16 32ratio of distance 00.10.20.30.40.50.60.70.80.91Portion of problem solved p ()Log2  Scaled Performance Profile for 50 iterarions (  =10%)MAPSymMAPCCS1CCS2CCS3CCS4Figure 11.11: tp,s = the smallest k s.t. a(k)p,s passing the test (11.4)Clearly, the performance profiles of the six algorithms are similar to theirperformance profile in Figure 11.8 and Figure 11.9.11.4.2 Comparisons of circumcenter methods with DRM or MAPIn the remaining sections, we use both of the performance measures(11.3) and (11.5) and show only the log2 scaled performance profiles.Below we see the performance profile of “CCS1 versus DRM”. BothFigures 11.12 and 11.13 show that circumcenter method induced by CCS1 isthe absolute winner.17211.4. Performance evaluation0 2 4 6 8 1000.10.20.30.40.50.60.70.80.91P(r p,s   : 1  s  ns)Log2  Scaled Performance Profile (  = 10-4 )DRMCCS1Figure 11.12: tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e1 2 4 8 16 32ratio of distance 00.10.20.30.40.50.60.70.80.91Portion of problem solved p ()Log2  Scaled Performance Profile for 50 iterarions (  =10%)DRMCCS1Figure 11.13: tp,s = smallest k s.t. a(k)p,s passing the convergence test (11.4)Below we show the performance profile of “CCS1 versus MAP and17311.4. Performance evaluationsymmetric MAP”. Clearly, MAP and symmetric MAP have much higherprobability to be the the optimal solver than CCS1 .0 0.2 0.4 0.6 0.8 1 1.200.10.20.30.40.50.60.70.80.91P(r p,s   : 1  s  ns)Log2  Scaled Performance Profile (  = 10-4 )MAPSymMAPCCS1Figure 11.14: tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e17411.4. Performance evaluation1 2 4 8 16ratio of distance 00.10.20.30.40.50.60.70.80.91Portion of problem solved p ()Log2  Scaled Performance Profile for 50 iterarions (  =10%)MAPSymMAPCCS1Figure 11.15: tp,s = the smallest k such that a(k)p,s passing the test (11.4)The following figures are the performance profiles of “CCS4 versus MAPand symmetric MAP”. Clearly, circumcenter method induced by CCS4 is theoptimal solver.17511.4. Performance evaluation0 0.5 1 1.5 2 2.500.10.20.30.40.50.60.70.80.91P(r p,s   : 1  s  ns)Log2  Scaled Performance Profile (  = 10-4 )MAPSymMAPCCS4Figure 11.16: tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e1 2 4 8 16 32ratio of distance 00.10.20.30.40.50.60.70.80.91Portion of problem solved p ()Log2  Scaled Performance Profile for 50 iterarions (  =10%)MAPSymMAPCCS4Figure 11.17: tp,s = the smallest k such that a(k)p,s passing the test (11.4)17611.4. Performance evaluation11.4.3 Evaluation of circumcenter methodsSimilarly with Section 11.4.2, in this section, we show only the log2scaled performance profile. The left subplots are from performance measure(11.3) and the right subplots are from performance measure (11.5).First, we plot the performance profile of all of the three circumcentermethods induced by S2, S3 and S4.0 0.5 1 1.5 200.10.20.30.40.50.60.70.80.91P(r p,s   : 1  s  ns)Log2  Scaled Performance Profile (  = 10-4 )CCS2CCS3CCS4Figure 11.18: tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e17711.4. Performance evaluation1 2 4 8 16ratio of distance 00.10.20.30.40.50.60.70.80.91Portion of problem solved p ()Log2  Scaled Performance Profile for 20 iterarions (  =10%)CCS2CCS3CCS4Figure 11.19: tp,s = the smallest k such that a(k)p,s passing the test (11.4)From Figure 11.18 and 11.19, comparing the ρs(1) we know all of thethree algorithms have the probability to be the fastest solver. All of thesolvers have higher probability to satisfy the convergence test with 50 itera-tions. In Figure 11.18, the optimal solver is circumcenter method inducedby 3. In Figure 11.19, the comparison of performance profile between CCS3and CCS4 is not clear.Below we compare the circumcenter methods induced by S2, S3 or S4two by two.17811.4. Performance evaluation0 0.2 0.4 0.6 0.8 100.10.20.30.40.50.60.70.80.91P(r p,s   : 1  s  ns)Log2  Scaled Performance Profile (  = 10-4 )CCS2CCS3Figure 11.20: tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e1 2 4 8 16 32ratio of distance 00.10.20.30.40.50.60.70.80.91Portion of problem solved p ()Log2  Scaled Performance Profile for 50 iterarions (  =10%)CCS2CCS3Figure 11.21: tp,s = the smallest k such that a(k)p,s passing the test (11.4)From Figure 11.20 and Figure 11.21, for both of the performance mea-17911.4. Performance evaluationsures defined by (11.3) and (11.5), the circumcenter method induced by CCS3has the higher probability of being the optimal solver and that in both plotsthe probability of CCS3 being the winner on a given problem are over 0.7.0 0.2 0.4 0.6 0.8 100.10.20.30.40.50.60.70.80.91P(r p,s   : 1  s  ns)Log2  Scaled Performance Profile (  = 10-4 )CCS2CCS4Figure 11.22: tp,s = smallest m s.t. ‖a(m)p,s − x‖ ≤ e18011.4. Performance evaluation1 2 4 8 16 32ratio of distance 00.10.20.30.40.50.60.70.80.91Portion of problem solved p ()Log2  Scaled Performance Profile for 50 iterarions (  =10%)CCS2CCS4Figure 11.23: tp,s = the smallest k such that a(k)p,s passing the test (11.4)Figure 11.22 and Figure 11.23 show that the performance of CCS2 andCCS4 has no large difference. But since S2 = {Id, RU1 , RU2 RU1} and S4 ={Id, RU1 , RU2 , RU2 RU1 , RU1 RU2 RU1}, the calculation of CCS2 is easier thanthat of CCS4 . Because the formula of CCS includes the calculation of theinverse of matrix, the circumcenter method is sensitive to the calculationerror. Hence, overall, on the left plots, CCS2 is better than CCS4 .According to our numerical experiments, when we use the performancemeasure,tp,s = the smallest k such that a(k)p,s passing the test (11.4).we find that as the upper bound of the number of iterations becomes large,the percentage of problems solved by circumcenter method induced by S2is larger than that of circumcenter method induced by S4. But in all casesthe circumcenter method induced by S3 has the highest probability to bethe optimal solver.181Chapter 12ConclusionIn this thesis we systematically studied the circumcenters of a set withfinitely many elements in real Hilbert space. We introduced several newnotions that are helpful to solve the best approximation problem and fea-sibility problem. Linear convergence was obtained by some of our newlyintroduced method.12.1 Main resultsWe presented several symmetrical and asymmetrical formulae of thecircumcenter of a set with finitely many elements. We explored the con-vergence of the circumcenters of sequences of sets. Moreover, we gave acharacterization of the existence of the circumcenter of three distinct points.We introduced two new concepts: circumcenter operator and circumcen-ter mapping induced by operators.When we specifically set the finitely many elements as the values offinitely many operators over a point x, we obtained the circumcenter map-ping induced by a set of operators. We defined the notion of proper thatmeans that the circumcenter exists for every point in the space. We exploredthe continuity of the mapping.When we chose the above operators from the set of compositions ofreflectors, we obtained the proper circumcenter mapping induced by reflec-tors. We deduced the circumcenter method induced by reflectors whichconsider the iteration sequence of the circumcenter mapping induced byreflectors. We obtained many pleasing properties of the iteration sequence.In addition, we gained some convergence results about the circumcentermethod induced by reflectors. Notably, we proved that for some special sets,the circumcenter operator induced by those sets converge to the solution ofbest approximation problem at least as fast as the MAP, symmetric MAP orone kind of their accelerated versions.We also considered the circumcenter operator induced by projectors. Wepresented both proper and improper examples of the circumcenter operator18212.2. Future workinduced by projectors.Although we showed that the circumcenter operators induced by reflec-tors perform very well to solve the best approximation problem or feasibilityproblem, we found some drawbacks of the circumcenter operator.Finally, performance profiles were used to compare four of the circum-center operators (we name them as CCS1 , CCS2 , CCS3 , CCS4) induced byreflectors, DRM, MAP, and symmetric MAP. In view of the performanceprofile plots, the circumcenter methods induced by the CCS1 dominated theDRM; The CCS2 , CCS3 , CCS4 performed much better than CCS1 and MAPand symmetric MAP operators. In fact, the CCS2 is the CRM operator in[12]. Moreover, as the performance profile plots illustrate, if we use theperformance profile generating file, the perf.m, in [17], then CCS3 performsbetter than CCS2 which is better than CCS4 . If we use the performance profilegenerating file, the perf_profile, in [26], then CCS4 is better than CCS2 . Butthe comparison between CCS2 and CCS3 is unclear.12.2 Future workIn the following, we discuss some possible directions of future research:1. As we mentioned in Section 11.2.3, we will try to prove(∀x ∈ U1 ∪U2)(∀k ∈N) ‖CCkS1 x− PU1∩U2‖ = ‖TkU1,U2 x− PU1∩U2‖,and(∀x ∈ H) (∀k ∈N) ‖CCkS1 x− PU1∩U2‖ ≤ ‖TkU1,U2 x− PU1∩U2‖.Lemma 8.2.5 may be used here.2. By the powerful Theorem 8.3.2 and Fact 2.4.9, in order to prove thelinear convergence of CCS , it suffices to find an TS ∈ aff(S) such thatTkS → P∩mi=1Ui µ-linearly in B(H) for some µ ∈ [0, 1[.Combining Fact 2.4.17 and Fact 2.4.18, we know in real Hilbert space,if T is linear and firmly nonexpansive, then Tn → PFix T pointwise.In addition, Fact 2.4.10 told us that in finite-dimensional space, Lk →L∞ pointwise imply that Lk → L∞ linearly. Hence, it is natural toask the question: How can we extend the Fact 2.4.10 from finite-dimensional space to infinite-dimensional space? If we can solve thisquestion, then we can generalize the linear convergence result, theProposition 8.3.3, of CCS from finite-dimensional space to infinite-dimensional space.18312.2. Future work3. In Section 8.6 we mentioned that the translation argument can be usedto generalize the results in Section 8.6 to the case when U1, U2 areaffine subspaces. To generalize the convergence results from m linearsubspaces to m affine subspaces and to give clear algebraical proof forthe generalization is also an interesting project. Here [12, Corollary 3]may be helpful.4. From Figures 11.20, 11.21, 11.22 and 11.23, we find that using theperf.m in [17], we obtain thatCCS3 is better than CCS2 which is better than CCS4 .But using the perf_profile.m in [26], we obtain thatCCS3 is better than CCS2 ,but the comparison between CCS3 and CCS4 is unclear.We shall try to find a clear explanation for this difference of the twocomparison results.184Bibliography[1] T. M. APOSTOL, Calculus. Vol. I: One-variable calculus, with an introduc-tion to linear algebra, Second edition, Blaisdell Publishing Co. Ginn andCo., Waltham, Mass.-Toronto, Ont.-London, 1967. → pages 23, 24[2] H. H. BAUSCHKE, J. Y. BELLO CRUZ, T. T. A. NGHIA, H. M. PHAN,AND X. WANG, The rate of linear convergence of the Douglas-Rachfordalgorithm for subspaces is the cosine of the Friedrichs angle, J. Approx.Theory, 185 (2014), pp. 63–79. → pages 10, 28, 29[3] , Optimal rates of linear convergence of relaxed alternating projectionsand generalized Douglas-Rachford methods for two subspaces, Numer. Algo-rithms, 73 (2016), pp. 33–76. → pages 17[4] H. H. BAUSCHKE AND J. M. BORWEIN, On projection algorithms forsolving convex feasibility problems, SIAM Rev., 38 (1996), pp. 367–426. →pages 13, 18, 100[5] H. H. BAUSCHKE, J. M. BORWEIN, AND A. S. LEWIS, The method ofcyclic projections for closed convex sets in Hilbert space, in Recent develop-ments in optimization theory and nonlinear analysis (Jerusalem, 1995),vol. 204 of Contemp. Math., Amer. Math. Soc., Providence, RI, 1997,pp. 1–38. → pages 26, 27[6] H. H. BAUSCHKE AND P. L. COMBETTES, Convex analysis and monotoneoperator theory in Hilbert spaces, CMS Books in Mathematics/Ouvragesde Mathématiques de la SMC, Springer, Cham, second ed., 2017. Witha foreword by Hédy Attouch. → pages 4, 7, 9, 12, 13, 14, 15, 16, 18, 32,42, 155, 156[7] H. H. BAUSCHKE, P. L. COMBETTES, AND D. R. LUKE, Finding bestapproximation pairs relative to two closed convex sets in Hilbert spaces, J.Approx. Theory, 127 (2004), pp. 178–192. → pages 148[8] H. H. BAUSCHKE, F. DEUTSCH, H. HUNDAL, AND S.-H. PARK, Accel-erating the convergence of the method of alternating projections, Trans. Amer.185BibliographyMath. Soc., 355 (2003), pp. 3433–3461. → pages 12, 25, 27, 28, 30, 31, 32,131, 136, 143[9] H. H. BAUSCHKE, B. LUKENS, AND W. M. MOURSI, Affine nonexpansiveoperators, Attouch-Théra duality and the Douglas-Rachford algorithm, Set-Valued Var. Anal., 25 (2017), pp. 481–505. → pages 16, 17, 19[10] H. H. BAUSCHKE, S. M. MOFFAT, AND X. WANG, Firmly nonexpansivemappings and maximally monotone operators: correspondence and duality,Set-Valued Var. Anal., 20 (2012), pp. 131–153. → pages 146[11] H. H. BAUSCHKE, H. OUYANG, AND X. WANG, On circumcenters offinite sets in Hilbert spaces, arXiv preprint arXiv:1807.02093, (2018). →pages v[12] R. BEHLING, J. Y. BELLO CRUZ, AND L.-R. SANTOS, Circumcenteringthe Douglas–Rachford method, Numer. Algorithms, 78 (2018), pp. 759–776.→ pages 1, 2, 25, 32, 33, 35, 88, 91, 93, 100, 123, 125, 158, 162, 183, 184[13] , On the linear convergence of the circumcentered-reflection method,Oper. Res. Lett., 46 (2018), pp. 159–162. → pages 2, 25, 32, 34, 35, 88,100, 120, 121, 158[14] J. M. BORWEIN AND M. K. TAM, A cyclic Douglas-Rachford iterationscheme, J. Optim. Theory Appl., 160 (2014), pp. 1–29. → pages 13[15] H. S. M. COXETER, Introduction to geometry, Wiley Classics Library,John Wiley & Sons, Inc., New York, 1989. Reprint of the 1969 edition.→ pages 23, 69[16] F. DEUTSCH, Best approximation in inner product spaces, vol. 7 ofCMS Books in Mathematics/Ouvrages de Mathématiques de la SMC,Springer-Verlag, New York, 2012. → pages 6, 9, 10, 11, 12, 22, 25, 26, 27,32[17] E. D. DOLAN AND J. J. MORÉ, COPS. See http://www.mcs.anl.gov/~more/cops/. → pages 159, 167, 168, 169, 183, 184, 200[18] , Benchmarking optimization software with performance profiles, Math.Program., 91 (2002), pp. 201–213. → pages 158, 167, 168[19] N. GOULD AND J. SCOTT, A note on performance profiles for benchmarkingsoftware, ACM Trans. Math. Software, 43 (2016), pp. Art. 15, 5. → pages158186Bibliography[20] L. HOGBEN, ed., Handbook of linear algebra, Discrete Mathematics andits Applications (Boca Raton), CRC Press, Boca Raton, FL, second ed.,2014. → pages 7[21] R. A. HORN AND C. R. JOHNSON, Matrix analysis, Cambridge Univer-sity Press, Cambridge, second ed., 2013. → pages 23[22] E. KREYSZIG, Introductory functional analysis with applications, WileyClassics Library, John Wiley & Sons, Inc., New York, 1989. → pages 5,6, 7, 8, 9, 15, 16, 17[23] S. J. LEON, Linear algebra with applications, Macmillan, Inc., New York;Collier-Macmillan Publishers, London, 1980. → pages 22[24] W. S. MASSEY, Cross products of vectors in higher-dimensional Euclideanspaces, Amer. Math. Monthly, 90 (1983), pp. 697–701. → pages 24[25] C. MEYER, Matrix analysis and applied linear algebra, Society for Indus-trial and Applied Mathematics (SIAM), Philadelphia, PA, 2000. →pages 20, 21, 22[26] J. J. MORÉ AND S. M. WILD, Benchmarking derivative-free optimizationalgorithms. See http://www.mcs.anl.gov/~more/dfo/. → pages159, 167, 168, 169, 183, 184, 200[27] R. T. ROCKAFELLAR, Convex analysis, Princeton Landmarks in Mathe-matics, Princeton University Press, Princeton, NJ, 1997. Reprint of the1970 original, Princeton Paperbacks. → pages 8[28] S. ROMAN, Advanced linear algebra, vol. 135 of Graduate Texts in Mathe-matics, Springer, New York, third ed., 2008. → pages 43[29] T. TOM DIECK, Algebraic topology, EMS Textbooks in Mathematics,European Mathematical Society (EMS), Zürich, 2008. → pages 23187Appendices188Appendix A: Matlab functionfilesIn this appendix, we show the Matlab function of our main actor inthis thesis, CCS . Below we assume S = {Id, RU1 , RU2}. In fact, for every Ssatisfying {Id} & S ⊆ Ω where Ω is defined in (7.1), the function definitionfor CCS is similar with our example below.In order to define the CCS , firstly we need the following projectionoperator P, the reflector R.1 func t ion [ y ] = P ( x ,A )2 % Ca l c u l a te the p r o j e c t i o n of a vec tor x onto the n u l l space ofthe matrix A3 y = x − pinv (A) ∗A∗x ;4 end1 func t ion [ y ] = R( x ,A )2 % Ca l c u l a te the r e f l e c t o r of a vec tor x onto the n u l l space of thematrix A3 y=2∗P ( x ,A)−x ;4 endNow we are ready to show the function CCS .1 func t ion [ p ] = CC_S( x , U1 , U2)2 % Ca l c u l a te the circumcenter of the s e t S ( x ) , CC_{ S } x .3 % Assume S={ Id , R_ {U_ { 1 } } , R_ {U_ { 2 } } } . ( i f necessary , we canchange the4 % d e f i n i t i o n of Sx accordingly )5 % generate the s e t S ( x )6 Sx =[x , R( x , U1) ,R( x , U2) ] ;7 % generate the matrix S ( x )−x and take 8 d i g i t s8 H=Sx−x ;9 K=round (H, 8 ) ;10 i f K==011 p=x ; % the c a r d i n a l i t y of S ( x ) i s 112 e l s e % the c a r d i n a l i t y of S ( x ) i s l a r g e r than or equal 213 [KK j b ]= r r e f (K) ;14 BK=H( : , j b ) ; % return the b a s i s of the range of H18915 GK=transpose (BK) ∗ BK ; % return the Gram matrix generated bythe columns of H16 f o r l =1: rank (K)17 b ( l ) =GK( l , l ) ; % get the diagonal elements of GK18 end19 p=x +(1/2)∗BK∗pinv (GK) ∗ t ranspose ( b ) ;20 endLetS1 = {Id, RU1 , RU2}S2 = {Id, RU1 , RU2 RU1}S3 = {Id, RU1 , RU2 , RU2 RU1}S4 = {Id, RU1 , RU2 , RU2 RU1 , RU1 RU2 RU1}.Notice that in all of our Matlab calculations, we name the CCS1 , CCS1 ,CCS1 ,CCS1by CC_Sa, CC_Sb, CC_Sc, CC_Sd respectively.190Appendix B: Calculation usedin Section 10.2Below we show the detailed calculation of Example 10.2.1 and 10.2.2 .Comparing the results we showed in Section 10.2, in the following pictureswe transform the column vectors to the corresponding row vectors to savespace.1 %%%%%% Example 1 0 . 1 . 12 A=[7 1 5 7 ] ;3 B=[10 5 5 1 0 ; 9 8 10 9 ] ;4 x =[4 5 3 8 ] ’ ;5 y=[9 10 6 6 ] ’ ;6 S11=CC_Sb ( x ,A, B ) ’7 S12=CC_Sb ( y ,A, B ) ’8 S13=CC_Sb ( x+y ,A, B ) ’S11 =-2.5976 1.7356 1.3262 1.4024S12 =0.7295 8.8048 0.3965 -2.2705S13 =-2.2681 11.8397 2.5827 -1.26811 % Veri fy add i t ive2 l o g i c a l ( S11+S12 == S13 )ans =1x4 logical array0 0 0 01 % Veri fy nonexpansive2 norm ( x−y )3 norm ( S11−S12 )191ans =7.9373ans =8.68321 % Veri fy s e l f−a d j o i n t2 S11 ∗ y3 x ’ ∗ S12 ’ans =10.3487ans =29.96731 %%%%%% Example 1 0 . 1 . 22 U1=[5 5 4 8 ] ;3 U2=[7 10 2 7 ; 8 10 2 1 ] ;4 x =[6 6 9 5 ] ’ ;5 y=[4 7 8 6 ] ’ ;6 S21=CC_Sa ( x , U1 , U2) ’7 S22=CC_Sa ( y , U1 , U2) ’8 S23=CC_Sa ( x+y , U1 , U2) ’S21 =0.6969 0.5650 4.9265 -3.2519S22 =-1.1533 2.0266 3.6855 -2.3886S23 =-0.3804 2.7325 8.5635 -5.75181 % Veri fy the add i t ive2 l o g i c a l ( S21+S22 == S23 )3 \end { verbatim }4 \ c o l o r { l i g h t g r a y } \begin { verbatim }5 ans =6 1x4 l o g i c a l array7 0 0 0 08 \end { verbatim } \ c o l o r { black }9 \begin { l s t l i s t i n g } [ language=Matlab ]10 % Veri fy nonexpansive11 norm ( x−y )12 norm ( S21−S22 )192ans =2.6458ans =2.80091 % Veri fy s e l f−a d j o i n t2 S21 ∗ y3 x ’ ∗ S22 ’ans =26.6430ans =26.4667193Appendix C: Main codesAppendix C.1: Codes to generate the Figure 11.31 % Randomly generate 10 p a i r s of subspaces and 10 i n i t i a l points ,comparing2 % the convergence r a t e of the 7 algori thms :DRM,MAP, symMAP,3 % CC_{ S_ { 1 } } ( c ircumcenter of { x , R_ {U_ { 1 } } x , R_ {U_ { 2 } } } ) ,4 % CC_{ S_ { 2 } } ( c ircumcenter of { x , R_ {U_ { 1 } } x , R_ {U_ { 2 } } R_ {U_ { 1 } } x } ) ,5 % CC_{ S_ { 3 } } ( c ircumcenter of { x , R_ {U_ { 1 } } x , R_ {U_ { 2 } } x , R_ {U_ { 2 } } R_{U_ { 1 } } x } )6 % CC_{ S_ { 4 } } ( c ircumcenter of { x , R_ {U_ { 1 } } x , R_ {U_ { 2 } } x , R_ {U_ { 2 } } R_{U_ { 1 } } x , R_ {U_ { 1 } } R_ {U_ { 2 } } x , R_ {U_ { 1 } } R_ {U_ { 2 } } R_ {U_ { 1 } } x )78 % We perform 100 i t e r a t i o n s and then c a l c u l a t e the d i s t a n c e sbetween the9 % i t e r a t i o n sequences with the s o l u t i o n point and the d i s t a n c ebetween the10 % sequences ’ p r o j e c t i o n s onto U_ { 1 } and U_ { 2 } to get the subplot 1and 211 % r e s p e c t i v e l y .1213 % Considering two stop c r i t e r i o n : t rue e r r o r and gap d i s t a n c e with\eps i lon14 % = 0 . 0 1 , we c a l c u l a t e the required number of i t e r a t i o n s and getthe subplots15 % 3 and 4 .1617 c l o s e a l l ; c l e a r ; c l c18 f o r k =1:1019 A=randi ( 1 0 0 , 3 , 1 0 ) ;20 B=randi ( 1 0 0 , 5 , 1 0 ) ;21 f o r l =1:102223 x0=randi ( 1 0 0 , 1 0 , 1 ) ;24 x=P ( x0 ,A ) ;2526 Barx=Psol ( x ,A, B ) ;27 DRM1x=DRM( x ,A, B ) ;DRM2x=DRM( x ,A, B ) ;DRM3x=DRM( x ,A, B ) ;28 MAP1x=MAP( x ,A, B ) ;MAP2x=MAP( x ,A, B ) ;MAP3x=MAP( x ,A, B ) ;19429 symMAP1x=symMAP( x ,A, B ) ; symMAP2x=symMAP( x ,A, B ) ; symMAP3x=symMAP( x ,A, B ) ;30 CC_Sa1x=CC_Sa ( x ,A, B ) ; CC_Sa2x=CC_Sa ( x ,A, B ) ; CC_Sa3x=CC_Sa ( x ,A, B ) ;31 CC_Sb1x=CC_Sb ( x ,A, B ) ; CC_Sb2x=CC_Sb ( x ,A, B ) ; CC_Sb3x=CC_Sb ( x ,A, B ) ;32 CC_Sc1x=CC_Sc ( x ,A, B ) ; CC_Sc2x=CC_Sc ( x ,A, B ) ; CC_Sc3x=CC_Sc ( x ,A, B ) ;33 CC_Sd1x=CC_Sd( x ,A, B ) ; CC_Sd2x=CC_Sd( x ,A, B ) ; CC_Sd3x=CC_Sd( x ,A, B ) ;34 ZDRMte=1;ZMAPte=1;ZsymMAPte=1; ZCC_Sate =1; ZCC_Sbte =1;ZCC_Scte =1; ZCC_Sdte =1;35 ZDRMgd=1;ZMAPgd=1;ZsymMAPgd=1;ZCC_Sagd=1;ZCC_Sbgd=1;ZCC_Scgd=1;ZCC_Sdgd=1;3637 % DRM38 f o r r =1:9939 DRM1x=DRM(DRM1x,A, B ) ;40 end41 DRMTE( ( k−1)∗10+ l ) =norm (DRM1x−Barx ) ;42 DRMGD( ( k−1)∗10+ l ) =norm ( P (DRM1x,A )−P (DRM1x, B ) ) ;43 while norm (DRM2x−Barx ) >0.01 && ZDRMte <=100044 DRM2x=DRM(DRM2x,A, B ) ;45 ZDRMte=ZDRMte+1;46 end47 ZDRMTE( ( k−1)∗10+ l ) =ZDRMte ;48 while norm ( P (DRM3x,A )−P ( DRM3x, B ) ) >0.01 && ZDRMgd <=100049 DRM3x=DRM(DRM3x,A, B ) ;50 ZDRMgd=ZDRMgd+1;51 end52 ZDRMGD( ( k−1)∗10+ l ) =ZDRMgd;5354 % MAP55 f o r r =1:9956 MAP1x=MAP(MAP1x,A, B ) ;57 end58 MAPTE( ( k−1)∗10+ l ) =norm (MAP1x−Barx ) ;59 MAPGD( ( k−1)∗10+ l ) =norm ( P (MAP1x,A )−P (MAP1x, B ) ) ;60 while norm (MAP2x−Barx ) >0.01 && ZMAPte <=100061 MAP2x=MAP(MAP2x,A, B ) ;62 ZMAPte=ZMAPte+1;63 end64 ZMAPTE( ( k−1)∗10+ l ) =ZMAPte ;65 while norm ( P (MAP3x,A )−P ( MAP3x, B ) ) >0.01 && ZMAPgd <=100066 MAP3x=MAP(MAP3x,A, B ) ;67 ZMAPgd=ZMAPgd+1;68 end69 ZMAPGD( ( k−1)∗10+ l ) =ZMAPgd;7019571 % SymMAP7273 f o r r =1:9974 symMAP1x=symMAP(symMAP1x,A, B ) ;75 end76 symMAPTE( ( k−1)∗10+ l ) =norm (symMAP1x−Barx ) ;77 symMAPGD( ( k−1)∗10+ l ) =norm ( P (symMAP1x,A )−P (symMAP1x, B ) ) ;78 while norm (symMAP2x−Barx ) >0.01 && ZsymMAPte <=100079 symMAP2x=symMAP(symMAP2x,A, B ) ;80 ZsymMAPte=ZsymMAPte+1;81 end82 ZsymMAPTE( ( k−1)∗10+ l ) =ZsymMAPte ;83 while norm ( P (symMAP3x,A )−P ( symMAP3x, B ) ) >0.01 &&ZsymMAPgd <=100084 symMAP3x=symMAP(symMAP3x,A, B ) ;85 ZsymMAPgd=ZsymMAPgd+1;86 end87 ZsymMAPGD( ( k−1)∗10+ l ) =ZsymMAPgd;8889 % CC_S_ { 1 }9091 f o r r =1:9992 CC_Sa1x=CC_Sa ( CC_Sa1x ,A, B ) ;93 end94 CC_SaTE ( ( k−1)∗10+ l ) =norm ( CC_Sa1x−Barx ) ;95 CC_SaGD ( ( k−1)∗10+ l ) =norm ( P ( CC_Sa1x ,A )−P ( CC_Sa1x , B ) ) ;96 while norm ( CC_Sa2x−Barx ) >0.01 && ZCC_Sate <=100097 CC_Sa2x=CC_Sa ( CC_Sa2x ,A, B ) ;98 ZCC_Sate=ZCC_Sate +1;99 end100 ZCC_SaTE ( ( k−1)∗10+ l ) =ZCC_Sate ;101 while norm ( P ( CC_Sa3x ,A )−P ( CC_Sa3x , B ) ) >0.01 && ZCC_Sagd<=1000102 CC_Sa3x=CC_Sa ( CC_Sa3x ,A, B ) ;103 ZCC_Sagd=ZCC_Sagd+1;104 end105 ZCC_SaGD ( ( k−1)∗10+ l ) =ZCC_Sagd ;106107108 % CC_S_ { 2 }109110 f o r r =1:99111 CC_Sb1x=CC_Sb ( CC_Sb1x ,A, B ) ;112 end113 CC_SbTE ( ( k−1)∗10+ l ) =norm ( CC_Sb1x−Barx ) ;114 CC_SbGD ( ( k−1)∗10+ l ) =norm ( P ( CC_Sb1x ,A )−P ( CC_Sb1x , B ) ) ;115 while norm ( CC_Sb2x−Barx ) >0.01 && ZCC_Sbte <=1000116 CC_Sb2x=CC_Sb ( CC_Sb2x ,A, B ) ;117 ZCC_Sbte=ZCC_Sbte +1;196118 end119 ZCC_SbTE ( ( k−1)∗10+ l ) =ZCC_Sbte ;120 while norm ( P ( CC_Sb3x ,A )−P ( CC_Sb3x , B ) ) >0.01 && ZCC_Sbgd<=1000121 CC_Sb3x=CC_Sb ( CC_Sb3x ,A, B ) ;122 ZCC_Sbgd=ZCC_Sbgd+1;123 end124 ZCC_SbGD ( ( k−1)∗10+ l ) =ZCC_Sbgd ;125126 % CC_S_ { 3 }127128 f o r r =1:99129 CC_Sc1x=CC_Sc ( CC_Sc1x ,A, B ) ;130 end131 CC_ScTE ( ( k−1)∗10+ l ) =norm ( CC_Sc1x−Barx ) ;132 CC_ScGD ( ( k−1)∗10+ l ) =norm ( P ( CC_Sc1x ,A )−P ( CC_Sc1x , B ) ) ;133 while norm ( CC_Sc2x−Barx ) >0.01 && ZCC_Scte <=1000134 CC_Sc2x=CC_Sc ( CC_Sc2x ,A, B ) ;135 ZCC_Scte=ZCC_Scte +1;136 end137 ZCC_ScTE ( ( k−1)∗10+ l ) =ZCC_Scte ;138 while norm ( P ( CC_Sc3x ,A )−P ( CC_Sc3x , B ) ) >0.01 && ZCC_Scgd<=1000139 CC_Sc3x=CC_Sc ( CC_Sc3x ,A, B ) ;140 ZCC_Scgd=ZCC_Scgd+1;141 end142 ZCC_ScGD ( ( k−1)∗10+ l ) =ZCC_Scgd ;143144 % CC_S_ { 4 }145146 f o r r =1:99147 CC_Sd1x=CC_Sd( CC_Sd1x ,A, B ) ;148 end149 CC_SdTE ( ( k−1)∗10+ l ) =norm ( CC_Sd1x−Barx ) ;150 CC_SdGD ( ( k−1)∗10+ l ) =norm ( P ( CC_Sd1x ,A )−P ( CC_Sd1x , B ) ) ;151 while norm ( CC_Sd2x−Barx ) >0.01 && ZCC_Sdte <=1000152 CC_Sd2x=CC_Sd( CC_Sd2x ,A, B ) ;153 ZCC_Sdte=ZCC_Sdte +1;154 end155 ZCC_SdTE ( ( k−1)∗10+ l ) =ZCC_Sdte ;156 while norm ( P ( CC_Sd3x ,A )−P ( CC_Sd3x , B ) ) >0.01 && ZCC_Sdgd<=1000157 CC_Sd3x=CC_Sd( CC_Sd3x ,A, B ) ;158 ZCC_Sdgd=ZCC_Sdgd+1;159 end160 ZCC_SdGD ( ( k−1)∗10+ l ) =ZCC_Sdgd ;161162 end163 end197164165 f i g u r e166167 subplot ( 2 , 2 , 1 )168 p l o t (DRMTE, ’ k+ ’ )169 hold on170 p l o t (MAPTE, ’ rd ’ )171 hold on172 p l o t (symMAPTE, ’m. ’ )173 hold on174 p l o t ( CC_SaTE , ’ go ’ )175 hold on176 p l o t ( CC_SbTE , ’ c∗ ’ )177 hold on178 p l o t ( CC_ScTE , ’ bs ’ )179 hold on180 p l o t (CC_SdTE , ’ y^ ’ )181 legend ( ’DRM’ , ’MAP’ , ’SymMAP’ , ’CC_{ S_ { 1 } } ’ , ’CC_{ S_ { 2 } } ’ , ’CC_{ S_ { 3 } } ’, ’CC_{ S_ { 4 } } ’ )182 y l a b e l ( ’|| s ^ { ( 1 0 0 ) } − P_ {U_ { 1 } \cap U_ { 2 } } x || ’ )183 x l a b e l ( ’ 100 i n s t a n c e s ( x_ { 0 } = P_ {U_ { 1 } } x ) ’ )184 t i t l e ( ’ D i f f e r e n c e s of i t e r a t i o n sequences and s o l u t i o n ’ )185 hold o f f186187 subplot ( 2 , 2 , 2 )188 p l o t (DRMGD, ’ k+ ’ )189 hold on190 p l o t (MAPGD, ’ rd ’ )191 hold on192 p l o t (symMAPGD, ’m. ’ )193 hold on194 p l o t (CC_SaGD, ’ go ’ )195 hold on196 p l o t (CC_SbGD, ’ c∗ ’ )197 hold on198 p l o t (CC_ScGD, ’ bs ’ )199 hold on200 p l o t (CC_SdGD, ’ y^ ’ )201 legend ( ’DRM’ , ’MAP’ , ’SymMAP’ , ’CC_{ S_ { 1 } } ’ , ’CC_{ S_ { 2 } } ’ , ’CC_{ S_ { 3 } } ’, ’CC_{ S_ { 4 } } ’ )202 y l a b e l ( ’|| P_ {U_ { 1 } } ( s ^ { ( 1 0 0 ) } )−P_ {U_ { 2 } } ( s ^ { ( 1 0 0 ) } ) || ’ )203 x l a b e l ( ’ 100 i n s t a n c e s ’ )204 t i t l e ( ’ D i f f e r e n c e s of sequences ’ ’ p r o j e c t i o n s onto U_ { 1 } and U_ { 2 }’ )205 hold o f f206207 subplot ( 2 , 2 , 3 )208 p l o t (ZDRMTE, ’ k+ ’ )209 hold on198210 p l o t (ZMAPTE, ’ rd ’ )211 hold on212 p l o t (ZsymMAPTE, ’m. ’ )213 hold on214 p l o t ( ZCC_SaTE , ’ go ’ )215 hold on216 p l o t (ZCC_SbTE , ’ c∗ ’ )217 hold on218 p l o t ( ZCC_ScTE , ’ bs ’ )219 hold on220 p l o t (ZCC_SdTE , ’ y^ ’ )221 legend ( ’DRM’ , ’MAP’ , ’SymMAP’ , ’CC_{ S_ { 1 } } ’ , ’CC_{ S_ { 2 } } ’ , ’CC_{ S_ { 3 } } ’, ’CC_{ S_ { 4 } } ’ )222 y l a b e l ( ’ Requied i t e r a t i o n numbers under e r r o r bound 0 . 0 1 ’ )223 x l a b e l ( ’ 100 i n s t a n c e s ’ )224 t i t l e ( ’ Stop C r i t e r i a : True Error (\ eps i lon = 0 . 0 1 ) ’ )225 hold o f f226227 subplot ( 2 , 2 , 4 )228 p l o t (ZDRMGD, ’ k+ ’ )229 hold on230 p l o t (ZMAPGD, ’ rd ’ )231 hold on232 p l o t (ZsymMAPGD, ’m. ’ )233 hold on234 p l o t (ZCC_SaGD, ’ go ’ )235 hold on236 p l o t (ZCC_SbGD, ’ c∗ ’ )237 hold on238 p l o t (ZCC_ScGD, ’ bs ’ )239 hold on240 p l o t (ZCC_SdGD, ’ y^ ’ )241 legend ( ’DRM’ , ’MAP’ , ’SymMAP’ , ’CC_{ S_ { 1 } } ’ , ’CC_{ S_ { 2 } } ’ , ’CC_{ S_ { 3 } } ’, ’CC_{ S_ { 4 } } ’ )242 y l a b e l ( ’ Requied i t e r a t i o n numbers under e r r o r bound 0 . 0 1 ’ )243 x l a b e l ( ’ 100 i n s t a n c e s ’ )244 t i t l e ( ’ Stop C r i t e r i a : Gap Dis tace (\ eps i lon = 0 . 0 1 ) ’ )245 hold o f f199Appendix C.2: Codes to generate performance matricesThe following codes generate the performance matrixes (tp,s)100×7 of thetwo plots in Figure 11.8. Once we have the performance matrix, it is easyto deduce the two performance profile plots in Figure 11.8 by the perf.mfile in [17]. The performance matrices (hp,i,s)100×50×7 to be used in [26] aregenerated similarly, to save the space, we don’t post them here.1 % Use Performance p r o f i l e to Compare the convergence r a t e of DRM,MAP, symMAP, CC_{ S_ { 1 } } , CC_{ S_ { 2 } } , CC_{ S_ { 3 } } , CC_{ S_ { 4 } }2 % Let ( s ^{k } ) be any of the seven sequences t h a t we monitor .34 % We use the number of i t e r a t i o n s to a t t a i n the t o l e r a n c e 0 .00015 % (\norm { s ^{k}−\bar { x } } <= 0 . 0 0 0 1 ) as the performance measure67 % We randomly generate 10 p a i r s of subspaces and 10 i n i t i a l points.89 % Record the number of i t e r a t i o n s of the s i x s o l v e r to solve the100 problems in the Matrix T to a t t a i n the true e r r o rc r i t e r i o n eps i lon =10^{−4}.1011 c l o s e a l l ; c l e a r ; c l c12 T=zeros ;13 f o r k =1:1014 A=randi ( 1 0 0 , 3 , 1 0 ) ;15 B=randi ( 1 0 0 , 5 , 1 0 ) ;16 f o r l =1:101718 x=randi ( 1 0 0 , 1 0 , 1 ) ;19 Barx=Psol ( x ,A, B ) ;20 DRMx=DRM( x ,A, B ) ;21 MAPx=MAP( x ,A, B ) ;22 symMAPx=symMAP( x ,A, B ) ;23 CC_Sax=CC_Sa ( x ,A, B ) ;24 CC_Sbx=CC_Sb ( x ,A, B ) ;25 CC_Scx=CC_Sc ( x ,A, B ) ;26 CC_Sdx=CC_Sd( x ,A, B ) ;27 ZDRMte=1;ZMAPte=1;ZsymMAPte=1;28 ZCC_Sate =1; ZCC_Sbte =1; ZCC_Scte =1; ZCC_Sdte =1;2930 % DRM recorded in the f i r s t column31 while norm (DRMx−Barx ) >0.0001 && ZDRMte <=100032 DRMx=DRM(DRMx,A, B ) ;33 ZDRMte=ZDRMte+1;34 end35 T ( ( k−1)∗10+ l , 1 ) =ZDRMte ;3620037 % MAP recorded in the second column38 while norm (MAPx−Barx ) >0.0001 && ZMAPte <=100039 MAPx=MAP(MAPx,A, B ) ;40 ZMAPte=ZMAPte+1;41 end42 T ( ( k−1)∗10+ l , 2 ) =ZMAPte ;43 % SymMAP recorded in the t h i r d column44 while norm (symMAPx−Barx ) >0.0001 && ZsymMAPte <=100045 symMAPx=symMAP(symMAPx,A, B ) ;46 ZsymMAPte=ZsymMAPte+1;47 end48 T ( ( k−1)∗10+ l , 3 ) =ZsymMAPte ;49 % CC_S_ { 1 } recorded in the fourth column50 while norm ( CC_Sax−Barx ) >0.0001 && ZCC_Sate <=100051 CC_Sax=CC_Sa ( CC_Sax ,A, B ) ;52 ZCC_Sate=ZCC_Sate +1;53 end54 T ( ( k−1)∗10+ l , 4 ) =ZCC_Sate ;55 % CC_S_ { 2 } recorded in the f i f t h column56 while norm ( CC_Sbx−Barx ) >0.0001 && ZCC_Sbte <=100057 CC_Sbx=CC_Sb ( CC_Sbx ,A, B ) ;58 ZCC_Sbte=ZCC_Sbte +1;59 end60 T ( ( k−1)∗10+ l , 5 ) =ZCC_Sbte ;61 % CC_S_ { 3 } recorded in the s i x t h column62 while norm ( CC_Scx−Barx ) >0.0001 && ZCC_Scte <=100063 CC_Scx=CC_Sc ( CC_Scx ,A, B ) ;64 ZCC_Scte=ZCC_Scte +1;65 end66 T ( ( k−1)∗10+ l , 6 ) =ZCC_Scte ;67 % CC_S_ { 4 } recorded in the seventh column68 while norm ( CC_Sdx−Barx ) >0.0001 && ZCC_Sdte <=100069 CC_Sdx=CC_Sd( CC_Sdx ,A, B ) ;70 ZCC_Sdte=ZCC_Sdte +1;71 end72 T ( ( k−1)∗10+ l , 7 ) =ZCC_Sdte ;7374 end75 end7677 % Log_ { 2 } sca led Performance p r o f i l e78 f i g u r e ( 1 )79 perf ( T , 1 ) ;8081 % Performance p r o f i l e82 f i g u r e ( 2 )83 perf ( T , 0 ) ;201

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.24.1-0371095/manifest

Comment

Related Items