{"Affiliation":[{"label":"Affiliation","value":"Graduate Studies, College of (Okanagan)","attrs":{"lang":"en","ns":"http:\/\/vivoweb.org\/ontology\/core#departmentOrSchool","classmap":"vivo:EducationalProcess","property":"vivo:departmentOrSchool"},"iri":"http:\/\/vivoweb.org\/ontology\/core#departmentOrSchool","explain":"VIVO-ISF Ontology V1.6 Property; The department or school name within institution; Not intended to be an institution name."}],"AggregatedSourceRepository":[{"label":"AggregatedSourceRepository","value":"DSpace","attrs":{"lang":"en","ns":"http:\/\/www.europeana.eu\/schemas\/edm\/dataProvider","classmap":"ore:Aggregation","property":"edm:dataProvider"},"iri":"http:\/\/www.europeana.eu\/schemas\/edm\/dataProvider","explain":"A Europeana Data Model Property; The name or identifier of the organization who contributes data indirectly to an aggregation service (e.g. Europeana)"}],"Campus":[{"label":"Campus","value":"UBCO","attrs":{"lang":"en","ns":"https:\/\/open.library.ubc.ca\/terms#degreeCampus","classmap":"oc:ThesisDescription","property":"oc:degreeCampus"},"iri":"https:\/\/open.library.ubc.ca\/terms#degreeCampus","explain":"UBC Open Collections Metadata Components; Local Field; Identifies the name of the campus from which the graduate completed their degree."}],"Creator":[{"label":"Creator","value":"Trienis, Michael Joseph","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/creator","classmap":"dpla:SourceResource","property":"dcterms:creator"},"iri":"http:\/\/purl.org\/dc\/terms\/creator","explain":"A Dublin Core Terms Property; An entity primarily responsible for making the resource.; Examples of a Contributor include a person, an organization, or a service."}],"DateAvailable":[{"label":"DateAvailable","value":"2008-11-20T18:44:39Z","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/issued","classmap":"edm:WebResource","property":"dcterms:issued"},"iri":"http:\/\/purl.org\/dc\/terms\/issued","explain":"A Dublin Core Terms Property; Date of formal issuance (e.g., publication) of the resource."}],"DateIssued":[{"label":"DateIssued","value":"2007","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/issued","classmap":"oc:SourceResource","property":"dcterms:issued"},"iri":"http:\/\/purl.org\/dc\/terms\/issued","explain":"A Dublin Core Terms Property; Date of formal issuance (e.g., publication) of the resource."}],"Degree":[{"label":"Degree","value":"Master of Science - MSc","attrs":{"lang":"en","ns":"http:\/\/vivoweb.org\/ontology\/core#relatedDegree","classmap":"vivo:ThesisDegree","property":"vivo:relatedDegree"},"iri":"http:\/\/vivoweb.org\/ontology\/core#relatedDegree","explain":"VIVO-ISF Ontology V1.6 Property; The thesis degree; Extended Property specified by UBC, as per https:\/\/wiki.duraspace.org\/display\/VIVO\/Ontology+Editor%27s+Guide"}],"DegreeGrantor":[{"label":"DegreeGrantor","value":"University of British Columbia","attrs":{"lang":"en","ns":"https:\/\/open.library.ubc.ca\/terms#degreeGrantor","classmap":"oc:ThesisDescription","property":"oc:degreeGrantor"},"iri":"https:\/\/open.library.ubc.ca\/terms#degreeGrantor","explain":"UBC Open Collections Metadata Components; Local Field; Indicates the institution where thesis was granted."}],"Description":[{"label":"Description","value":"After introducing concepts from convex analysis, we study how to continuously transform one convex\nfunction into another. A natural choice is the arithmetic average, as it is pointwise continuous;\nhowever, this choice fails to average functions with different domains. On the contrary, the proximal\naverage is not only continuous (in the epi-topology) but can actually average functions with\ndisjoint domains. In fact, the proximal average not only inherits strict convexity (like the arithmetic\naverage) but also inherits smoothness and differentiability (unlike the arithmetic average).\nThen we introduce a computational framework for computer-aided convex analysis. Motivated\nby the proximal average, we notice that the class of piecewise linear-quadratic (PLQ) functions is\nclosed under (positive) scalar multiplication, addition, Fenchel conjugation, and Moreau envelope.\nAs a result, the PLQ framework gives rise to linear-time and linear-space algorithms for convex\nPLQ functions. We extend this framework to nonconvex PLQ functions and present an explicit\nconvex hull algorithm.\nFinally, we discuss a method to find primal-dual symmetric antiderivatives from cyclically monotone\noperators. As these antiderivatives depend on the minimal and maximal Rockafellar functions\n[5, Theorem 3.5, Corollary 3.10], it turns out that the minimal and maximal function in [12,\np.132,p.136] are indeed the same functions. Algorithms used to compute these antiderivatives can\nbe formulated as shortest path problems.","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/description","classmap":"dpla:SourceResource","property":"dcterms:description"},"iri":"http:\/\/purl.org\/dc\/terms\/description","explain":"A Dublin Core Terms Property; An account of the resource.; Description may include but is not limited to: an abstract, a table of contents, a graphical representation, or a free-text account of the resource."}],"DigitalResourceOriginalRecord":[{"label":"DigitalResourceOriginalRecord","value":"https:\/\/circle.library.ubc.ca\/rest\/handle\/2429\/2799?expand=metadata","attrs":{"lang":"en","ns":"http:\/\/www.europeana.eu\/schemas\/edm\/aggregatedCHO","classmap":"ore:Aggregation","property":"edm:aggregatedCHO"},"iri":"http:\/\/www.europeana.eu\/schemas\/edm\/aggregatedCHO","explain":"A Europeana Data Model Property; The identifier of the source object, e.g. the Mona Lisa itself. This could be a full linked open date URI or an internal identifier"}],"Extent":[{"label":"Extent","value":"6563254 bytes","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/extent","classmap":"dpla:SourceResource","property":"dcterms:extent"},"iri":"http:\/\/purl.org\/dc\/terms\/extent","explain":"A Dublin Core Terms Property; The size or duration of the resource."}],"FileFormat":[{"label":"FileFormat","value":"application\/pdf","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/elements\/1.1\/format","classmap":"edm:WebResource","property":"dc:format"},"iri":"http:\/\/purl.org\/dc\/elements\/1.1\/format","explain":"A Dublin Core Elements Property; The file format, physical medium, or dimensions of the resource.; Examples of dimensions include size and duration. Recommended best practice is to use a controlled vocabulary such as the list of Internet Media Types [MIME]."}],"FullText":[{"label":"FullText","value":"Computational Convex Analysis: From Continuous Deformation to Finite Convex Integration by Michael Joseph Trienis B.Sc., The University of British Columbia Okanagan, 2006 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF Master of Science in College of Graduate Studies (Interdisciplinary) The University Of British Columbia Okanagan January 1st, 2007 c\u00a9 Michael Joseph Trienis 2007 Abstract After introducing concepts from convex analysis, we study how to continuously transform one con- vex function into another. A natural choice is the arithmetic average, as it is pointwise continuous; however, this choice fails to average functions with different domains. On the contrary, the prox- imal average is not only continuous (in the epi-topology) but can actually average functions with disjoint domains. In fact, the proximal average not only inherits strict convexity (like the arithmetic average) but also inherits smoothness and differentiability (unlike the arithmetic average). Then we introduce a computational framework for computer-aided convex analysis. Motivated by the proximal average, we notice that the class of piecewise linear-quadratic (PLQ) functions is closed under (positive) scalar multiplication, addition, Fenchel conjugation, and Moreau envelope. As a result, the PLQ framework gives rise to linear-time and linear-space algorithms for convex PLQ functions. We extend this framework to nonconvex PLQ functions and present an explicit convex hull algorithm. Finally, we discuss a method to find primal-dual symmetric antiderivatives from cyclically mono- tone operators. As these antiderivatives depend on the minimal and maximal Rockafellar functions [5, Theorem 3.5, Corollary 3.10], it turns out that the minimal and maximal function in [12, p.132,p.136] are indeed the same functions. Algorithms used to compute these antiderivatives can be formulated as shortest path problems. ii Table of Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix 1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Convex Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2 Convex Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 Fenchel Conjugate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.4 Moreau Envelope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.5 Convex Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.6 Subdifferential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.7 Continuity Notions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.8 Convex Hull . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2 Proximal Average . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 iii Table of Contents 2.1 Arithmetic Average . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.2 Continuity and Homotopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.3 Strict Convexity and Smoothness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3 Piecewise Linear-Quadratic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.1 Numerical Methods for Convex Transforms . . . . . . . . . . . . . . . . . . . . . . . 30 3.2 Function Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.2.1 The Class of Piecewise Linear Functions . . . . . . . . . . . . . . . . . . . . 32 3.2.2 The Class of Piecewise Linear-Quadratic Functions . . . . . . . . . . . . . . 35 3.3 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.3.1 Piecewise Linear-Quadratic Algorithms . . . . . . . . . . . . . . . . . . . . . 37 3.3.2 Extending the Piecewise Linear-Quadratic Algorithms . . . . . . . . . . . . . 45 3.3.3 Fast Algorithms for Convex Transforms . . . . . . . . . . . . . . . . . . . . . 55 3.4 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 3.4.1 Fast Algorithms for Convex Transforms . . . . . . . . . . . . . . . . . . . . . 58 3.4.2 Piecewise Linear-Quadratic Algorithms . . . . . . . . . . . . . . . . . . . . . 59 4 Finite Convex Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.2 Cyclic monotonicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.3 Antiderivatives and their properties . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 4.4 Relationship between [12] and [5] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.4.1 Minimal antiderivative in higher dimension . . . . . . . . . . . . . . . . . . . 69 4.4.2 Minimal antiderivative in one dimension . . . . . . . . . . . . . . . . . . . . 72 4.5 Linking Antiderivative Algorithms to Network Flow Problems . . . . . . . . . . . . 74 iv Table of Contents 5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 Appendices A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 v List of Tables 1.1 Extended-valued convention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.1 Special cases for the convex hull of a PLQ function . . . . . . . . . . . . . . . . . . . 48 vi List of Figures 1.1 Identifying a convex set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2 The epigraph of x 7\u2192 sin(x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Identifying a convex function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.4 The Moreau envelope of the absolute value function . . . . . . . . . . . . . . . . . . 12 1.5 A set subtangents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.6 First-order convexity characterization . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.7 A lower semi-continuous function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 1.8 The closed convex hull . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.1 Averages of the linear function f0(x) = x+ 2 and the quadratic function f1(x) = x 2. 25 2.2 The proximal averaging of two functions with different domains. . . . . . . . . . . . 26 2.3 Proximal averages of nonconvex functions . . . . . . . . . . . . . . . . . . . . . . . . 28 3.1 A zeroth-order model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.2 A first-order model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.3 A second-order model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 3.4 Multiplying a PLQ function by a scalar . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.5 The addition of two PLQ functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.6 The biconjugate of a nonconvex function . . . . . . . . . . . . . . . . . . . . . . . . . 46 vii List of Figures 3.7 Back-tracking with the PLQ convex hull algorithm . . . . . . . . . . . . . . . . . . . 49 3.8 The convex hull of a piecewise \u201dquadratic-quadratic\u201d function . . . . . . . . . . . . . 51 3.9 The convex hull of a piecewise \u201dquadratic-linear\u201d function . . . . . . . . . . . . . . . 52 3.10 The convex hull of a piecewise \u201dlinear-quadratic\u201d function . . . . . . . . . . . . . . . 53 3.11 The convex hull of a piecewise \u201dlinear-linear\u201d function . . . . . . . . . . . . . . . . . 54 3.12 The limitations of discrete addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.13 Convergence of the discrete Legendre transform . . . . . . . . . . . . . . . . . . . . . 60 4.1 Finite convex integration is not unique up to a constant . . . . . . . . . . . . . . . . 62 4.2 Constructing a primal-dual symmetric method . . . . . . . . . . . . . . . . . . . . . 68 4.3 Graph associated with a system of difference constraints . . . . . . . . . . . . . . . . 75 4.4 Minimal and maximal antiderivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 viii Acknowledgements I would like to thank my supervisor, Dr. Yves Lucet, for giving me the opportunity to pursue research in the area of optimization and convex analysis. His infinite patience and guidance were vital in every aspect of this work. As well, I sincerely thank Dr. Donovan Hare for being the first professor to spark my interest in optimization from a real world perspective, and to Dr. Heinz Bauschke, whose passion for mathematics inspires us all. I also give thanks to Liangjin Yao and Melisa Lavallee, who helped in revising my thesis. Everyone who attended the OCANA seminar provided invaluable knowledge and constructive criticism. This thesis was partially supported by the Pacific Institute for the Mathematical Sciences, the National Science and Engineering Research Council of Canada, and UBC Okanagan Internal Grant program. ix Introduction The main content of this thesis is derived from the papers [4], [5], and [19]. Contributions not contained in the previous papers include the extension to nonconvex PLQ functions, and the link with the all-pairs shortest path problem. We now summarize the contributions of each research paper. Firstly, article [4] states that the proximal average allows a parametric family of convex functions to continuously transform one convex function into another, even when the domains of the two functions do not intersect. The proximal average operator is also shown to be an homotopy with respect to the epi-topology. Moreover, the paper also shows that the parametric family inherits desirable properties such as differentiability and strict convexity. Next, article [19] presents a new computational framework for computer-aided convex analysis, and states that the class of piecewise linear-quadratic functions improves convergence and stability with respect to current models. A stable convex calculus is achieved by using symbolic-numeric algorithms to compute all fundamental convex transforms. The main result states the existence of efficient (linear-time) algorithms for the class of PLQ functions. These results are extended in the present thesis to nonconvex functions. Finally, article [5] discloses a new method which always produces primal-dual symmetric an- tiderivatives using Fitzpatrick functions and the proximal midpoint average. A link with the all- pairs shortest path algorithms is given at the end of this thesis. We first introduce fundamental convex notations in Chapter 1, then summarize the main results 1 Acknowledgements of the paper [4] in Chapter 2. In Chapter 3, we sum up and expand on several contributions in [19], which include explicitly defining the (linear-time) PLQ algorithms and extending the framework for those algorithms to nonconvex functions. Chapter 4 provides some notes on primal-dual symmetric methods, and links the minimal and maximal antiderivatives in [5, Theorem 3.5, Corollary 3.10] with [12, p.132, p.136]. The last chapter concludes the thesis with a brief summary and suggestions for future research. 2 Chapter 1 Preliminaries In this section we recall basic notions from convex analysis. Throughout, we assume the reader has a basic knowledge in set theory as well as fundamental results of calculus. We name X = X\u2217 = Rd, and R\u0304 = R \u222a {+\u221e} with inner product < \u00b7|\u00b7 >, and norm \u2016 \u00b7 \u2016. We extend any proper function f : dom f \u2192 R with f\u0304(x) = \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 f(x) if x \u2208 dom f, \u221e if x 6\u2208 dom f. We can recover the domain of the original function f from f\u0304 using the set dom f := {x : f\u0304(x) < +\u221e}. As our framework involves functions with infinite values, we need to acquire a convention for algebra that involves infinite values. Table 1.1 summarizes the convention which will be used throughout. For the sake of simplicity, we define positive and negative reals R\u2212\u2212 := ]\u2212\u221e, 0[ and R++ := ]0,+\u221e[. Definition 1.1. A function f : X \u2192 R\u0304 is proper if there is an x \u2208 X such that f(x) <\u221e. 3 Chapter 1. Preliminaries Table 1.1: Let \u03b1 \u2208 R++ and \u03b2 \u2208 R\u2212\u2212. Then we have the following extended-valued convention. Arithmetic Multiplication \u03b1+\u221e = +\u221e \u03b1(+\u221e) = +\u221e \u03b1\u2212\u221e = \u2212\u221e \u03b1(\u2212\u221e) = \u2212\u221e \u03b2 +\u221e = +\u221e \u03b2(+\u221e) = \u2212\u221e \u03b2 \u2212\u221e = \u2212\u221e \u03b2(\u2212\u221e) = +\u221e +\u221e+\u221e = +\u221e 0(+\u221e) = 0 0(\u2212\u221e) = 0 1.1 Convex Sets Let x1, x2 \u2208 X. Then the point x is on the line segment [x1, x2] if x is a convex combination of x1 and x2 i.e. [x1, x2] = {x : \u03bb1 + \u03bb2 = 1, \u03bb1, \u03bb2 \u2265 0 and x = \u03bb1x1 + \u03bb2x2}. See Figure 1.1. Definition 1.2. A set C is convex if for any x1, x2 \u2208 C \u2282 X, x1, x2 \u2208 C \u21d2 [x1, x2] \u2286 C holds. The epigraph is a notion which relates convex sets to convex functions. Definition 1.3. The epigraph of a function f is the set of all points on or above the graph of f . That is, epi f = {(x, r) : x \u2208 X, r \u2208 R, r \u2265 f(x)} . The above definition is illustrated on Figure 1.2. We can also describe a convex set C as the convex combination of all points in C. 4 Chapter 1. Preliminaries \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 Figure 1.1: If the points x1, x2 \u2208 C \u2282 R 2, and \u03bb \u2208 [0, 1] then \u03bbx1 + (1 \u2212 \u03bb)x2 \u2208 C. So C is a convex set. \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 Figure 1.2: The epigraph of x 7\u2192 sin(x) is the set of all points on and above the graph. 5 Chapter 1. Preliminaries \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 Figure 1.3: The segment [(x, f(x)), (y, f(y))] is always above the graph of f . Fact 1.4. [20, Theorem 2.2] A set C is convex if and only if \u2200n \u2208 N x1, \u00b7 \u00b7 \u00b7 , xn \u2208 C we have n\u2211 i=1 \u03bbixi \u2208 C for all \u03bbi \u2265 0 where n\u2211 i=1 \u03bbi = 1. 1.2 Convex Functions We now define convex functions. Definition 1.5. A function f : X \u2192 R\u0304 is convex if dom f is a convex set and f(\u03bbx+ (1\u2212 \u03bb)y) \u2264 \u03bbf(x) + (1\u2212 \u03bb)f(y) ( \u2200 x, y \u2208 dom f ) , where \u03bb \u2208 [0, 1]. Geometrically, the inequality can be described as the line segment connecting (x, f(x)) and (y, f(y)) being always on or above the graph of the function f , as seen on Figure 1.3. The inequality 6 Chapter 1. Preliminaries becomes an equality when considering affine functions; therefore, affine functions are also convex. The notion of strict convexity is almost identical to the previous definition except that the inequality is strict. Definition 1.6. The function f is strictly convex if domf is convex and f(\u03bbx+ (1\u2212 \u03bb)y) < \u03bbf(x) + (1\u2212 \u03bb)f(y) ( \u2200 x, y \u2208 dom f ) , whenever x 6= y, and \u03bb \u2208 ]0, 1[ . Remark 1.7. Strict convexity is a stronger notion than convexity as all strictly convex functions are convex but the converse is false. A strictly convex function is a convex function without any linear parts. It is an important notion as strictly convex functions have at most one minimizer (see Figure 1.3). Notation 1.8. The set of all convex functions on X is denoted by Conv X. Notation 1.9. Any vector x \u2208 X = Rd is identified as a column vector; the transpose, of x \u2208 Rd\u00d71, is a new vector in R1\u00d7d denoted by xT . Finally, we denote \u3008\u00b7, \u00b7\u3009 as the standard dot product \u3008x, y\u3009 = yTx. Definition 1.10. The n \u00d7 n matrix M is positive-semidefinite if xTMx \u2265 0 for all x 6= 0. Fur- thermore, a matrix M is positive-definite if xTMx > 0 for all x 6= 0. Example 1.11. Quadratic functions f(x) = \u3008Ax, x\u3009 + \u3008b, x\u3009 + c with A positive-semidefinite, are convex. If A is positive definite, then f is strictly convex. Example 1.12. The exponential function f(x) = exp(x) for x \u2208 R is strictly convex with no minimizer. Fact 1.13. [20, Theorem 4.1] The function f is convex, if and only if epi f is a convex set. 7 Chapter 1. Preliminaries Fact 1.14. [7, p.639] A function f : X \u2192 R is said to be closed if and only if epi f is closed. 1.3 Fenchel Conjugate The convex conjugate, also known as the Fenchel conjugate (we will refer to it simply as the conjugate) is an important operation in convex analysis. It is used as an intermediate transform for other more advanced operations like the Moreau envelope and the proximal average. Definition 1.15. Let f : X \u2192 R\u0304. The function f\u2217 is defined as f\u2217(y) = sup x\u2208dom f {\u3008y, x\u3009 \u2212 f(x)} . We notice that f\u2217 is a convex function as the supremum of affine (convex) functions is a convex function. Definition 1.16. A set is closed if every limit point of the set is a point in the set. Proposition 1.17. [7, p.91] The function f\u2217 is always convex and lsc, (see Definition 1.40) even if f is not convex. Proof. The conjugate is the pointwise supremum of a family of affine functions of y, therefore the epigraph of the conjugate is an intersection of a family of closed half spaces (closed convex sets). Since the intersection of closed convex sets is a closed convex set, the epigraph is a closed convex set. So the conjugate is lsc and convex. 8 Chapter 1. Preliminaries Example 1.18. The conjugate of a quadratic function f(x) = a0x 2 + b0x+ c0, the conjugate is f\u2217(y) = \uf8f1\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f3 1 4a0 (y \u2212 b0) 2 \u2212 c0 if a0 > 0, I{b0}(y)\u2212 c0 if a0 = 0, \u221e if a0 < 0. The conjugate function has nice duality properties with respect to convexity. Fact 1.19. [20, Corollary 12.2] The biconjugate f\u2217\u2217 = f , if and only if f is proper, convex and lsc. Example 1.20. Let f(x) = b0x+ c0. Then f \u2217(y) = I{b0}(y) \u2212 c0. Taking the conjugate again we get f\u2217\u2217(x) = b0x+ c0 = f(x). Proposition 1.21. [20, p.105] Given a function f : X \u2192 R\u0304 and x \u2208 dom f , Fenchel\u2019s inequality holds: \u3008p, x\u3009 \u2264 f(x) + f\u2217(p) ( \u2200 p, x \u2208 X ) . Proof. Directly from the definition of the Fenchel conjugate f\u2217(p) := supx{\u3008p, x\u3009 \u2212 f(x)}, gives f\u2217(p) \u2265 \u3008p, x\u3009 \u2212 f(x). The energy function is the only function whose conjugate is itself. Fact 1.22. [20, p.106] The only self conjugate function is the energy function 12\u2016 \u00b7 \u2016 2. Notation 1.23. The set of all closed convex functions on X is denoted by Conv X. As the conjugate function is always convex, taking the double conjugate yields the closed convex hull. Fact 1.24. [20, p.36] Let f be a proper function. Then c\u0304o f = f\u2217\u2217. 9 Chapter 1. Preliminaries Proposition 1.25. [11, Corollary 1.4.4] Let f \u2208 Conv X and x \u2208 dom f . Then \u3008s, x\u3009 = f(x) + f\u2217(s)\u21d4 s \u2208 \u2202f(x)\u21d4 x \u2208 \u2202f\u2217(s). (1.1) Proof. By definition of the conjugate we have \u2212 f\u2217(s) = inf x\u2208X [f(x)\u2212 \u3008s, x\u3009]. (1.2) So the lsc proper convex function g: x 7\u2192 f(x) \u2212 \u3008s, x\u3009, achieves its infimum at x\u0302 if and only if 0 \u2208 \u2202g(x\u0302) = \u2202f(x\u0302)\u2212 s. That is, \u2212f\u2217(s) = [f(x\u0302)\u2212 \u3008s, x\u3009] if and only if s \u2208 \u2202f(x\u0302). Applying this same result to f\u2217, we obtain x \u2208 \u2202f\u2217(s) if and only if \u3008s, x\u3009 = f\u2217(s) + f\u2217\u2217(x), which is again Equation (1.1) since f\u2217\u2217 = f. 1.4 Moreau Envelope Let us first define an operation which is used in the Moreau envelope. Definition 1.26. Let f1 and f2 be two functions from X to R\u0304. Their infimal convolutions is defined 10 Chapter 1. Preliminaries by f12f2(x) := inf{f1(x) + f2(x2) : x1 + x2 = x}, = inf y\u2208X [f1(y) + f2(x\u2212 y)]. Definition 1.27. [20, Theorem 31.5.] Let \u03bb \u2208 R++ and s \u2208 X. Then the Moreau envelope, also called the Moreau-Yosida regularization is defined as M\u03bb(s) = f2 1 2\u03bb \u2016 \u00b7 \u20162(s) = inf x\u2208X f(x) + \u2016s\u2212 x\u20162 2\u03bb . (1.3) We summarize some of its key properties. Fact 1.28. (i.) [21, Theorem 1.25] M\u03bb converges pointwise to f as \u03bb decreases to 0. (ii.) [21, Theorem 13.37] The Moreau envelope is smooth and continuous. (iii.) [18, p.2] The functions M\u03bb(x) and f(x) share the same critical points. It is also important due to its regularization properties, in particular, with nondifferentiable functions as seen in Figure 1.4. Using Equation (1.3) and expanding \u2016 \u00b7 \u20162 we obtain M\u03bb(s) = \u2016s\u20162 2\u03bb \u2212 1 \u03bb g\u2217\u03bb(s), (1.4) where g\u2217\u03bb(s) = sup {\u3008s, x\u3009 \u2212 g\u03bb(x)} and g\u03bb(x) = \u2016x\u20162 2 + \u03bbf(x) (see [17]). Remark 1.29. Formula (1.4) is important, as any algorithm used to compute the conjugate can also be used to compute the Moreau envelope. As the Moreau envelope is decomposed into conjugation, addition, and (positive) scalar multiplication, all algorithms and models need only to accommodate 11 Chapter 1. Preliminaries \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 0 0.25 0.5 0.75 1 Figure 1.4: The Moreau envelope of f(x) = |x| is a smooth PLQ function. these operations. In other words, the Moreau envelope (regularization) depends entirely on the three operations in Equation (1.4). 1.5 Convex Optimization Duality theory in optimization associates another optimization problem (the dual) to a given prob- lem (the primal). The primal problem is the initial model which typically consists of an objective function and a series of constraints. The conjugate plays a critical role in convex duality because it gives a dual representation (D) of the primal (P). Let h and k be proper, lsc and convex on Rn 12 Chapter 1. Preliminaries and Rm respectively. Then (P) inf {\u03c6(x) : \u03c6(x) := \u3008c, x\u3009+ k(x) + h(b\u2212Ax), and x \u2208 Rn}, (1.5) (D) sup {\u03c8(y) : \u03c8(y) := \u3008b, y\u3009 \u2212 h\u2217(y)\u2212 k\u2217(A\u2217y \u2212 c), and y \u2208 Rm}. (1.6) The dual problem clearly shows the importance of the conjugate function as Equation (1.6) relies directly on the conjugate of h and k. Remark 1.30. A version of Fenchel Duality Theorem involving the primal and dual problems, can be found in [20, Theorem 31.1]. The notion of duality has had a huge impact as some dual problems are much easier to solve than their primal counterparts. The primal and the dual are intrinsically linked to each other such that the solution to the dual provides insight into the solution of the primal. Under some conditions, the optimal value of both problems is the same. The indicator function of a convex set C \u2282 X is the bridge between functions and sets. It is defined as x 7\u2192 IC(x) = \uf8f1\uf8f4\uf8f2 \uf8f4\uf8f3 0 if x \u2208 C, +\u221e otherwise. The indicator function will allow us to translate results on convex sets to convex functions. Example 1.31. In optimization we are concerned with the following problem minimize f over C. This problem can be equivalently rewritten as the unconstrained optimization problem minimize f + IC over X, 13 Chapter 1. Preliminaries \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 \u22122 \u22121 0 1 2 3 4 Figure 1.5: A set of subtangents for smooth and non-smooth parts. as indicator functions act as infinite penalization. Definition 1.32. We say that a function is smooth if it is differentiable everywhere. In optimization we use critical points to locate minimum or maximum values. Definition 1.33. Let f be a differentiable function then a critical point is a point in the interior of the domain of the function where the derivative equals zero. As the gradient is undefined for nonsmooth functions, we require some notion of generalized differentiability; this is known as the subdifferential operator. 14 Chapter 1. Preliminaries 1.6 Subdifferential In order to properly describe the construction of the subdifferential operator, we need to define some other notions. Given a proper convex function f , we can construct a hyperplane f(x0)+ \u3008s, x\u2212 x0\u3009 which passes through (x0, f(x0)) that always lies on or below the function f (seen in Figure 1.5). The slope of this supporting hyperplane is called a subgradient. Definition 1.34. The vector s is a subgradient of f at x0 \u2208 dom f if and only if f(x) \u2265 f(x0) + \u3008s, x\u2212 x0\u3009 ( \u2200 x \u2208 X ) . In the case where the function f has a corner at x0, there may exist infinitely many supporting hyperplanes, and thus many subgradients. Figure 1.5 illustrates multiple subgradients at the corner (0, 0) and only a single subgradient otherwise. The subdifferential is simply the set of all possible subgradients at all points. Definition 1.35. The subdifferential of a proper function f at x is \u2202f(x) = {s \u2208 X : f(y) \u2265 f(x) + \u3008s, y \u2212 x\u3009, \u2200 y \u2208 X} . It is constructed from hyperplanes which support epi f at (x, f(x)); therefore the subdifferential may be empty for nonconvex functions. Recall that the subdifferential is a generalization of the derivative and in the case when f is convex and smooth, {\u2207f(x)} = \u2202f(x). Definition 1.36. We say that x\u0304 is a critical point of f if 0 \u2208 \u2202f(x\u0304). Remark 1.37. When considering the class of conjugate functions which have closed form solutions, one strategy is to determine if \u3008y, x\u3009 \u2212 f(x) is concave (in term of x). If this is the case then the critical point will maximize \u3008y, x\u3009 \u2212 f(x). This is the approach taken by the symbolic convex 15 Chapter 1. Preliminaries \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 \u22121 0 1 2 3 4 5 Figure 1.6: First-order convexity characterization. analysis toolkit (SCAT) package [9, p.72]. However, when no closed form exists we require numerical methods to find a solution. Convexity is characterized by the following first order condition. Observation 1.38. Assume f is differentiable at each point in dom f , and that dom f is open. Then the function f is convex, if and only if, dom f is convex and f(y) \u2265 f(x) + \u3008\u2207f(x), y \u2212 x\u3009 ( \u2200 x, y \u2208 dom f ) . We can see from Figure 1.6 that the open circle is the point where the subtangent f(x) + f \u2032(x)(y \u2212 x) passes through (x, f(x)), and the constraint f(y) \u2265 f(x) + f \u2032(x)(y \u2212 x) means that the subtangent must lie on or below the function f(y). The following is a second-order characterization of convexity. 16 Chapter 1. Preliminaries Fact 1.39. [20, Theorem 4.5] Assume f is proper, twice continuously differentiable at each point in dom f , where the dom f is open and convex. Then the function f is convex on dom f , if and only if, for all x \u2208 dom f,\u22072f(x) is positive semi-definite. If \u22072f(x) is positive-definite for all x \u2208 dom f , then f is strictly convex. 1.7 Continuity Notions Definition 1.40. A function f is lower semicontinuous (lsc) at x0 \u2208 dom f if for every \u03b5 > 0, there exists a neighbourhood U of x0 such that f(x) > f(x0)\u2212 \u03b5, for all x in U . This definition can be equivalently expressed as lim inf x\u2192x0 f(x) \u2265 f(x0). Lower semicontinuity is a weaker notion than continuity in the sense that continuity implies lower semicontinuity. In other words, if a function f is continuous at a point x0 then it is also lsc at x0 but the converse is not true, as seen in Figure 1.7. If a function f is lsc at every point in its domain then f is a lsc function. The notion of upper semicontinuous (usc) is analogous to that of lsc. Definition 1.41. The function f is usc at x0 \u2208 dom f if for every \u03b5 > 0, there exists a neighborhood U of x0 such that f(x) < f(x0) + \u03b5 for all x in U . This definition can be equivalently expressed as lim sup x\u2192x0 f(x) \u2264 f(x0). 17 Chapter 1. Preliminaries \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 \u22122.5 \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 Figure 1.7: The function is lsc at x0 = 0, and continuous otherwise therefore it is lsc everywhere. A function f is lsc if and only if \u2212f is usc. Moreover, the notion of being continuous at x0 is equivalent to being both lsc and usc at x0. Fact 1.42. [20, Theorem 7.1] A function f is lsc if and only if the epigraph of f is closed. Definition 1.43 (continuous function). Assume that xn, x0 belong to dom f . Then a real function f is continuous if for any sequence (xn) such that lim n\u2192\u221e xn = x0, it holds that lim n\u2192\u221e f(xn) = f(x0). Equivalently, we can express the above definition as the following. 18 Chapter 1. Preliminaries Notation 1.44. For all \u01eb > 0, there exists an \u03c3 > 0 such that |x\u2212 x0| < \u03c3 \u21d2 |f(x)\u2212 f(x0)| < \u01eb. Fact 1.45. [20, p.51] A function f is lsc and usc at x0 if and only if f is continuous at x0. 1.8 Convex Hull Given a nonconvex function g, the convex hull (denoted by co) is found by taking the supremum of all convex functions which minorize g. Definition 1.46. If two functions f and g from X to R satisfy f(x) \u2264 g(x) for all x \u2208 X, we say that f minorizes g (on X). Fact 1.47. [11, Proposition 2.5.1, Proposition 2.5.2] Let g : X \u2192 R\u0304, not identically +\u221e, be minorized by an affine function: for some (s, b) \u2208 X \u00d7 R g(x) \u2265 \u3008s, x\u3009 \u2212 b ( \u2200 x \u2208 X ) . Then co g(x) := sup{h(x) : h \u2208 Conv X,h \u2264 g}. (1.7) Similarly, the closed convex hull of g is co g(x) := sup{h(x) : h \u2208 ConvX, h \u2264 g}. (1.8) The above definition of a closed convex hull is illustrated on Figure 1.8. The concept of convex functions is intrinsically linked to convex sets using the epigraph; there- fore, we also define the convex hull for sets. If we are given any set S, not necessarily convex, we can construct a convex set by taking the intersection of all convex sets that contain S. 19 Chapter 1. Preliminaries Fact 1.48. [11, Proposition 1.3.4, Definition 1.4.1] The convex hull is the intersection of all convex sets which are supersets of S: coS := \u2229{C : C is convex and contains S }. (1.9) The closed convex hull of a nonempty set S \u2282 X is the intersection of all closed convex sets containing S. It is also the intersection of all closed half-spaces containing S. We can also find the convex hull of a set of discrete points by finding the set of all convex combinations. Definition 1.49. A convex combination of x1, \u00b7 \u00b7 \u00b7 , xk is the point x = \u03bb1x1 + \u03bb2x2 + \u00b7 \u00b7 \u00b7 + \u03bbkxk such that \u2211k i=1 \u03bbi = 1, and \u03bbi \u2265 0 for i = 1, \u00b7 \u00b7 \u00b7 , k. Fact 1.50. [7, p.34] The convex hull of a finite set of points {x1, \u00b7 \u00b7 \u00b7 , xk} is co{x1, \u00b7 \u00b7 \u00b7 , xk} = {\u03bb1x1 + \u00b7 \u00b7 \u00b7 + \u03bbkxk : \u03bb1 + \u00b7 \u00b7 \u00b7+ \u03bbk = 1, \u03bbi \u2265 0 for i = 1, \u00b7 \u00b7 \u00b7 , k}. 20 Chapter 1. Preliminaries \u22126 \u22124 \u22122 0 2 4 6 \u22123 \u22122 \u22121 0 1 2 3 4 Figure 1.8: The closed convex hull. 21 Chapter 2 Proximal Average Throughout this chapter we will assume f0, and f1 belong to Conv X. 2.1 Arithmetic Average The main motivation for the proximal average is to determine how to continuously transform f0 into f1. The first and most natural way is the arithmetic average function x 7\u2192 (1\u2212 \u03bb)f0(x) + \u03bbf1(x) ( \u2200\u03bb \u2208 ]0, 1[ ) . The arithmetic average is the convex combination of f0 and f1 taken pointwise. In fact, when f0 and f1 are finite-valued the arithmetic average is pointwise continuous. Remark 2.1. We say that a function is finite if it does not take the values +\u221e and \u2212\u221e. Modern convex analysis uses extended-valued functions for many valid reasons. Indeed, many constrained optimization problems are often reformulated as unconstrained optimization problems by introducing indicator functions in the objective (see Example 1.31). As many functions (includ- ing indicator functions) consider infinite values, we encounter our first shortcoming: when we take the arithmetic average of two functions which do not have identical domains, the domain of the average may be empty. 22 Chapter 2. Proximal Average Example 2.2. Let f0(x) := x ln(x)\u2212x and f1(x) := exp(x). Then the arithmetic average function is a\u03bb(x) := (1\u2212 \u03bb)f0(x) + \u03bbf1(x) ( \u2200\u03bb \u2208 ]0, 1[ ) . As illustrated by Figure 4.2(a), x 7\u2192 a\u03bb(x) is only continuous on the interval dom f0 \u2229 dom f1 = [0,+\u221e[. When dom f0 \u2229 dom f1 = \u2205 the arithmetic average is not even defined anywhere and will be exactly +\u221e everywhere while the proximal average always has full domain. In contrast to the arithmetic average, let us consider the proximal average. The proximal average originates from the convex combination of two proximal maps, (which turns out to be a proximal map) Prox(f\u03bb) = (1\u2212 \u03bb) Prox(f0) + \u03bbProx(f1), where the proximal mapping of f \u2208 ConvX defined by Prox(f)(x) = Argmin f(y) + \u2016x\u2212 y\u20162 2\u03bb , (2.1) is the set of minimizers of the Moreau envelope (see Equation (1.3)). Definition 2.3. The proximal average operator P is defined as P : Conv\u00d7[0, 1] \u00d7 Conv\u2192 Conv (f0, \u03bb, f1) 7\u2192 ( (1\u2212 \u03bb)(f0 + 1 2 \u2016 \u00b7 \u20162)\u2217 + \u03bb(f1 + 1 2 \u2016 \u00b7 \u20162)\u2217 )\u2217 \u2212 1 2 \u2016 \u00b7 \u20162. The arithmetic average is convex as it is composed of operations which are convexity preserving. But it is unclear from the definition, whether the proximal average is convex, as the difference of convex functions is not always convex. 23 Chapter 2. Proximal Average Fact 2.4. [6] Set f\u03bb = P(f0, \u03bb, f1). Then ( P(f0, \u03bb, f1) )\u2217 = P(f\u22170 , \u03bb, f \u2217 1 ). Using the Biconjugate Theorem with Fact 2.4 and conjugating the proximal average twice, gives f\u2217\u2217\u03bb = ( P(f0, \u03bb, f1) )\u2217\u2217 = ( P(f\u22170 , \u03bb, f \u2217 1 ) )\u2217 = P(f\u2217\u22170 , \u03bb, f \u2217\u2217 1 ) = f\u03bb. Thus, f\u03bb \u2208 Conv X. Fact 2.5. [4, Proposition 2.2] The following properties always hold for the proximal average oper- ator: (i.) P(f0, \u03bb, f1) = P(f1, 1\u2212 \u03bb, f0), (ii.) P(f0, 0, f1) = f0, (iii.) P(f0, 1, f1) = f1. Proposition 2.6. [4, Proposition 2.8] Let f \u2208 Conv X, then P(f, 12 , f \u2217) = 12\u2016 \u00b7 \u2016 2. Proof. In fact, ( P(f, 12 , f \u2217) )\u2217 = P(f\u2217, 12 , f \u2217\u2217) = P(f\u2217, 1\u2212 12 , f) = P(f, 1 2 , f \u2217). As the conjugate of P(f, 12 , f \u2217) is itself, by Fact 1.22 the proximal midpoint average P(f, 12 , 1 2 , f \u2217) is 12\u2016 \u00b7 \u2016 2. 2.2 Continuity and Homotopy In contrast to the arithmetic average the proximal average is not pointwise continuous. 24 Chapter 2. Proximal Average Example 2.7. Let f0(x) := I{b0}(x), and f1 := I{b1}(x). Then f\u03bb(x) = I{(1\u2212\u03bb)b0+\u03bbb1}(x) + (1\u2212 \u03bb)\u03bb 2 \u2016b0 \u2212 b1\u2016 2. Let \u03bb0 > 0 and set x0 = (1\u2212 \u03bb0)b0 + \u03bb0b1. Then for \u03bb 6= \u03bb0 we have f\u03bb(x0) = +\u221e, but for \u03bb = \u03bb0 we obtain f\u03bb(x0) = (1\u2212\u03bb)\u03bb 2 \u2016b0 \u2212 b1\u2016 2. So the proximal average operator \u03bb 7\u2192 P(f0, \u03bb, f1)(x) is not pointwise continuous. Figure 2.2(b) illustrates that there may exist some type of continuity with the proximal average. \u22123 \u22122 \u22121 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 0.25 0.5 0.75 1 (a) From f0 to f1: Arithmetic Average \u22123 \u22122 \u22121 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 0.25 0.5 0.75 1 (b) From f0 to f1: Proximal Average Figure 2.1: Averages of the linear function f0(x) = x+ 2 and the quadratic function f1(x) = x 2. Next, we quote several results from [4] that detail the setting in which the proximal average is continuous. Definition 2.8 (epi-convergence and epi-topology). Let g and (gn)n\u2208N be functions from X to ]\u2212\u221e,+\u221e]. Then (gn)n\u2208N epi-converges to g, if the following properties hold for every x \u2208 X: for every sequence (xn)n\u2208N in X converging to x, one has g(x) \u2264 lim inf n\u2192+\u221e gn(xn), there exists a sequence (xn)n\u2208N in X converging to x such that lim sup n\u2192+\u221e gn(xn) \u2264 g(x). 25 Chapter 2. Proximal Average The epi-topology is the topology induced by epi-convergence. Fact 2.9. [4, Theorem 4.2] If Conv X is equipped with the epi-topology, then the proximal average operator P : Conv X \u00d7 [0, 1] \u00d7Conv X \u2192 Conv X is continuous. \u22123 \u22122 \u22121 0 1 2 3 4 0 5 10 15 20 0 0.25 0.5 0.75 1 (a) Deforming f0(x) = x ln(x) \u2212 x into f1(x) = exp(x). \u22125 \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 5 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 0 0.25 0.5 0.75 1 (b) Deforming f0(x) = \u2212 ln(\u2212x) into f1(x) = \u2212 ln(x). Figure 2.2: The proximal averaging of two functions with different domains. Fact 2.10. [4, Corollary 3.3] Let f0 \u2208 Conv X and f1 \u2208 Conv X. Then P(f0, \u00b7, f1) : [0, 1]\u2192 Conv X provides a homotopy between f0 and f1. In fact, all functions in Conv X are homotopic for the epi-topology. Next, we define the domain of the proximal average operator. Fact 2.11. [6] Let C0 and C1 be nonempty closed convex subsets of X, and \u03bb \u2208 ]0, 1[. Set f0 := IC0 , 26 Chapter 2. Proximal Average f1 := IC1 , f\u03bb := P(f0, \u03bb, f1). Then for every x \u2208 X, f\u03bb(x) = (1\u2212 \u03bb)\u03bb inf { 1 2\u2016c0 \u2212 c1\u2016 2 : (1\u2212 \u03bb)c0 + \u03bbc1 = x, ci \u2208 Ci } . Consequently, dom f\u03bb = (1\u2212 \u03bb)C0 + \u03bbC1. Definition 2.12. The set C \u2286 X is the set of all x such that there exists a sequence xn \u2208 C such that xn \u2192 x. Fact 2.13. [4, Theorem 2.13] For any f0 \u2208 Conv X and f1 \u2208 Conv X, dom f\u03bb = (1\u2212 \u03bb) dom f0 + \u03bbdom f1, int dom f\u03bb = int ( (1\u2212 \u03bb) dom f0 + \u03bbdom f1 ) . The result was later refined in [3, Theorem 4.6] as the domain is the convex combination of the domains. As a consequence of Fact 2.13 the proximal average remedies the shortcomings of Example 2.2. This is illustrated on Figure 2.2. 2.3 Strict Convexity and Smoothness The proximal average inherits nice properties from the two functions it averages. Notation 2.14. We say a function has full domain if dom f = X = Rd is the whole space. Fact 2.15. [4, Theorem 3.2] Let f0 \u2208 Conv X, f1 \u2208 Conv X, \u03bb \u2208 ]0, 1[, f\u03bb := P(f0, \u03bb, f1). Suppose that f0 or f1 has full domain, and that f \u2217 0 or f \u2217 1 has full domain. Then the following hold: (i.) Both f\u03bb and f \u2217 \u03bb have full domain. (ii.) If f0 or f1 is smooth (i.e. differentiable everywhere), then so is f\u03bb. 27 Chapter 2. Proximal Average (iii.) If f0 or f1 is strictly convex and its Fenchel conjugate has full domain, then f\u03bb is strictly convex. The arithmetic average preserves strict convexity if one function is strictly convex; however, if one function is not smooth then the arithmetic average will not be smooth. Remark 2.16. It is possible to find the proximal average of nonconvex functions; however, in order to avoid the degenerate cases of +\u221e or \u2212\u221e, each nonconvex function must be bounded below by at least one affine function. As a consequence of Section 3.3.2 we can now compute the proximal average of nonconvex functions. Figure 2.16 illustrates the proximal average between convex and nonconvex functions. \u22126 \u22124 \u22122 0 2 4 6 \u22122 0 2 4 6 8 0 0.25 0.5 0.75 1 (a) \u22125 \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 5 \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 0 0.25 0.5 0.75 1 (b) \u22126 \u22124 \u22122 0 2 4 6 \u22123 \u22122 \u22121 0 1 2 3 0 0.25 0.5 0.75 1 (c) Figure 2.3: Proximal averages of nonconvex functions 28 Chapter 3 PLQ Model Definition 3.1. Let PLQ be the set of extended valued functions which are convex, lsc and proper with a piecewise linear subdifferential mapping \u2202f : R \u2192 2R. The set PLQ contains piecewise linear functions, piecewise quadratic functions and the sum of any such function with an indicator function. There are several standard operations which arise frequently in convex analysis. Definition 3.2. The standard convex operations are (positive) scalar multiplication, addition, con- jugation, and regularization (Moreau envelope). Throughout we consider only these operations as benchmarks for algorithms and model ac- curacy. These operations provide a natural set of tools for convex problems, as each operation preserve convexity. Proposition 3.3. The following operations are convexity preserving: (positive) scalar multiplica- tion, addition, conjugation, and regularization. Proof. (Positive) scalar multiplication and addition clearly preserve convexity. The conjugate is convexity preserving as it is always convex by Proposition 1.17. The Moreau envelope of a convex function f is convex by Formula (1.3). 29 Chapter 3. Piecewise Linear-Quadratic Model 3.1 Numerical Methods for Convex Transforms In convex analysis many transforms are difficult to compute or do not have a closed form. Example 3.4. [15, p.46] Let W (the Lambert function) be the inverse function of x 7\u2192 xex defined on the interval ]0,+\u221e[ . Then the conjugate of f(x) = x log(x) + x2 2 \u2212 x is f\u2217(y) = 1 2 [W(ey)]2 +W(ey). As the conjugate of f cannot be written with standard functions (i.e. a combination of trigono- metric or polynomial functions) , we use W to represent such functions (it is well known that W does not admit a closed form). Notation 3.5. Let I := {0, \u00b7 \u00b7 \u00b7 , n}, be a finite set of ordered integers. Fast algorithms which efficiently approximate these transforms, depend on discretization of a continuous model. Discretization is the process of converting a continuous model into discrete counterparts. Example 3.6. The continuous model for the Fenchel conjugate is f\u2217(y) = sup x\u2208R [yx\u2212 f(x)], (3.1) and the discretization of this model is f\u2217n(yj) = max i\u2208I {yjxi \u2212 f(xi)}, j \u2208 I. (3.2) 30 Chapter 3. Piecewise Linear-Quadratic Model The main idea in approximating the continuous model is to evaluate the discrete model over a grid and not on a single point. Many fast algorithms utilize the fact that the domain to approximate the operation and the domain on which the function is defined, is known prior to computation. This prior knowledge allows for efficient computation and can lead to a decrease in time complexity. Remark 3.7. Computing grids for fast algorithms requires prior knowledge of not only the domain of f but also the domain of the resulting function. For example, the function f(x) = \u2212 log(x) is defined only on the positive reals. The framework is the same for all fast algorithms. First, we discretize our model and compute the transform using an efficient algorithm. Convergence results from Section 3.4 allow us to recover the continuous model. 3.2 Function Approximation Given a discrete set of points {(xi, f(xi))}i\u2208I sampled from a function f (typically the function is not known) such that x0 < x1 < x2 < \u00b7 \u00b7 \u00b7 < xn, what is the closest approximating function f\u0303 (from a well defined class of functions) that passes through our discrete set of points? The process used to solve this problem is known as interpolation. When considering the class of piecewise polynomial functions there are various orders of ap- proximation which correspond to the order of differentiability used in the model (approximation). Example 3.8. The following implications describe the orders of approximation: Zeroth-order Model \u21d2 f(xi) = f\u0303(xi), First-order Model \u21d2 f(xi) = f\u0303(xi) and f \u2032(xi) = f\u0303 \u2032(xi), Second-order Model \u21d2 f(xi) = f\u0303(xi), f \u2032(xi) = f\u0303 \u2032(xi), and f \u2032\u2032(xi) = f\u0303 \u2032\u2032(xi). 31 Chapter 3. Piecewise Linear-Quadratic Model Remark 3.9. The word model and approximation can be used interchangeably. 3.2.1 The Class of PL Functions The class of piecewise linear (PL) functions accommodates a variety of models of finite order. The following models take a discrete set of points and output a continuous piecewise linear approxima- tion of the function f . Consider the zeroth-order model f\u03030(x) := max i [fi + si(x\u2212 xi)], (3.3) where si = fi+1\u2212fi xi+1\u2212xi and fi = f(xi) = f\u03030(xi). The model f\u0303 is convex, lsc, and piecewise linear as Equation (3.3) takes the maximum of a finite set of affine functions. Moreover, f\u03030 \u2265 f as it is an inner approximation of the epigraph (see Figure 3.1). Lemma 3.10. Let f \u2208 PLQ such that dom f := {x : x0 \u2264 x \u2264 xn}. Then, for x \u2208 dom f f(x) \u2264 max i\u2208{0,\u00b7\u00b7\u00b7 ,n\u22121} [ f(xi) + f(xi+1)\u2212 f(xi) xi+1 \u2212 xi (x\u2212 xi) ] . (3.4) Proof. Let h := max i\u2208{0,\u00b7\u00b7\u00b7 ,n\u22121} [ f(xi) + f(xi+1)\u2212 f(xi) xi+1 \u2212 xi (x\u2212 xi) ] . First we show that f(xi) = h(xi) for i = 0, . . . , n. Since f is convex we have f(x1)\u2212 f(x0) x1 \u2212 x0 \u2264 f(x2)\u2212 f(x1) x2 \u2212 x1 \u2264 \u00b7 \u00b7 \u00b7 \u2264 f(xn)\u2212 f(xn\u22121) xn \u2212 xn\u22121 . (3.5) Then denote li(x) := f(xi) + f(xi+1)\u2212 f(xi) xi+1 \u2212 xi (x\u2212 xi) ( \u2200 i \u2208 I ) . 32 Chapter 3. Piecewise Linear-Quadratic Model Now we have li(xi+1) = f(xi+1) = li+1(xi). Using Equation (3.5) and lk(xi+1) \u2264 li(xi+1) if k \u2264 i, lk(xi+1 \u2264 li(xi+1) if k \u2265 i, we obtain h(xi+1) = max 0\u2264k\u2264n lk(xi+1) = f(xi+1). By Definition 1.5 all segments which join any two points on a convex function f must lay on or above f . Hence, there exists an index i0 such that x \u2208 [xi0 , xi0+1], as xi < xi+1. There also exists \u03bb such that x = \u03bbxi0 + (1\u2212 \u03bb)xi0+1 where \u03bb \u2208 [0, 1]. Then f(xi0) + f(xi0+1)\u2212 f(xi0) xi0+1 \u2212 xi0 (x\u2212 xi0) = f(xi0) + f(xi0+1)\u2212 f(xi0) xi0+1 \u2212 xi0 (\u03bbxi0 + (1\u2212 \u03bb)xi0+1 \u2212 xi0), = f(xi0) + f(xi0+1)\u2212 f(xi0) xi0+1 \u2212 xi0 (1\u2212 \u03bb)(xi0+1 \u2212 xi0), = f(xi0) + (1\u2212 \u03bb)[f(xi0+1)\u2212 f(xi0)], = \u03bbf(xi0) + (1\u2212 \u03bb)f(xi0+1) \u2265 f(\u03bbxi0 + (1\u2212 \u03bb)xi0+1) = f(x). Consider the first-order model f\u03031(x) := max i [fi + gi(x\u2212 xi)], (3.6) where gi \u2208 \u2202f(xi). This model is also convex, continuous, and piecewise linear as Equation (3.6) is 33 Chapter 3. Piecewise Linear-Quadratic Model \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 f(x) f0(x) Figure 3.1: The zeroth-order model takes the maximum of a set of affine functions. taking the maximum of a finite set of linear functions as seen on Figure 3.2. The main difference between f\u03030 and f\u03031 is that this model is an outer approximation of the epigraph with f\u03031 \u2264 f . Lemma 3.11. Let f \u2208 PLQ and gi \u2208 \u2202f(xi). Then f(x) \u2265 max i\u2208{0,\u00b7\u00b7\u00b7 ,n} [ f(xi) + gi(x\u2212 xi) ] ( \u2200 x \u2208 R ) . (3.7) Proof. From Definition 1.34, we have f(x) \u2265 f(xi) + gi(x\u2212 xi). Therefore the supremum of these minorizing functions will also minorize f . Theorem 3.12. Let f \u2208 PLQ, then f\u03031(x) \u2264 f(x) \u2264 f\u03030(x). Proof. This is a direct consequence of Lemma 3.10, and Lemma 3.11. The problem with the zeroth-order and first-order models is that the class of piecewise linear 34 Chapter 3. Piecewise Linear-Quadratic Model \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 f(x) f1(x) Figure 3.2: The first-order model takes the maximum of a set of subtangents. (PL) functions is not closed under standard convex operations. Example 3.13. The following example is motivated by the proximal average. Let f1(x) = x. Then f\u22171 (y) = I{1}(y) =\u21d2 ( f\u22171\u0003 1 2 \u2016 \u00b7 \u20162 ) (y) = (y \u2212 1)2 2 . So a simple combination of standard convex operations results in a function which is quadratic. Therefore all quadratic pieces cannot be computed exactly, we have to rely on convergence of piecewise linear approximations. 3.2.2 The Class of PLQ Functions The class of piecewise linear-quadratic (PLQ) functions remedies the shortcomings of Example 3.13. In fact, the class of PLQ functions is closed under all the standard convex operations. 35 Chapter 3. Piecewise Linear-Quadratic Model \u22123 \u22122 \u22121 0 1 2 3 0 2 4 6 8 10 12 f(x) f2(x) Figure 3.3: Second-order model of the function f(x) = x 4 4 . Proposition 3.14. The class of PLQ functions is closed under (positive) scalar multiplication, addition, conjugation, and regularization. Proof. It is clear that the class of PLQ functions is closed under (positive) scalar multiplication and addition. The conjugate of a PLQ function f is a PLQ function. We notice that the graph of \u2202f\u2217 is piecewise linear, and is symmetric (with respect to the line y = x) to the graph of \u2202f . Therefore f\u2217 is PLQ [21, p.484]. As a consequence of Equation (1.4) and Remark 1.29, the class of PLQ functions is closed under the Moreau envelope operation. The PLQ class is robust enough to approximate any convex function using a zeroth-order, first-order or even second-order model. An example of a second order model is seen in Figure 3.3. 36 Chapter 3. Piecewise Linear-Quadratic Model Remark 3.15. It is important that we use a higher-order model because as the order increases, the approximating function deviates less from the actual function. In fact, as the degree of the Taylor series rises, the approximating function is closer to the correct function in a neighborhood of the given discrete set of points (see Subsection 3.4). 3.3 Algorithms The objective of this section is to provide a detailed comparison between Fast and PLQ algorithms (see [16\u201319]). Moreover, we intend to discuss the advantages and drawbacks of each algorithm with respect to the standard convex operations. We will also provide examples and detailed steps which will emphasize the main idea behind each algorithm. In order to compare algorithms we need a measure of efficiency. Let us consider time-complexity and space-complexity as the main benchmarks for practical use. The time and space complexity of an algorithm can be determined by counting the number of times we look at each point and its corresponding memory allocation. Moreover, we can verify the complexity analysis by increasing the size of input and checking the rate at which the CPU time or memory allocation increases. From here we can categorize an algorithm into certain classes of efficiency. 3.3.1 PLQ Algorithms Not only does the class of PLQ functions provide closure for the standard convex operations but the class also gives rise to linear-time and linear-space algorithms. Moreover, these algorithms do not require prior knowledge of the domain (unlike fast algorithms). They provide exact solutions when performing standard convex operations on PLQ functions. An efficient way to store the PLQ function is a PLQ matrix. To illustrate, the unbounded PLQ 37 Chapter 3. Piecewise Linear-Quadratic Model function f(x) := \uf8f1\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f3 a0x 2 + b0x+ c0 if x \u2264 x0, a1x 2 + b1x+ c1 if x0 < x \u2264 x1, ... an\u22121x 2 + bn\u22121x+ cn\u22121 if xn\u22122 < x \u2264 xn\u22121, anx 2 + bnx+ cn otherwise. (3.8) is stored as a matrix plqf := \uf8ee \uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8f0 x0 a0 b0 c0 x1 a1 b1 c1 ... ... ... ... xn\u22121 an\u22121 bn\u22121 cn\u22121 +\u221e an bn cn \uf8f9 \uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fb . (3.9) The bounded-domain version of f(x) is stored as plqf := \uf8ee \uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8f0 x0 0 0 +\u221e x1 a1 b1 c1 ... ... ... ... xn\u22121 an\u22121 bn\u22121 cn\u22121 +\u221e 0 0 +\u221e \uf8f9 \uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fb . Half-bounded functions are stored similarly. Remark 3.16. The indicator function f(x) := I{x0}(x) of a single point x0 is a special case which is stored as the row vector plqf := [x0 0 0 +\u221e]. Definition 3.17. Let PLQE denote the set of all lsc extended-valued functions which are PLQ and 38 Chapter 3. Piecewise Linear-Quadratic Model continuous on the interior of their domains. Remark 3.18. The class of functions in PLQE is just the set of functions in PLQ extended to nonconvex functions. A PLQ matrix can only store functions in PLQE due to the following limitations. Example 3.19. The usc function f(x) = \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 x2 if x < 0, x2 + 1 if x \u2265 0, cannot be stored as a PLQ matrix as \uf8ee \uf8ef\uf8f0 0 1 0 0 +\u221e 1 0 1 \uf8f9 \uf8fa\uf8fb\u21d4 \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 x2 if x \u2264 0, x2 + 1 if x > 0. This limitation is due to the fact that the data structure is not robust enough to represent all combinations of inequalities and strict inequalities. Similarly, functions which are neither lsc nor usc cannot be stored as a PLQ matrix. Example 3.20. A discontinuous function f(x) = \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 x2 if x < 0, x2 + 1 if x > 0, does not fit into our model as the quadratic, x2 for x < 0 is defined by a strict inequality. Another limitation of the model is that we cannot explicitly store vertical lines. 39 Chapter 3. Piecewise Linear-Quadratic Model Example 3.21. The PLQ operator f(x) = \uf8f1\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f3 {\u22121} if x < 0, [\u2212 1, 1] if x = 0, {1} if x > 0, cannot be represented by PLQ matrix due to the vertical line at x = 0. Remark 3.22. The limitations of the PLQ matrix in Example 3.19 and 3.20 could be remedied by adding a point between each interval. Also, in Example 3.21 we could store vertical lines implicitly: store all nonvertical lines, and complete the graph by continuity\/maximality. We will not consider such models in the present thesis. Next we present a linear-time algorithm for each standard convex operation in Definition 3.2, where each algorithm only considers functions from the set PLQ. (Positive) Scalar Multiplication (PLQ Algorithm) Scalar multiplication of a PLQ function amounts to multiplying each piece of a PLQ function by a scalar (see Figure 3.4). As multiplying each piece takes constant time, multiplying n pieces will take n steps; hence the algorithm runs in linear-time. Let \u03bb \u2208 [0,+\u221e[. Then \u03bb\u2297 \uf8ee \uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8f0 x0 a0 b0 c0 x1 a1 b1 c1 ... ... ... ... xn\u22121 an\u22121 bn\u22121 cn\u22121 +\u221e an bn cn \uf8f9 \uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fb = \uf8ee \uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8f0 x0 \u03bba0 \u03bbb0 \u03bbc0 x1 \u03bba1 \u03bbb1 \u03bbc1 ... ... ... ... xn\u22121 \u03bban\u22121 \u03bbbn\u22121 \u03bbcn\u22121 +\u221e \u03bban \u03bbbn \u03bbcn \uf8f9 \uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fb . 40 Chapter 3. Piecewise Linear-Quadratic Model \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 0 1 2 3 4 5 6 7 Figure 3.4: The dashed line is the function resulting from multiplying a PLQ function by a scalar. Notation 3.23. The symbol \u2297 denotes PLQ multiplication, and is not the usual matrix multipli- cation. Addition (PLQ Algorithm) Now we will present an algorithm to compute the sum of two PLQ functions (which could be extended to the sum of a finite number of PLQ functions). Definition 3.24. The partition points are the xi of the first column in the PLQ matrix (3.9). Moreover, the set of partition points is the whole column vector. Algorithm 3.25. The algorithm sorts all the partition points on the x-axis (visualize an x-y plane) where along each interval [xi, xi+1] we compute the sum of two quadratic (or linear) pieces which are active in [xi, xi+1]. 41 Chapter 3. Piecewise Linear-Quadratic Model The addition of two PLQ functions is not trivial unless both functions have identical sets of partition points as in Example 3.26. Example 3.26. \uf8ee \uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8f0 x0 a0 b0 c0 ... ... ... ... xn\u22121 an\u22121 bn\u22121 cn\u22121 +\u221e an bn cn \uf8f9 \uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fb \u2295 \uf8ee \uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8f0 x0 d0 e0 f0 ... ... ... ... xn\u22121 dn\u22121 en\u22121 fn\u22121 +\u221e dn en fn \uf8f9 \uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fb = \uf8ee \uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8f0 x0 a0 + d0 b0 + e0 c0 + f0 ... ... ... ... xn\u22121 an\u22121 + dn\u22121 bn\u22121 + en\u22121 cn\u22121 + fn\u22121 +\u221e an + dn bn + en cn + fn \uf8f9 \uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fb . Remark 3.27. We use the symbol \u2295 to denote the addition of two PLQ functions, which is not the usual matrix addition. In the extreme case, if both PLQ matrices (or functions) have disjoint sets of partitioning points, the matrix resulting from the addition operation will have the union of both sets as partition points (see Example 3.28). Example 3.28. The addition of the following PLQ matrices (or PLQ functions) is shown in Figure 3.5. \uf8ee \uf8ef\uf8ef\uf8ef\uf8ef\uf8f0 \u22121 1 0 0 1 0 0 1 +\u221e 1 0 0 \uf8f9 \uf8fa\uf8fa\uf8fa\uf8fa\uf8fb \u2295 \uf8ee \uf8ef\uf8ef\uf8ef\uf8ef\uf8f0 0 1 0 0 2 0 1 0 +\u221e 1 0 \u22122 \uf8f9 \uf8fa\uf8fa\uf8fa\uf8fa\uf8fb = \uf8ee \uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8ef\uf8f0 \u22121 2 0 0 0 1 0 1 1 0 1 1 2 1 1 0 +\u221e 2 0 \u22122 \uf8f9 \uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fa\uf8fb . So the general case requires us to keep track of each partitioned interval and determine which quadratic (or linear) pieces are added together. As we are adding at most n + m quadratic (or linear) pieces together the running time is 42 Chapter 3. Piecewise Linear-Quadratic Model \u22122.5 \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 0 2 4 6 8 10 12 14 Figure 3.5: The addition of two PLQ matrices (or functions) with disjoint sets of partitioning points results in the \u201ddashed\u201d function. 43 Chapter 3. Piecewise Linear-Quadratic Model O(n + m), where n, (resp. m) is the size of the sets of partitioning points for the first (resp. second) function. Conjugation (PLQ Algorithm) Next we present an algorithm which finds the conjugate of a function f(x) \u2208 PLQ. Visualizing f(x) as Equation (3.8) we consider a pair of quadratic functions from f(x): f\u0304(x) := \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 \u03d5i(x) := aix 2 + bix+ ci if xi\u22121 < x \u2264 xi, \u03d5i+1(x) := ai+1x 2 + bi+1x+ ci+1 if xi < x \u2264 xi+1. It is convex if and only if \uf8f1\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f3 \u03d5i(xi) = \u03d5i+1(xi) (continuity criterion), \u03d5\u2032i(xi) \u2264 \u03d5 \u2032 i+1(xi) (increasing \u201dslopes\u201d criterion ), ai, ai+1 \u2265 0 (convexity criterion). From these assumptions, we can compute the conjugate directly as f\u0304\u2217(y) = \uf8f1\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f3 \u03d5\u2217i (y) if \u03d5 \u2032 i(xi\u22121) < y \u2264 \u03d5 \u2032 i(xi), y\u0304(y \u2212 \u03d5\u2032i(xi)) + \u03d5 \u2217 i (\u03d5 \u2032 i(xi)) if \u03d5 \u2032 i(xi) \u2264 y \u2264 \u03d5 \u2032 i+1(xi), \u03d5\u2217i+1(y) if \u03d5 \u2032 i+1(xi) < y \u2264 \u03d5 \u2032 i+1(xi+1), (3.10) where y\u0304 := \u03d5\u2032i+1(xi)xi \u2212 \u03d5i+1(xi)\u2212 \u03d5 \u2032 i(xi)xi + \u03d5i(xi)) \u03d5\u2032i+1(xi)\u2212 \u03d5 \u2032 i(xi) . The middle case (which only arises when the function f\u0304(x) is nonsmooth at xi) is an affine function 44 Chapter 3. Piecewise Linear-Quadratic Model bridging the graph of \u03d5\u2217i (y) with \u03d5 \u2217 i+1(y). Computing f\u0304 \u2217(y) can be done in linear-time and space since the computation of \u03d5\u2217i (y) and \u03d5 \u2217 i+1(y) can be performed directly (see Example 1.18). Remark 3.29. The conjugate in Equation 3.10 was computed directly using the Maple package \u201dsymbolic convex analysis toolkit\u201d (SCAT) [9]. As f\u0304(x) is convex we can compute the conjugate supy\u3008x, y\u3009\u2212 f\u0304(x) by solving for the critical point of the inner function \u3008y, x\u3009\u2212 f\u0304(x) and substituting it back into the conjugate equation. Algorithm 3.30. The general strategy is to iteratively take pairs of quadratic (or linear) functions from f(x) (from left to right, consecutively) and apply the explicit closed form of the conjugate operation from Equation (3.10) to f\u0304(x). One drawback of Equation (3.10) is that we assumed f\u0304(x) is convex. Therefore the PLQ conjugate and Moreau envelope algorithms are restricted to convex functions. The next section removes the convexity assumption. 3.3.2 Extending the PLQ Algorithms We recall the fact that the PLQ conjugate, and Moreau envelope algorithms are restricted to convex functions. From Formula (1.4), and Remark 1.29 we need only to extend the conjugate algorithm to nonconvex functions. In fact, the relationship between the Fenchel conjugate and the convex hull provides some insight into conjugating a nonconvex PLQ function. Remark 3.31. The class of functions in PLQE will be use the same data structure as in the set PLQ. Fact 3.32. [11, p.44] For any function f minorized by at least one affine function, the biconjugate of f is the closed convex hull: f\u2217\u2217 = co f . 45 Chapter 3. Piecewise Linear-Quadratic Model \u22126 \u22124 \u22122 0 2 4 6 \u22123 \u22122 \u22121 0 1 2 3 4 Figure 3.6: The biconjugate (f\u2217\u2217) yields the closed convex hull. As the Fenchel conjugate and convex hull depend on the pointwise supremum of a family of affine functions, the Fenchel conjugate is exactly the conjugate of the convex hull (see Facts 3.32 and 1.47). Theorem 3.33. For any proper function f , we have f\u2217 = (co f)\u2217 and f\u2217\u2217\u2217 = f\u2217. Proof. The function f\u2217 is lsc and convex so we have f\u2217\u2217\u2217 = f\u2217. The conjugate of Fact 3.32, yields f\u2217\u2217\u2217 = (co f)\u2217. Since f\u2217\u2217\u2217 = f\u2217 we have f\u2217 = (co f)\u2217. So regardless whether we take the conjugate of the convex hull or the conjugate itself, the conjugate function will be the same. Therefore, the PLQ conjugate algorithm can be extended to nonconvex functions by first taking the convex hull. 46 Chapter 3. Piecewise Linear-Quadratic Model Convex Hull Now we present an algorithm which finds the convex hull of a function f(x) in PLQE. Let f(x) be defined as Equation (3.8). Then iteratively take pairs of PLQ functions f\u0304(x) := \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 \u03d5i(x) = aix 2 + bix+ ci if xi\u22121 < x \u2264 xi, \u03d5i+1(x) = ai+1x 2 + bi+1x+ ci+1 if xi < x \u2264 xi+1, from f(x) (i.e. for i = 1, . . . , n), and determine if they are nonconvex. The result below is important, but we omit its proof as it is trivial. Theorem 3.34. The PLQ function f\u0304(x) is nonconvex if at least one of the following conditions hold. I. ai < 0 (Concavity Criterion), II. ai+1 < 0 (Concavity Criterion), III. \u03d5i(xi) < \u03d5i+1(xi) (Discontinuity Criterion), IV. \u03d5i(xi) > \u03d5i+1(xi) (Discontinuity Criterion), V. \u03d5\u2032i(xi) > \u03d5 \u2032 i+1(xi) (Decreasing \u201dSlope\u201d Criterion). Remark 3.35. Although we have restricted our framework to the class of functions which are continuous (see the set PLQE). We still consider cases III. and IV. as a more general setting for the PLQ convex hull algorithm. First, we address several special cases. 47 Chapter 3. Piecewise Linear-Quadratic Model Table 3.1: The following special cases arise when taking the convex hull of f(x). Case 1. a0 < 0 or an < 0 co f(x) = \u2212\u221e Case 2. a0 \u2265 0 and an \u2265 0 Case 2.a. a0 \u2265 0 and an > 0 co f(x) is PLQ Case 2.b. a0 > 0 and an \u2265 0 co f(x) is PLQ Case 2.c. a0 = 0 and an > 0 co f(x) is PLQ Case 2.d. a0 > 0 and an = 0 co f(x) is PLQ Case 2.e a0 = an = 0 and b0 > bn co f(x) = \u2212\u221e Case 2.f a0 = an = 0 and b0 \u2264 bn co f(x) is PLQ Remark 3.36. The above special cases arise from the fact that the convex hull requires at least one supporting affine function which is finite. If we cannot find at least one, then the convex hull is exactly minus infinity. Theorem 3.37. Let f(x) \u2208 PLQE, and be defined as Equation (3.8). Then the convex hull has several special cases co f(x) = \uf8f1\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f3 \u2212\u221e if a0 < 0, \u2212\u221e if an < 0, \u2212\u221e if a0 = an = 0 and \u03d5 \u2032 0(x0) > \u03d5 \u2032 n(xn\u22121), PLQ otherwise. Proof. Table 3.1 considers all possible cases concerning the first and last quadratics of f , which determine if f(x) can be supported by an affine function. Algorithm 3.38. If f(x) falls under one of the special cases in Theorem 3.37 we stop with co f(x) = \u2212\u221e. As the unbounded concave functions are noted in Theorem 3.37 we address the case of bounded concave functions. 48 Chapter 3. Piecewise Linear-Quadratic Model If f(x) contains any bounded concave functions (i.e. satisfying I or II in Theorem 3.34 with xi\u22121, and xi+1 finite) then replace each concave part by an affine function interpolating the endpoints (See Equation (3.11) in Solution 3.43). Next, iteratively grab pairs of functions f\u0304(x) from f(x) and determine if they are \u201dquadratic-quadratic\u201d, \u201dquadratic-linear\u201d, \u201dlinear-quadratic\u201d or \u201dlinear- linear\u201d. Then proceed to the appropriate Problem (resp. 3.39, 3.41, 3.40, or 3.43). If the pair turns out to be nonconvex by one of the criteria in Theorem 3.34, then we backtrack by decrementing the index i and repeating the above process. The algorithm is complete when there no longer exists any nonconvex pairs by Theorem 3.34. \u22125 \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 5 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 (a) \u22125 \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 5 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 (b) \u22125 \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 5 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 (c) \u22125 \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 5 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 (d) \u22125 \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 5 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 (e) Figure 3.7: The above sequence of images, demonstrate the required steps in computing the convex hull of a PLQ function. We see from Figure (c) that it is necessary to back-track at most once per iteration to ensure a convex function. Indeed a single pass may not produce a convex function, and as a result the algorithm looks at each partition point at most twice. Hence, the algorithm takes at most 2n steps and runs in linear time. 49 Chapter 3. Piecewise Linear-Quadratic Model Problem 3.39 (quadratic-quadratic). Given two quadratic (or linear) functions \u03d5i and \u03d5i+1 with domain [xi\u22121, xi+1], partitioned at xi, solve the following system for x\u03b1 and x\u03b2. \u03d5\u2032i(x\u03b1) = \u03d5 \u2032 i+1(x\u03b2) \u03d5\u2032i(x\u03b1)(x\u2212 x\u03b1) + \u03d5i+1(x\u03b1) = \u03d5 \u2032 i+1(x\u03b2)(x\u2212 x\u03b2) + \u03d5i+1(x\u03b2) \u2200 x \u2208 R xi\u22121 \u2264 x\u03b1 \u2264 xi xi \u2264 x\u03b2 \u2264 xi+1. If there exists a solution, then the convex hull is co f\u0304(x) = \uf8f1\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f3 \u03d5i(x) if xi\u22121 \u2264 x < x\u03b1, \u03d5\u2032i(x\u03b1)(x\u2212 x\u03b2) + \u03d5i+1(x\u03b2) if x\u03b1 \u2264 x \u2264 x\u03b2, \u03d5i+1(x) if x\u03b2 < x < xi+1. If the above system of equations has no solution, we solve Problem 3.41. 50 Chapter 3. Piecewise Linear-Quadratic Model \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 \u22121 0 1 2 3 4 5 Figure 3.8: The \u201ddashed\u201d function is an instance of co f\u0304(x), and the partition points x\u03b1 and x\u03b2 are where the affine function \u03d5\u2032i(x\u03b1)(x\u2212 x\u03b2) +\u03d5i+1(x\u03b2) bridges the gap between the left quadratic \u03d5i(x), and the right quadratic \u03d5i+1(x). Note that we do not need to consider the bounded and unbounded cases as we can always find an affine function minorizing f\u0304(x) and supporting both quadratic pieces. Problem 3.40 (quadratic-linear). Given two quadratic (or linear) functions \u03d5i and \u03d5i+1 joined at xi with domain [xi\u22121, xi+1], find an x\u03b1 such that \u03d5\u2032i(x\u03b1)(x\u2212 x\u03b1) + \u03d5i(x\u03b1) = \u03d5 \u2032 i(x\u03b1)(x\u2212 xi+1) + \u03d5i(xi+1), \u2200 x \u2208 R xi\u22121 \u2264 x\u03b1 \u2264 xi. 51 Chapter 3. Piecewise Linear-Quadratic Model If a solution exists then the convex hull is co f\u0304(x) = \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 \u03d5i(x) if xi\u22121 \u2264 x < x\u03b1, \u03d5\u2032i(x\u03b1)(x\u2212 xi+1) + \u03d5i(xi+1) if x\u03b1 \u2264 x \u2264 xi+1. If the above system of equations has no solution, we solve Problem 3.40. \u22122 \u22121 0 1 2 3 4 5 \u22120.5 0.0 0.5 1.0 1.5 2.0 (a) \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 (b) Figure 3.9: The \u201ddashed\u201d function is an instanceco f\u0304(x) and the partition point x\u03b1 is where the affine functions \u03d5\u2032i+1(x\u03b1)(x\u2212 xi\u22121) + \u03d5i(xi\u22121) on the right joins the quadratic \u03d5i+1(x) on the left. The two graphs (a) and (b) represent the difference between a bounded and unbounded convex hull. If xi+1 is finite then our graph will look like (a) but if xi+1 = +\u221e then it will look like (b). Problem 3.41 (linear-quadratic). Given two quadratic (or linear) functions \u03d5i and \u03d5i+1 defined on the interval [xi\u22121, xi+1] and joined at xi, find an x\u03b2 such that \u03d5\u2032i(x\u03b2)(x\u2212 xi\u22121) + \u03d5i(xi\u22121) = \u03d5 \u2032 i+1(x\u03b2)(x\u2212 x\u03b2) + \u03d5i+1(x\u03b2) \u2200 x \u2208 R, xi \u2264 x\u03b2 \u2264 xi+1. 52 Chapter 3. Piecewise Linear-Quadratic Model If we find a solution to the above equations the convex hull is co f\u0304(x) = \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 \u03d5\u2032i+1(x\u03b2)(x\u2212 xi\u22121) + \u03d5i(xi\u22121) if xi\u22121 \u2264 x \u2264 x\u03b2, \u03d5i+1(x) if x\u03b2 < x \u2264 xi+1. If the above system of equations has no solution, we solve Problem 3.43. \u22125 \u22124 \u22123 \u22122 \u22121 0 1 2 \u22120.5 0.0 0.5 1.0 1.5 2.0 (a) \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 (b) Figure 3.10: The \u201ddashed\u201d function is an instance co f\u0304(x), and the partition point at x\u03b2 is where the affine function \u03d5\u2032i(x\u03b2)(x\u2212 xi+1) +\u03d5i+1(xi+1) on the left joins the quadratic \u03d5i(x) on the right. The two graphs (a) and (b) represent the difference between a bounded and unbounded convex hull. If xi\u22121 is finite then our graph will look like (a) but if xi\u22121 = \u2212\u221e then it will look like (b). Notation 3.42. Assume we have a bounded PLQ function f\u0304(x) discontinuous at xi. Then cont f\u0304(x) returns the continuous version of f\u0304(x). The function f\u0304(x) is discontinuous by case III or IV of Theorem 3.34. These cases are addressed below. Problem 3.43 (linear-linear). If f\u0304(x) satisfies III of Theorem 3.34, then the function cont f\u0304(x) 53 Chapter 3. Piecewise Linear-Quadratic Model is the line interpolating (xi, \u03d5i(xi)) and (xi+1, \u03d5i+1(xi+1)). Hence, cont f\u0304(x) = \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 \u03d5i(x) if xi\u22121 \u2264 x \u2264 xi, \u03d5i+1(xi+1)\u2212\u03d5i(xi) xi+1\u2212xi (x\u2212 xi) + \u03d5i(xi) if xi < x \u2264 xi+1. If f\u0304(x) satisfies IV of Theorem 3.34, then the function cont f\u0304(x) is the line interpolating (xi\u22121, \u03d5i(xi\u22121)) and (xi, \u03d5i+1(xi)). Hence, cont f\u0304(x) = \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 \u03d5i+1(xi)\u2212\u03d5i(xi\u22121) xi+1\u2212xi (x\u2212 xi) + \u03d5i(xi) if xi\u22121 < x \u2264 xi, \u03d5i+1(x) if xi < x \u2264 xi+1. If f\u0304(x) satisfies V of Theorem 3.34, then the function co f\u0304(x) is the line interpolating (xi\u22121, \u03d5(xi\u22121)) and (xi+1, \u03d5(xi+1)). Hence, co f\u0304(x) = \u03d5i+1(xi+1)\u2212 \u03d5i(xi\u22121) xi+1 \u2212 xi\u22121 (x\u2212 xi\u22121) + \u03d5i(xi\u22121) if xi\u22121 \u2264 x \u2264 xi+1. (3.11) \u22123.0 \u22122.5 \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 \u22126 \u22125 \u22124 \u22123 \u22122 \u22121 0 1 (a) \u22123.0 \u22122.5 \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 \u22126 \u22125 \u22124 \u22123 \u22122 \u22121 0 1 (b) \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 (c) Figure 3.11: The \u201ddashed\u201d curves represent different instances of the function co f\u0304(x). Figure (a) is the convex hull when the function f\u0304(x) is bounded from the right. Figure (b) is the convex hull when f\u0304(x) is bounded from the left, and Figure (c) is the convex hull when f\u0304(x) is bounded from both sides. 54 Chapter 3. Piecewise Linear-Quadratic Model 3.3.3 Fast Algorithms for Convex Transforms Let us recall fast algorithms, and compare them with PLQ algorithms using the same standard convex operations as in Definition 3.2. (Positive) Scalar Multiplication (Fast Algorithm) Multiplying a function by a (positive) scalar, amounts to evaluating a function on a grid and multiplying each sample value by the scalar value. Notation 3.44. The value X[i] is the ith term in the grid X. For example if X := (5, 8, 9, 2) then X[1] = 5. Moreover, if we are given a function f(x) := x2 for x \u2208 R, then f(X[1]) = 25, and f(X) = (X)2 = (25, 64, 81, 4). Remark 3.45. In the following Fast algorithms, we do not require a uniform grid and use it for the sake of simplicity. Algorithm 3.46. Let X := {a + i b\u2212a n : i = 1, 2, \u00b7 \u00b7 \u00b7 , n} be a discrete grid on the interval [a, b] with n points. Then scalar multiplication amounts to multiplying each value f(X[i]) by a scalar \u03bb. Fast scalar multiplication runs in linear-time and is an equivalent algorithm to PLQ scalar multiplication in the sense that they both use the same set of steps. Addition (Fast Algorithm) Adding two continuous functions numerically calls for the evaluation of each function, and comput- ing the sum of their values. The addition of two grids is not necessarily straightforward. In order to add two functions evaluated on different grids, we need to model our discrete set of points and compute the sum along interpolated values. 55 Chapter 3. Piecewise Linear-Quadratic Model \u22125 \u22124 \u22123 \u22122 \u22121 0 1 2 3 4 5 0 50 100 150 200 250 300 Figure 3.12: The limitations of discrete addition. The addition of two functions on disjoint grids is shown in Figure 3.12. The function marked by \u201dcrosses\u201d is f1(X1) = (X1) 2 \u2212 2(X1) + 1, and the function marked by \u201dcircle-crosses\u201d is f2(X2) = 10(X2) 2. They are evaluated on the grids X2 := {\u22125 + i 5+5 5 : i = 1, 2, \u00b7 \u00b7 \u00b7 , 5}, and X1 := {\u22125 + i5+515 : i = 1, 2, \u00b7 \u00b7 \u00b7 , 15} respectively. The addition of f1(X1) with f2(X2) is the curve marked by \u201dstars\u201d. When considering needle functions, any grid points which do not match up, sum to positive infinity. The extreme case is when the two grids are disjoint; then the resulting function is exactly positive infinity. Conjugation (Fast Algorithm) One of the first fast algorithms used to approximate the conjugate was the Fast Legendre Transform (FLT) algorithm [8, 13]. It was introduced to numerically approximate the conjugate of a function 56 Chapter 3. Piecewise Linear-Quadratic Model with a log-linear worst-case complexity. A more efficient algorithm, conveniently named the Linear- time Legendre transform (LLT) improved the complexity to linear-time [14]. The Linear-Time Legendre Transform (LLT) is a fast algorithm which takes as input the sets X, Y , S where Y [i] is an approximation of f(X[i]). The input is a set of points and slopes in R. The output is a set Zm where Z[j] is an approximation of f \u2217(S[j]). The LLT algorithm can be considered a black box which computes the conjugate at various points in space. Computing the approximation f\u2217(S[j]) involves two different steps depending on the convexity of f . Step 1. Assuming f is nonconvex and X[i] is increasing, we use the Beneath-Beyond algorithm (see [2]) to compute the convex hull of the planar set (X[i], Y [i]) in linear-time. After obtaining the convex hull, we discard all points inside the convex hull and proceed to Step 2. Step 2. Assuming f is convex (otherwise complete Step 1), we compute the slopes C := ( ci = yi+1 \u2212 yi xi+1 \u2212 xi ( \u2200 i \u2208 I )) . Finding a solution for Equation (3.2) is equivalent to finding the supremum of all affine functions that support f . As we are given a set of slopes to approximate the Fenchel conjugate, we need only to find the support point (or where the function is maximized). Using the fact that f is convex and therefore has increasing slopes, we only need to find index i such that C[i] \u2264 S[j] \u2264 C[i+ 1], as c1 \u2264 c2 \u2264 \u00b7 \u00b7 \u00b7 \u2264 cn. Thus, X[i] is the supporting point and maximizes i 7\u2192 S[j]X[i] \u2212 Y [i] for slope S[j]. As a result we have an approximation of the conjugate Z[j] = S[j]X[i] \u2212 Y [i] \u2248 f\u2217(S(j)), 57 Chapter 3. Piecewise Linear-Quadratic Model at S(j). For further information see [15, p.27]. Remark 3.47. The LLT algorithm utilizes the fact that the supremum of Formula (3.1) is attained if f is convex and x \u2208 dom f and s \u2208 \u2202f(x). See Proposition 1.25. 3.4 Convergence We recall the definition of pointwise convergence. Definition 3.48. Suppose {fn} is a sequence of functions with common domain. If lim n\u2192\u221e fn(x) = f(x) holds for all x \u2208 dom f then the sequence {fn} converges pointwise to f . 3.4.1 Fast Algorithms for Convex Transforms (Convergence) All fast algorithms depend on discretizing a continuous model in order to approximate a convex operation. Since fast algorithms output a discrete set of data points, all fast algorithms implicitly rely on a zeroth order model. In order for these computational algorithms to converge, we need to either increase the number of points or enlarge the domain. For example, when we consider the discrete Legendre transform in Equation (3.2), we have the following convergence results [19, p.3]: Notation 3.49. For the sake of simplicity we consider equidistant points: X := (a+ i b\u2212 a n : i = 1, 2, \u00b7 \u00b7 \u00b7 , n). We define the discrete approximation of f as fX with linear interpolation defining fX at points not in X. 58 Chapter 3. Piecewise Linear-Quadratic Model Notation 3.50. The function fn is the function f evaluated on a grid X := {a + i (b\u2212a) n : i = 1, 2, . . . , n} where the n corresponds to the size of the grid X. Moreover, fn is the n \u2212 th term in a sequence of functions (fn). Fact 3.51. Assume f : R \u2192 R \u222a {+\u221e} is proper. Convergence on a bounded domain (see [8, 13, 14]). (i) If f is usc on [a, b], then f\u2217n converges pointwise to f \u2217 [a,b](s) := supx\u2208[a,b][sx\u2212 f(x)]. (ii) If f is twice continuously differentiable on an open interval containing [a, b], then max [a,b] |f\u2217[a,b] \u2212 f \u2217 n| \u2264 1 2 (b\u2212 a)2 n2 max [a,b] f \u2032\u2032. Convergence on unbounded domains (see [10, 14]) The following equivalence holds for any s \u2208 R, and any a > 0: \u2202f\u2217(s) \u2229 [\u2212a, a] 6= \u2205 \u21d4 f\u2217[\u2212a,a](s) = f \u2217(s). 3.4.2 PLQ Algorithms (Convergence) Let us recall the fact that PLQ algorithms provide exact solutions within the class of PLQ functions, and that functions outside this class must be modeled via PL or PLQ approximation. Theorem 3.52. Assume f is proper, modeled by a PL approximation f\u0303n, and satisfies condition (i) or (ii) of Fact 3.51. Then f\u0303\u2217n converges pointwise to f \u2217. Proof. Assume f\u0303n is modeled via PL approximation, then it falls under the same framework as the LLT algorithm. As consequence of Fact 3.51, f\u0303n converges pointwise to f \u2217 on both bounded and unbounded domain. 59 Chapter 3. Piecewise Linear-Quadratic Model \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 dom=[\u22121,1] dom=[\u22122,2] dom=[\u22123,3] f* (a) Convergence by enlarging the domain: the con- jugate of f(x) := |x| converges to I[\u22121,1](x) the indicator function of the interval [\u22121, 1] \u22122.0 \u22121.5 \u22121.0 \u22120.5 0.0 0.5 1.0 1.5 2.0 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 N=3 N=5 N=9 f* (b) Convergence by decreasing the grid spacing: the discrete conjugate of the function f(x) := x2\/2 converges to f\u2217 = f . Figure 3.13: Convergence of the discrete Legendre transform. 60 Chapter 4 Finite Convex Integration This chapter provides additional examples and details for [12]. Throughout we will also describe and illustrate the relationship between the two papers, [12] and [5]. As notations differ, our purpose is to explain both styles in an effort to provide additional perspectives and to aid understanding. 4.1 Introduction Notation 4.1. Define I := {0, 1, . . . , n}. We are concerned with the following problem. Problem 4.2. Given a finite family {(xi, x \u2217 i )}i\u2208I \u2282 X \u00d7X \u2217. find a lsc convex function f : X \u2192 R such that x\u2217i \u2208 \u2202f(xi) \u2200i \u2208 I or determine no such function exists. Consider an operator A where gra A := {(x, x\u2217) \u2208 X \u00d7 X\u2217 : x\u2217 \u2208 Ax}. The finite family {(xi, x \u2217 i )}i\u2208I is interpreted as the graph of a finite operator. Finite operators which have certain properties can admit antiderivatives. Definition 4.3. The function f is an antiderivative of A if graA \u2286 gra \u2202f . 61 Chapter 4. Finite Convex Integration \u22123 \u22122 \u22121 0 1 2 3 \u22123 \u22122 \u22121 0 1 2 3 (a) \u2202f1(x) \u22123 \u22122 \u22121 0 1 2 3 \u22123 \u22122 \u22121 0 1 2 3 (b) \u2202f2(x) Figure 4.1: Finite convex integration is not unique up to a constant. The continuous integration problem is well known to be unique up to a constant. For example, the antiderivative of the continuous function f(x) = x is the function F (x) = 12x 2 + K, where K \u2208 R. However, the finite integration problem may have many solutions. Example 4.4. Let graA = {(\u22121,\u22122), (0, 0), (1, 2)} and f(0) = 0 (initial condition). Then both f1(x) = x 2 and f2(x) = 2|x| are convex, lsc antiderivatives as they satisfy gra A \u2286 gra \u2202f1(x) and graA \u2286 gra \u2202f2(x) (see Problem 4.2 and Figure 4.2). Hence, given \u03bb0, we define the solution set F as the set of all lsc convex functions that satisfy f(x0) = \u03bb0 and x \u2217 i \u2208 \u2202f(xi) \u2200i \u2208 I. Altogether, f \u2208 F \u21d4 \uf8f1\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f3 f is lsc, convex, f(x0) = \u03bb0, and x\u2217i \u2208 \u2202f(xi) \u2200i \u2208 I. Next we discuss the relationship between cyclic monotonicity and the existence of antideriva- tives. 62 Chapter 4. Finite Convex Integration 4.2 Cyclic monotonicity Finite operators can have a special property called cyclic monotonicity (CM), which is necessary for an operator to admit a convex antiderivative. Definition 4.5. The family {(xi, x \u2217 i )}i\u2208I is said to be CM if for all j0, j1, . . . , jk, jk+1 \u2208 I with j0 = jk+1, the inequality \u2211 l=0,...,k \u3008x\u2217jl , xjl+1 \u2212 xjl\u3009 \u2264 0 holds. Proposition 4.6. For every convex lsc function, the subdifferential operator \u2202f is CM. Proof. Take x\u2217i \u2208 \u2202f(xi) for all i \u2208 I. We have f(xj1)\u2212 f(xj0) \u2265 \u3008x \u2217 j0 , xj1 \u2212 xj0\u3009, f(xj2)\u2212 f(xj1) \u2265 \u3008x \u2217 j1 , xj2 \u2212 xj1\u3009, ... f(xjk)\u2212 f(xjk\u22121) \u2265 \u3008x \u2217 jk\u22121 , xjk \u2212 xjk\u22121\u3009, f(xjk+1)\u2212 f(xjk) \u2265 \u3008x \u2217 jk , xjk+1 \u2212 xjk\u3009. Adding up all inequalities and setting xjk+1 = xj0 , the LHS becomes [f(xj1)\u2212 f(xj0)] + [f(xj2)\u2212 f(xj1)] + \u00b7 \u00b7 \u00b7+ [f(xjk)\u2212 f(xjk\u22121)] + [f(xj0)\u2212 f(xjk)] = 0. As a result, we get \u2211 l=0,...,k \u3008x\u2217jl , xjl+1 \u2212 xjl\u3009 \u2264 0. 63 Chapter 4. Finite Convex Integration Definition 4.7. A multivalued mapping \u03c1 from X to X is said to be monotone if \u3008x1 \u2212 x0, x \u2217 1 \u2212 x \u2217 0\u3009 \u2265 0 holds for every (x0, x \u2217 0) and (x1, x \u2217 1) in the graph of \u03c1. This definition corresponds to the case where k = 1 from the definition of CM. Thus, if \u03c1 is a CM mapping then \u03c1 is a monotone mapping. Geometrically, monotonicity describes a function which is \u201dnon-decreasing\u201d. This means that the graph of the function is either \u201dincreasing\u201d or \u201dflat\u201d. However, when the dimension of X is greater than one, there exists monotone mappings which are not cyclically monotone (for an example, see [20, p.240]). Remark 4.8. The most efficient way to verify that a finite operator is cyclically monotone, is to use an algorithm (such as [12, p.140]) to compute an antiderivative. If the algorithm cannot find an antiderivative then the given operator is not cyclically monotone. However, if it does find a convex antiderivative, this implies that the operator is cyclically monotone. Note that the algorithm stated in [12, p.140] runs in O(n3) time. 4.3 Antiderivatives and their properties A CM operator may admit many antiderivatives using a variety of different methods. Notation 4.9. We say that mA is a method m : A\u2192 ConvX applied to an operator A, and mA is the resulting antiderivative. These antiderivatives may have a variety of different properties. Definition 4.10. Let A be a finite CM operator with graA containing the points (ai, a \u2217 i )i\u2208I . The 64 Chapter 4. Finite Convex Integration method m is primal-dual symmetric if its resulting antiderivative mA satisfies m \u2217 A = mA\u22121. Definition 4.11. [5, Theorem 3.5] Let A be a CM operator with graA containing n points (ai, a \u2217 i ). Then the Rockafellar function is defined by RA,(a1,a\u22171)(x) = max (a2,a\u22172)\u2208graA, ... (an,a\u2217n)\u2208graA \u3008a\u22171, a2 \u2212 a1\u3009+ \u00b7 \u00b7 \u00b7+ \u3008a \u2217 n\u22121, an \u2212 an\u22121\u3009+ \u3008a \u2217 n, x\u2212 an\u3009. (4.1) Remark 4.12. We refine the above definition to be more precise. Let \u03c3 := n+1\u2211 i=1 \u3008ai+1 \u2212 ai, a \u2217 i \u3009 \u2264 0, where an+2 = a1 Then graA contains at most n points, and there exists integers k and l such that ak = al with 1 \u2264 k < l \u2264 n+ 1 and \u03c3 := \u03c31 + \u03c32 with \u03c31 := l\u22121\u2211 i=k \u3008ai+1 \u2212 ai, a \u2217 i \u3009, and \u03c32 := n+1\u2211 i=l \u3008ai+1 \u2212 ai, a \u2217 i \u3009+ k\u2212l\u2211 i=1 \u3008ai+1 \u2212 ai, a \u2217 i \u3009. Since \u03c31 \u2264 0 we have \u03c3 \u2264 \u03c32. If we repeat the above argument k times we obtain \u03c3 \u2264 \u03c32 \u2264 \u00b7 \u00b7 \u00b7 \u2264 \u03c3k. 65 Chapter 4. Finite Convex Integration As a result, we now have RA,(a1,a\u22171)(x) = max (a2,a\u22172)\u2208graA, ... (an,a\u2217n)\u2208graA \u3008a\u22171, a2 \u2212 a1\u3009+ \u00b7 \u00b7 \u00b7+ \u3008a \u2217 n\u22121, an \u2212 an\u22121\u3009+ \u3008a \u2217 n, x\u2212 an\u3009. (4.2) such that (ar, a \u2217 r) 6= (as, a \u2217 s) for r 6= s. Definition 4.13. Let A be a CM operator, and let a \u2208 domA. Then we set RA,a := RA,(a,a\u2217) where a\u2217 is an arbitrary point in Aa. Definition 4.14. Let A be CM. We say that a function f : X \u2192 R\u0304 is an intrinsic antiderivative if graA \u2286 gra \u2202f and f depends only on graA. The following example is not an intrinsic method (i.e. does not always produce intrinsic an- tiderivatives). Example 4.15. Let e \u2208 X be such that \u2016e\u2016 = 1 and define A via graA := {(\u2212e,\u2212e), (e, e)}. Then for every x \u2208 X, we have RA,\u2212e(x) = max { \u2212 \u3008x|e\u3009 \u2212 1, \u3008x|e\u3009 \u2212 3 } = \u22122 + | \u3008x|e\u3009 \u2212 1| (4.3) and RA,e(x) = max { \u3008x|e\u3009 \u2212 1,\u2212\u3008x|e\u3009 \u2212 3 } = \u22122 + | \u3008x|e\u3009+ 1|. (4.4) Consequently, RA,e 6\u2265 RA,\u2212e and RA,e 6\u2264 RA,\u2212e. 66 Chapter 4. Finite Convex Integration There are several types of intrinsic methods when the graph of A is finite. Example 4.16. Let A be cyclically monotone such that graA is finite. Then the two methods max (a,a\u2217)\u2208graA RA,(a,a\u2217), and \u2211 (a,a\u2217)\u2208graA 1 nA RA,(a,a\u2217) produce intrinsic methods of A (nA is the number of points in graA). Let us consider a process to build a primal-dual symmetric method from any intrinsic antideriva- tive. Definition 4.17. Let f0, f1 \u2208 F, then the proximal midpoint average of f0 and f1 is the function P(f0, 1 2 , f1) := ( 1 2 ( f0 + 1 2\u2016 \u00b7 \u2016 2 )\u2217 + 12 ( f1 + 1 2\u2016 \u00b7 \u2016 2 )\u2217)\u2217 \u2212 12\u2016 \u00b7 \u2016 2. (4.5) Fact 4.18. [5, Corollary 4.17] Let m be a method which produces an intrinsic method with full domain. Then P ( mA, 1 2 ,m \u2217 A\u22121 ) and P ( mA\u22121 , 1 2 ,m \u2217 A ) are both primal-dual symmetric antiderivatives for A and A\u22121 respectively. Moreover, they have full domain. The importance of primal-dual symmetric methods can be illustrated by the following diagram. A \u2217 \u2212\u2212\u2212\u2212\u2192 A\u22121 m \uf8e6\uf8e6y m \uf8e6\uf8e6y mA = P ( mA, 1 2 ,m \u2217 A\u22121 ) \u2217 \u2212\u2212\u2212\u2212\u2192 mA\u22121 = P ( mA\u22121, 1 2 ,m \u2217 A ) 67 Chapter 4. Finite Convex Integration In general (within our framework), we always have primal-dual symmetry at the discrete level: x\u2217i \u2208 A(xi) \u21d4 xi \u2208 A \u22121(x\u2217i ); however, there is no reason to expect this type of symmetry at the continuous level, i.e. m\u2217A = mA\u22121. It turns out that when the antiderivatives mA, and m \u2217 A\u22121 are intrinsic methods, the proximal midpoint average P ( mA, 1 2 ,m \u2217 A\u22121 ) produces an antiderivative which is primal-dual symmetric at the continuous level [5, Example 4.19 and Example 4.20]. \u22123 \u22122 \u22121 0 1 2 3 \u22121 0 1 2 3 4 (a) Convex antiderivatives. \u22123 \u22122 \u22121 0 1 2 3 \u22121 0 1 2 3 4 (b) The subdifferential. Figure 4.2: Constructing a primal-dual symmetric method. In Figure 4.2(a), the family graA := { (a, exp(a)) : a \u2208 {\u22121,\u221212 , 0, 1 2 , 1} } is denoted by the \u201dcircle-cross\u201d marks on the function f(x) = exp(x). The \u201ddashed\u201d function and the \u201ddashed- dotted\u201d function are two antiderivatives mA, and m \u2217 A\u22121 , respectively. As we are interested in finding an antiderivative between these two functions, we take the proximal midpoint average P ( mA, 1 2 ,m \u2217 A\u22121 ) represented by the \u201dthick\u201d function. This function is primal-dual symmetric. We see from Figure 4.2(b) that the \u201dthick\u201d function is \u2202P ( mA, 1 2 ,m \u2217 A\u22121 ) and is an antideriva- tive (as graA \u2282 gra \u2202P). Any convex antiderivative will have a monotone subdifferential, hence \u2202P ( mA, 1 2 ,m \u2217 A\u22121 ) lies within the \u201ddashed\u201d rectangles. 68 Chapter 4. Finite Convex Integration 4.4 Relationship between [12] and [5] 4.4.1 Minimal antiderivative in higher dimension Fact 4.19. [12, p.132] Let \u0393 be the set of all ordered subsets J = {j0, \u00b7 \u00b7 \u00b7 , jk} of I with j0 = 0, and for all r 6= s, jr 6= js. Then, for each J \u2208 \u0393, we define the function fJ(x) := \u3008x \u2217 j0 , xj1 \u2212 xj0\u3009+ \u00b7 \u00b7 \u00b7 + \u3008x \u2217 jk\u22121 , xk \u2212 xjk\u22121\u3009+ \u3008x \u2217 jk , x\u2212 xjk\u3009 (4.6) on X. Then for a given \u03bb0 and x0, the minimal antiderivative f \u2212 is defined as f\u2212(x) := \u03bb0 +max J\u2208\u0393 fJ(x). (4.7) The Rockafellar function RA,(a,a\u2217) is a special method that produces minimal antiderivatives. Remark 4.20. We say that an antiderivative is minimal if it is the greatest piecewise linear function which bounds all other antiderivatives from below. Similarly, the maximal antiderivative is the smallest piecewise linear function which bounds all other antiderivatives from above. Fact 4.21. [5, Theorem 3.5] Let A be CM and a \u2208 domA. Then RA,(a,a\u2217) = min{f \u2208 F : f is an antiderivative of A and f(a) = 0}. As both f\u2212 and RA,(a,a\u2217) are minimal antiderivatives, they must be equal. Transformation 4.22. Let \u0393 be the collection of all subsets J = {j0, \u00b7 \u00b7 \u00b7 , jk} of I such that 69 Chapter 4. Finite Convex Integration andj0 = 0, and for all r 6= s, jr 6= js. Let us rename the following indexes and variables: J = {j0, j1, \u00b7 \u00b7 \u00b7 , jk} := {1, 2, \u00b7 \u00b7 \u00b7 , n}, xi := ai for i = 1, \u00b7 \u00b7 \u00b7 , n \u2212 1, x\u2217i := a \u2217 i for i = 1, \u00b7 \u00b7 \u00b7 , n\u2212 1, x0 := a1, and x\u22170 := a \u2217 1. Theorem 4.23. Assume that A is a CM operator and that \u03bb0 = 0 = f \u2212(a). Then RA,(a,a\u2217) = f \u2212, where f\u2212 \u2208 F and (a, a\u2217) \u2208 graA. Proof. Recall, that f\u2212(x) := \u03bb0 +max J\u2208\u0393 fJ(x), where \u03bb0 = 0. For each J \u2208 \u0393 we have fJ(x) := \u3008x \u2217 j0 , xj1 \u2212 xj0\u3009+ \u00b7 \u00b7 \u00b7+ \u3008x \u2217 jk\u22121 , xjk \u2212 xjk\u22121\u3009+ \u3008x \u2217 jk , x\u2212 xjk\u3009. (4.8) After redefining Equation (4.8) by Transformation 4.22, we get fJ(x) = \u3008a \u2217, a2 \u2212 a\u3009+ \u00b7 \u00b7 \u00b7+ \u3008a \u2217 n\u22121, an \u2212 an\u22121\u3009+ \u3008a \u2217 n, x\u2212 an\u3009. Taking the maximum gives the announced result. Similarly, the maximal antiderivatives are the same. 70 Chapter 4. Finite Convex Integration Fact 4.24. [12, p.137] Let H := {h \u2208 X : h(x\u22170) = \u03bb \u2217 0, \u03bb \u2217 0 = \u3008x0, x \u2217 0\u3009 \u2212 \u03bb0, xi \u2208 \u2202h \u2212(x\u2217i ) \u2200i \u2208 I} Then, by [12, Theorem 3.4] there exists at most one minimal antiderivative h\u2212 which is piecewise linear with full domain. Remark 4.25. Note that h\u2212 is the minimal antiderivative in the dual space H. It is unique as it is the greatest piecewise linear function which bounds all other antiderivatives (in H) from below. If we wish to compute f+ := (h\u2212)\u2217, it is completely characterized in [12, Theorem 4.3]. Observation 4.26. Since h\u2212 \u2208 H we have the following constraints: i. h\u2212(x\u22170) = \u03bb \u2217 0 : \u03bb \u2217 0 = \u3008x0, x \u2217 0\u3009 \u2212 \u03bb0, ii. xi \u2208 \u2202h \u2212(x\u2217i ) \u2200i \u2208 I. Taking the conjugate of the minimal antiderivative in the dual space will yield the maximal antiderivative in the primal space. Hence (h\u2212)\u2217 is maximal in F. Notation 4.27. Throughout we will denote the maximal antiderivative (h\u2212)\u2217 as f+. Fact 4.28. [5, Corollary 3.10] Let A be CM and a \u2208 domA, R\u2217A\u22121,(a\u2217,a) \u2212 \u3008a, a \u2217\u3009 = max{f \u2208 F : f is an antiderivative of A and f(a) = 0}. Theorem 4.29. Assume A is a CM operator and that f+(a) = \u03bb\u22170 = 0. Then R\u2217A\u22121,(a\u2217,a) \u2212 \u3008a, a \u2217\u3009 = f+, where f+ := (h\u2212)\u2217, h\u2212 \u2208 H, and (a, a\u2217) \u2208 graA. Proof. It suffices to show h\u2212 = RA\u22121,(a\u2217,a) + \u3008a, a \u2217\u3009. 71 Chapter 4. Finite Convex Integration Recall, h\u2212 \u2208 H \u21d4 \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 h\u2212(x\u22170) = \u03bb \u2217 0 : \u03bb \u2217 0 = \u3008x0, x \u2217 0\u3009 \u2212 \u03bb0, xi \u2208 \u2202h \u2212(x\u2217i ) \u2200i \u2208 I. From \u03bb0 = \u03bb \u2217 0 \u2212 \u3008x0, x \u2217 0\u3009 and \u03bb \u2217 0 = 0 we have h\u2212(x) = \u03bb0 +max J\u2208\u0393 hJ(x) (4.9) = \u03bb\u22170 \u2212 \u3008x0, x \u2217 0\u3009+max J\u2208\u0393 hJ(x) (4.10) = max J\u2208\u0393 hJ (x)\u2212 \u3008x0, x \u2217 0\u3009 (4.11) where for each J \u2208 \u0393 we have hJ (x) := \u3008xj0 , x \u2217 j1 \u2212 x\u2217j0\u3009+ \u00b7 \u00b7 \u00b7 + \u3008xjk\u22121 , x \u2217 k \u2212 x \u2217 jk\u22121 \u3009+ \u3008xjk , x\u2212 x \u2217 jk \u3009. After redefining Equation (4.11) by Transformation 4.22, we get h\u2212(x) = max J\u2208\u0393 hJ(x)\u2212 \u3008a, a \u2217\u3009 where for each J \u2208 \u0393 we have hJ(x) = \u3008a, a \u2217 2 \u2212 a \u2217\u3009+ \u00b7 \u00b7 \u00b7+ \u3008an\u22121, a \u2217 n \u2212 a \u2217 n\u22121\u3009+ \u3008an, x\u2212 a \u2217 n\u3009. 4.4.2 Minimal antiderivative in one dimension In one dimension Problem 4.2 has a closed form for the minimal antiderivative. 72 Chapter 4. Finite Convex Integration Fact 4.30. [5, Theorem 3.14] Let A have a finite graph and suppose that the graph of B : conv(Ax) is \u222ani=1({ai} \u00d7 [b \u2212 i , b + i ]), where n \u2208 {1, 2, \u00b7 \u00b7 \u00b7 }, a1 < a2 < \u00b7 \u00b7 \u00b7 < an, and b \u2212 1 \u2264 b + 1 \u2264 b \u2212 2 \u2264 \u00b7 \u00b7 \u00b7 \u2264 b \u2212 n \u2264 b + n . Set a0 := \u2212\u221e and an+1 := +\u221e. Suppose that k \u2208 {1, \u00b7 \u00b7 \u00b7 , n}. Then RA,(ak ,a\u2217k) is given by x 7\u2192 \uf8f1\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f3 (x\u2212 ai)b \u2212 i + \u2211k j=i+1(aj\u22121 \u2212 aj)b \u2212 j , if ai\u22121 < x \u2264 ai \u2264 ak; (x\u2212 ai)b + i + \u2211i\u22121 j=k(aj+1 \u2212 aj)b + j , if ak \u2264 ai \u2264 x < ai+1; and \u2202RA,(ak,a\u2217k) is given by x 7\u2192 \uf8f1\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f2 \uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f4\uf8f3 {b\u2212i }, if ai\u22121 < x \u2264 ai \u2264 ak; [b\u2212i , b \u2212 i+1], if x = ai < ak; [b\u2212k , b + k ], if x = ak; [b+i\u22121, b + i ], if ak < x = ai; {b+i }, if ak \u2264 ai < x < ai+1; Remark 4.31. In one dimension, the closed form of the minimal antiderivative in [12, p.145] is exactly the same closed form of the Rockafellar function in [5, Theorem 3.14] since both closed forms have full domain and are piecewise linear minimal antiderivatives. In one dimension we have a closed form of the minimal antiderivative. Therefore the time complexity is linear. 73 Chapter 4. Finite Convex Integration 4.5 Computational Algorithms for Antiderivatives The most efficient algorithms used to compute minimal and maximal antiderivatives are described in [12, p.139-140]. They are specialized algorithms driven by the following LP formulation. Fact 4.32. [12, Proposition 3.5] The optimal solutions of min \u03bb { \u2211 i\u2208I,i6=0 \u03bbi : \u03bbj \u2212 \u03bbi \u2265 \u3008x \u2217 i , xj \u2212 xi\u3009 ( \u2200 i, j \u2208 I ) } (4.12) correspond to the values f\u2212(xi) (i.e. \u03bbi = f \u2212(xi) for i = 1, . . . , n), which completely determines the minimal antiderivative RA,(a,a\u2217). Remark 4.33. We only need to discuss the minimal antiderivative, as the maximal antiderivative is determined through conjugate duality, see [12, p.137] and [5, Corollary 3.10]. Definition 4.34. Considering the set of linear inequalities Ax \u2264 b, we say that the system Ax \u2264 b is a difference of constraints when the constraint matrix A contains one +1 and one \u22121 in each row; all other entries are zero, for all rows. As xi, x \u2217 i , xj are given, the constraint matrix in Equation (4.12) is a system of difference constraints. It turns out that we can find a feasible solution (an antiderivative) to the Linear program in Equation (4.12) by finding a solution to the shortest path problem in the corresponding graph formulation (see [1, p.103-105]). Example 4.35. Let graA := {(0, 0), (\u22121,\u22122), (1, 2)}. Then by Equation (4.12) we have the fol- 74 Chapter 4. Finite Convex Integration lowing constraints: \u03bb0 \u2212 \u03bb1 \u2264 0, \u03bb0 \u2212 \u03bb2 \u2264 0, \u03bb1 \u2212 \u03bb0 \u2264 2, \u03bb1 \u2212 \u03bb2 \u2264 4, \u03bb2 \u2212 \u03bb0 \u2264 2, \u03bb2 \u2212 \u03bb1 \u2264 4. Each system of difference constraints is associated with a graph whose nodes are \u03bbi and each con- straint \u03bbi \u2212 \u03bbj \u2264 \u3008x \u2217 i , xi \u2212 xj\u3009 represents the arc (i, j) with the weighed value \u3008x \u2217 i , xj \u2212 xi\u3009. Hence, we obtain the following graph. 2 0 Figure 4.3: Graph associated with a system of difference constraints. 75 Chapter 4. Finite Convex Integration In this specific example we notice that all weights are nonnegative, therefore we can use Dijkstra\u2019s algorithm to obtain the shortest path distances from a particular node to every other node. Thus the following table represents the shortest path distances from node i to node j. d(i, j) Node 0 Node 1 Node 2 Node 0 0 2 2 Node 1 0 0 2 Node 2 0 2 0 By [1, p.104] and Fact 4.32 we have d(i, j) = \u03bbj = f(xj). Hence, i. f1(x0 = 0) = \u03bb0 = 0, f1(x1 = \u22121) = \u03bb1 = 2, f1(x2 = 1) = \u03bb2 = 2, ii. f2(x0 = 0) = \u03bb0 = 0, f2(x1 = \u22121) = \u03bb1 = 0, f2(x2 = 1) = \u03bb2 = 2, iii. f3(x0 = 0) = \u03bb0 = 0, f3(x1 = \u22121) = \u03bb1 = 2, f3(x2 = 1) = \u03bb2 = 0. We recall that \u03bbj = f(xj) for j = 0, . . . , n completely determines an antiderivative f . Therefore the above list determines exactly three antiderivatives (f1, f2, and f3). However, not one of these antiderivatives is the minimal antiderivative as the minimal antiderivative corresponds to the case \u03bbi = f(xi) as illustrated on Figure 4.4. 76 Chapter 4. Finite Convex Integration \u22123 \u22122 \u22121 0 1 2 3 0 1 2 3 4 5 x2 2|x| f\u2212 f+ Figure 4.4: Minimal and maximal antiderivatives. Therefore, generally we cannot find a minimal antiderivative by solving its corresponding short- est path problem, but we can find several antiderivatives. In fact, for each node in the graph G we can find at most one antiderivative. Moreover, if the weights \u3008x\u2217i , xj \u2212 xi\u3009 are all nonnegative we can find an antiderivative in O(n2) time using Dijkstra\u2019s algorithm. On the other hand, if we remove the nonnegative weight assumption, we can find several antiderivatives in O(n3) using Floyd-Warshall algorithm which is described in [1, p.147]. Note that the graph of G is always full (i.e. n2 arcs), so labeling algorithms [1, p.155] (whose complexity depends on the number of arcs) will not help us compute an antiderivative more efficiently. Remark 4.36. As many convex transforms rely on the subgradient constraint in Equation (4.12), they may also be linked to the shortest path problem. 77 Chapter 5 Conclusion We visited the proximal average which is an advanced convex transform. It inherits a much richer set of properties (continuity, differentiability, full domain, and strict convexity properties) than the arithmetic average. Due to the complicated nature of the proximal average, existing computa- tional algorithms failed to accurately approximate nested transforms (without complications). This shortcoming is remedied by the PLQ model and its corresponding algorithms. As these algorithms depend on convexity, we extended their framework to nonconvex functions using the convex hull. We also explicitly defined each algorithm and stated its general strategy. We showed that a method used to produce primal-dual symmetric antiderivatives depends on the proximal midpoint average and Rockafellar\u2019s function. The latter is a minimal antiderivative, and through conjugate duality, it also provides a maximal antiderivative. Therefore, we showed that the minimal and maximal antiderivatives in [5, Theorem 3.5, Corollary 3.10] are the same minimal and maximal antiderivatives as in [12, p.132, p.136]. So the algorithms in [12, p.139, p.140] can be used to compute primal-dual symmetric antiderivatives. Moreover, these minimal and maximal antiderivatives can be computed by solving a shortest path problem. In fact, the specialized algorithms used to solve the all-pairs shortest path problem [1, p.144] can be used to find many antiderivatives (including Rockafellar\u2019s functions). Future work may focus on the following extensions. \u2022 Developing a second-order model to accommodate the PLQ algorithms. 78 Chapter 5. Conclusion \u2022 Analyzing convergence for a second-order model. \u2022 Extending the PLQ model to higher dimension. \u2022 Generalizing the PLQ matrix to represent a more general class of functions. \u2022 Expanding the PLQ model to include other convex operations which have the same closure properties. \u2022 Exploring the link between convex transforms and shortest path problems. 79 Bibliography [1] R. K. Ahuja, T. L. Magnanti, and J. B. Orlin, Network flows, Prentice Hall Inc., Englewood Cliffs, NJ, 1993. Theory, algorithms, and applications. [2] C. B. Barber, D. P. Dobkin, and H. Huhdanpaa, The quickhull algorithm for convex hulls, ACM Transactions on Mathematical Software, 22 (1996), pp. 469\u2013483. [3] H. H. Bauschke, R. Goebel, Y. Lucet, and X. Wang, The proximal average: Basic properties, tech. report, University of British Columbia, 2007. [4] H. H. Bauschke, Y. Lucet, and M. Trienis, How to transform one convex function continuously into another, tech. report, University of British Columbia, July 2006. Accepted for publication in SIAM Review. [5] H. H. Bauschke, Y. Lucet, and X. Wang, Primal-dual symmetric intrinsic methods for finding antiderivatives of cyclically monotone operators, SIAM J. Control Optim., 46 (2007), pp. 2031\u20132051. [6] H. H. Bauschke, E. Matous\u030ckova\u0301, and S. Reich, Projection and proximal point methods: Convergence results and counterexamples, Nonlinear Anal., 56 (2004), pp. 715\u2013738. [7] S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, Cam- bridge, 2004. 80 Bibliography [8] L. Corrias, Fast Legendre\u2013Fenchel transform and applications to Hamilton\u2013Jacobi equations and conservation laws, SIAM J. Numer. Anal., 33 (1996), pp. 1534\u20131558. [9] C. H. Hamilton, Symbolic convex analysis, master\u2019s thesis, Simon Fraser University, 2005. [10] J.-B. Hiriart-Urruty, Lipschitz r-continuity of the approximate subdifferential of a convex function, Math. Scand., 47 (1980), pp. 123\u2013134. [11] J.-B. Hiriart-Urruty and C. Lemare\u0301chal, Convex Analysis and Minimization Algo- rithms, vol. 305\u2013306 of Grundlehren der Mathematischen Wissenschaften [Fundamental Prin- ciples of Mathematical Sciences], Springer-Verlag, Berlin, 1993. Vol I: Fundamentals, Vol II: Advanced theory and bundle methods. [12] D. Lambert, J.-P. Crouzeix, V. H. Nguyen, and J.-J. Strodiot, Finite convex inte- gration, J. Convex Anal., 11 (2004), pp. 131\u2013146. [13] Y. Lucet, A fast computational algorithm for the Legendre\u2013Fenchel transform, Computa- tional Optimization and Applications, 6 (1996), pp. 27\u201357. [14] , Faster than the Fast Legendre Transform, the Linear-time Legendre Transform, Numer. Algorithms, 16 (1997), pp. 171\u2013185. [15] , La Transforme\u0301e de Legendre\u2013Fenchel et la convexifie\u0301e d\u2019une fonction : algorithmes rapides de calcul, analyse et re\u0301gularite\u0301 du second ordre, PhD thesis, Laboratoire Approximation et Optimisation, UFR MIG, Universite\u0301 Paul Sabatier, Feb. 1997. [16] , Fast Moreau Envelope computation II: Applications, tech. report, University of British Columbia, 2005. 81 Bibliography [17] , A linear Euclidean distance transform algorithm based on the Linear-time Legendre Transform, in Proceedings of the Second Canadian Conference on Computer and Robot Vision (CRV 2005), Victoria BC, May 2005, IEEE Computer Society Press. [18] , Fast Moreau envelope computation I: Numerical algorithms, Numerical Algorithms, 43 (2006), pp. 235\u2013249. DOI - 10.1007\/s11075-006-9056-0. [19] Y. Lucet, H. H. Bauschke, and M. Trienis, The piecewise linear-quadratic model for computational convex analysis, Comput. Optim. Appl., (2006). Accepted for publication. [20] R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, New York, 1970. [21] R. T. Rockafellar and R. J.-B. Wets, Variational Analysis, Springer-Verlag, Berlin, 1998. 82 Appendix A \/\/PROBLEM 1. (1) f1\u2019(x2) = f2\u2019(x4), \/\/ (2) m1(x-x2) + f1(x2) = m2(x-x4) + f2(x4), where \/\/ (3) x1 < x2 < x3 \/\/ (4) x3 < x4 < x5 \/\/ \/\/PROBLEM 2. (1) m1(x-x2) + f1(x2) = m1(x-x5) + f2(x5), where \/\/ (2) x1 < x2 < x3 \/\/ (3) x3 < x4 < x5, and \/\/ (4) m1=f1\u2019(x2) \/\/ \/\/PROBLEM 3. (1) m2(x-x1) + f1(x1) = m2(x-x4) + f2(x4), where \/\/ (2) x1 < x2 < x3 \/\/ (3) x3 < x4 < x5, and \/\/ (4) m2=f2\u2019(x4) \/\/ \/\/PROBLEM 4. (1) m1(x-x1) + f1(x1) = m1(x-x5) + f2(x5), where \/\/ (2) x1 < x2 < x3 \/\/ (3) x3 < x4 < x5 83 Appendix A. function plqco=plq_co(plqf) EPS=1E-15; plqf_original=plqf; plqfLHS=[];plqfRHS=[]; \/\/Bounded plq functions will have an infinity value as a coefficient \/\/in either the first row or the last row, simply remove these pieces \/\/and add these pieces after the convex hull has been computed. if or(plqf(1,2:$) == %inf) then plqfLHS = plqf(1,:); plqf=plqf(2:$,:); end; if or(plqf($,2:$) == %inf) then plqfRHS = plqf($,:); plqf=plqf(1:$-1,:); end; n = size(plqf,1); x=plqf(:,1);a=plqf(:,2);b=plqf(:,3);c=plqf(:,4); \/\/update the coefficients if a(1) < 0 | a(n) < 0 | (a(1)==0 & a(n) == 0 & b(n) < b(1) ) then plqco=[%inf,0,0,-%inf]; return; end; \/\/special case i=1; while i <= n-1 & n >= 2, a=clean(a(:,1)); b=clean(b(:,1)); c=clean(c(:,1)); \/\/Considering a PLQ function with two pieces, it is nonconvex \/\/if the following condition holds. if (2*a(i)*x(i)+b(i)) - (2*a(i+1)*x(i)+b(i+1)) > EPS | a(i) < 0 | a(i+1) < 0 then if n >=3 & i >= 2 then x1 = plqf(i-1,1); else x1 = -%inf; end; f1=plqf(i,:); f2=plqf(i+1,:);x3=f1(1,1); x5=f2(1,1); 84 Appendix A. qplot(plqf_original,plqf,-%inf,%inf); plqf = [plqf(1:(i-1),:);plq_conv_on_interval(f1,f2,x1,x3,x5);plqf((i+2):$,:)]; plqf = plq_clean(plqf); if n <= 1 then plqco=[plqf]; break; end; if i > 1 then i=i-1; end; \/\/backtrack else i=i+1; end; n = size(plqf,1); \/\/update size of the PLQ function x=plqf(:,1);a=plqf(:,2);b=plqf(:,3);c=plqf(:,4); \/\/update the coefficients end; plqco=[plqfLHS;plqf;plqfRHS]; qplot(plqf_original,plqco,-%inf,%inf); return; endfunction 85 Appendix A. \/\/We assume x(1) < x(2) < x(3) < x(4) < x(5) are ordered and finite \/\/(no infinity values). In creating the PLQ hull we may need to create \/\/points x(2) and x(4). Note that x(1) is the LEFTBOUND, x(3) is the \/\/partition point, and x(5) is the RIGHTBOUND in our PLQ interval. \/\/plq_conv_on_interval finds the convex hull on the interval [x1,x5]. function [plqco] = plq_conv_on_interval(f1,f2,x1,x3,x5) \/\/Assume f1 is to the left of f2 x(1) = x1; a(1)=f1(1,2); a(2)=f2(1,2); x(3) = x3; b(1)=f1(1,3); b(2)=f2(1,3); x(5) = x5; c(1)=f1(1,4); c(2)=f2(1,4); ieee(2); \/\/allow infinity i.e. 1\/0 plqco=[]; if a(1) < 0 then \/\/concave pieces are replaced by a linear function. f1=_plq_conv_buildl(x,a,b,c,1,1,1,3); f2=[x(5),a(2),b(2),c(2)]; plqco=plq_conv_on_interval(f1,f2,x(1),x(3),x(5)); return; elseif a(2) < 0 then f1=[x(3),a(1),b(1),c(1)]; f2=_plq_conv_buildl(x,a,b,c,2,5,2,3); plqco=plq_conv_on_interval(f1,f2,x(1),x(3),x(5)); return; elseif 2*a(1)*x(3)+b(1) <= 2*a(2)*x(3)+b(2) then \/\/triveral case plqco=[x(3),a(1),b(1),c(1);x(5),a(2),b(2),c(2)]; return; elseif a(1) == 0 & a(2) == 0 then \/\/(LINEAR-LINEAR) f1 is linear and f2 is linear. 86 Appendix A. [plqco]=_plq_conv_buildl(x,a,b,c,1,1,2,5); return; elseif a(1) == 0 & a(2) <> 0 then \/\/(LINEAR-QUDARTIC) if f1 is linear and f2 is quadratic then x(4) is determined. plqco=[_conv_interval_routine(x,a,b,c,x1,x3,x5)]; return; elseif a(1) <> 0 & a(2) == 0 then \/\/(QUADRATIC-LINEAR) if f1 is quadratic and f2 is linear then x(2) is determined. plqco=[_conv_interval_routine(x,a,b,c,x1,x3,x5)]; return; elseif a(1) <> 0 & a(2) <> 0 then \/\/(QUADRATIC-QUADRATIC) if f1 is quadratic and f2 is quadratic \/\/we need to find solutions to A*x(4)^2 + B*x(4) + C. \/\/The following code solves [PROBLEM 1.], A=(1\/4)*(-4*a(2)^2+4*a(2)*a(1))\/a(1); B=(1\/4)*(-4*a(2)*b(2)+4*b(1)*a(2))\/a(1); C=(1\/4)*(-b(2)^2-b(1)^2+4*c(1)*a(1)+2*b(1)*b(2)-4*c(2)*a(1))\/a(1); D = B^2 - 4*A*C; \/\/Discriminant. if A == 0 then \/\/Linear case, so we have Bx+C=0 with one zero. x(4) = (-C\/B); x(2) = (-1\/2)*(b(1)-2*a(2)*x(4)-b(2))\/a(1); if x(2) < x(1) | x(3) < x(2) then x(2)=%inf; end; if x(4) < x(3) | x(4) > x(5) then x(4)=%inf; end; \/\/if x(2) or x(4) are outside the interval, send the x(i) to infinity. \/\/In otherwords these solutions provide no information and thus we \/\/should just ignore them. Similarly if D < 0, just ignore x(2) and x(4) \/\/as they don\u2019t help with the hull. else \/\/ A <> 0, Quadratic case, so we have Ax^2+Bx+C=0, with one or two zeros. 87 Appendix A. \/\/Solving for x(4). if D>0 then nr = (-B - sqrt(D))\/(2*A); \/\/negative root pr = (-B + sqrt(D))\/(2*A); \/\/positive root if D==0 then nr=pr; end; if x(3) < pr & pr < x(5) then x(4) = pr; elseif x(3) < nr & nr < x(5) then x(4) = nr; else x(4)=%inf; end; x(2) = (-1\/2)*(b(1)-2*a(2)*x(4)-b(2))\/a(1); if x(1) > x(2) | x(2) > x(3) then x(2)=%inf; end; else x(2)=%inf; x(4)=%inf; end end if x(2) < %inf & x(4) < %inf then plqco=[x(2),a(1),b(1),c(1); _plq_conv_buildl(x,a,b,c,1,2,2,4);x(5),a(2),b(2),c(2)]; return; else \/\/x(2) == %inf | x(4) == %inf, \/\/Here we solve the SECOND PROBLEM: as x(2) or x(4) is infeasible in [PROBLEM 1.] plqco=[_conv_interval_routine(x,a,b,c,x1,x3,x5)]; return; end; else printf(\"Error: Case does not exist\"); end endfunction 88 Appendix A. \/\/if we can\u2019t find a solution to [PROBLEM 1.] then we need to find a solution to [PROBLEM 2.]. \/\/The following routine solves [PROBLEM 2.]. function [plqco] = _conv_interval_routine(x,a,b,c,x1,x3,x5), EPS=1E-15; x(1)=x1;x(3)=x3;x(5)=x5; \/\/solving the quadratic for x(2) if a(1) <> 0 & x(5) < %inf then A1 = -a(1); B1 = 2*a(1)*x5; C1 = c(1)+b(1)*x5-a(2)*x5^2-b(2)*x5-c(2); D1 = B1^2 - 4*A1*C1; pr1 = (-B1 + sqrt(D1))\/(2*A1); \/\/solution is x2 nr1 = (-B1 - sqrt(D1))\/(2*A1); \/\/negative root for the LINEAR-QUADRATIC CASE. if x(1) <= pr1 & pr1 <= x(3) then x(2) = pr1; plqco=[x(2),a(1),b(1),c(1);_plq_conv_buildl(x,a,b,c,1,2,2,5)]; return; elseif x(3) <= pr1 & pr1 <= x(5) then x(4)=pr1; plqco=[_plq_conv_buildl(x,a,b,c,1,1,2,4);x(5),a(2),b(2),c(2)]; return; elseif x(1) <= nr1 & nr1 <= x(3) then x(2) =pr1; plqco=[x(2),a(1),b(1),c(1);_plq_conv_buildl(x,a,b,c,1,2,2,5)]; return; elseif x(3) <= nr1 & nr1 <= x(5) then x(4)=pr1; plqco=[_plq_conv_buildl(x,a,b,c,1,1,2,4);x(5),a(2),b(2),c(2)]; return; else plqco=_plq_conv_buildl(x,a,b,c,1,1,2,5); return; end; \/\/solving the quadratic for x(4) elseif a(2) <> 0 & x(1) > -%inf then 89 Appendix A. A2 = a(2); B2 = -2*a(2)*x(1); C2 = -b(2)*x(1)+a(1)*x(1)^2+b(1)*x(1)+c(1)-c(2); D2 = B2^2 - 4*A2*C2; pr2 = (-B2 + sqrt(D2))\/(2*A2); nr2 = (-B2 - sqrt(D2))\/(2*A2); if x(3) <= pr2 & pr2 <= x(5) then x(4)=pr2; plqco=[_plq_conv_buildl(x,a,b,c,1,1,2,4);x(5),a(2),b(2),c(2)]; return; elseif x(1) <= pr2 & pr2 <= x(3) then x(2)=pr2; plqco=[x(2),a(1),b(1),c(1);_plq_conv_buildl(x,a,b,c,1,2,2,5)]; return; elseif x(3) <= nr2 & nr2 <= x(5) then x(4)=nr2; plqco=[_plq_conv_buildl(x,a,b,c,1,1,2,4);x(5),a(2),b(2),c(2)]; return; elseif x(1) <= nr2 & nr2 <= x(3) then x(2)=nr2; plqco=[x(2),a(1),b(1),c(1);_plq_conv_buildl(x,a,b,c,1,2,2,5)]; return; else plqco=_plq_conv_buildl(x,a,b,c,1,1,2,5); return; end; else plqco=_plq_conv_buildl(x,a,b,c,1,1,2,5); return; end; endfunction 90","attrs":{"lang":"en","ns":"http:\/\/www.w3.org\/2009\/08\/skos-reference\/skos.html#note","classmap":"oc:AnnotationContainer"},"iri":"http:\/\/www.w3.org\/2009\/08\/skos-reference\/skos.html#note","explain":"Simple Knowledge Organisation System; Notes are used to provide information relating to SKOS concepts. There is no restriction on the nature of this information, e.g., it could be plain text, hypertext, or an image; it could be a definition, information about the scope of a concept, editorial information, or any other type of information."}],"Genre":[{"label":"Genre","value":"Thesis\/Dissertation","attrs":{"lang":"en","ns":"http:\/\/www.europeana.eu\/schemas\/edm\/hasType","classmap":"dpla:SourceResource","property":"edm:hasType"},"iri":"http:\/\/www.europeana.eu\/schemas\/edm\/hasType","explain":"A Europeana Data Model Property; This property relates a resource with the concepts it belongs to in a suitable type system such as MIME or any thesaurus that captures categories of objects in a given field. It does NOT capture aboutness"}],"GraduationDate":[{"label":"GraduationDate","value":"2007-05","attrs":{"lang":"en","ns":"http:\/\/vivoweb.org\/ontology\/core#dateIssued","classmap":"vivo:DateTimeValue","property":"vivo:dateIssued"},"iri":"http:\/\/vivoweb.org\/ontology\/core#dateIssued","explain":"VIVO-ISF Ontology V1.6 Property; Date Optional Time Value, DateTime+Timezone Preferred "}],"IsShownAt":[{"label":"IsShownAt","value":"10.14288\/1.0066799","attrs":{"lang":"en","ns":"http:\/\/www.europeana.eu\/schemas\/edm\/isShownAt","classmap":"edm:WebResource","property":"edm:isShownAt"},"iri":"http:\/\/www.europeana.eu\/schemas\/edm\/isShownAt","explain":"A Europeana Data Model Property; An unambiguous URL reference to the digital object on the provider\u2019s website in its full information context."}],"Language":[{"label":"Language","value":"eng","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/language","classmap":"dpla:SourceResource","property":"dcterms:language"},"iri":"http:\/\/purl.org\/dc\/terms\/language","explain":"A Dublin Core Terms Property; A language of the resource.; Recommended best practice is to use a controlled vocabulary such as RFC 4646 [RFC4646]."}],"Program":[{"label":"Program","value":"Interdisciplinary Studies","attrs":{"lang":"en","ns":"https:\/\/open.library.ubc.ca\/terms#degreeDiscipline","classmap":"oc:ThesisDescription","property":"oc:degreeDiscipline"},"iri":"https:\/\/open.library.ubc.ca\/terms#degreeDiscipline","explain":"UBC Open Collections Metadata Components; Local Field; Indicates the program for which the degree was granted."}],"Provider":[{"label":"Provider","value":"Vancouver : University of British Columbia Library","attrs":{"lang":"en","ns":"http:\/\/www.europeana.eu\/schemas\/edm\/provider","classmap":"ore:Aggregation","property":"edm:provider"},"iri":"http:\/\/www.europeana.eu\/schemas\/edm\/provider","explain":"A Europeana Data Model Property; The name or identifier of the organization who delivers data directly to an aggregation service (e.g. Europeana)"}],"Publisher":[{"label":"Publisher","value":"University of British Columbia","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/publisher","classmap":"dpla:SourceResource","property":"dcterms:publisher"},"iri":"http:\/\/purl.org\/dc\/terms\/publisher","explain":"A Dublin Core Terms Property; An entity responsible for making the resource available.; Examples of a Publisher include a person, an organization, or a service."}],"Rights":[{"label":"Rights","value":"Attribution-NonCommercial-NoDerivatives 4.0 International","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/rights","classmap":"edm:WebResource","property":"dcterms:rights"},"iri":"http:\/\/purl.org\/dc\/terms\/rights","explain":"A Dublin Core Terms Property; Information about rights held in and over the resource.; Typically, rights information includes a statement about various property rights associated with the resource, including intellectual property rights."}],"RightsURI":[{"label":"RightsURI","value":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/","attrs":{"lang":"en","ns":"https:\/\/open.library.ubc.ca\/terms#rightsURI","classmap":"oc:PublicationDescription","property":"oc:rightsURI"},"iri":"https:\/\/open.library.ubc.ca\/terms#rightsURI","explain":"UBC Open Collections Metadata Components; Local Field; Indicates the Creative Commons license url."}],"ScholarlyLevel":[{"label":"ScholarlyLevel","value":"Graduate","attrs":{"lang":"en","ns":"https:\/\/open.library.ubc.ca\/terms#scholarLevel","classmap":"oc:PublicationDescription","property":"oc:scholarLevel"},"iri":"https:\/\/open.library.ubc.ca\/terms#scholarLevel","explain":"UBC Open Collections Metadata Components; Local Field; Identifies the scholarly level of the author(s)\/creator(s)."}],"Subject":[{"label":"Subject","value":"Convex analysis","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/subject","classmap":"dpla:SourceResource","property":"dcterms:subject"},"iri":"http:\/\/purl.org\/dc\/terms\/subject","explain":"A Dublin Core Terms Property; The topic of the resource.; Typically, the subject will be represented using keywords, key phrases, or classification codes. Recommended best practice is to use a controlled vocabulary."},{"label":"Subject","value":"Convex function","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/subject","classmap":"dpla:SourceResource","property":"dcterms:subject"},"iri":"http:\/\/purl.org\/dc\/terms\/subject","explain":"A Dublin Core Terms Property; The topic of the resource.; Typically, the subject will be represented using keywords, key phrases, or classification codes. Recommended best practice is to use a controlled vocabulary."},{"label":"Subject","value":"Proximal average","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/subject","classmap":"dpla:SourceResource","property":"dcterms:subject"},"iri":"http:\/\/purl.org\/dc\/terms\/subject","explain":"A Dublin Core Terms Property; The topic of the resource.; Typically, the subject will be represented using keywords, key phrases, or classification codes. Recommended best practice is to use a controlled vocabulary."},{"label":"Subject","value":"PLQ (piecewise linear-quadratic)","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/subject","classmap":"dpla:SourceResource","property":"dcterms:subject"},"iri":"http:\/\/purl.org\/dc\/terms\/subject","explain":"A Dublin Core Terms Property; The topic of the resource.; Typically, the subject will be represented using keywords, key phrases, or classification codes. Recommended best practice is to use a controlled vocabulary."},{"label":"Subject","value":"Nonconvex","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/subject","classmap":"dpla:SourceResource","property":"dcterms:subject"},"iri":"http:\/\/purl.org\/dc\/terms\/subject","explain":"A Dublin Core Terms Property; The topic of the resource.; Typically, the subject will be represented using keywords, key phrases, or classification codes. Recommended best practice is to use a controlled vocabulary."},{"label":"Subject","value":"Algorithm","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/subject","classmap":"dpla:SourceResource","property":"dcterms:subject"},"iri":"http:\/\/purl.org\/dc\/terms\/subject","explain":"A Dublin Core Terms Property; The topic of the resource.; Typically, the subject will be represented using keywords, key phrases, or classification codes. Recommended best practice is to use a controlled vocabulary."},{"label":"Subject","value":"Primal-dual symmetric antiderivatives","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/subject","classmap":"dpla:SourceResource","property":"dcterms:subject"},"iri":"http:\/\/purl.org\/dc\/terms\/subject","explain":"A Dublin Core Terms Property; The topic of the resource.; Typically, the subject will be represented using keywords, key phrases, or classification codes. Recommended best practice is to use a controlled vocabulary."},{"label":"Subject","value":"Rockafellar functions","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/subject","classmap":"dpla:SourceResource","property":"dcterms:subject"},"iri":"http:\/\/purl.org\/dc\/terms\/subject","explain":"A Dublin Core Terms Property; The topic of the resource.; Typically, the subject will be represented using keywords, key phrases, or classification codes. Recommended best practice is to use a controlled vocabulary."}],"Title":[{"label":"Title","value":"Computational convex analysis : from continuous deformation to finite convex integration","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/title","classmap":"dpla:SourceResource","property":"dcterms:title"},"iri":"http:\/\/purl.org\/dc\/terms\/title","explain":"A Dublin Core Terms Property; The name given to the resource."}],"Type":[{"label":"Type","value":"Text","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/type","classmap":"dpla:SourceResource","property":"dcterms:type"},"iri":"http:\/\/purl.org\/dc\/terms\/type","explain":"A Dublin Core Terms Property; The nature or genre of the resource.; Recommended best practice is to use a controlled vocabulary such as the DCMI Type Vocabulary [DCMITYPE]. To describe the file format, physical medium, or dimensions of the resource, use the Format element."}],"URI":[{"label":"URI","value":"http:\/\/hdl.handle.net\/2429\/2799","attrs":{"lang":"en","ns":"https:\/\/open.library.ubc.ca\/terms#identifierURI","classmap":"oc:PublicationDescription","property":"oc:identifierURI"},"iri":"https:\/\/open.library.ubc.ca\/terms#identifierURI","explain":"UBC Open Collections Metadata Components; Local Field; Indicates the handle for item record."}],"SortDate":[{"label":"Sort Date","value":"2007-12-31 AD","attrs":{"lang":"en","ns":"http:\/\/purl.org\/dc\/terms\/date","classmap":"oc:InternalResource","property":"dcterms:date"},"iri":"http:\/\/purl.org\/dc\/terms\/date","explain":"A Dublin Core Elements Property; A point or period of time associated with an event in the lifecycle of the resource.; Date may be used to express temporal information at any level of granularity. Recommended best practice is to use an encoding scheme, such as the W3CDTF profile of ISO 8601 [W3CDTF].; A point or period of time associated with an event in the lifecycle of the resource.; Date may be used to express temporal information at any level of granularity. Recommended best practice is to use an encoding scheme, such as the W3CDTF profile of ISO 8601 [W3CDTF]."}]}