{"http:\/\/dx.doi.org\/10.14288\/1.0381024":{"http:\/\/vivoweb.org\/ontology\/core#departmentOrSchool":[{"value":"Science, Faculty of","type":"literal","lang":"en"},{"value":"Mathematics, Department of","type":"literal","lang":"en"}],"http:\/\/www.europeana.eu\/schemas\/edm\/dataProvider":[{"value":"DSpace","type":"literal","lang":"en"}],"https:\/\/open.library.ubc.ca\/terms#degreeCampus":[{"value":"UBCV","type":"literal","lang":"en"}],"http:\/\/purl.org\/dc\/terms\/creator":[{"value":"Arian, Arman","type":"literal","lang":"en"}],"http:\/\/purl.org\/dc\/terms\/issued":[{"value":"2019-09-27T15:43:19Z","type":"literal","lang":"en"},{"value":"2019","type":"literal","lang":"en"}],"http:\/\/vivoweb.org\/ontology\/core#relatedDegree":[{"value":"Doctor of Philosophy - PhD","type":"literal","lang":"en"}],"https:\/\/open.library.ubc.ca\/terms#degreeGrantor":[{"value":"University of British Columbia","type":"literal","lang":"en"}],"http:\/\/purl.org\/dc\/terms\/description":[{"value":"Compressed sensing (CS) is a signal acquisition paradigm to simultaneously\r\nacquire and reduce dimension of signals that admit sparse representations.\r\nThis is achieved by collecting linear, non-adaptive measurements of a signal,\r\nwhich can be formalized as multiplying the signal with a \u201cmeasurement matrix\".\r\nIf the measurement matrix satisfies the so-called restricted isometry\r\nproperty (RIP), then it will be appropriate for compressed sensing. While\r\nwide classes of random matrices provably satisfy the RIP with high probability,\r\nexplicit and deterministic constructions have been shown (so far) to\r\nsatisfy the RIP only in a significantly suboptimal regime.\r\nIn this thesis, our focus is on deterministic measurement matrices in\r\ncompressed sensing. In a nutshell, we investigate quantization methods for a\r\nclass of deterministic matrices (Chapter 2); introduce a novel deterministic\r\nconstruction of a family of binary, circulant measurement matrices using\r\nthe Legendre symbol (Chapter 3); and propose two novel approaches for\r\nimproving the RIP constant estimates based on Gershgorin circle theorem,\r\nobtaining an improvement over the Gershgorin bounds by a multiplicative\r\nconstant. One of our approaches here, together with a conjecture we make\r\nregarding the distribution of quadratic residues in a finite field provides a\r\npotential path to break the so-called \u201csquare-root barrier\"\u2013we give a proof\r\nbased on the assumption that the conjecture holds.","type":"literal","lang":"en"}],"http:\/\/www.europeana.eu\/schemas\/edm\/aggregatedCHO":[{"value":"https:\/\/circle.library.ubc.ca\/rest\/handle\/2429\/71789?expand=metadata","type":"literal","lang":"en"}],"http:\/\/www.w3.org\/2009\/08\/skos-reference\/skos.html#note":[{"value":"Compressed Sensing with DeterministicMeasurement MatricesbyArman ArianB.Sc., Sharif University of Technology, 2008M.Sc., Sharif University of Technology, 2011A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinThe Faculty of Graduate and Postdoctoral Studies(Mathematics)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)August 2019c\u0000 Arman Arian 2019The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the dissertation entitled: Examining Committee: submitted byin partial fulfillment of the requirements forthe degree ofinSupervisor Supervisory Committee Member Supervisory Committee MemberUniversity ExaminerUniversity ExaminerArman ArianMathematicsOzgur YilmazYaniv PlanBrian MarcusJulia Yulia GordonLutz Hans-Joachim LampeCompressed Sensing with Deterministic Measurement MatricesDoctor of PhilosophyiiAbstractCompressed sensing (CS) is a signal acquisition paradigm to simultaneouslyacquire and reduce dimension of signals that admit sparse representations.This is achieved by collecting linear, non-adaptive measurements of a signal,which can be formalized as multiplying the signal with a \u201cmeasurement ma-trix\". If the measurement matrix satisfies the so-called restricted isometryproperty (RIP), then it will be appropriate for compressed sensing. Whilewide classes of random matrices provably satisfy the RIP with high proba-bility, explicit and deterministic constructions have been shown (so far) tosatisfy the RIP only in a significantly suboptimal regime.In this thesis, our focus is on deterministic measurement matrices incompressed sensing. In a nutshell, we investigate quantization methods for aclass of deterministic matrices (Chapter 2); introduce a novel deterministicconstruction of a family of binary, circulant measurement matrices usingthe Legendre symbol (Chapter 3); and propose two novel approaches forimproving the RIP constant estimates based on Gershgorin circle theorem,obtaining an improvement over the Gershgorin bounds by a multiplicativeconstant. One of our approaches here, together with a conjecture we makeregarding the distribution of quadratic residues in a finite field provides apotential path to break the so-called \u201csquare-root barrier\"\u2013we give a proofbased on the assumption that the conjecture holds.iiiLay SummaryCompressed sensing (CS) is a revolutionary sampling paradigm born in 2006.In this field, signals are viewed as vectors, and measurements are obtained viaa measurement matrix. The theory in CS provides tools to prove that certainrandom matrices perform \u201cwell\" as measurement matrices with high proba-bility, and the performance of deterministic matrices have been considered(so far) to be \u201cweak\". In this thesis, we propose a novel class of measurementmatrices which has certain advantages compared to the structures given inthe literature so far. We will also improve (a bit) the \u201cweakness\" of deter-ministic matrices mentioned above for a particular construction. Moreover,we will derive an approach to show how one can quantize the collected mea-surements obtained from a deterministic matrix efficiently, in order to storeor transmit data.ivPrefaceThis thesis consists of my original research, conducted at the Departmentof Mathematics at the University of British Columbia, Vancouver, Canada,under the supervision of Prof. Ozgur Yilmaz. The results in Chapter 2 havebeen submitted as a conference paper to the 13th International conferenceon Sampling Theory and Applications (SampTA 2019).vTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . x1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Compressed sensing\u2013 a new sampling paradigm . . . . . . . . 11.1.1 Basic idea . . . . . . . . . . . . . . . . . . . . . . . . 11.1.2 Measurement matrices in compressed sensing . . . . . 31.1.3 Restricted Isometry Property (RIP) . . . . . . . . . . 61.1.4 RIP for random matrices . . . . . . . . . . . . . . . . 81.1.5 Optimality of random matrices and Gelfand widths . 101.2 Deterministic constructions in compressed sensing . . . . . . 111.2.1 Coherence . . . . . . . . . . . . . . . . . . . . . . . . 141.2.2 Measurement matrices based on Reed-Muller codes . 161.2.3 DeVore\u2019s construction . . . . . . . . . . . . . . . . . 171.2.4 Chirp sensing measurement matrices . . . . . . . . . 181.2.5 The construction of Bourgain et al. . . . . . . . . . . 211.3 Quantization in compressed sensing . . . . . . . . . . . . . . 241.3.1 Memoryless scalar quantization . . . . . . . . . . . . . 251.3.2 Sigma-Delta (\u2303\u0000) quantization . . . . . . . . . . . . 271.3.3 Approximating signals using \u2303\u0000 quantization . . . . 271.3.4 Higher order \u2303\u0000 quantization . . . . . . . . . . . . . 301.3.5 One-stage recovery for \u2303\u0000 quantization . . . . . . . . 341.4 Organization of the thesis . . . . . . . . . . . . . . . . . . . . 36viTable of Contents2 One-stage recovery for \u2303\u0000-quantized compressed sensing 372.1 Approach 1: Modifying the measurement matrix . . . . . . . 402.1.1 Implications for bounded orthonormal systems . . . . 442.1.2 Numerical experiments . . . . . . . . . . . . . . . . . 472.2 Approach 2: Using a digital buffer . . . . . . . . . . . . . . . 472.2.1 Numerical experiments . . . . . . . . . . . . . . . . . 532.3 \u2303\u0000-quantized compressed sensing with chirp sensing matrices 542.3.1 Approximation error as the number of measurementsgrows . . . . . . . . . . . . . . . . . . . . . . . . . . . 552.3.2 Approximation error as the sparsity level varies . . . 592.3.3 Numerical experiments . . . . . . . . . . . . . . . . . 612.4 Further encoding of \u2303\u0000-quantized compressive measurements 623 Deterministic partial binary circulant compressed sensingmatrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 673.1 Introduction and preliminaries . . . . . . . . . . . . . . . . . 673.2 Compressed sensing matrices using Legendre sequence . . . . 693.3 Circulant matrices . . . . . . . . . . . . . . . . . . . . . . . . 703.4 A novel, explicit construction . . . . . . . . . . . . . . . . . . 733.5 One-stage recovery for \u2303\u0000-quantized compressed sensing withdeterministic partial circulant matrices . . . . . . . . . . . . 773.6 Numerical experiments . . . . . . . . . . . . . . . . . . . . . 784 RIP constants for deterministic compressed sensing matrices-beyond Gershgorin . . . . . . . . . . . . . . . . . . . . . . . . 834.1 Paley tight frames for compressed sensing . . . . . . . . . . . 854.2 Improving the Gershgorin bound using skew-adjacency matri-ces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 894.3 Improving the Gershgorin bound using Dembo bounds . . . . 914.4 A generalized Dembo approach . . . . . . . . . . . . . . . . . 994.5 A path to break the square-root barrier using Dembo bounds 1075 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . 116Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118viiList of Figures1.1 Error in approximation using first order and second order \u2303\u0000quantization versus number of measurements m. The graphsare plotted in log-log scale and they are compared with theupper bounds on the error, namely, f(m) = C\/pm, andg(m) = D\/pm3 (the graphs of f(m) = 1pmand g(m) = 1mpmare multiplied with proper constants to shift them close to thenumerical graph) respectively. . . . . . . . . . . . . . . . . . 332.1 Error in approximation using first order and second order \u2303\u0000quantization with one-stage reconstruction scheme and with a\u201cmodified\" random partial Fourier matrix for 10-sparse signalsand the comparison with the graphs of f(m) = Cm1\/2andg(m) = Dm3\/2in log-log scale. . . . . . . . . . . . . . . . . . . 482.2 Quantizing the signal x by first using MSQ with a very smallstep size \u00000, then applying U which is a fast transform followedby a \u2303\u0000 quantization scheme. . . . . . . . . . . . . . . . . . . 482.3 Error in approximation using first order and second order \u2303\u0000quantization with one-stage reconstruction scheme and withan extra MSQ step (before applying the matrix U on the mea-surement vector). . . . . . . . . . . . . . . . . . . . . . . . . 532.4 Error in approximation using first order and second order \u2303\u0000quantization with one-stage reconstruction scheme for a 4-sparse signal and the comparison with the graphs of f(p) =Cpp and g(p) =Dp3\/2(each one shifted properly to match theoriginal graphs) in log-log scale. . . . . . . . . . . . . . . . . 622.5 Error in approximation using first order and second order \u2303\u0000quantization with one-stage reconstruction scheme with fixednumber of measurements (p = 541) and the comparison withthe graphs of f(k0) = 1pk0and g(k0) = 1pk03. . . . . . . . . . . 633.1 The binary matrix given by the new construction for p = 997. 80viiiList of Figures3.2 Coherence of the matrices introduced in this chapter, and alsochirp sensing matrices in log-log scale, accompanied with bestfitted lines. As seen in this Figure, the coherence of our con-struction behaves as \u21e0 m\u00001\/3, and for chirp sensing matricesbehaves as \u21e0 p\u00001\/2 = m\u00001\/2. . . . . . . . . . . . . . . . . . . 803.3 The fraction of exactly recovered vectors versus sparsity for afixed number of measurements. The number of measurementsis chosen as m = 256 for the Reed-Muller matrix, m = 178for the new construction and also for the Bernoulli matrix,and m = 169 for the DeVore\u2019s construction. The ambientdimension of all signals is n = 300 . . . . . . . . . . . . . . . 813.4 The graph of fraction of exactly recovered vectors (for 10 ex-periments) versus prime number p for a fixed level of sparsity(k = 10 or 20) for the new construction and the Bernoullimatrices. Note that only three graphs are shown because thegraphs corresponding to k = 20 for the new construction andrandom Bernoulli exactly overlap with each other. This sug-gests that our proposed deterministic construction has a verysimilar performance to random Bernoulli. . . . . . . . . . . . 823.5 The maximum sparsity level of recoverable signals g(p) versusthe prime number p compared with the graph of f(p) = p3\/4. 824.1 The graph of lower bound of the RIP constants, comparedwith the Gershgorin bound and the new improved bound, asgiven in Section 4.2, on the RIP constants. . . . . . . . . . . 874.2 Comparison of lower bounds of the RIP constants obtainedfrom a single random support set and using the worst caseamong 1000 random support sets. We observe that the slopeof the graph (in log-log scale) obtained from a single supportset almost remains constant, as we increase the number ofsupport sets from 1 to 1000. . . . . . . . . . . . . . . . . . . 884.3 Comparing the sharpness of Dembo bounds and Gershgorinbounds for estimating the maximum eigenvalue of a semi-positive definite Hermitian matrix by considering the ratioof these bounds over the actual maximum eigenvalues for afixed Paley matrix with p = 103. . . . . . . . . . . . . . . . . 93ixAcknowledgementsFirst and foremost, I express my most sincere gratitude to my supervisorProfessor Ozgur Yilmaz. I learned so many things from him during the yearsof completing my Ph.D. degree. In addition to learning a lot about topicsrelated to my research area, I learned how to enjoy beauty of Mathematicsmore deeply by tackling the topics and problems using a \u201cbroad perspective\".I am deeply thankful to him because of all the insightful ideas he gave meand because of all the time he spent for me to help me develop those ideas.I would like to thank the supervisory committee for their time, and theiradvice that always guided me through this journey.Lastly, I would like to thank my family who have always given me loveand support, and helped me a lot emotionally during the time of writing thisdissertation.xChapter 1IntroductionThis thesis focuses on several problems in compressed sensing (CS)\u2013 a novelsignal acquisition paradigm that relies upon a new sampling theory for\u201csparse signals\". One of the main problems in compressed sensing is the de-sign of appropriate measurement matrices with (nearly) optimal dimensionrelations. While nearly optimal measurement matrices are ubiquitous (anyrandom matrix with i.i.d. Gaussian entries will do with high probability), noexplicit deterministic construction is known that produces (nearly) optimalmeasurement matrices. In this thesis, we address several problems relatedto CS with deterministic matrices including a novel construction, quantiza-tion, and improved bounds on the RIP constants of certain deterministicconstructions.In this chapter, we give an overview of the theory of CS, deterministicconstructions in CS, and quantization in CS.1.1 Compressed sensing\u2013 a new samplingparadigm1.1.1 Basic ideaOne of the key ingredients of today\u2019s digital technology is our ability toacquire, store, process, and transmit signals digitally. The signals of interestcan be images, audio, video, text, etc., which can be modelled as finite (buttypically high) dimensional vectors. Furthermore, signals often possess low-dimensional structure that can be exploited to accomplish the various taskswe listed above.Compressed sensing relies on such a structural property that variousclasses of signals enjoy: often signals are sparse or compressible with respectto some basis (i.e., transform). To be more precise, a signal x 2 Rn is k-sparseif it has at most k non-zero entries. The support of x = (x1, x2, \u00b7 \u00b7 \u00b7 , xn) isthe set of indices j such that xj 6= 0, and is denoted by supp(x). We alsodefine kxk0 = |supp(x)|, and \u2303nk := {x : kxk0 \uf8ff k} denotes the set of allk-sparse vectors.11.1. Compressed sensing\u2013 a new sampling paradigmModelling a given signal class, say the collection of all natural images, as\u2303nk , for k \u2327 n, is naive at best. However, we can generalize this approachby allowing two additional relaxations on the requirement of sparsity:(i) Signals may not often satisfy the sparsity assumption, but may admitsparse representations with respect to a known basis. More precisely,a signal x is said to be sparse with respect to a basis B (here B is aunitary matrix, i.e., its columns form an orthonormal basis for Rn) ifx = Bu where u 2 Rn is sparse.(ii) A signal x may not be sparse (with respect to a basis B) but compress-ible. This means that there is a u 2 \u2303nk such that kx \u0000 Buk is smallin some norm. This would be the case if, e.g., the sorted entries of udecay rapidly.Once we make these more relaxed assumptions, we obtain a realisticmodel that is indeed encountered in many practical scenarios. For example,natural images are compressible with respect to wavelets and audio signalswith respect to Fourier. For various other classes of signals, one needs torelax the requirement in the condition (i) (mentioned above) that B is abasis and work with frames instead. But for our purposes in this thesis, wewill assume both (i) and (ii) hold. Often, we will also make the assumptionthat B = I, and work directly with sparse vectors in \u2303nk .Next, we describe the main idea behind CS. Suppose x 2 \u2303nk with k \u2327 n.A generalized sample of x is given by its inner product with a measurementvector \u0000i 2 Rn. In CS, we pick m \u2327 n non-adaptive measurement vec-tors \u00001, ...\u0000m 2 Rn (which we now assume are row vectors) and collect themeasurementsyj = h\u0000j , xi, j = 1, 2, ....,mor equivalently, setting y = [y1, \u00b7 \u00b7 \u00b7 , ym]T ,y = \u0000xwhere \u0000 is the m \u21e5 n matrix whose jth row is \u0000j . The measurementsyj collected this way are called compressed measurements or compressiblesamples of x. The m \u21e5 n matrix \u0000 where m \u2327 n, is the (compressed)measurement matrix. Next, we summarize the important features that aproper CS measurement matrix must satisfy in order to guarantee that aconvex minimization algorithm can be used for the recovery of sparse orcompressible signals.21.1. Compressed sensing\u2013 a new sampling paradigm1.1.2 Measurement matrices in compressed sensingOne of the main goals in CS is to design measurement matrices that preservethe information that identifies the original signal x. One of the main toolsfor assessing if a matrix is an appropriate CS measurement matrix is theso-called Restricted Isometry Property (RIP).Definition 1. A matrix \u0000 is said to satisfy RIP of order k if there exists aconstant \u0000 2 (0, 1) satisfying(1\u0000 \u0000)kxk22 \uf8ff k\u0000xk22 \uf8ff (1 + \u0000)kxk22 (1.1)for every k-sparse vector x. The RIP constant of order k for a matrix \u0000,denoted by \u0000k, is defined as the minimum over all numbers \u0000 satisfying (1.1).One of the important implications of RIP is the uniqueness property in thefollowing sense.Lemma 1. Let \u0000 2 Rm\u21e5n be a matrix satisfying RIP of order 2k. Let xand y be k-sparse vectors and assume that \u0000x = \u0000y, then x = y.Proof. Let z = x\u0000y. Since x and y are k-sparse vectors, each of them have atmost k non-zero entries. Thus, z has at most 2k non-zero entries which meansthat z is a 2k-sparse vector. On the other hand, we can use the assumption\u0000x = \u0000y to conclude \u0000z = 0. Therefore by (1.1), (1 \u0000 \u00002k)kzk22 = 0 andsince 1\u0000 \u00002k > 0 this implies z = 0, i.e., x = y as desired.Based on Lemma 1 that guarantees injectivity of \u0000 on \u2303nk , the theorembelow gives an algorithm to reconstruct the original k-sparse signal from themeasurement vector.Theorem 1. Let \u0000 be a matrix satisfying RIP of order 2k. Suppose that xis a k-sparse vector and let y = \u0000x be the measurement vector. If x\u02c6 is thereconstructed vector for x defined byx\u02c6 = argmin kzk0 such that y = \u0000z, (1.2)then x\u02c6 = x.Proof. Since x is a k-sparse vector satisfying y = \u0000x, we can conclude thatany solution to (1.2) is also k-sparse. Therefore, by Lemma 1, the solutionto (1.2) is unique, and it must be equal to x, i.e., x\u02c6 = x.31.1. Compressed sensing\u2013 a new sampling paradigmThe reconstruction scheme given in Theorem 1 is in fact intractable and isNP-hard [59]. On the other hand, the convex relaxation of this algorithmwhich is computationally efficient, also reconstructs the original signal per-fectly in the absence of noise. In their 2006 breakthrough article, Cand\u00e8s,Romberg, and Tao [20] proved the following theorem.Theorem 2. [20] Let \u0000 be a matrix satisfying RIP of order 4k, with theproperty that \u00003k + 3\u00004k < 2. Let x be a k-sparse vector and let y = \u0000x bethe measurement vector. If x\u02c6 is the reconstruction vector for x defined byx\u02c6 = argmin kzk1 such that y = \u0000z,then x\u02c6 = x.This theorem is of extreme importance since we can say based on it, thefield of CS was born. We should mention, however, that parallel to Cand\u00e8set al. [20] in 2006, Donoho [30] also published a breakthrough article in thesame year that can also be considered as the starting point of CS.The condition given in this theorem, i.e., \u00003k + 3\u00004k < 2 was improvedgradually between 2006 and 2014. In 2014, Cai et al. improved this conditionto \u00002k < 1\/p2, and more importantly they proved that this condition isoptimal and can not be improved further. In particular, they proved thefollowing theorem.Theorem 3. [18] Assume that the conditions of Theorem 1 hold and that\u00002k <1p2\u21e1 0.7071. Let x\u02c6 be the reconstruction vector obtained by BasisPursuit (BP) algorithm given byx\u02c6 = argmin kzk1 such that y = \u0000z, (1.3)then x\u02c6 = x.Note that, as shown in [18], the condition of this theorem can be gener-alized to \u0000tk 0. Inthis case, the vector x can be well approximated with the solution to BasisPursuit Denoising (BPDN) algorithm defined asx\u02c6 = argmin kzk1 subject to ky \u0000 \u0000zk2 \uf8ff \u270f (1.5)where \u270f is an upper bound on the size of the noisy contribution. Moreprecisely, we have the following theorem, due to Cand\u00e8s, Romberg, and Tao.Below, xk := argminu2\u2303nk kx\u0000uk denotes the best k-term approximationof x, and can be obtained by keeping the largest (in absolute value) k entriesof x, and replacing the rest be zeros. The error in approximation (in `p norm)using the best k-term approximation is denoted by \u0000k(x)p, and is defined as\u0000k(x)p := kx\u0000 xkkp (1.6)Theorem 4. [20] Assume that \u00003k + 3\u00004k < 2 and kwk2 \uf8ff \u270f. Then, thesolution to (1.5) obeyskx\u0000 x\u02c6k2 \uf8ff D0pk\u0000k(x)1 +D1\u270f (1.7)for some universal constants D0 and D1 that depend only on the value of\u00004k.Similar to what we explained for the noise-free case, the condition \u00003k+3\u00004k <2 was later improved to the optimized condition \u00002k < 1\/p2 [18] in thefollowing sense.Theorem 5. [18] Assume that \u00002k < 1p2 and kwk2 \uf8ff \u270f. Then, the solutionto (1.5) obeyskx\u0000 x\u02c6k2 \uf8ff C0pk\u0000k(x)1 + C1\u270f (1.8)for some universal constants C0 and C1 which depend only on the value of\u00002k.Remark 1. Explicit formulas for the constants C0 and C1 mentioned inTheorem 5 were derived in [18]. For the sake of completeness, we mentionthem here:C0 = 2\u21e3p2\u00002k +q2( 1p2 \u0000 \u00002k)\u00002k2( 1p2\u0000 \u00002k)+ 1\u2318,andC1 =2p2(1 + \u00002k)1\u0000p2\u00002k.51.1. Compressed sensing\u2013 a new sampling paradigmRemark 2. The Basis Pursuit and Basis Pursuit Denoising algorithms de-fined in (1.3) and (1.5) are indeed two of the most important algorithms inCS, but they are not the only ones. In fact, finding a fast algorithm for re-construction is one of the two major problems in CS (the other problem isfinding an appropriate measurement matrix). The major algorithms in CScan be categorized into two groups. The first class is convex optimization al-gorithms; BP and BPDN are important examples in this group. The secondclass is greedy algorithms, and important algorithms of this group includeOrthogonal Matching Pursuit (OMP), Compressive Sampling Matching Pur-suit (CoSaMP), and Iterative Hard Thresholding (IHT). Note that this thesisfocuses on the design and analysis of CS measurement matrices, and we willnot go through different types of algorithms in detail.1.1.3 Restricted Isometry Property (RIP)Let x 2 Rn be a given signal, and let \u0000 be an m \u21e5 n measurement matrix.In this section, we will review an important approach to evaluate the RIPconstants that can be exploited directly from the matrix and without con-sidering k-sparse signals. Let T be a subset of {1, 2, 3, ..., n} with k elements.By xT , we mean the vector in Rk obtained by restricting x to the entriesindexed by the elements of T . Similarly, by \u0000T we mean the m\u21e5 |T | matrixdefined by the restriction of \u0000 to the columns indexed by the elements ofthe set T . Define \u0000kmin and \u0000kmax as follows.\u0000kmax = maxT :|T |\uf8ffk\u0000max(\u0000\u21e4T\u0000T ),\u0000kmin = minT :|T |\uf8ffk\u0000min(\u0000\u21e4T\u0000T ),(1.9)where \u0000max and \u0000min denote the maximum and minimum eigenvalue of amatrix respectively, and G := \u0000\u21e4T\u0000T is called the Gramian matrix corre-sponding to \u0000T .Proposition 1. For every matrix \u0000, the RIP constant \u0000k of the matrix canbe computed by the following formula\u0000k = max{1\u0000 \u0000kmin,\u0000kmax \u0000 1} (1.10)Proof. Let T0 to be the index set for which k\u0000\u21e4T0\u0000T0\u0000Ik2 becomes maximum,i.e., k\u0000\u21e4T0\u0000T0 \u0000 Ik2 = maxT :|T |\uf8ffk k\u0000\u21e4T\u0000T \u0000 Ik2. We know that the singularvalues of a Hermitian matrix are the absolute values of eigenvalues of thesame matrix. Hence, to obtain k\u0000\u21e4T0\u0000T0 \u0000 Ik2 (the maximum singular value61.1. Compressed sensing\u2013 a new sampling paradigmof (\u0000\u21e4T0\u0000T0 \u0000 I)), we should simply consider |\u00001|, where \u00001 is the eigenvalueof this matrix with the largest absolute value. Let v1 2 Ck be the eigenvectorof this matrix corresponding to \u00001, then(\u0000\u21e4T0\u0000T0 \u0000 I)v1 = \u00001v1 = \u00b1k\u0000\u21e4T0\u0000T0 \u0000 Ik2v1Therefore,k\u0000T0v1k22 \u0000 kv1k22 = h\u0000\u21e4T0\u0000T0v1, v1i \u0000 hv1, v1i= h(\u0000\u21e4T0\u0000T0 \u0000 I)v1, v1i = \u00b1k\u0000\u21e4T0\u0000T0 \u0000 Ik2kv1k22(1.11)So(1\u0000 k\u0000\u21e4T0\u0000T0 \u0000 Ik2))kv1k22 \uf8ff k\u0000T0v1k22 \uf8ff (1 + k\u0000\u21e4T0\u0000T0 \u0000 Ik2)kv1k22which implies(1\u0000 k\u0000\u21e4T0\u0000T0 \u0000 Ik2))kw1k22 \uf8ff k\u0000w1k22 \uf8ff (1 + k\u0000\u21e4T0\u0000T0 \u0000 Ik2)kw1k22where w1 2 Cn is a k-sparse vector obtained by padding zeros to v1, i.e.,v1 = (w1)T0 . Accordingly, \u0000k \uf8ff k\u0000\u21e4T0\u0000T0 \u0000 Ik2 = maxT :|T |\uf8ffk k\u0000\u21e4T\u0000T \u0000 ISk2.On the other hand, according to (1.1), we have|k\u0000w1k22 \u0000 kw1k22| = |k\u0000T0v1k22 \u0000 kv1k22| \uf8ff \u0000kkv1k22Thus,|k\u0000T0v1k22 \u0000 kv1k22|kv1k22\uf8ff \u0000kSo by using (1.11), we havek\u0000\u21e4T0\u0000T0 \u0000 Ik2 = maxT :|T |\uf8ffk k\u0000\u21e4T\u0000T \u0000 Ik2 \uf8ff \u0000kTherefore,\u0000k = maxT :|T |\uf8ffkk\u0000\u21e4T\u0000T \u0000 ISk2,which implies (1.10).Note that verifying whether a given matrix is RIP with a small constant ornot is intractable because we need to calculate the minimum and maximumeigenvalues of\u0000nk\u0000matrices. This problem is circumvented by using randommatrices.71.1. Compressed sensing\u2013 a new sampling paradigm1.1.4 RIP for random matricesConsider a matrix \u0000 2 Rm\u21e5n with the property that each entry of \u0000 is arandom variable. Such a matrix \u0000 is called a random matrix. As stated in[37], the random matrices were initially introduced in the context of Statisticsby Wishart and in Mathematical Physics by Wigner.Definition 2. Let \u0000 be an m\u21e5 n random matrix.(i) The matrix \u0000 is called a Bernoulli random matrix if the entries of \u0000are i.i.d. Rademacher random variables, i.e., random variables taking1 or -1, each with probability p = 1\/2.(ii) The matrix \u0000 is called a Gaussian random matrix if the entries of \u0000are i.i.d. standard Gaussian random variables.(iii) The matrix \u0000 is called a sub-Gaussian random matrix if the entries of \u0000are independent (not necessarily identically distributed) sub-Gaussianrandom variables with zero mean and variance one.The random variable X is defined to be a sub-Gaussian random variablewith variance proxy \u0000 denoted by X \u21e0 subG(\u00002) if E(X) = 0, and if themoment-generating function of X satisfiesmX(t) = E(etX) \uf8ff et2\u00002\/2for any t 2 R. The name sub-Gaussian should becomes clear from this defi-nition as et2\u00002\/2 is the moment generating function of the Gaussian randomvariable with mean zero and variance \u00002. Note that unlike the ordinaryconvention in probability theory, the sub-Gaussian random variable with agiven variance proxy determines infinitely many distributions. In fact, if Xis a sub-Gaussian random variable with variance proxy \u00002, then it is also asub-Gaussian random variable with variance proxy \u00002 with any \u0000 \u0000 \u0000.Remark 3. It is obvious that a Gaussian random variable is a sub-Gaussianrandom variable. Also, if X is a Rademacher random variable, then themoment generating function of X would bemX(t) =et + e\u0000t2= cosh t \uf8ff et2\/2and therefore, X is sub-Gaussian. Thus, a random matrix whose some ofits entries are standard Gaussian and some of its entries are Rademacherrandom variables is considered to be a sub-Gaussian random matrix. (notethat as stated in Definition 2, the entries of a sub-Gaussian matrix do nothave to be identically distributed).81.1. Compressed sensing\u2013 a new sampling paradigmStudying random matrices and their properties have been of a great inter-est in the past few years because of interesting properties of these matrices.For example, if A is a matrix, one could always ask about finding or estimat-ing the extreme singular value of A. The reason for interest in these quan-tities is that the maximum singular value i.e., \u0000max(A) is same as operatornorm of A, and the condition number of A, i.e., cond(A) = \u0000max(A)\/\u0000min(A)measures how far the matrix is from being an isometry. The closer to onethis number is, the closer the matrix is to be an isometry. In general, an-swering these question is difficult, but important results have been derivedin the case that A is a random matrix, e.g., [23, 55, 70, 74].In the context of CS, the sub-Gaussian random matrices satisfy the RIPwith high probability, if the sparsity level k is slightly less than m. Specifi-cally, the following theorem holds.Theorem 6. [37] Let \u0000 be an m\u21e5n sub-Gaussian random matrix. Then forany 0 < \u0000 < 1 there exists a constant C > 0 (depending only on sub-Gaussianvariance proxy) such that restricted isometry constant of 1pm\u0000 satisfies \u0000k \uf8ff\u0000 with probability at least 1 \u0000 2 exp (\u0000\u00002m\/(2C)) provided that the numberof measurements satisfiesm \u0000 2C\u0000\u00002k log (en\/k) (1.12)Note that a factor 1pmis appeared in front of the measurement matrix inthe theorem above because if a matrix satisfies RIP, then it is expected to beclose to be an isometry. Hence we would expect the columns of the measure-ment matrix to be normalized to one. Since the entries of the measurementmatrix are random variables with variance one, we multiply the matrix withthe factor 1pmso that the columns become unit-norm in expectation.As we can see in (1.12), the bound in minimum number of measurementsm is just slightly larger than k. Recall that in practice the signal x livesin Rn where the ambient dimension n is a large number, while the sparsitylevel k is small for many classes of signals. Therefore, based on theoremabove, we can reconstruct these signals with high probability when we usesub-Gaussian random matrices as the measurement matrix with using onlymmeasurements satisfying (1.12), and this number of measurements is seem-ingly insufficient. It can be shown [36, 38] that the bound (1.12) is optimal.In particular, as we will discuss in the next section, the factor log (n\/k) cannot be improved.91.1. Compressed sensing\u2013 a new sampling paradigm1.1.5 Optimality of random matrices and Gelfand widthsHere, we consider the measurement matrix \u0000 2 Rm\u21e5n as an encoder (for thesignal x) and the algorithm that reconstructs x as a decoder and we denoteit by \u0000 : Rm ! Rn. For a set K \u2713 Rn, \u0000k(K)p is defined as follows.\u0000k(K)p := supx2K\u0000k(x)pwhere \u0000k(x)p is as defined in (1.6). Moreover, the worst reconstruction errorfor the best pair of encoder\/decoder is denoted by Em(K, `p), and is definedasEm(K, `p) := inf supx2Kkx\u0000\u0000(Ax)kpwhere the infimum is taken over all pairs of (A,\u0000) with A 2 Rm\u21e5n and\u0000 : Rm ! Rn. The bounds for Em(K, `p) depend on a quantity calledGelfand width of a set. To define the Gelfand widths, first we need to definethe codimension of a subspace of a normed space.Definition 3. If W is a subspace of a vector space V , then the codimensionof W is defined as the dimension of the quotient space V\/W which is givenbycodim(W ) = dim(V\/W ) = dimV \u0000 dimWDefinition 4. If K is a compact set in a normed space X, then the Gelfandwidth of K of order m, denoted by dm(K)X , dm(K,X), or simply dm(K) ifthe normed space X is clear, is defined asdm(K) := inf sup{kxk : x 2 K \\ Y }where the infimum is taken over all subsets Y of X with codim Y \uf8ff m.Proposition 2. [33] Let K \u2713 X = Rn (where X = Rn is equipped with `pnorm) be a compact set with K = \u0000K and K+K \u2713 C0K for some constantC0. Then,dm(K) \uf8ff Em(K, `p) \uf8ff C0dm(K)Now, assume that Bnp denotes the unit ball respect to the `p norm, i.e.,Bnp := {x 2 Rn : kxkp \uf8ff 1}Also, suppose that the vector space X = Rn is equipped with `2 norm. Thenext theorem provides bounds for dm(Bnp ).101.2. Deterministic constructions in compressed sensingTheorem 7. [36] Let 0 < p \uf8ff 1. There exist universal constants Cp andDp > 0 such that the Gelfand width of dm(Bnp ) satisfiesCpmin{1, log(en\/m)m}1\/p\u00001\/2 \uf8ff dm(Bnp ) \uf8ff Dpmin{1,log(en\/m)m}1\/p\u00001\/2Now, if we set p = 1, for large enough m we will obtainC1rlog(en\/m)m\uf8ff dm(Bn1 ) \uf8ff D1rlog(en\/m)mTherefore, by setting K = Bn1 in Proposition 2, we obtainC1rlog(en\/m)m\uf8ff dm(Bn1 ) \uf8ff Em(Bn1 , `2) \uf8ff 2dm(Bn1 ) \uf8ff 2D1rlog(en\/m)m(1.13)where we used the fact that Bn1 +Bn1 \u2713 2Bn1 , and hence, we can use C0 = 2.Next, assume that \u0000 2 Rm\u21e5n satisfies RIP with \u00002k < 1p2 , then for x 2 Bn1we havekx\u0000\u0000(Ax)k2 \uf8ff C0\u0000k(x)1pkwhere \u0000 : Rm ! Rn is the `1 minimization reconstruction algorithm. Then,since \u0000k(x) \uf8ff kxk1 \uf8ff 1, we will have kx \u0000 \u0000(\u0000x)k2 \uf8ff C0 1pk . Hence,Em(Bn1 , `2) \uf8ff C0pk . Combining this inequality with (1.13), we concludeC1rlog(en\/m)m\uf8ff Em(Bn1 , `2) \uf8ffC0pkand this implies m \u0000 C20C21k log(en\/m) which proves that the bound (1.12) isoptimal for the minimum number of measurements in order to have a matrixwith small enough RIP constant.1.2 Deterministic constructions in compressedsensingIn Section 1.1, we saw that random matrices are important examples ofRIP matrices. However, there are two major drawbacks for these matrices.First, storing the entries of a random matrix and performing matrix-vector111.2. Deterministic constructions in compressed sensingmultiplication require significant computational resources. However, this isnot the case for deterministic matrices, since as we will see below, most ofthese matrices are structured matrices and the full matrix does not have tobe stored. Second, even though sub-Gaussian matrices are RIP, with RIPconstants satisfying \u0000k \uf8ff s with high probability in the optimal regime m\u0000ks\u00002 log n, there is no tractable algorithm to check whether a given randommatrix is RIP or not in that regime, and the RIP verification is an NP-hardproblem in such a regime. This has been shown in [76] by Wang et al. andin particular, it has been shown that there is no polynomial time algorithmfor RIP verification\u2013even in an average case scenario\u2013when the parameterssatisfy m \u0000 k1+\u21b5s\u00002 log n, for any \u21b5 2 [0, 1). In fact, the polynomial timealgorithm exists only if the parameters satisfy m \u0000 k2s\u00002 log n. In thefollowing, we explain briefly what is meant by average case RIP verificationand we summarize the main results of [76].The NP-hardness mentioned above occurs when a \u201cdesigned\" set of ma-trices, e.g., a set of random matrices with i.i.d. entries, or a structured classof deterministic matrices is given and we want to know whether the RIPconstant of the matrix with largest RIP constant, i.e., the worst case sce-nario, satisfies \u0000k \uf8ff s (for a fixed value of s). However, it is useful to providea tool (called certifier) that determines whether the RIP constant of givenmatrices satisfies \u0000k \uf8ff s in the average case. To formulate this idea, for0 \uf8ff \u21b5 \uf8ff 1, define the set of R\u21b5 as the set containing all quadruples of thefollowing form.R\u21b5 := {(m,n, k, s) : m\u0000 k1+\u21b5s\u00002 log n}Also, let RIPm,n(k, s) denote the set of all m \u21e5 n matrices whose RIP con-stants of order k satisfies \u0000k \uf8ff s. With this notation, Theorem 6 can bestated as follows. Fix s 2 (0, 1), and let (m,n, k, s) 2 R0. In the asymptoticregime m,n, k !1, if X is a sub-Gaussian random matrix of the size m\u21e5n,then P(X 2 RIPm,n(k, s))! 1.Now, the question is for a given set of designed matrices, can we find a toolthat verifies whether most of the matrices in that set with the parameters(m,n, k, s) 2 R\u21b5 have the RIP constants satisfying \u0000k \uf8ff s. To do so, asequence of certifiers can be defined as follows.Definition 5. Given parameters m,n, k,s, and a set of of m \u21e5 n designedmatrices A, a certifier for A is defined as the sequence of measurable func-tions m : Rm\u21e5n ! {0, 1} such that(i) m(X) = 1 iff X 2 RIPm,n(k, s).121.2. Deterministic constructions in compressed sensing(ii) P( m(X) = 0, X 2 A) \uf8ff 1\/3 in the asymptotic regime m,n, k !1.Note that the certifier above depends on the parameters m,n, k, and sas well as the set A. In [76], the set A is considered to be the set of allm\u21e5n sub-Gaussian random matrices. It would be ideal to obtain a certifierthat is computationally tractable and works for the parameters in the setR\u21b5, with \u21b5 as small as possible. In Proposition 1, we saw that the RIPconstant of a matrix can be computed via (1.10). This provides a certifier m : Rm\u21e5n ! {0, 1} defined by m(X) =\u21e21, if maxT :|T |\uf8ffk kX\u21e4TXT \u0000 Ikk2\uf8ff s0, if maxT :|T |\uf8ffk k\u0000\u21e4T\u0000T \u0000 Ikk2 > sAlthough this certifier works for the class of sub-Gaussian matrices withthe parameters chosen from R0, but it is computationally intractable sincecomputing the operator norm mentioned above is NP-hard. However, if theparameters are chosen from R1 (specifically, for m \u0000 196\u00004k2s\u00002 log n), andif we set the set of design matrices to be the set of sub-Gaussian matrices,there is a polynomial time certifier m : Rm\u21e5n ! {0, 1} given in [37] whichcan be defined as follows. m(X) =8<: 1, if kX\u21e4X \u0000 Ink1 \uf8ff 14\u00002qlognm0, if kX\u21e4X \u0000 Ink1 > 14\u00002qlognmNow, the important question is whether a polynomial time certifier exists,when the parameters are chosen from the set R\u21b5 for \u21b5 2 [0, 1), and the setA is set to be the set of m \u21e5 n random matrices with i.i.d. entries with adistribution Q. It is shown in [76] that the answer to this question is \u201cno\"assuming that a version of planted clique hypothesis holds. In particular,they prove the following Theorem under the assumption mentioned above.Theorem 8. Fix \u21b5 2 [0, 1). Then, there exists parameters (m,n, k, s) 2 R\u21b5such that no certifier\/distribution couple ( , Q) (with being a polynomialtime certifier) exists for this sequence of parameters.These caveats of random matrices explained above leads us to consider\u201cdeterministic matrices\" as an alternative for measurement matrices. How-ever, note that if by doing so, we want to resolve the non-existence oftractable algorithm for RIP verification as explained above, we need to con-struct a class of deterministic measurement matrices that satisfy RIP andhave the parameters satisfying m \u0000 k1+\u21b5s\u00002 log n for \u21b5 2 [0, 1) (since as131.2. Deterministic constructions in compressed sensingmentioned above for m \u0000 k2s\u00002 log n, a polynomial time algorithm existsfor RIP verification of random matrices). This means that to solve this is-sue, we need to construct a class of deterministic matrices that breaks thesquare-root barrier, see Section 1.2.1. Next, we explain one of the main toolsused for deterministic CS matrices which is a property called coherence.1.2.1 CoherenceThe coherence of a measurement matrix \u0000 with unit norm columns is definedas\u00b5(\u0000) = max1\uf8ffi 6=j\uf8ffn|h\u0000i,\u0000ji|where \u0000i is the ith column of \u0000. A matrix can be used for measurementof sparse signals if the coherence of that matrix is small. More precisely, if(2k \u0000 1)\u00b5 < 1, there will be a reconstruction algorithm for the recovery ofevery k-sparse signal in k iterations [31].For any CS matrix, the coherence and the RIP constant can be related toeach other. These quantities are in fact related to each other via a theoremcalled Gershgorin circle theorem which provides bounds on the eigenvaluesof a matrix. This theorem can be stated as follows [8, 39].Theorem 9. Suppose that A = (aij) is an n\u21e5 n matrix. LetRi =nXj=1,j 6=i|aij |,then each eigenvalue of A is in at lease one of the disks{z : |z \u0000 aii| \uf8ff Ri}Proposition 3. Suppose \u0000 is a matrix with coherence \u00b5. Then \u0000 satisfiesthe RIP of order k with\u0000k \uf8ff \u00b5(k \u0000 1) (1.14)whenever k < 1\u00b5 + 1.Proof. Consider an index set T \u2713 {1, 2, ..., n} with |T | = k. Also, assumethat \u0000max,T and \u0000min,T are the maximum and minimum eigenvalues of \u0000\u21e4T\u0000Trespectively. Since all columns of \u0000 have unit norm, the diagonal elementsof \u0000\u21e4T\u0000T are all one. Hence, by using Gershgorin circle theorem we concludethat141.2. Deterministic constructions in compressed sensing|\u0000max,T \u0000 1| \uf8ff max1\uf8ffi\uf8ffkXj 6=i|h\u0000j |\u0000ii| \uf8ff \u00b5(k \u0000 1)and similarly|\u0000min,T \u0000 1| \uf8ff max1\uf8ffi\uf8ffkXj 6=i|h\u0000j |\u0000ii| \uf8ff \u00b5(k \u0000 1)Since the inequalities above are valid for any index set T , we conclude thatthe quantities \u0000kmax and \u0000kmin defined as \u0000kmax := maxT,|T |\uf8ffk \u0000max,T , and\u0000kmin := minT,|T |\uf8ffk \u0000min,T satisfy |\u0000kmax \u0000 1| \uf8ff \u00b5(k \u0000 1) and |\u0000kmin \u0000 1| \uf8ff\u00b5(k \u0000 1) respectively. Therefore,\u0000k = max{\u0000kmax \u0000 1, 1\u0000 \u0000kmin} \uf8ff \u00b5(k \u0000 1)Since computing the coherence of a deterministic matrix is rather a simpleproblem, we can use the proposition above to determine values of k for whichthe RIP constant \u0000k satisfies \u0000k < 1\/p2 which implies that we can use thosematrices as CS measurement matrices for reconstruction of k-sparse signals.Here, of course we prefer the value of k to be as large as possible which canbe achieved for matrices with small coherence. Unfortunately, there exists alower bound for the coherence of any matrix called the Welch bound [77].Theorem 10. (Welch Bound) Let \u0000 2 Rm\u21e5n and assume that columns of\u0000 have unit norm. Then\u00b5(\u0000) \u0000rn\u0000mm(n\u0000 1)Based on this theorem, we can derive an upper bound for the maximumsparsity level of signals in the case that coherence is used along with Propo-sition 3 to prove that a matrix is RIP with small enough isometry constant.To do that, we consider a matrix that satisfies the Welch bound, and in thatcase, in order to have \u00002k < 1\/p2, it is enough to have \u00b5(2k \u0000 1) < 1\/p2.This inequality will hold ifk . 1\/p22\u00b5=12p2rm(n\u0000 1)n\u0000mwhere the notation f . g means |f | \uf8ff C|g| for some universal constantC. In the context of CS, where n is a large number, and m \u2327 n, we have151.2. Deterministic constructions in compressed sensingn\u00001n\u0000m \u21e1 1, and hence, the maximum sparsity level of the signals satisfiesk = O(pm). This barrier on the sparsity of signals using deterministic CSis called the square-root barrier for deterministic CS, and is indeed muchworse than sub-Gaussian case, where as we saw, the maximum sparsity levelis given by k = O( mlog(nk )).Note that this bound is imposed on the maximum sparsity level by com-bining Welch bound with the bound on the RIP constant given by Propo-sition 3. While the Welch bound can not be improved, as we will see inChapter 4, the bounds on RIP constants obtained from Gershgorin circletheorem (Proposition 3) can be improved. This can suggest a novel waythat leads to breaking the square-root barrier.There have been several deterministic CS constructions in the literature.Most of these constructions rely on graph theory, coding theory, or polyno-mials. There are also some individual constructions. In the following, wesummarize some of the major deterministic constructions. In Chapter 3, wewill construct a novel deterministic CS and we will compare this construc-tion with the existing constructions and we will provide the advantages ofthis construction.1.2.2 Measurement matrices based on Reed-Muller codesLet P be a fixed s\u21e5s binary symmetric matrix and let a = (a0, a1, ..., as\u00001)Tand b = (b0, b1, ..., bs\u00001)T be arbitrary binary vectors in Zs2. A second-orderReed-Muller (RM) sequence of length 2s is obtained from these parametersby\u0000P,b(a) =(\u00001)w(b)p2si(2b+Pa)T a (1.15)where w(b) denotes the weight of b, i.e., the number of ones in b. In [41], thedeterministic measurement matrix \u0000RM is constructed as\u0000RM = [UP1 UP2 .... UP2s(s\u00001)\/2 ]where each UPj is a 2s\u21e52s orthogonal matrix whose columns are the second-order RM sequence obtained by fixing (any binary zero-diagonal symmetrics \u21e5 s matrix) Pj in (1.15) and letting b range through all possible binarys-vectors. Since there are 2s(s\u00001)\/2 possible s\u21e5 s binary zero-diagonal sym-metric matrices Pj , and each of them are 2s \u21e5 2s, the associated full RMmatrix has the dimension 2s \u21e5 2s(s+1)\/2.161.2. Deterministic constructions in compressed sensingThe inner products between columns of \u0000RM have been calculated in [53].If the first column, \u0000Pi,. is chosen from UPi and the second column, \u0000Pj,. ischosen from UPj then|h\u0000Pi,. ,\u0000Pj,.i| =\u21e21\/p2l, 2l times;0, 2s \u0000 2l times. (1.16)where l = rank(Pi \u0000 Pj). Note that for large values of s, the Welch Boundis given by \u00b5(\u0000RM ) =qn\u0000mm(n\u00001) ' 1pm = 1p2s where n = 2s(s+1)2 is theambient dimension and m = 2s is the number of measurements. Therefore,Reed-Muller matrices nearly reach the Welch bound. In [41], Howard et al.provide a fast reconstruction algorithm when Reed-Muller matrices are usedas measurement matrices. However, Ni et al. state in [61] that the algorithmgiven in [41] is not efficient enough for 2D signals such as medical images,and so they propose an algorithm that is more useful in this context. Intheir paper, they introduce a reconstruction algorithm when a Reed-Mullermatrix of the following form is used as the measurement matrix.\u0000 = [UP1 UP2 UP3 UP4 ]Here, the matrices P1, P2, P3, and P4 are s \u21e5 s zero-diagonal binary sym-metric matrices and the difference of any two of these matrices is full rank.1.2.3 DeVore\u2019s constructionOne of the most important deterministic class of matrices in CS is the classof binary matrices introduced by DeVore in [28]. The significance of thisconstruction is because it is a binary deterministic construction, and alsothere is an explicit and simple formula for entries of these matrices, i.e., it isnot recursive, existential, or dependent of other entities such as special typesof graphs. Moreover, unlike most deterministic constructions, this construc-tion has the property that for a fixed number of measurements, the ambientdimension can be chosen as large as we want which is an ideal feature inthe context of CS. In particular, Devore\u2019s construction provides a class ofp2 \u21e5 pr+1 matrices, where p is a prime number, and r \u0000 2 is an integer. Todefine these matrices, fix a postive integer r \u0000 2 and a polynomial q(x) inZp[x] with deg(q) = r. Here, Zp = {0, 1, 2, ..., p \u0000 1} is the field of integersmod p. For each i 2 Zp we know that q(i) 2 Zp. Consider the column of themeasurement matrix \u0000 as a p2 dimensional vector vq whose (p` + s + 1)-stentry (for `, s 2 Zp) is defined as 1\/pp if vq(`) = s and 0 otherwise.171.2. Deterministic constructions in compressed sensingSince there are pr+1 polynomials of degree r, this matrix is of the size p2 \u21e5pr+1. If vq and vs are two different columns of \u0000, then vq(j)vs(j) = 0 unlessq(j) = s(j), i.e. (q \u0000 s)(j) = 0. Note that deg(q \u0000 s) \uf8ff r, thus, there are atmost r elements j with this property. Therefore,|hvq, vsi| \uf8ff rpAccordingly, by Proposition 3, RIP constant of these matrices satisfies\u0000k \uf8ff (k \u0000 1)rpfor k < p\/r + 1. A generalization of DeVore\u2019s construction is given in [51],and includes evaluation of functions at the points on algebraic curves. InChapter 3, we will also introduce a binary deterministic construction whichhas the advantage of being a partial circulant matrix and hence, we can do afast matrix-vector multiplication or fast reconstruction using these matrices.1.2.4 Chirp sensing measurement matricesThis is a class of deterministic CS matrices, first introduced by Applebaumet al. [2]. There are different aspects for significance of these matrices. Oneaspect is that each entry (before normalization) is a root of unity with anexplicit formula and as we will see these matrices almost reach the Welchbound. Another aspect lies in the fact that there is a specific fast recon-struction algorithm for these matrices. The standard recovery algorithmsin CS are the basis pursuit (BP) which has the computational complexityof O(n3), and the matching pursuit which has the computational algorithmof O(kmn). However, Applebaum et al. [2] suggested a new algorithm forchirp sensing matrices that includes fast Fourier transform (FFT) and has acomputational complexity of O(mk2 log k). Another aspect is that Bourgainet al.\u2019s deterministic construction [16], which is the only construction knownso far that breaks the square-root barrier for deterministic matrices is in facta submatrix of chirp sensing matrices. The definition and properties of chirpsensing matrices is related to Gauss sums as defined as follows.Definition 6. Let p be a prime number, and let ! be a primitive pth root ofunity. The quadratic Gauss sum mod p is defined asGp :=p\u00001Xn=0!n2181.2. Deterministic constructions in compressed sensingThe exact value of quadratic Gauss sums which were originally computed byGauss, are given by the following formula [43].Gp =\u21e2p1\/2 if p \u2318 1 mod 4 ;ip1\/2 if p \u2318 3 mod 4 ; (1.17)where i =p\u00001. Applebaum et al. [2] used this fact to construct the classof chirp sensing matrices, of the size p \u21e5 p2. To define these matrices, let0 \uf8ff r \uf8ff p\u0000 1 and 0 \uf8ff m \uf8ff p\u0000 1 be two arbitrary integers in Zp. Define thecolumns of \u0000 as follows\u0000(r,m) = \u0000rp+m+1 =1pp26666664!r\u00b702+m\u00b70!r\u00b712+m\u00b71...!r\u00b7(p\u00001)2+m\u00b7(p\u00001)37777775 (1.18)Here, \u0000j denotes the jth column of \u0000. As r and m range between 0 andp\u0000 1 they generate p2 columns of \u0000. By this definition the (`, j)th entry ofthe matrix \u0000 is defined as \u0000`,j = !r`2+m` with rp+m+ 1 = j. The p\u21e5 p2matrices that are constructed this way are called chirp sensing matrices.Next, in order to compute the coherence of these matrices, one needs ageneralization of the Gauss sums (1.17) in the following sense.Proposition 4. Let p be a prime number, ! be the primitive pth root ofunity, and r1, r2 2 Zp be two distinct numbers. Then,p\u00001Xx=0!(r1\u0000r2)x2=8<:\u21e3r1\u0000r2p\u2318p1\/2 if p \u2318 1 mod 4 ;\u21e3r1\u0000r2p\u2318ip1\/2 if p \u2318 3 mod 4 ;(1.19)Here,\u21e3ap\u2318denotes the Legendre symbol which is equal to 1 if r1\u0000 r2 is aquadratic residue, and is equal to -1 if r1 \u0000 r2 is a quadratic non-residue.Proof. Case (I) : r1\u0000r2 is a quadratic residue. In this case, the sum in (1.19)is the sum of all quadratic residues whose value was given in (1.17).Case (II) : r1 \u0000 r2 is a quadratic non-residue. Note that if a and b are aquadratic residue and quadratic non-residue respectively, thenp\u00001Xx=0!ax2+p\u00001Xx=0!bx2= 2\u21e31 + ! + !2 + ...!p\u00001\u2318= 0191.2. Deterministic constructions in compressed sensingHere, we used the fact that for any 0 6= j 2 Zp, this number is either aquadratic residue in which case, !j will be appeared twice in the first sumabove, or is a quadratic non-residue in which case it will be appeared twicein the second sum above. Therefore, if r1 \u0000 r2 is a quadratic non-residue,thenp\u00001Xx=0!(r1\u0000r2)x2= \u0000p\u00001Xx=0!x2as desired.Now, the coherence of these matrices can be computed easily since as thefollowing Proposition suggests, there is only two possibility for the magnitudeof inner product of two distinct columns of these matrices.Proposition 5. Suppose \u0000 is a p \u21e5 p2 chirp sensing matrix. If \u0000j and \u0000kare two distinct columns of \u0000, then h\u0000j ,\u0000ki = 0 or |h\u0000j ,\u0000ki| = 1pp . Inparticular, \u00b5 = 1pp .Proof. The norm of inner product of \u0000j and \u0000k can be written as :|h\u0000j ,\u0000ki| = 1p\u0000\u0000\u0000 p\u00001X`=0!(r1\u0000r2)`2+(m1\u0000m2)`\u0000\u0000\u0000where r1, m1 correspond to the jth column and r2, m2 correspond to thekth column respectively. If r1 = r2 and m1 6= m2, then h\u0000j ,\u0000ki = 0 sincefor any prime p, the sum of all pth roots of unity is zero. If r1 6= r2, then|h\u0000j ,\u0000ki| = 1p\u0000\u0000\u0000 p\u00001X`=0!(r1\u0000r2)`2+(m1\u0000m2)`\u0000\u0000\u0000=1p\u0000\u0000\u0000!\u0000(r1\u0000r2)\u21b52 p\u00001X`=0!(r1\u0000r2)(`+\u21b5)2\u0000\u0000\u0000=1p\u0000\u0000\u0000 p\u00001X`=0!(r1\u0000r2)(`+\u21b5)2\u0000\u0000\u0000 = 1p\u0000\u0000\u0000 p\u00001X`=0!(r1\u0000r2)`2\u0000\u0000\u0000 = 1ppwhere \u21b5 2 Zp is the solution to equation 2(r1\u0000r2)x = m1\u0000m2 mod p, andwhere we used Proposition 4. Hence, for a chirp sensing matrix,201.2. Deterministic constructions in compressed sensing\u00b5 = max1\uf8ffj 6=k\uf8ffn|h\u0000j ,\u0000ki| = 1ppNote that the Welch bound for chirp sensing matrices is given byrn\u0000mm(n\u0000 1) =sp2 \u0000 pp(p2 \u0000 1)which is almost same as 1pp for large enough p, and is same as coherence ofthese matrices.A generalization of chirp sensing codes was given by Nelson et al. [60].They constructed a class of p \u21e5 pr CS matrices, where r is an integer withr \u0000 2. Their construction is based on Weil Theorem which as stated asfollows is a generalization for quadratic Gauss sum formula (1.17).Theorem 11. Let r \u0000 2 be an integer and let p > r be a prime number. Leta := (a1, a2, ..., ar), where each aj is an integer in Zp, and letF (a, u) := arur + ...+ a1uThen for a 6= (0, ..., 0), we have|S(a)| \uf8ff (r \u0000 1)ppwhere S(a) =Ppu=1 e2\u21e1iF (a,u)p .Using this theorem, Nelson et al. defined a class of p\u21e5pr matrices (similar towhat we saw for defining the p\u21e5p2 chirp sensing matrices), and the coherenceof matrices in their construction satisfies \u00b5 \uf8ff r\u00001pp .The last construction we review in this chapter, is the deterministic con-struction given by Bourgain et al. [16] which is a class of submatrices of chirpsensing matrices and breaks the square-root barrier although by a small im-provement, i.e., m1\/2+\u270f for a very small number \u270f.1.2.5 The construction of Bourgain et al.In [15, 16], Bourgain et al. give an explicit construction of deterministicmatrices of the size p\u21e5 p1+\u270f (for a small number \u270f) that breaks the square-root barrier. Below, we give a brief overview of the construction given in211.2. Deterministic constructions in compressed sensing[16]. We also refer to [56] for a detailed discussion and review of [16] andimportant extensions.Fix a prime number p, and an even number m \u0000 100. Let ! = e2\u21e1i\/p,and let Zp be the set of integers mod p as usual. Set\u21b5 :=12m, \u0000 :=12.01m, r := b\u0000 log plog 2c, M := b22.01m\u00001cand define the sets A and B as follows.A := {1, 2, ..., bp\u21b5c}, B :=n rXj=1xj(2M)j\u00001 : x1, ..., xr 2 {0, 1, ...,M\u00001}oNext, define the p \u21e5 n measurement matrix \u0000\u02dc to be the submatrix of chirpsensing matrix \u0000 as follows.ua,b =1pp(!ax2+bx)x2Zp (1.20)where a 2 A , and b 2 B. The number of columns of \u0000\u02dc, n, is clearly |A ||B|.Now, the number of elements of B can be counted in the following way.|B| \u21e3M r \u21e3 (2 1\u0000\u00001)\u0000 log2 p = 2log2 p2log2 p\u0000\u0000 = p.p\u0000\u0000 = p1\u0000\u0000where we used Hardy notation a \u21e3 b, i.e., a \u21e3 b if there are positive constantsC1 and C2 such that C1b < a < C2b. Hence,n = |A ||B| \u21e3 p\u21b5p1\u0000\u0000 = p1+ 1402mIt turns out that the matrix \u0000 as defined above breaks the square-root bar-rier. The proof is based on the following lemma.Lemma 2. [16] Let k \u0000 210 and s be a positive integer. Assume that thecoherence parameter of the matrix \u0000 is \u00b5 \uf8ff 1\/k. Also, assume that for some\u0000 \u0000 0 and any disjoint J1, J2 \u2713 {1, 2, ..., n} with |J1| \uf8ff k, |J2| \uf8ff k we haveDXj2J1uj ,Xj2J2ujE\uf8ff \u0000k.Then, \u0000 = [u1 u2 \u00b7 \u00b7 \u00b7 un] satisfies the RIP of order 2sk with constant44sp\u0000 log k.221.2. Deterministic constructions in compressed sensingNext, it is shown in [16] that for sufficiently large enough p, the matrix \u0000\u02dcas defined in (1.20) satisfies the condition of Lemma 2 with k = bppc, and\u0000 = O(p\u0000\u270f1(log p)2). Here, \u270f1 is a constant defined as\u270f1 :=c0\u00008 \u0000 47\u21b5\u000023\u00002m1 + 93\/m+ c0\/2where c0 is a constant that originates from additive combinatorics and isestimated as c0 = 110430 [17] (though this estimate is not necessarily sharp),\u0000 is a constant that can be set to \u0000 := 141.4m . This reduces the expressionabove for \u270f1 to the following expression.\u270f1 =c0m \u0000 3799.6m2331.2(1 + 93\/m+ c0\/2)(1.21)Therefore, if we set s = 2bp\u270f0c , with \u270f0 < \u270f1\/2, then Lemma 2 implies that \u0000\u02dcsatisfies RIP with order \u0000 p1\/2+\u270f0 , and constantO(p\u0000\u270f1\/2+\u270f0(log p)2). Hence,The square-root barrier would be broken for the CS matrix \u0000\u02dc as definedin (1.20). Note that in [16], the value of m is set to be m = 2b3799.6c0 c.Using the approximation for the value of c0 mentioned above, we obtainm = 79, 259, 656, which yields \u270f1 \u21e1 1.8261\u21e5 10\u000015.Discussion about the construction of Bourgain et al.The construction by Bourgain et al. constitutes a landmark (and the only onethus far) in the effort for breaking the so-called square-root barrier. One ofour main goals in this thesis is to propose two novel approaches that improvethe Gershgorin bound by a multiplicative or additive constant, respectively,in the case of a specific construction. We will also propose a conjecture re-garding the distribution of quadratic residues in Zp and we will show thatthis conjecture combined with one of the two approaches mentioned aboveleads to breaking the square-root barrier with much milder conditions onparameters compared to the construction given in [16]. Specifically, thereare three aspects that we should consider when we talk about breaking thesquare-root barrier. The first (and probably the most important one) is howclose to the unity the power \u21b5 in k = O(m\u21b5) can be chosen. In the con-struction given in [16], \u21b5 = 1\/2 + \u270f0, with \u270f0 < \u270f1\/2 < 10\u000015 in the instanceconsidered above. If our proposed conjecture holds, then our result suggestsk = O(m5\/7). Another aspect that potentially can be improved is the boundon the minimum number of measurements p. In the construction given in[16], the RIP constant is O(p\u0000\u270f1\/2+\u270f0(log p)2). Recall that the minimum231.3. Quantization in compressed sensingconstant that guarantees `1 recovery through BP or BPDN is \u00002k < 1\/p2.Note that in order to havep\u0000\u270f1\/2(log p)2 \u21e1 p\u00009.1304\u21e510\u000016(log p)2 < 1\/p2,we must have p > 101016 = 1010000000000000000, since one can verify thatp = 101016 is not large enough to satisfy the inequality above. It can beseen that a similar bound on the minimum number of measurements musthold in order for the aspect ratio of the measurement matrices (n\/p) to bereasonably large, e.g., n\/p = 2, and also in order for the bound on theRIP constant given in [16] to be smaller (tighter) compared to the classicalGershgorin bound \u0000k \uf8ff k\u00001pp . Therefore, our goal in Chapter 4 is to proposean approach that can lead to improved version of breaking the so calledsquare-root barrier (assuming our conjecture holds).1.3 Quantization in compressed sensingQuantization of compressive samples. As summarized in Section 1, us-ing compressed sensing one can reconstruct a given sparse signal using fewmeasurements. In fact, we saw that for a given ambient dimension n, andsparsity level k, one can construct m\u21e5n measurement matrices \u0000 that guar-antee that (1.3) holds ifm \u0000 Ck log (n\/k) for some constant C. For example,certain sub-Gaussian matrices satisfy this condition with high probability, asmentioned in Theorem 6. So, effectively compressed sensing transforms thesignal in high-dimensional space to a space of lower dimension. On the otherhand, one of the original motivations of compressed sensing has been com-pression. In the context of signal acquisition, we must not only \u201csample\" (ormeasure) the signal in such a way that we can accurately reconstruct it later,but we must also quantize the measurements so that we may store\/transmitthem using digital devices. Quantization makes the range of a signal discrete,so that the quantized signal takes on only a discrete, usually finite, set ofvalues. Unlike sampling (where we saw that under suitable conditions exactreconstruction is possible), quantization is generally irreversible and resultsin loss of information. It therefore introduces a distortion into the quantizedsignal that can not be eliminated. A quantizer is a device or a function thatperforms quantization. The round-off error introduced by quantization iscalled quantization error.Main Problem: Given a vector y, which, in our context will be thecompressed measurements of a sparse or compressible signal, we want to241.3. Quantization in compressed sensingreplace it by a vector whose elements are chosen from a discrete set A, calledthe quantization alphabet. We consider the general form of A = \u0000Z. Forexample for \u0000 = 12 our alphabet will be, A = 12Z = {....,\u00001,\u000012 , 0, 12 , 1, ....}1.3.1 Memoryless scalar quantizationOne of the main goals in quantization is to obtain an approximation ofthe original signal as accurately as possible given a fixed alphabet A. It isalso important to have a \u201cprogressive\" approximation scheme in the sensethat once more measurements are collected, i.e., the larger m is, the betterresulting approximation is. Consider a signal x 2 Rn and a measurementvector y = \u0000x 2 Rn. One of the easiest approaches to quantization is amethod called memoryless scalar quantization (MSQ). In this method, wereplace each measurement y(i), i 2 {1, 2, ...,m}, by its closest neighbourqMSQ(i) in A = \u0000Z. By the definition of our alphabet we have |y(i) \u0000qMSQ(i)| < \u00002 and therefore,ky \u0000 qMSQk2 < \u00002pm (1.22)Next, we observe that we can interpret the error due to quantization as\u201cnoise\", i.e., in (1.4) we set ! = y \u0000 qMSQ. In other words,qMSQ = \u0000x+ !where k!k2 = ky \u0000 qMSQk2 \uf8ff \u0000pm2 . Using Theorem 5, it can be concludedthat the solution x\u02c6 tox\u02c6 = argmin kzk1 subject to kqMSQ \u0000 \u0000zk2 \uf8ff \u0000pm2satisfieskx\u0000 x\u02c6k2 \uf8ff C1\u0000pmTherefore, as we increase the number of measurements m, the upper boundon the error in reconstruction increases, which is contrary to what we expect.Note here, that as a convention, we have worked so far with matrices withexpected unit-norm columns. This is to ensure that the matrices satisfy theRIP condition. Since the entries of m\u21e5n measurement matrices are randomvariables with unit variance, we need a normalization factor 1\/pm so thatthe columns have unit-norm (in expectation).251.3. Quantization in compressed sensingIn the context of quantization, if we want to work with the same nor-malization convention, we have to decrease the quantizer step size \u0000 as weincrease the number of measurements m. For example, consider a sig-nal x and the measurement vector y = 1pm\u00000x where \u00000 is an unnor-malized measurement matrix. Suppose that when m = 1, y(1) = 2.3478and the quantization alphabet size is \u0000 = 0.5. Therefore, we will haveq(1) = 2.5. Now suppose that we increase the number of measurementsto m0 = 10000. Assuming the same values for entries of \u00000 as above, wewill have y(1) = 0.023478. Therefore, we must choose the new quantizationalphabet size as \u00000 = \u0000\/pm = \u0000\/100 = 0.005 to map this entry to 0.025and be able to keep the scaled original information about the value of y(1),otherwise all entries of the measurement vector including y(1) will map to 0for sufficiently large m and we will lose all the information.Therefore, if we want to keep the quantizer step size constant as we in-crease m, we need to change the normalization convention for the measure-ment matrices and work with unnormalized measurement matrices whoseentries are random variables with the unit variance (or unit-magnitude inthe case of deterministic matrices).Using this convention, we still haveky \u0000 qk2 = k\u00000x\u0000 qk2 \uf8ff \u0000pm (1.23)(where \u00000 is the unnormalized measurement matrix), but in order to be ableto use Theorem 5 we need to divide both sides of (1.23) by the factor ofpmto obtaink 1pm\u00000x\u0000 qpmk2 \uf8ff \u0000and we conclude that the solution x\u02c6 to the `1 minimization problemx\u02c6 = argmin kzk1 subject to k 1pm\u00000z \u0000 qpmk2 \uf8ff \u0000satisfies kx\u0000 x\u02c6k2 \uf8ff C1\u0000.Therefore, we observe that if we use MSQ, as we increase the numberof measurements m, even after resolving the normalization issue, the recon-struction error does not decrease as we expect. Hence, as we want to use aquantization scheme that has this property, we introduce another method ofquantization called \u201cSigma-Delta quantization\". In the following, we give abrief summary about this method of quantization in CS.261.3. Quantization in compressed sensing1.3.2 Sigma-Delta (\u2303\u0000) quantization\u2303\u0000 Quantization is a quantization scheme used predominantly in quantizingovercomplete bandlimited functions and redundant frame expansions [9, 10,12, 40, 42, 48, 49, 73, 79, 80]. The 1st order \u2303\u0000 Quantization scheme isdefined as follows.Let u0 = 0, and define uj and qj for j \u0000 1 recursively as follows.qj = argminp2A|uj\u00001 + yj \u0000 p|uj = uj\u00001 + yj \u0000 qj ,(1.24)When the input y = (yj) is a finite sequence, i.e. y 2 Rm, we can use matrixnotation to abbreviate (1.24). In this case, we can represent (1.24) asy \u0000 q = Du (1.25)where D = [Dij ]m\u21e5m is the difference matrix whose entries satisfyDij =8<: 1 if i = j;\u00001 if i = j + 1;0 otherwise.1.3.3 Approximating signals using \u2303\u0000 quantizationAs we saw before, if we perform MSQ as the method of quantization, and ifwe assume that the entries of our measurement matrices have the unit norm(if the matrix is deterministic) or unit-norm in expectation (if the matrix israndom), the reconstruction vector x\u02c6 satisfieskx\u0000 x\u02c6k2 \uf8ff C1\u0000for some constant C1. Consequently, the error bound does not improve aswe increase m. In this section, we compute the error in approximating theoriginal signal using \u2303\u0000 quantization with the same normalization conditionas above.We know that y\u0000q = \u0000x\u0000q = Du. As before, let x be a k-sparse vectorand assume that we have recovered support of x, i.e., we know T = supp(x).Also, let \u0000T be the restriction of \u0000 to its columns indexed by the set T . Wecan view \u0000T as a frame matrix. The concept of \u201cframes\" is a generalizationof the concept of \u201cbasis\". Specifically, a set of vectors {fn} in a Hilbert spaceH is called a frame if there exists positive constants A and B called framebounds satisfyingAkxk2 \uf8ffX|hx, fii|2 \uf8ff Bkxk2271.3. Quantization in compressed sensingfor every x 2 H, and here, kxk denotes the norm induced by the innerproduct of the Hilbert space. In finite dimensional case, where H = Rk,if we have a set of vectors {fn} with n = 1, 2, ...,m, the frame matrix Eis obtained by considering the transpose of these vectors as the rows of thismatrix. It can be seen that the set {fn} is a frame if and only if the resultingframe matrix is full rank. Any left inverse of the frame matrix E is calleda dual of E, and the Moore-Penrose pseudo-inverse is called the canonicaldual.In the context of CS, we have y = \u0000x = \u0000TxT , where \u0000T 2 Rm\u21e5kwhich is a tall and full rank matrix, and can be viewed as a frame matrix.Thus, it admits a dual matrix; let F be a dual (left inverse) of \u0000T . Sincey = \u0000x = \u0000TxT , multiplying both sides of (1.25) from left by F impliesxT \u0000 Fq = FDuNow define x\u02c60\u2303\u0000 = Fq . Then the error in approximation using \u2303\u0000 quanti-zation will be bounded bykxT \u0000 x\u02c60\u2303\u0000k \uf8ff kFDuk2 \uf8ff kFDk2kuk2On the other hand, it is shown in [47] that there exists a \u2303\u0000 scheme suchthat for every quantization alphabet A = \u0000Z and for every input signal y,kuk1 \uf8ff C\u0000 for a universal constant C not depending on m (see Proposition6). Using this inequality we can conclude kx\u0000 x\u02c6\u2303\u0000k2 \uf8ff C\u0000kFDk2pm. Thismotivates minimizing kFDk2 in order to minimize the upper bound on theerror.The frame F that minimizes kFDk2 subject to F\u0000T = I is called the(first order) Sobolev dual frame of \u0000T and is given by F = (D\u00001\u0000T )\u2020D\u00001[11], where \u2020 denotes the Moore-Penrose inverse of a matrix. The Moore-Penrose inverse of a tall matrix A is given by A\u2020 = (A\u21e4A)\u00001A\u21e4. Knowingthat the operator norm of a matrix is its maximum singular value, for thischoice of F we can writekFDk2 = k(D\u00001\u0000T )\u2020k2 = \u0000max[(D\u00001\u0000T )\u2020] = [\u0000min(D\u00001\u0000T )]\u00001Therefore, the error in approximation is bounded bykxT \u0000 x\u02c60\u2303\u0000k2 \uf8ffC\u0000pm\u0000min(D\u00001\u0000T )(1.26)On the other hand, kx\u0000 x\u02c6\u2303\u0000k2 = kxT \u0000 x\u02c60\u2303\u0000k2 where x\u02c6\u2303\u0000 2 Rn is the vectorobtained from x\u02c60\u2303\u0000 by padding zeros in the components that are not indexed281.3. Quantization in compressed sensingby T = supp(x). Therefore,kx\u0000 x\u02c6\u2303\u0000k2 \uf8ff C\u0000pm\u0000min(D\u00001\u0000T ). (1.27)Approximation via Two-Stage Reconstruction \u2303\u0000 Quantization:Given the quantized measurement vector q 2 Rm, approximation of x using\u2303\u0000 quantization includes two major steps. In the first step, the supportof x is recovered using `1 minimization problem. In the second stage, ap-proximation to vector x is calculated using Sobolev-dual of a submatrix of\u0000 determined by the support recovered in stage 1.Stage 1: Support Recovery: Let x be a given k-sparse signal, \u0000be an m \u21e5 n sub-Gaussian unnormalized measurement matrix. We set aconstant smaller than 1\/p2, say, \u00002k = 0.5 in Theorem 6. We also assumethat m \u0000 16Ck log(en\/(2k)), where C is the constant given by this Theorem6. Let q be the corresponding quantized measurement vector. In order toestimate the support of x, consider the following `1 minimization problemx0 := argminkzk1 subject to k\u0000z \u0000 qk2 \uf8ff \u270f := \u0000pm (1.28)By Theorem 5, with probability at least 1\u0000 2 exp(\u0000 m8C ), we havekx\u0000 x0k2 \uf8ff C1\u0000 \uf8ff 12\u0000 := \u2318where we used the fact that for the constant C1 whose expression is given inRemark 1, and for \u00002k that satisfies \u00002k \uf8ff 0.5, we have C1 \uf8ff 12. Therefore,if the smallest entry of x is greater than 2\u2318 in magnitude, then with theprobability mentioned above, picking the positions of k largest entries of x0,recovers the support of x perfectly.Stage 2: Signal Approximation: The support of x is recovered inStage 1. Now, let T = supp(x), and E = \u0000T . If we put F 0 = (D\u00001E)\u2020D\u00001,and x\u02c60\u2303\u0000 = F 0q, then by (1.27) we havekx\u0000 x\u02c6\u2303\u0000k2 = kxT \u0000 x\u02c60\u2303\u0000k2 \uf8ffC\u0000pm\u0000min(D\u00001E)Hence, to bound the error in approximation, it remains to find lower boundfor \u0000min(D\u00001E). To that end, Gunturk et al. [40] proved the followingtheorem.Theorem 12. Let E be an m \u21e5 k random matrix whose entries are i.i.dN (0, 1). Given \u21b5 2 (0, 1), there exist constants c1, c2, and c3 such that ifm \u0000 k(c1 logm)11\u0000\u21b5 , then with probability at least 1\u0000 exp(c2 k\u21b5m1\u0000\u21b5 ),\u0000min(D\u00001E) \u0000 c3m\u21b5+12k\u21b52.291.3. Quantization in compressed sensingTherefore, if \u0000 is a Gaussian matrix with entries N (0, 1) and with the min-imum number of measurements mentioned above, then for a fixed signal xand for any choice of \u0000T , with high probability \u0000min(D\u00001E) \u0000 c00m\u21b5+12 , fora constant c00. Therefore, the bound on the error in approximation can bewritten as follows.kxT \u0000 x\u02c60\u2303\u0000k2 \uf8ffC\u0000pmc00m\u21b5+12=C\u0000c001m\u21b52with \u21b5 2 (0, 1). Comparing the above error with the error of approximationusing MSQ which as we saw, could be written in the formkx\u0000 x\u02c6MSQk2 \uf8ff C1\u0000we see that this time the error of approximation decreases as the numberof measurements increases. Accordingly, we expect a better performance inthe sense of having smaller error in approximation using \u2303\u0000 quantizationinstead of MSQ, especially when the number of measurements is large.1.3.4 Higher order \u2303\u0000 quantizationAbove, we summarized the first order \u2303\u0000 quantization scheme. One canalso define higher-order \u2303\u0000 quantizers. For example, a second-order \u2303\u0000quantizer with alphabet A is defined as follows.Given a vector y = (y1, ..., yn), the quantization vector q is defined byrunning the iterationqi = QA(\u21e2(ui\u00001, ui\u00002, yi))ui = yi \u0000 qi + 2ui\u00001 \u0000 ui\u00002(1.29)where i \u0000 1, and u0 = u\u00001 = 0. Above, QA : R! R is the scalar quantizerwith alphabet A as defined in (1.24), and \u21e2 : R3 ! R is a specified quanti-zation rule. Note that if we set vi := ui \u0000 ui\u00001 (for i \u0000 0), then (1.29) canbe rewritten in the following form.qi = QA(\u21e2(ui\u00001, vi\u00001, yi))vi = vi\u00001 + yi \u0000 qiui = ui\u00001 + vi(1.30)with u0 = v0 = 0.In (1.29), we can set \u21e2(ui\u00001, ui\u00002, yi) := 2ui\u00001\u0000ui\u00002+ yi (see (1.34) below),or alternatively, we can set \u21e2(ui\u00001, vi\u00001, yi) := ui\u00001 + vi\u00001 + yi in (1.30).301.3. Quantization in compressed sensingWe also set QA(x) = b10xe10 where b...e is the nearest integer function andQA(x) rounds a number to its first decimal place. Thus, the second order\u2303\u0000 quantization can be written in the following form.q1 =b10y1e10v1 = y1 \u0000 q1u1 = v1(1.31)And for i \u0000 2 :qi =b10(ui\u00001 + vi\u00001 + yi)e10vi = vi\u00001 + yi \u0000 qiui = ui\u00001 + vi(1.32)Using matrix notation, we observe that y, q, u, and v in (1.29) satisfyy \u0000 q = Dv , v = Du) y \u0000 q = D2u.Similar to what we observed for defining the second order \u2303\u0000 quantizationin (1.29), the rth order \u2303\u0000 quantization can be defined as follows.qi = QA(\u21e2(yi, ui\u00001, ui\u00002, ..., ui\u0000r))ui = yi \u0000 qi \u0000rXj=1\u2713nj\u25c6(\u00001)iui\u0000j . (1.33)Above \u21e2 : Rr+1 7! R is an appropriate quantization rule, and can be consid-ered as [72]\u21e2(yi, ui\u00001, ui\u00002, ..., ui\u0000r) :=rXj=1(\u00001)j\u00001\u2713rj\u25c6ui\u0000j + yi. (1.34)Also, QA is the scalar quantizer with alphabet A as defined in (1.24) withthe initial condition u0 = u\u00001 = .... = u\u0000r+1 = 0. In the case of an rth-order \u2303\u0000 quantizer, the relationship between y, q, and u can be expressedusing matrix notation as y \u0000 q = Dru. It is shown in [48] that the \u2303\u0000quantization scheme as given in (1.33), is stable in the sense that the socalled state variable u remains bounded in infinity norm (uniformly in m).In other words, the following proposition has been proved.311.3. Quantization in compressed sensingProposition 6. [48] Set the quantization alphabet to be A = \u0000Z, then foreach r 2 N, there exists a constant Cr such that the \u2303\u0000 scheme given in(1.33) is stable for all input signals y with kuk1 \uf8ff Cr\u0000. Furthermore, forthe quantization rule given in (1.34), one can assume Cr = 1\/2.Replacing D with Dr in our equations for first-order \u2303\u0000 quantization, wesee that in the case of rth order \u2303\u0000 quantization, the reconstruction vectoris given by x\u02c6\u2303\u0000 = Fq = (D\u0000rE)\u2020D\u0000rq , and the bound in error of approx-imation is proportional to 1\u0000min(D\u0000rE) . In particular, the following theoremholds for estimating the error in reconstruction of signals using rth order \u2303\u0000quantization.Theorem 13. [40] Let E be an m\u21e5k random matrix whose entries are i.i.d.N (0, 1). Given r 2 N and \u21b5 2 (0, 1), there exist constants c = c(r) > 0, andc0 = c0(r) > 0 such that if we set \u0000 := mk , and if \u0000 \u0000 (c logm)1\/(1\u0000\u21b5), thenwith probability at least 1\u0000 exp(\u0000c0m\u0000\u0000\u21b5), we havekx\u0000 x\u02c6\u2303\u0000k2 . \u0000\u0000\u21b5(r\u00001\/2)\u0000Therefore, if we fix the value of k, then the error in reconstruction shoulddecay like f(m) = 1pmin the case of first order \u2303\u0000 quantization, and itshould decay like g(m) = 1mpmin the case of second order \u2303\u0000 quantization.Figure 1.1 confirms this fact. In this experiment, the sparsity level k = 10 isbeing fixed, and for each m, 200 experiments is performed. In each experi-ment, a 10-sparse signal x 2 R200 with a random support T \u2713 {1, 2, ..., 200},and standard normal entries is chosen, and an m \u21e5 200 Gaussian matrix isconsidered as the measurement matrix. Then, x is quantized using first or-der and second order \u2303\u0000 quantization schemes (with the step size \u0000 = 0.1).Let q(r) denote the rth order quantized vector. Then the reconstruction vec-tors are computed using via x\u02c6 := F 0q(r) = (D\u0000r\u0000T )\u2020D\u0000r, and the error inreconstruction, i.e., kx \u0000 x\u02c6k2 is computed. The average errors in these 200experiments are plotted versus number of measurements m in the log-logscale.Unfortunately, there are a number of caveats for the two-stage recon-struction for \u2303\u0000 quantization. First, in order to recover the support of asignal x, apart from a universal constant, the quantizer step \u0000 must be assmall (in magnitude) as the lower bound on the smallest non-zero entries ofthe signal x (see Theorem B in [40]). Furthermore, two-stage reconstructionfor \u2303\u0000 quantization is not stable and robust (unlike the usual situation inCS). For these reasons, another reconstruction method, this time one stage,321.3. Quantization in compressed sensing50 100 150Number of measurements m10-310-2Mean error in approximationFigure 1.1: Error in approximation using first order and second order \u2303\u0000quantization versus number of measurements m. The graphs are plotted inlog-log scale and they are compared with the upper bounds on the error,namely, f(m) = C\/pm, and g(m) = D\/pm3 (the graphs of f(m) = 1pmand g(m) = 1mpmare multiplied with proper constants to shift them closeto the numerical graph) respectively.331.3. Quantization in compressed sensingwas later introduced in [72], which we review next.1.3.5 One-stage recovery for \u2303\u0000 quantizationAs an alternative to the two-stage reconstruction method of [40] and [48],[72] introduced a one-stage reconstruction scheme. Specifically, if q is the\u2303\u0000 quantization vector obtained from noisy measurements \u0000x + \u2318 withk\u2318k1 \uf8ff \u270f, then an approximation x\u02dc to the signal x is recovered by solving(x\u02c6, \u232b\u02c6) := argmin(z,\u232b)kzk1 subject to kD\u0000r(\u0000z + \u232b \u0000 q)k2 \uf8ff Cr\u0000pmand k\u232bk2 \uf8ff \u270fpm(1.35)where D denotes the difference matrix as above, r denotes the order of the\u2303\u0000 scheme, and Cr is the constant obtained from Proposition 6.Theorem 14. (Theorem 1 in [72]) Let k, `, m, N be integers satisfyingm \u0000 ` \u0000 ck log(n\/k) and let \u0000 be an m \u21e5 n sub-Gaussian matrix. Let q bethe \u2303\u0000 quantization of \u0000x+! (with k!k1 \uf8ff \u270f). Then with high probabilityon the draw of \u0000, for every signal x 2 Rn, the solution x\u02c6 of (2.10) satisfieskx\u02c6\u0000 xk2 \uf8ff C\u21e3\u0000(m`)\u0000r+1\/2 +\u0000k(x)pk+rm`\u270f\u2318(1.36)where c, C are constants that do not depend on m, `, n.Note that the parameter ` can be chosen as any number in the interval[ck log(n\/k),m] and various choices of ` have different implications. Forthe case where there is no noise, ` would be chosen as small as possible tominimize the error in approximation. Also, note that in practice we aregiven a k-sparse signal x, and we know the sparsity level k. Based on thisk, we have to choose at least mmin = O(k log(n\/k)) measurements in orderto obtain an RIP matrix with small enough constant (with high probability)and hence, use the standard CS results. In the context of quantization, wecan choose the parameter ` = O(k log(n\/k)) (similar to mmin). Now, if weuse m = mmin measurements, we can not obtain a guarantee for the error ofsignal reconstruction being small enough, but as we increase the number ofmeasurements (while we keep ` fixed), the error becomes smaller and smaller(decaying like m\u0000r+1\/2 if there is no noise). If there is noise, or if the signalis not k-sparse, the bound on the error can still be found according to (1.36).341.3. Quantization in compressed sensingUsing one-stage reconstruction method has important advantages. First,unlike the two-stage reconstruction method, this method is robust with re-spect to noise. This is a significant advantage of this method since in practicethere is always noise present. Second, as we saw before, two-stage reconstruc-tion method requires the minimum entry of x to be at least 24\u0000. However,the one-stage reconstruction method does not have any restrictions of thissort. For these reasons, this method is a preferred method to use for re-construction. Another advantage of using one-stage reconstruction methodis that by choosing an appropriate value for ` in (1.36) and assuming that\u0000 = m\/k is constant, it can be shown [72] that the error in reconstructiondecays root-exponentially in the case with no noise involved.Further encoding and exponential accuracyAs mentioned above if one-stage quantization is used, in the absence of noisea root-exponential accuracy can be achieved. However, it was shown in [71]that the exponential error decay can be achieved if the quantization stage isfollowed by an encoding stage. We explain the idea as follows.We start with a compressible signal x 2 Rn. Then, we measure this signalusing a measurement matrix \u0000, and we obtain the measurement vector y =\u0000x. Next, using a quantization alphabet A, we quantize the measurementsusing a \u2303\u0000 quantization scheme Q : Rm ! Am, obtaining the quantizedvector q = Q(y) = Q(\u0000x) 2 Am. In the following step, we encode thequantized vector using an encoding map E : Am ! C, where C is a finite set,called, the codebook satisfying log2 |C| \u2327 log2 |Am|. This way, to representthe quantized measurements we will need much less number of bits. Weshould however, encode in such a way that we can still obtain an accuratereconstruction using an appropriate reconstruction algorithm. In fact, theencoding map is defined as follows.E : q ! BD\u0000rqwhere D is, as usual, the difference matrix, B is an `\u21e5m matrix with i.i.d.Bernoulli random entries, andm \u0000 ` \u0000 ck log(n\/k)for an appropriate constant c. The algorithms to find the reconstructionvector x\u02c6 are given in (20) and (21) in [71] for the cases where the noise isabsent or present respectively. In particular, the reconstruction algorithmfor the noise-free case is as follows.x\u02c6 := argminzkzk1, subject to kBD\u0000r(\u0000z \u0000 q)k2 \uf8ff 3Cm351.4. Organization of the thesiswhere C is a constant that depends on the \u2303\u0000 quantizer.Next, let \u0000 : C ! Rn denote the reconstruction algorithm. Also, theworst case error of reconstruction, the distortion D, is defined viaD := sup k\u0000\u21e3E(Q(\u0000x))\u2318\u0000 xk2where the supremum is over all signals under consideration, which is eitherthe set of all k-sparse signals or the set of all compressible signals. Further-more, the bit rate R is defined asR := log2 |C|Using this notation, it was proved in [71] that the exponential accuracy canbe achieved in the following sense.Theorem 15. [71] Let k0 = b `C0 lognc, and assume that x is a k-sparse signalwith k \uf8ff k0. Then,D . 2\u0000C2 Rk0 logn1.4 Organization of the thesisIn this thesis, we focus on three important problems related to compressedsensing with explicit, deterministic constructions.In Chapter 2, we investigate how to extend existing results on \u2303\u0000 quan-tization to the cases when compressed measurements are obtained usingmatrices that do not belong to sub-Gaussian ensemble.In Chapter 3, we present a novel construction that results in a binary,partially circulant measrement matrix that is easy to store and fast to apply.In Chapter 4, we develop techniques that can be used to improve boundson the RIP constants obtained by using the Gershgorin circle Theorem. Weend by making a conjecture, which, we prove, will lead to breaking the squareroot barrier if true.36Chapter 2One-stage recovery for\u2303\u0000-quantized compressedsensingThe quantization problem is an important and difficult problem in the con-text of CS. CS is a signal acquisition paradigm that performs dimensionreduction when the acquired signals live in a high-dimensional ambient spacebut have low-dimensional structure. Specifically, the focus is (approximately)sparse signals in Rn where n is large. Consequently, the \u201ccompressed\" rep-resentations that result are still real valued vectors, this time in Rm, withm\u2327 n.While quantization was mostly omitted in the early CS literature, therehas been several recent papers that address this problem. The approaches inthe literature focus mostly on either \u201cmemoryless scalar quantizers\" (MSQ)or \u201cnoise-shaping quantizers\".MSQ for CS: Suppose that the entries of y = \u0000x+\u2318 2 Rm are (possiblynoisy) measurements of an (approximately) sparse x 2 Rn. An MSQ withalphabet A rounds off each entry of y (independently) to the closest elementof A. [13, 24, 65].A special case of MSQ is the 1-bit quantizers, where each measurementis replaced by its sign [14, 44, 63, 64]. While such quantizers typically loseinformation regarding the magnitude of x, they are still interesting in certainapplications due to their simplicity.As discussed in Section 1.3, a direct interpretation of the quantizationerror as additive noise leads to an estimate x\u02dc (via, for example Basis PursuitDenoise [22, 30]), where kx\u0000 x\u02dck is proportional to the quantizer step size \u0000,and does not improve by increasing the number of measurements m. On theother hand, it was observed empirically in [40] that in a two-stage recoverymethod where the Penrose-Moore pseudo-inverse is used in the second stage(after support recovery), the error kx\u0000 x\u02dck is O( 1pm).Motivated by this, [54] shows that kx \u0000 x\u02dck is bounded by the sum of a37Chapter 2. One-stage recovery for \u2303\u0000-quantized compressed sensingterm that does not depend on m, but unobservably small in any realisticsetting, and a term that is indeed O( 1pm), at least for a wide class of sub-Gaussian matrices with high probability. Similarly, it was also shown inthe 1-bit CS context, that the approximation error decays as m increases.Specifically, it was shown in [64] that for a fixed level of sparsity, the errorin approximation using a specific convex minimization program decays asO( 1m1\/5) up to a logarithmic factor.While these improved results show some decay as a function of m, thisdecay is mild, suggesting that MSQ does not utilize extra measurementsefficiently. This leads us to noise-shaping quantizers.Noise-shaping quantizers for CS: Noise-shaping quantizers were orig-inally introduced in the context of analogue-to-digital (A\/D) conversion ofbandlimited signals [42]. These A\/D convertors, called \u2303\u0000 quantizers be-come popular in A\/D conversion technology [73] as they can be implementedusing low-accuracy circuit elements and still produce high-accuracy approxi-mations by oversampling the bandlimited signals at a rate much higher thanthe Nyquist rate. For many classes of signals oversampling is much easier (toimplement on circuitry) compared to using high-accuracy circuit elements,for example a scalar quantizers Q\u0000 with very small \u0000.Motivated by their efficiency in exploiting redundancy, \u2303\u0000 quantizerswere considered in the context of frame expansions (which are inherentlyredundant). Indeed, they were shown to yield approximations that improveas the redundancy increases in the contexts of Gabor frames [79, 80], fi-nite frames in Rd with certain regularity assumptions [9, 10, 12] , Gaussianrandom frames [40] and sub-Gaussian random frames [48, 49].As explained in Section 1.3, the frame theory setting is inherently con-nected to CS when the underlying signal is sparse. This observation moti-vated the \u201ctwo-stage reconstruction method\" in CS after the measurementsare quantized using an rth order \u2303\u0000 scheme. In a nutshell, suppose x 2 \u2303nk ,\u0000 2 Rm\u21e5n be an appropriate CS measurement matrix, and y = \u0000x be thenoise free compressive measurements. Also, let q be obtained by quantizingy using an rth order \u2303\u0000 scheme and let D be the difference matrix as in(1.3.2). In order to perform two-stage \u2303\u0000 quantization, we have to firstrecover the support set T = supp(x). Then, the reconstruction vector x\u02c6 isgiven by x\u02c6\u2303\u0000 = Fq where F = (D\u0000r\u0000T )\u2020D\u0000r. Under such assumptions, theerror in approximation satisfieskx\u0000 x\u02c6\u2303\u0000k2 \uf8ff C\u0000pm\u0000min(D\u0000r\u0000T ).However, performing two-stage \u2303\u0000 quantization for sub-Gaussian matrices38Chapter 2. One-stage recovery for \u2303\u0000-quantized compressed sensinghave two main disadvantages :(1) The recovery of signals is not necessarily robust respect to additive noise(unlike the usual situation in CS).(2) The smallest entry of x must be at least 2\u2318 := 24\u0000 in magnitude. There-fore, in general we have to work with a fine alphabet so that the smallestentry of the signal becomes greater than 2\u2318. Hence, two-stage \u2303\u0000 quan-tization is not adaptable to coarse quantization.As discussed in Section 1.3, to solve these issues, and to recover theoriginal signal from quantized measurements simply in one stage and bysolving a convex minimization problem, the following algorithm is proposedin [72].(x\u02c6, \u232b\u02c6) := argmin(z,\u232b)kzk1 s.t. kD\u0000r(\u0000z + \u232b \u0000 q)k2 \uf8ff Cr\u0000pmand k\u232bk2 \uf8ff \u270fpm(2.1)Here Cr is a constant that can depend on the order r and \u0000 and in the specificcase of an rth order greedy \u2303\u0000 quantizer, Cr = 1\/2 [72]. In this thesis wemake the latter assumption, i.e., we assume Cr = 1\/2. As seen in Section1.3, if x is k-sparse, and if the value of ` is large enough, then for any mwhich is at least as large as `, the error in approximation satisfieskx\u02c6\u0000 xk2 \uf8ff C\u21e3(m`)\u0000r+1\/2 +rm`\u270f\u2318(2.2)where c, C are constants that do not depend on m, `, n.Although this method solves the issues mentioned above for the two-stage \u2303\u0000 quantization, it can be applied only for the class of sub-Gaussianmatrices. However, there are other important class of measurement matri-ces that are used in CS such as random restrictions of Fourier matrices orbounded orthonormal systems in general, and deterministic measurementmatrices. Therefore, one important task is to propose a method to perform\u2303\u0000 quantization using these type of matrices.The idea in this chapter is to abstract out the underlying property thatenables us to perform one-stage quantization for sub-Gaussian matrices andto make it work for other CS random matrices and for deterministic CS ma-trices. Specifically, we will propose two novel approaches by using a modifiedmeasurement matrix and by using a digital buffer given in Sections 2.1 and2.2 respectively.392.1. Approach 1: Modifying the measurement matrix2.1 Approach 1: Modifying the measurementmatrixIn [72], a wide class of sub-Gaussian matrices was considered and it wasshown that one-stage reconstruction \u2303\u0000 quantization can be performed us-ing (2.1). In order to generalize the results of [72] to other classes of randommatrices and also certain deterministic matrices, we next isolate one mainproperty, which we call (P1), that the measurement matrices must satisfy forsuch a generalization. Specifically, we define the property (P1) as follows.Property (P1). Suppose that \u0000 is an m\u21e5 n unnormalized CS matrix,with (expected) column norm ofpm. We say that \u0000 satisfies the prop-erty (P1) of order (k, `) if the RIP constant of 1p`(\u0000)`\u2013 where (\u0000)` is therestriction of \u0000 to its first ` rows\u2013 satisfies \u00002k < 1\/9.Note that sub-Gaussian matrices, and random restrictions of BoundedOrthonormal Systems (including the DFT matrix) satisfy this property withhigh probability for appropriate choices of k and `.Next, let y = \u0000x+\u2318, and k\u2318k1 \uf8ff \u270f. Set H := [CrDr \u270f\u0000 I], i.e., an m\u21e52mmatrix whose first m columns are obtained from CrDr and whose next mcolumns are obtained from \u270f\u0000 I. Suppose that H = U\u2303VT is the singularvalue decomposition of H. We shall prove that one-stage reconstructionfollowing \u2303\u0000 quantization can be performed if :(i) \u0000 satisfies (P1), and(ii) the compressed measurements are obtained using U\u0000 as our measure-ment matrix (as opposed to \u0000).Remark 4. If U is a unitary matrix and \u0000 is a CS measurement matrix,then 1pmU\u0000 can also be used as a CS measurement matrix for the recovery ofsignals with the same level of sparsity. This is because multiplying a matrixwith a unitary matrix does not change its RIP constant.The following results are instrumental.Proposition 7. [35] Let f, g 2 Cn, and \u0000 2 Cm,n. Suppose that \u0000 is RIPwith constant \u00002k < 1\/9. Then for any 1 \uf8ff p \uf8ff 2, we havekf \u0000 gkp \uf8ff C4k1\/p\u00001\/2k\u0000(f \u0000 g)k2 + C5k1\u00001\/p(kfk1 \u0000 kgk1 + 2\u0000k(g)1)where C4 and C5 are constants that only depend on \u00002k.402.1. Approach 1: Modifying the measurement matrixProposition 8. [48] The jth singular value of the matrix D\u0000r satisfies1(3\u21e1r)r(mj)r \uf8ff \u0000j(D\u0000r) \uf8ff (6r)r(mj)rNow, we are ready to show that if \u0000 is a matrix that satisfies the property(P1), then certain unitary transform of \u0000 can be used as the measurementmatrix to perform \u2303\u0000 quantization followed by a one-stage reconstructionscheme.Theorem 16. Suppose that \u0000 is an m \u21e5 n CS matrix, x 2 \u2303nk , and H =U\u2303V T as above. Assume that k and ` are chosen so that \u0000 satisfies (P1)of order (k, `). Suppose we measure the signal x using the matrix \u0000\u02dc = U\u0000,yielding the measurements y = \u0000\u02dcx, and perform the \u2303\u0000 quantization to yto obtain the vector q. Then x\u02c6, obtained from (2.1), approximates x with theerror satisfyingkx\u0000 x\u02c6k2 \uf8ff 2CrC4\u0000(3\u21e1r)r(m`)\u0000r+1\/2 + 2C4rm`\u270f+2C5pk\u0000k(x) (2.3)where C4 and C5 are constants only depending on RIP constant of \u0000.The proof of this theorem when \u0000 is a sub-Gaussian matrix is given in[72]. However, the only condition used in that proof is in fact property (P1)as defined above. For the sake of clarity and completeness, we next give theproof of this theorem.Proof of Theorem 16. Let x\u02c6 and \u232b\u02c6 be the solutions to the minimization prob-lem (2.1). Also, let \u0000\u02dc := U\u0000 and w := D\u0000r(\u0000\u02dcx\u02c6\u0000 \u232b\u02c6 \u0000 q). Then\u0000\u02dcx\u02c6\u0000 q = Drw + \u232b\u02c6 = [CrDr, \u270f\u0000I]24 1Crw\u0000\u270f \u232b\u02c635 = Hpwhere p :=24 1Crw\u0000\u270f \u232b\u02c635 is the vector whose first m entries are obtained from1Crw and whose next m rows are obtained from \u0000\u270f \u232b\u02c6, and H = [CrDr, \u270f\u0000 I] isas defined earlier.Next, let H = U\u2303V T be the singular value decomposition of H (whereU is m\u21e5m, \u2303 is m\u21e5 2m, and V is 2m\u21e5 2m). The pseudo-inverse of H will412.1. Approach 1: Modifying the measurement matrixbe given by H\u2020 = V \u2303\u00001UT where \u2303\u00001 is the matrix whose (i, i)th entry (for1 \uf8ff i \uf8ff m) (\u2303\u00001)ii satisfies (\u2303\u00001)ii = 1\u2303ii , so that\u2303\u00001\u2303 =\uf8ffIm\u21e5m 0m\u21e5m0m\u21e5m 0m\u21e5m\u0000Hence,H\u2020H = V\uf8ffIm\u21e5m 0m\u21e5m0m\u21e5m 0m\u21e5m\u0000V T = [v1 v2 . . . vm 0 \u00b7 \u00b7 \u00b7 0]26664vT1vT2...vT2m37775where vi denotes the ith column of the matrix V . Accordingly, for any vectorx 2 R2m,H\u2020Hx = hv1, xiv1 + . . .+ hvm, xivmThat is, H\u2020H is the orthogonal projection operator into the subspaceW = span{v1, v2, ..., vm} and so kH\u2020Hkop \uf8ff 1. Now,kH\u2020(\u0000\u02dcx\u02c6\u0000 q)k2 = kH\u2020Hpk2 \uf8ff kH\u2020Hkopkpk2 \uf8ff kpk2 \uf8ff \u0000p2m (2.4)where we used the fact that kwk2 \uf8ff Cr\u0000pm and k\u232b\u02c6k2 \uf8ff \u270fpm (by (2.1)).Next, we find a lower bound on the singular values of H\u2020. Note that H =[CrDr\u270f\u0000 I]. Thus,HH\u21e4 = [CrDr,\u270f\u0000I]\uf8ffCr(Dr)T\u270f\u0000 I\u0000= C2r (Dr)(Dr)T + (\u270f\u0000)2I.Let \u0000j(Dr) denote the jth largest singular value ofDr. Then, the eigenvaluesof HH\u21e4 are given by C2r\u00002j (Dr) + (\u270f\u0000 )2. Using the fact that singular valuesof a matrix A are given as the square roots of eigenvalues of AA\u21e4, andalso using the connection between the singular values of a matrix A and itspseudo-inverse \u0000`(A\u2020) = 1\u0000m\u0000`+1(A) (with ` = 1, 2, ...,m\u0000 1,m), we obtain :\u0000`(H\u2020) =\u21e3C2r\u00002m\u0000`+1(Dr) + (\u270f\u0000)2\u2318\u00001\/2=\u21e3C2r1\u00002` (D\u0000r)+ (\u270f\u0000)2\u2318\u00001\/2 \u0000 \u21e3C2r (3\u21e1r`m )2r + ( \u270f\u0000 )2\u2318\u00001\/2. (2.5)422.1. Approach 1: Modifying the measurement matrixOn the other hand, we havekH\u2020(\u0000\u02dcx\u0000 q)k2 = kH\u2020(\u0000\u02dcx+ \u2318 \u0000 \u2318 \u0000 q)k2 = kH\u2020(y \u0000 q \u0000 \u2318)k2 = kH\u2020(Dru\u0000 \u2318)k2= kH\u2020[CrDr \u270f\u0000I]24 1Cr u\u0000\u0000\u270f \u231835k2 \uf8ff kHTHkopk24 1Cr u\u0000\u0000\u270f \u231835k2 \uf8ff \u0000p2mwhere we used kHTHkop \uf8ff 1 , y \u0000 q = Dru with kuk2 \uf8ff Cr\u0000pm (byProposition 6) and k\u2318k2 \uf8ff \u270fpm. Combining the inequality above with(2.4), we obtainkH\u2020\u0000\u02dc(x\u0000 x\u02c6)k2 \uf8ff kH\u2020(\u0000\u02dcx\u0000 q)k2 + kH\u2020(\u0000\u02dcx\u02c6\u0000 q)k2\uf8ff \u0000p2m+ \u0000p2m = 2\u0000p2m, (2.6)Therefore,2\u0000p2m \u0000 kH\u2020\u0000\u02dc(x\u0000 x\u02c6)k2 = kV \u2303\u00001UTU\u0000(x\u0000 x\u02c6)k2= k\u2303\u00001\u0000(x\u0000 x\u02c6)k2 \u0000 \u0000`(H\u2020)k\u0000`(x\u0000 x\u02c6)k2where \u0000`(H\u2020) is the `th singular value of H\u2020 and \u0000` is the matrix obtainedby considering the first ` rows of \u0000. Hence,k\u0000`(x\u0000 x\u02c6)k2 \uf8ff 2\u0000p2m\u0000`(H\u2020).Now by hypothesis, k and ` are chosen so that 1p`\u0000` satisfies RIP with\u00002k < 1\/9. Thus, by using p = 2 in Proposition 7, we obtainkx\u0000 x\u02c6k2 \uf8ff C4p`k\u0000`(x\u0000 x\u02c6)k2 + 2C5pk\u0000k(x)`1where we used the fact that kx\u02c6k1 \uf8ff kxk1. Hence, for such ` and k,kx\u0000 x\u02c6k2 \uf8ff C4p`k\u0000`(x\u0000 x\u02c6)k2 + 2C5pk\u0000k(x)\uf8ff 2C4\u0000pm\u0000`(H\u2020)p`+2C5pk\u0000k(x)\uf8ff 2C4\u0000rm`\u21e3C2r (3\u21e1r`m)2r + (\u270f\u0000)2\u23181\/2+2C5pk\u0000k(x)\uf8ff 2C4\u0000rm`\u21e3Cr(3\u21e1r`m)r +\u270f\u0000\u2318+2C5pk\u0000k(x)= 2C4Cr(3\u21e1r)r\u0000(m`)\u0000r+12 + 2C4rm`\u270f+2C5pk\u0000k(x)432.1. Approach 1: Modifying the measurement matrixwhere we used the inequality (t2 + s2)1\/2 \uf8ff t+ s for t, s > 0.2.1.1 Implications for bounded orthonormal systemsThe initial matrices used in CS were all non-structured random matrices suchas sub-Gaussian matrices. Using them came with at least two importantcaveats, namely, multiplying non-structured matrices with vectors is a longprocess and also storing them is costly and difficult. For these reasons, animportant class of random matrices in CS are considered choosing randomrows of Fourier matrices. Since these random matrices are structured, theysolve the issues mentioned above. Another reason for using these matrices isthat in some applications such as MRI [52] or tomographic imaging [19] thedevices are designed in a way that they measure the coefficients of signals inthe transform domain. Using these matrices was first suggested by Cand\u00e8set al. [21] to recover sparse signals using few measurements. The number ofmeasurements was later improved by Rudelson et al. [69]. Specifically, it isshown in [69] that for a normalized n\u21e5 n discrete Fourier transform (DFT)matrix F (n) whose (k, j)th entry is given byF (n)k,j =1pne2\u21e1i(j\u00001)(k\u00001)n (2.7)If the number of measurements m satisfies m = O(k log4 n) , then the sub-matrix \u0000 consisting of m rows of F (n) satisfies RIP condition with highprobability.A generalization of Fourier matrices is given in [34], and is defined asBounded Orthonormal Systems (BOS). Let D \u2713 Rd be equipped with aprobability measure \u232b. A BOS is defined as follows.Definition 7. Let \u0000 = [\u00001,\u00002, . . . ,\u0000n] be a system of complex-valued func-tions on D. We call \u0000 a Bounded Orthonormal System (BOS) if(i) \u0000 is an orthonormal system, that is, for all j, k 2 {1, 2, ..., n},Z\u0000j(t)\u0000k(t)d\u232b(t) = \u0000j,k =\u21e21 if j = k0 otherwise , and(ii) \u0000 is uniformly bounded, that is, for all j 2 {1, 2, ..., n},k\u0000jk1 = supt2D|\u0000j(t)| \uf8ff Kfor some constant K.442.1. Approach 1: Modifying the measurement matrixIn the context of CS, we are mostly interested in discrete orthonormal sys-tems. In this case, D = {1, 2, ..., n} and the discrete uniform measure \u232b isgiven by \u232b(B) = card(B)n for B \u2713 {1, 2, ..., n}. Then, for a unitary matrixU = [Ut,k] = [u1, u2, ..., un] 2 Cn\u21e5n, the normalized vectors pnuk 2 Cn (fork = 1, 2, ..., n) form a BOS ifpn maxt,k2{1,2,...,n}|Ut,k| \uf8ff K (2.8)for some constant K.One important example of BOS is the DFT matrix defined in (2.7). Itis obvious that this matrix is unitary and it satisfies (2.8). Other importantexamples of BOS are discrete cosine transform (DCT) C(n) and discrete sinetransform (DST) S(n) defined byC(n)j,k =r2ncos\u21e3\u21e1n(j\u0000 12)(k\u0000 12)\u2318, S(n)j,k =r2nsin\u21e3(2j \u0000 1)k\u21e12n\u2318(2.9)It is routine to check that C(n) and S(n) are unitary matrices and they satisfy(2.8) with K =p2. Another important example of BOS is the class ofHadamard matrices defined as follows. A Hadamard matrix H of order2n \u21e5 2n is the matrix whose (j, `)th entry is given byHj,` =12n\/2(\u00001)Pnk=1 jk`kwhere for any j 2 {1, 2, ..., 2n}, jk 2 {0, 1} is defined such thatj =Pnk=1 jk2k\u00001 + 1. It can be shown that H is unitary and satisfies(2.8) with K = 1.If U is a discrete BOS, by choosing m random rows ofpnU , one canobtain the random matrix A =pnRTU where RT : Cn ! Cm is the randomoperator that samples m rows of U . After proper normalization, such matrixA satisfies RIP with high probability if the number of measurements is largeenough and thus it can be used as a CS measurement matrix.Theorem 17. [37] Let A 2 Cm\u21e5n be the random sampling matrix associatedwith a BOS with constant K \u0000 1. If for \u0000 2 (0, 1),m \u0000 CK2\u0000\u00002k ln4(n)(for a universal constant C > 0), then with probability at least 1 \u0000 n\u0000 ln3 nthe restricted isometry constant \u0000k of 1pmA satisfies \u0000k \uf8ff \u0000.452.1. Approach 1: Modifying the measurement matrixCorollary 1. For a k-sparse signal x 2 Rn, we can use a Fourier matrixF (n), Discrete Cosine Transform matrix C(n), or Discrete Sine TransformS(n) and consider m0 to be the smallest value (obtained by Theorem 17) forwhich the corresponding measurement matrix satisfies RIP with \u00002k < 1\/9with high probability. Next, set ` := m0, and choose m \u0000 ` rows of F (n), C(n),S(n) randomly and denote them by F (m,n), C(m,n), and S(m,n) respectively.Then, measure x using UF (m,n) , UC(m,n), or US(m,n). Let x\u02c6 be the solutionto (2.1) with \u0000 replaced by one of the matrices mentioned here. Then, theerror in approximation using one-stage \u2303\u0000 quantization satisfies (2.3) as weincrease the number of measurements m.Remark 5. Here, we show that computing the signal with UF (m,n), UC(m,n),or US(m,n) is a fast process at least when r = 1. First, note that an explicitformula for entries of U is given in the case of r = 1 in [47] :Uk` =s2n+ 1\/2cos\u21e32(k \u0000 1\/2)(n\u0000 `+ 1\/2)\u21e12n+ 1\u2318=s2n+ 1\/2cos\u21e3(2k \u0000 1)(2n\u0000 2`+ 1)\u21e1\/22n+ 1\u2318=s2n+ 1\/2cos\u21e3(2k \u0000 1)\u21e1\/2\u0000 (2k \u0000 1)`\u21e12n+ 1\u2318=s2n+ 1\/2(\u00001)k+1 sin\u21e3(2k \u0000 1)`\u21e12n+ 1\u2318On the other hand, Discrete Sine Transform (DST) of type III is given by[62]S(n)k` =r2nsin\u21e3(2k \u0000 1)`\u21e12n\u2318Therefore, we can obtain entries of U using a submatrix of S(2n+1). Thereason is that we can write (k, 2`)th element of S(2n+1) asS(2n+1)k,2` =r22n+ 1sin\u21e32(2k \u0000 1)`\u21e14n+ 2\u2318=r22n+ 1sin\u21e3(2k \u0000 1)`\u21e12n+ 1\u2318which is same as (k, `)th entry of matrix U in absolute value up to a constant.We will also use the expression above for the entries of U in order to to showthat evaluating Uy for a vector y is fast. See Remark 9.462.2. Approach 2: Using a digital bufferRemark 6. While the singular value decomposition of D can be computedexplicitly, to our knowledge an explicit formula for singular value decompo-sition of Dr with r \u0000 2 is not known. Note, however, that one can estimatethe singular values of Dr using Weyl\u2019s inequalities [40, 47].Remark 7. Alternatively, one could apply U after collecting the measure-ments using F (m,n), C(m,n), or S(m,n). Of course, this would require that wekeep all m analogue measurements in memory, at least until we apply U stillin analogue domains which is not practically feasible in applications when mis large. We will propose a remedy in Section 2.2.2.1.2 Numerical experimentsIn order to verify the results given in Theorem 16, and in particular, given inCorollary 1, we perform a numerical experiment. In this experiment, we fixthe ambient dimension of signals to n = 200, the sparsity level to k = 5, andthe quantization step to \u0000 = 0.1. We consider them\u21e5200 matrix UFm,200 assuggested by Corollary 1 withm 2 {20, 30, 40, 50, 60, 70} as the measurementmatrix. For each value of m, we consider 20 signals in R200, random supportT \u2713 {1, 2, ..., 200}, and with non-zero entries chosen from normal Gaussiandistribution. For each of these signals, we find the measurement vector, andsubsequently quantize it using first or second order \u2303\u0000 quantization. Next,we find x\u02c6, the solution to (2.1), and we find the error in approximation. Wetake an average for the error for all 20 signals and move to the next value ofm. The results are plotted in Figure 2.1 in log-log scale. As we observe inthis Figure, the error bounds decays as predicted in (2.3).2.2 Approach 2: Using a digital bufferAside from the issues raised in Remark 7, the above approach is not ideal alsobecause the measurement matrix U\u0000 (specifically U) depends on m. Thismeans that we must use a different measurement matrix if we wish to increasethe number of measurements m, i.e., we can not \u201creuse\u201d the measurementsalready collected. This problem would be resolved if we could modify thescheme so that\u2022 We first collect y = \u0000x and quantize y;\u2022 We then use U (or any other matrix that admits a fast implementation)on the quantized measurements, which are now in the digital domain.472.2. Approach 2: Using a digital buffer20 30 40 50 60 700.050.10.150.20.25Figure 2.1: Error in approximation using first order and second order \u2303\u0000quantization with one-stage reconstruction scheme and with a \u201cmodified\"random partial Fourier matrix for 10-sparse signals and the comparison withthe graphs of f(m) = Cm1\/2and g(m) = Dm3\/2in log-log scale.Figure 2.2: Quantizing the signal x by first using MSQ with a very smallstep size \u00000, then applying U which is a fast transform followed by a \u2303\u0000quantization scheme.482.2. Approach 2: Using a digital bufferTo that end, we propose the following scheme.(1) Given a standard CS measurement matrix \u0000, we collect the compressedmeasurements y = \u0000x + \u2318, where \u2318, as before, denotes the noise suchthat k\u2318k1 \uf8ff \u270f.(2) We fix a small \u00000 (much smaller than the desired final accuracy) andquantize y using an MSQ with step size \u00000 resulting in yMSQ. This is ahigh bit-budget representation of y and will be discarded after the nextstages so, it is just kept in a buffer (with sufficiently large memory).(3) We compute UyMSQ, which finely approximates Uy = U\u0000x as U is anisometry.(4) We use a \u2303\u0000 quantizer (of appropriate order r that matches the matrixU in step (3)) with step size \u0000 to quantize UyMSQ. This will be thedigital representation of x that we will keep.Finally, we will reconstruct an approximation to x by means of convex opti-mization problem similar to (2.1) given by(x\u02c6, \u232b\u02c6) := argmin(z,\u232b)kzk1 subject to kD\u0000r(U\u0000z + \u232b \u0000 q)k2 \uf8ff Cr\u0000pmand k\u232bk2 \uf8ff \u000000pm(2.10)with \u000000 defined as \u000000 = \u270f+ \u00000\/2.Note that this method will be successful provided \u00000 in step (2) is suf-ficiently small to match the quantization error corresponding to the \u2303\u0000quantization of step (4). Thus, we will have to ensure that m \uf8ff mmax where\u00000 will be chosen depending on mmax (or vice versa). Collecting all these, wehave the following theorem, which we will prove after stating few remarks.Theorem 18. Let x 2 Rn, \u0000 be a CS measurement matrix, k and ` be suchthat \u0000 satisfies (P1), suppose that q is obtained from x following the schemesuggested above where\u2022 U is tailored to a \u2303\u0000 quantizer of order r (as described in section 2.1).\u2022 \u00000 := \u0000(3\u21e1r)rmrmax492.2. Approach 2: Using a digital buffer\u2022 k\u2318k1 \uf8ff \u270fIf x\u02c6 is obtained via (2.10), the approximation error satisfieskx\u0000 x\u02c6k2 \uf8ff 2C4\u0000(3\u21e1r)r(m`)\u0000r+1\/2 + 2C4rm`\u270f+2C5pk\u0000k(x) (2.11)for ` \uf8ff m \uf8ff mmax. Here C4 and C5 depend only on the RIP constants of \u0000.Remark 8. Since in practice, the original measurements in CS are physicalquantities (such as currents), the MSQ step mentioned in Theorem 18 wasperformed in order to assign numbers to the measurements which enables usto store the measurements in the processor and multiply with U later.Remark 9. In step (3) above, we need to compute UyMSQ. Here, we showthat this computation can be done fast. To that end, we use the fact that forS(m) as defined in (2.9), computing S(m)y is fast for any positive integer m,and any vector y. Let y = (y1, y2, ..., ym), then(Uy)1 = U11y1 + U12y2 + ....+ U1mym(Uy)2 = U21y1 + U22y2 + ...+ U2mym...(Uy)m = Um1y1 + Um2y2...+ UmmymThus,(Uy)1 =s2m+ 1\/2\u21e3sin(\u21e12m+ 1)y1 + sin(2\u21e12m+ 1)y2 + ....+ sin(m\u21e12m+ 1)ym\u2318(Uy)2 = \u0000s2m+ 1\/2\u21e3sin(3\u21e12m+ 1)y1 + sin(6\u21e12m+ 1)y2 + ...+ sin(3m\u21e12m+ 1)ym\u2318...(Uy)m = (\u00001)m+1s2m+ 1\/2\u21e3sin((2m\u0000 1)\u21e12m+ 1)y1 + ...+ sin(m(2m\u0000 1)\u21e12m+ 1)ym\u2318502.2. Approach 2: Using a digital bufferThus, we can write the above equations in the following form.(Uy)1 =p2S(2m+1)12 y1 +p2S(2m+1)14 y2 + ...+p2S(2m+1)1,2m ym(Uy)2 = \u0000p2S(2m+1)22 y1 \u0000p2S(2m+1)24 y2 \u0000 ...\u0000p2S(2m+1)2,2m ym...(Uy)m = (\u00001)m+1p2S(2m+1)m2 y1 + (\u00001)m+1p2S(2m+1)m4 y2 + ...+ (\u00001)m+1p2S(2m+1)m,2m ymTherefore, we have(Uy)j = (\u00001)j+1p2(S(2m+1)y\u02dc)jfor j = 1, 2, ...,m, and where y\u02dc 2 R2m+1 is a vector whose odd entries arezero, and whose (2k)th entry (k = 1, 2, ...,m) is defined as yk. Accordingly,computing Uy is a fast process.Proof of Theorem 18. Let x 2 Rn be the given signal, y = \u0000x + \u2318 be themeasurement vector (as usual), and y\u02dc := yMSQ be the vector obtained from ybe performing MSQ (with the step size \u00000 mentioned above). Then, we havey\u02dc = y + \u23180 = \u0000x+ \u2318 + \u23180 with k\u2318k2 \uf8ff \u270fpm and k\u23180k2 = ky \u0000 y\u02dck2 \uf8ff \u000002pm.Moreover, since we apply \u2303\u0000 quantization scheme on the vector Uy\u02dc (toobtain the quantized vector q), we can writeUy\u02dc \u0000 q = Druwith kuk2 \uf8ff Cr\u0000pm (by Proposition 6). Thus,U\u0000x\u0000 q = Dru+ \u00b5 = Hp0where \u00b5 = U(\u2318 + \u23180), H = [CrDr \u000000\u0000 I] and p0 =24 1Cr u\u0000\u000000\u00b535. Note thatk\u00b5k2 \uf8ff k\u2318k2 + k\u23180k2 \uf8ff (\u270f+ \u000002)pm = \u000000pmwhere \u000000 is defined as \u000000 = \u270f+ \u000002 . Hence, kp0k22 \uf8ff \u00002m+ \u00002m = 2\u00002m.Therefore, by defining w := D\u0000r(\u0000x\u02c6\u0000 \u232b\u02c6\u0000q) where x\u02c6 and \u232b\u02c6 are the solutionsto minimization problem (2.10) , we havekH\u2020U\u0000(x\u0000 x\u02c6)k2 \uf8ff kH\u2020(U\u0000x\u0000 q)k2 + kH\u2020(U\u0000x\u02c6\u0000 q)k2= kH\u2020Hp0k2 + kH\u2020(Drw + \u232b\u02c6)k2\uf8ff kH\u2020Hkopkp0k2 + kH\u2020Hkopkp00k2\uf8ff \u0000p2m+ \u0000p2m = 2\u0000p2m(2.12)512.2. Approach 2: Using a digital bufferwhere p00 =24 1Crw\u0000\u000000 \u232b\u02c635 , and kp00k2 \uf8ff \u0000p2m (since kwk2 \uf8ff Cr\u0000pm andk\u232b\u02c6k2 \uf8ff \u000000pm by (2.10)).On the other hand for every 1 \uf8ff ` \uf8ff m,2\u0000p2m \u0000 kH\u2020U\u0000(x\u0000 x\u02c6)k2 = kV \u2303\u00001UTU\u0000(x\u0000 x\u02c6)k2= k\u2303\u00001\u0000(x\u0000 x\u02c6)k2 \u0000 \u0000`(H\u2020)k\u0000`(x\u0000 x\u02c6)k2where \u0000`(H\u2020) is the `th singular value of H\u2020. Hence,k\u0000`(x\u0000 x\u02c6)k2 \uf8ff 2\u0000pm\u0000`(H\u2020)and the lower bound for \u0000`(H\u2020) is given in (2.5) (with \u270f replaced by \u000000).Now by Proposition 7 , if k and ` are chosen so that 1p`\u0000` satisfies RIP with\u00002k < 1\/9, then by using p = 2 we obtainkx\u0000 x\u02c6k2 \uf8ff C4p`k\u0000`(x\u0000 x\u02c6)k2 + 2C5pk\u0000k(x)where we used the fact that kx\u02c6k1 \uf8ff kxk1. Hence, for such ` and k :kx\u0000 x\u02c6k2 \uf8ff C4p`k\u0000`(x\u0000 x\u02c6)k2 + 2C5pk\u0000k(x)\uf8ff 2C4\u0000pm\u0000`(H\u2020)p`+2C5pk\u0000k(x)Now, we use (2.5), with \u270f\u0000 replaced by\u000000\u0000 =\u270f+\u00000\/2\u0000 to simplify the boundabove.kx\u0000 x\u02c6k2 \uf8ff 2C4Cr\u0000(3\u21e1r)r(m`)\u0000r+1\/2 + 2C4rm`(\u270f+ \u00000\/2) +2C5pk\u0000k(x)\uf8ff 2C4\u0000(3\u21e1r)r(m`)\u0000r+1\/2 + 2C4rm`\u270f+2C5pk\u0000k(x)for values of m satisfying ` \uf8ff m \uf8ff mmax. In the last inequality above, weused ` \u0000 1, and we assumed Cr = 1\/2, and \u00000 = \u0000(3\u21e1r)rmrmax.522.2. Approach 2: Using a digital buffer20 30 40 50 60 7010-1Figure 2.3: Error in approximation using first order and second order \u2303\u0000quantization with one-stage reconstruction scheme and with an extra MSQstep (before applying the matrix U on the measurement vector).2.2.1 Numerical experimentsIn this section, we verify the result given in Theorem 18 empirically. In orderto do that we repeat the experiment explained in Section 2.1.2. The onlydifference is that in this experiment, to obtain the measurement vector y,we use the original random partial Fourier matrix Fm\u21e5200 (as opposed toUFm\u21e5200), then we use the step size \u00000 = (3\u21e1r)r\u0000mrmax to obtain the high-budgetquantized vector yMSQ (which will be stored in the buffer and will be dis-carded later). Next, we find UyMSQ and quantize it using rth order (r = 1, 2)\u2303\u0000 quantization (with the step size \u0000 = 0.1) to obtain the vector q. Next,we use (2.3) to obtain the vector x\u02c6 and we find the error in approximation.Similar to what we did in Section 2.1.2, we repeat the experiment for 20signals, and we take an average for the error in approximation. The graphof errors along with the reference graphs f(m) = Cpmand g(m) = Dmpmareshown in Figure 2.3 in log-log scale.532.3. \u2303\u0000-quantized compressed sensing with chirp sensing matrices2.3 \u2303\u0000-quantized compressed sensing with chirpsensing matricesAs we saw before, the chirp sensing matrices are defined as follows. For aprime number p and the primitive root of unity ! = e2\u21e1ip , the columns ofp\u21e5 p2 matrix \u0000 are defined via\u0000rp+m+1 =26666664!r\u00b702+m\u00b70!r\u00b712+m\u00b71...!r\u00b7(p\u00001)2+m\u00b7(p\u00001)37777775 (2.13)In above, \u0000j denotes the jth column of \u0000, and r and m range between 0and p \u0000 1. By this definition, the (k, j)th entry of the matrix \u0000 is definedas \u0000k,j = !rk2+mk with rp +m + 1 = j. Motivated by the fact that chirpsensing matrices can be used as CS measurement matrices, we try to use \u2303\u0000schemes to quantize CS measurements obtained using these matrices. Weknow that we can do so if they satisfy (P1). However, we observe that (P1)does not hold for these matrices. Consider a p \u21e5 p2 chirp sensing matrix \u0000and let T = {1, 2} (hence, we shall consider the first and second columns ofthis matrix). Note that we prefer the parameter ` in (2.3) to be as small aspossible in order to minimize the error in approximation, but as we illustratebelow the property (P1) does not hold even for ` = p1\u0000\u270f (for any \u270f > 0),and large enough p. Set E = \u0000` (where as above, \u0000` denotes the restrictionof \u0000 to its first ` rows). Next, consider the matrix A = 1`E\u21e4TET . Obviously,A11 = A22 = 1 and for any given \u270f0 > 0,|A12| = |A21| =\u0000\u0000\u00001`\u21e31 + e2\u21e1ip + e4\u21e1ip + e6\u21e1ip + ...+ e2`\u21e1ip\u2318\u0000\u0000\u0000 \u0000 1\u0000 \u270f0for large enough p since each term in the sum above goes to 1 as p!1. Theeigenvalues of this matrix satisfy (1\u0000\u0000)2 = |A12|2 and so \u0000min = 1\u0000 |A12| \uf8ff\u270f0. Hence, \u00002 = max{1 \u0000 \u0000min,\u0000max \u0000 1} \u0000 1 \u0000 \u270f0 for large enough p, andtherefore \u00002 < 1\/9 can not hold.However, this issue can be resolved if we use a certain submatrix of thechirp sensing matrix by choosing certain values of m. Specifically, we definea p\u21e5 pbppc matrix \u0000\u00af as follows.Definition 8. Let p be a prime number, and \u0000 = (!rx2+mx)x2Zp be ap \u21e5 p2 chirp sensing matrix, where the columns are indexed by two param-542.3. \u2303\u0000-quantized compressed sensing with chirp sensing matriceseters r and m in Zp. Define \u0000\u00af = (!rx2+mx)x2Zp as a p \u21e5 pbppc subma-trix of \u0000 if the values of r and m are chosen from {0, 1, 2., , , p \u0000 1}, and{bppc, 2bppc, ...., (bppc)2} respectively.We will show that such matrices satisfy (P1) and hence, one can performone-stage \u2303\u0000 quantization using them as measurement matrices. We willanalyze the corresponding approximation error in two scenarios: First, wefix the sparsity level and vary the number of measurements. Next, we fix thenumber of measurements and vary the sparsity level.2.3.1 Approximation error as the number of measurementsgrowsIn this case, we fix the signal x and we will increase the ambient dimensionand the number of measurements while fixing the vector x by embeddingx into higher dimensional space. This is because for the class of matricesdefined above, to increase the number of measurements p, we must also in-crease the ambient dimension, which is equal to pbppc. As such, we evaluatethe error in quantization using one-stage \u2303\u0000 quantization as the number ofmeasurements p increases.First, we prove that the class of matrices defined in Definition 8 satisfythe property (P1 ) of order (k, `) for appropriate choices of k, and `.Theorem 19. Consider the p\u21e5 pbppc matrix \u0000\u00af as defined in Definition 8.Then there exists a prime number p0 such that for p \u0000 p0, the matrix \u0000 sat-isfies the property (P1) of order (k, `) for k \uf8ff 4pp log p and ` = bp3\/4 log2 pc.To prove this theorem, we will use the following result about an estimatefor exponential sums, given by Weyl [57, p. 41].Theorem 20. (a) Suppose that P (x) = \u21b5x2 + \u0000x+ \u0000 where \u21b5 satisfies\u0000\u0000\u0000\u21b5\u0000 aq\u0000\u0000\u0000 \uf8ff 1q2for some relatively prime integers a and q. Then\u0000\u0000\u0000 NXn=1e(P (n))\u0000\u0000\u0000 . Npq+pN log q +pq log qwhere e(x) = e2\u21e1ix and the notation f . g means |f | \uf8ff Cg for a constant Cand for all values of the free variables under consideration.552.3. \u2303\u0000-quantized compressed sensing with chirp sensing matrices(b) Suppose that P (x) = \u0000x+ \u0000. Then\u0000\u0000\u0000 NXn=1e(P (n))\u0000\u0000\u0000 \uf8ff 12k\u0000kwhere k\u0000k is the distance to the nearest integer.Proof of Theorem 19. First we define the incomplete Gauss type sum S(r,m, p, `)for r, m, p, and ` as given in the theorem viaS(r,m, p, `) := e2\u21e1ip [r+m] + e2\u21e1ip [4r+2m] + ...+ e2\u21e1ip [r`2+m`]Suppose that va and vb are two distinct columns of 1p`\u0000` corresponding tothe values of r1,m1 and r2,m2 (i.e., a = r1p+m1+1 and b = r2p+m2+1).Then|hva, vbi| = 1` |S(r2 \u0000 r1,m2 \u0000m1, p, `)| (2.14)To bound the RHS in above, we need to consider two cases.Case 1. If r1 6= r2, we bound |hva, vbi| by setting ` = bp3\/4 log2 pc.For this purpose, we apply part (a) of Theorem 20 mentioned above with\u21b5 = r2\u0000r1p which is of the formaq for relatively prime integers a = r2 \u0000 r1and q = p. Hence, by using part (a) of Theorem 20, and using the fact thatp3\/4 log2 p2 \uf8ff p3\/4 log2 p\u0000 1 \uf8ff ` \uf8ff p3\/4 log2 p for p \u0000 3, we obtain|hva, vbi| . 1p3\/4 log2 p|S(r2 \u0000 r1,m2 \u0000m1, p, `)|. 1p3\/4 log2 p\u21e3p1\/4 log2 p+ p3\/8 log3\/2 p+pp log p\u2318. 1p1\/4 log3\/2 p(2.15)Case 2. If r1 = r2, we set ` = bp3\/4 log2 pc and we use the fact that theset of possible values of m are {bppc, 2bppc, 3bppc, ...., (bppc)2}, and weuse part (b) of Theorem 20. Note that in our problem \u0000 = m2\u0000m1p , and\u00001 \uf8ff \u0000 \uf8ff 1. Accordingly, to evaluate k\u0000k, we evaluate min{|\u0000|, 1 \u0000 |\u0000|}.Next,|\u0000| \u0000 bppcp\u0000pp\u0000 1p\u0000 0.1ppp=0.1ppfor any prime number p. Also,|\u0000| \uf8ff (bppc \u0000 1)bppcp\uf8ff (pp\u0000 1)ppp= 1\u0000 1pp562.3. \u2303\u0000-quantized compressed sensing with chirp sensing matriceswhich implies 1\u0000 |\u0000| \u0000 1pp . Therefore, in any case, k\u0000k \u0000 0.1pp , and since weset the value of ` to be ` = bp3\/4 log2 pc, we will have|hva, vbi| . 1p3\/4 log2 ppp . 1p1\/4 log2 p. (2.16)Combining the equations (2.15) and (2.16) we obtain|hva, vbi| . 1p1\/4 log3\/2 pSince the columns of the matrix \u0000` have unit norm, we can conclude thatthe coherence of this matrix, \u00b5, satisfies \u00b5 . 1p1\/4 log3\/2 p.Therefore, there exists a prime number p0 such that for p \u0000 p0 :\u0000k \uf8ff k\u00b5 . p1\/4 log p 1p1\/4 log3\/2 p=1plog p<19Next, we prove the following corollary which shows that we can indeed use\u0000\u00af along with a rth-order \u2303\u0000 quantizer.Corollary 2. Let x 2 \u2303nk , let p0 be as defined in Theorem 19, and supposethat p1 > p0 is a prime number such that k \uf8ff 4pp1 log p1. Then, for anyp \u0000 p1, x can be approximated by x\u02c6, the solution to (2.1), if(i) the measurement matrix is U \u0000\u00af, where \u0000\u00af is the p\u21e5pbppc matrix definedas in Definition 8, and(ii) q is obtained by quantizing U \u0000\u00af using an rth order \u2303\u0000 scheme.In the noise-free case, as we increase the number of measurements p, theapproximation error satisfieskx\u0000 x\u02c6k2 \uf8ff C(3\u21e1r)r\u0000(log p)2r\u00001p\u0000 14 (r\u0000 12 ) (2.17)where C is a constant that does not depend on r, p0, p, and p1.Proof. Set ` = bp3\/4 log2 pc. Then, since p > p1, we have k \uf8ff 4pp log p.Thus, by Theorem 19, the p\u21e5 pbppc matrix \u0000\u00af satisfies (P1) of order (k, `),and hence the vector x can be approximated by x\u02c6. Moreover, by Theorem16, as p increases in the noise-free case, the error in approximation satisfieskx\u0000 x\u02c6k2 \uf8ff C(3\u21e1r)r\u0000(log p)2r\u00001p\u0000 14 (r\u0000 12 ) (2.18)where C is a constant that does not depend on r, p0, p, and p1.572.3. \u2303\u0000-quantized compressed sensing with chirp sensing matricesNote that the error decay rate O(p\u000014 (r\u0000 12 )) (up to a factor logarithmic inp) given in Corollary 2 is inferior to O(p\u0000(r\u000012 )) which we obtain with randommatrices (with m = p measurements). This behaviour is due to the fact thatthe both dimensions of \u0000\u00af increase as we increase p. One way to circumventthis issue is to restrict the maximum number of measurements to some pmax.In the following theorem, we will prove that under such circumstances, theapproximation error behaves like p\u0000(r\u000012 ), similar to the case with randommatrices.Theorem 21. Fix \u21b5,\u0000 > 0, with \u21b5 + \u0000\/2 < 1\/2. Let x 2 \u2303nk , and assumethat p0 be as defined in Theorem 19. Suppose that p1 > p0 is a prime numbersuch that k \uf8ff p\u21b51 . Then, for any p1 \uf8ff p \uf8ff pmax, where pmax = O(p1+\u00001 ), thesignal x can be approximated by x\u02c6, the solution to (2.1), if(i) the measurement matrix is U \u0000\u00af, where \u0000\u00af is the p\u21e5pbppc matrix definedas in Definition 8, and(ii) q is obtained by quantizing U\u0000x using an rth order \u2303\u0000 scheme.In the noise-free case, as we increase the number of measurements p, theapproximation error satisfieskx\u0000 x\u02c6k2 \uf8ff D\u0000p\u0000(r\u00001\/2) (2.19)where D is a constant that depends on p1, and order r, but does not dependon p0 or p.Proof. Set ` = bp1\/2+\u21b5+\u0000\/21 log2 p1c. Then, by using Theorem 20, and similarto the argument given for the proof of Theorem 19, we conclude that thecoherence of \u0000\u02dc satisfies\u00b5 . 1`\u21e3 `pp+p` log p+pp log p+pp\u2318. 1`pp log p . p1\/2+\u0000\/21plog p1p1\/2+\u21b5+\u0000\/21 log2 p1Hence, the RIP constant of 1p`\u0000\u00af satisfies\u0000k < k\u00b5 .p1\/2+\u0000\/2+\u21b51plog pp1\/2+\u21b5+\u0000\/21 log2 p1=1log3\/2 p1< 1\/9where we used the fact that 1log3\/2 p0< 1\/9, and p1 \u0000 p0. This means that\u0000\u00af satisfies the property (P1) of order (k, `), and hence the vector x can bereconstructed using the solution of (2.1) if U \u0000\u00af is used as the measurementmatrix. Also, by using ` = bp1\/2+\u21b5+\u0000\/21 log2 p1c, and m = p in (2.1) weobtain the bound on the error in approximation (2.20) as desired.582.3. \u2303\u0000-quantized compressed sensing with chirp sensing matricesAs an example of Theorem above, we can set k = 4, \u21b5 = 0.34, and\u0000 = 0.3. Then \u21b5 + \u0000\/2 = 0.49 < 1\/2, and we must choose p1 such thatk = 4 < p0.341 . We can observe that p1 = 61 satisfies this inequality. Hence,if the number of measurements satisfies 61 \uf8ff p \uf8ff 611.3, then the guaranteeon the error bound (2.20) will hold.Combining Theorem 21 and Corollary 2, we obtain the following result.Corollary 3. Fix \u21b5,\u0000 > 0, with \u21b5 + \u0000\/2 < 1\/2. Let x 2 \u2303nk , and assumethat p0 be as defined in Theorem 19. Suppose that p1 > p0 is a prime numbersuch that k \uf8ff p\u21b51 . Then, for any p \u0000 p1, the signal x can be approximatedby x\u02c6, the solution to (2.1), if(i) the measurement matrix is U \u0000\u00af, where \u0000\u00af is the p\u21e5pbppc matrix definedas in Definition 8, and(ii) q is obtained by quantizing U\u0000x using an rth order \u2303\u0000 scheme.In the noise-free case, as we increase the number of measurements p, theapproximation error satisfieskx\u0000 x\u02c6k2 \uf8ff D\u0000p\u0000(r\u00001\/2) (2.20)if p \uf8ff p1+\u00001 , andkx\u0000 x\u02c6k2 \uf8ff C\u0000(log p)2r\u00001p\u0000 14 (r\u0000 12 ) (2.21)if p > p1+\u00001 .2.3.2 Approximation error as the sparsity level variesIn the previous section, we saw that if we use an appropriate measurementmatrix, and an appropriate approximation scheme, then as we increase thenumber of measurements, the error in approximation decreases. Our objec-tive in this section is to fix the number of measurement (which also fixes theambient dimension) and reduce the sparsity level k. We expect to observea similar behaviour to what we observed above, and see a decay in error inapproximation.Theorem 22. Consider the CS matrix \u0000\u00af as defined in Definition 8. Thereexists a prime number p1 such that for a fixed number of measurements p,with p \u0000 p1, 1 \uf8ff k \uf8ffpplog p and ` = bkpp log pc, the matrix \u0000\u00af satisfies theproperty (P1) of order (k, `).592.3. \u2303\u0000-quantized compressed sensing with chirp sensing matricesProof. By doing a similar calculation to the one given in the proof of The-orem 19, and using ` = bkpp log pc, the equations (2.15) and (2.16) will bereplaced by|hva, vbi| . 1kpp log p\u21e3k log p+pk 4pp log p+pp log p\u2318and|hva, vbi| .ppkpp log prespectively. This implies|hva, vbi| . max{ 1pp,1pk 4pp,1kplog p,1k log p}Therefore there exists a prime number p1 such that for p \u0000 p1, the RIPconstant of the matrix \u0000` satisfies\u0000k < k\u00b5 . max{ kpp,pk4pp,1plog p,1log p} . 1plog p< 1\/9where we used the assumption on the sparsity level k \uf8ffpplog p .Similar to what we observed in Section 2.3.1, we state a corollary regardingthe bound on the error term when the matrix U \u0000\u00af is used as the measurementmatrix, and one-stage recovery scheme is used to reconstruct x. To matchthis corollary with the similar results, where we had a decreasing functionfor the error term, we consider the error as a function of k0 = 1\/k. Note thatwe expect the error term to decay as we decrease the value of k, i.e., as weincrease the value of k0.Corollary 4. There exists a prime number p0 such that for a fixed primenumber p with p \u0000 p0, any k-sparse signal x can be approximated with thevector x\u02c6, the solution to (2.1), provided that the following holds.(i) The sparsity level satisfies k \uf8ff bkmax :=pplog pc.(ii) The measurement matrix is U \u0000\u00af, with \u0000\u00af defined as in Definition 8;(iii) q is obtained by quantizing U \u0000\u00afx using an rth order \u2303\u0000 scheme \u2013 as in(1.33).602.3. \u2303\u0000-quantized compressed sensing with chirp sensing matricesThe error in approximation satisfieskx\u0000 x\u02c6k2 \uf8ff 2CrC4(3\u21e1r)r(pplog p)\u0000r+1\/2!(k0)\u0000r+1\/2 (2.22)assuming that no noise is present. Here, k0 = 1\/k, and the constant C4 onlydepends on RIP constant of \u0000.Proof. Let x be a k-sparse signal, and x\u02c6 be the approximation vector. Also,let p0 be the prime number given by Theorem 19. Replace the value ofm = pand ` = bkpp log pc \uf8ff kpp log p into (2.3) and use the fact that \u0000k(x) = 0for a k-sparse signal to concludekx\u0000 x\u02c6k2 \uf8ff 2CrC4(3\u21e1r)r(pplog p)\u0000r+1\/2!(k)r\u00001\/2as desired.2.3.3 Numerical experimentsIn this section, we verify the results we obtained in Sections 2.3.1 and 2.3.2.We run two numerical experiments. In the first experiment, we considerprime numbers p = 61, 137, 223, 307, 397, 487, 593, 677, 787, and foreach prime p, we draw 20 signals, each of which is a 4-sparse signal witha random support chosen from the set {1, 2, \u00b7 \u00b7 \u00b7 , 61\u2305p61\u21e7}, and whose en-tries are chosen independently from a standard Gaussian distribution. Inother words, the actual ambient dimension of signals that are consideredis 61\u2305p61\u21e7= 427. For each such signal, we compute the CS measurementsy = U \u0000\u00af which we subsequently quantize using a stable rth-order \u2303\u0000 schemeto obtain q with r = 1 or r = 2. Next, we reconstruct an approximation x\u02c6of x using (2.1) where we set \u0000 = U \u0000\u00af, \u0000 = 0.1, r = 1, 2, and \u270f = 0. Finally,for each p, we compute the average kx \u0000 x\u02c6k2. We plot the average error asa function of p in log-log scale in Figure 2.4. As mentioned in Section 2.3.1,for 4-sparse signals, we expect the bound on the error in approximation tobehave like p\u0000(r\u00001\/2) at least for 61 \uf8ff p \uf8ff 611.3. Figure 2.4 confirms thisfact and shows the p\u0000(r\u00001\/2) behaviour even for p values beyond this range.In the second experiment, we fixed the number of measurements to bep = 541, and we considered k-sparse signals with 3 \uf8ff k \uf8ff 15. Then foreach k0 = 1k , we consider 50 signals which are k-sparse and have a randomsupport T \u2713 {1, 2, ..., 1400} and have entries chosen independently from the612.4. Further encoding of \u2303\u0000-quantized compressive measurements102 Number of measurements p10-310-2Mean Error in ApproximationFigure 2.4: Error in approximation using first order and second order \u2303\u0000quantization with one-stage reconstruction scheme for a 4-sparse signal andthe comparison with the graphs of f(p) = Cpp and g(p) =Dp3\/2(each oneshifted properly to match the original graphs) in log-log scale.standard Gaussian distribution. For each of these signals, the reconstructionvector x\u02c6 is obtained from (2.1) with r = 1 or r = 2. We average over all theerrors for each value of k, and we plot the graph of average errors as well asthe upper bounds on the error obtained in Section 2.3.2 in log-log scale inFigure 2.5.2.4 Further encoding of \u2303\u0000-quantized compressivemeasurementsIn one-stage recovery of \u2303\u0000 quantized measurements, we start with a mea-surement vector y and since we have to store\/transmit data we quantize thisvector using an alphabet A to obtain a quantized vector q 2 Am. To en-code q, we need log2 |A|m = m log2 |A| bits. In [71], Saab et al. proposed amethod to encode using much less number of bits without affecting the errorin reconstruction significantly. In the following, we give a brief review abouttheir result.In a nutshell, they reduce the dimension of q to encode using less numberof bits. In particular, suppose that L \uf8ff m, and consider the encoder E :Am ! C defined as E(q) = BD\u0000rq. where B is an L \u21e5m Bernoulli matrix622.4. Further encoding of \u2303\u0000-quantized compressive measurements0.1 0.15 0.2 0.25 0.3k'=1\/k10-310-2Error in ApproximationFigure 2.5: Error in approximation using first order and second order \u2303\u0000quantization with one-stage reconstruction scheme with fixed number of mea-surements (p = 541) and the comparison with the graphs of f(k0) = 1pk0andg(k0) = 1pk03.with i.i.d. equiprobable entries.First, we find how many bits we are saving by using this encoder. Weconsider the alphabet AK\u0000 := {\u0000K\u0000, ...,\u0000\u0000, \u0000, ...,K\u0000}. Since kD\u0000rk1 \uf8ff mr,and kBk \uf8ff m [71], we obtain kBD\u0000rqk1 \uf8ff mr+1kqk1 \uf8ff mr+1K\u0000. Thus,for each entry of E we need an alphabet of the formA0 = AKmr+1\u0000There are L such entries, so in total we should useL log2 |A0| = L(r + 1) log2m+ L log2 2Kbits to represent E(q). Thus, by enlarging the size of alphabet and reducingthe dimension, Saab et al. [71] reduced the number of bits because the sizeof alphabet appears only as logarithmic factor.Now, the goal is to find an algorithm to reconstruct x with the vector x\u02c6given the encoded vector q\u02dc = E(q) = BD\u0000rq, and with kx \u0000 x\u02c6k2 to be assmall as possible. This algorithm is given in [71] as follows.(x\u02c6, u\u02c6, e\u02c6) = argminkx\u02dck1 subject to BD\u0000r(\u0000x\u02dc+ e\u02dc)\u0000Bu\u02dc = BD\u0000rqand kBu\u02dck2 \uf8ff 3Cm and ke\u02dck \uf8ffpm\u270f(2.23)632.4. Further encoding of \u2303\u0000-quantized compressive measurementsNext, we prove that this algorithm can be applied using the measurementmatrix defined in Definition 8. In order to do so, first we choose a Bernoullimatrix B of the size L \u21e5 p with L = bp5\/8 log2 pc and consider the p \u21e5 pmatrix D\u0000r. Then, write the singular value decomposition of BD\u0000r in theform BD\u0000r = TSRT . Using this notation, we prove the following theorem.Theorem 23. Consider a k-sparse signal x 2 Rn, with k \uf8ff b 8pp log pc.Suppose that we use the matrix R\u0000\u00af as the measurement matrix, where R isas above, and \u0000\u00af is the matrix given in Definition 8, to find the measurementvector y. Then, we use the rth order \u2303\u0000 quantization to obtain the quantizedvector q. Next, find the reconstruction vector x\u02c6 via (2.23). The error inreconstruction satisfieskx\u0000 x\u02c6k \uf8ff C1\u21e3 log ppp)r\/2\u00003\/4 + C2s pplog p\u270f+ C3\u0000k(x)1pkwith probability at least 1\u0000C5e\u0000c6p11\/16 log p for some constants C1, C2, C3, C5,and c6.Note that if we want to have decreasing bound (as a function of p) for theerror in approximation in the noise-free case, we need to have r\/2\u00003\/4 > 0.This means we must have r \u0000 2.Proof. First, let L = bp5\/8 log2 pc, and we verify that 1pL\u0000\u00afL satisfies theRIP with \u00002k < 1\/9, if k \uf8ff 4pp log p. To that end, we use (2.14) along withTheorem 20, to conclude that|hva, vbi| . 1p5\/8 log2 p\u21e3p1\/8 log2 p+ p5\/16 log3\/2 p+pp log p\u2318.pp log pp5\/8 log2 p=1p1\/8 log3\/2 pHence, the coherence of 1pL\u0000\u00afL satisfies \u00b5 . 1p1\/8 log3\/2 p , and this implies thatthe RIP constant satisfies\u0000k < k\u00b5 \uf8ff p1\/8 log pp1\/8 log3\/2 p< 1\/9for large enough p.Similar to the what mentioned in the proof of Theorem 18, if we usep = 2 in Proposition 7, and the value of L as stated above, since 1pL\u0000\u00afL642.4. Further encoding of \u2303\u0000-quantized compressive measurementssatisfies the RIP with \u00002k < 1\/9, we can conclude thatkx\u0000 x\u02c6k2 \uf8ff d1pLk\u0000\u00afL(x\u0000 x\u02c6)k2 + d2\u0000k(x)1pk(2.24)for some constants d1 and d2. Next, we find an upper bound for k 1pL \u0000\u00afL(x\u0000x\u02c6)k2. To do that, we consider the set E = E1 \\ E2 whereE1 := {B 2 Bern(L,m) : \u0000L(BD\u0000r) \u0000\u21e3mL\u2318r\/2\u00001\/4pm},andE2 := {B 2 Bern(L,m) : kBk`2!`2 \uf8ffpL+ 2pm}It is shown in [71] thatP (E) \u0000 1\u0000 2e\u0000c1pmL \u0000 \u0000e\u0000c2L, (2.25)for some constants \u0000, c1, and c2. It is also shown that for any B 2 E , if wedecompose BD\u0000r in the form BD\u0000r = TSRT , and if we set \u0000\u02dc = RT\u0000 (here,\u0000 is the measurement matrix, and in our case, \u0000 = R\u0000\u00af with \u0000\u00af as given inDefinition 8, and so \u0000\u02dc = RT (R\u0000\u00af) = \u0000\u00af), then we have1pLk\u0000\u02dcL(x\u0000 x\u02c6)k2 \uf8ff 6C\u21e3 Lm\u2318r\/2\u00003\/4+ 2rmL\u270ffor a constant C. Hence, using the value of L as given above, we obtaink 1pL\u0000\u00afL(x\u0000 x\u02c6)k2 \uf8ff 6C\u21e3 log2 p8pp\u2318r\/2\u00003\/4+ 2s8pplog2 p\u270f (2.26)Accordingly, by combining (2.24) and (2.26), we obtainkx\u0000 x\u02c6k2 \uf8ff C1\u21e3 log2 p8pp)r\/2\u00003\/4 + C2s8pplog2 p\u270f+ C3\u0000k(x)1pkNoting that m = dp3\/4e, and L = dp5\/8 log2 pe, we conclude that e\u0000c1pmL .e\u0000c2L, which implies that 1 \u0000 e\u0000c2L . 1 \u0000 e\u0000c1pmL. Therefore, by (2.25),the inequality above holds with probability at least 1 \u0000 C5e\u0000c6pmL, i.e.,1\u0000 C5e\u0000c6p11\/16 log p, for some constants C1, C2, C3, C5, and c6.652.4. Further encoding of \u2303\u0000-quantized compressive measurementsTherefore, based on what we summarized from [71] in Section 1.3.5, thefollowing corollary holds regarding the exponential accuracy. Here, RT \u0000\u00af,with \u0000\u00af as defined in Definition 8 is used as the measurement matrix, themeasurements are quantized using rth order \u2303\u0000 quantization, and furtherencoding is performed at the end.Corollary 5. There exist constants C0, C2 such that in the noise-free case,and for k0 = bC0p5\/8 log pc, the distortion rate D, as defined in Section 1.3.5,satisfiesD . 2\u0000C2 Rk0 log pwhere R is the bit rate defined as R := log |C|.66Chapter 3Deterministic partial binarycirculant compressed sensingmatrices3.1 Introduction and preliminariesIn this chapter, we present a novel construction for deterministic CS matri-ces based on decimated Legendre sequences. As outlined below, Legendresequence provides a binary sequence with \u00b11 entries which initially seemsideal to use in the context of CS. However, in order to be able to use thesesequences as rows or columns of a measurement matrix, one has to guaranteea low maximum correlation between two such sequences. This was done firstby Zhang et al. [82] in 2002 (before the birth of CS) in the context of codingtheory, and by considering decimated Legendre sequences. The use of Leg-endre symbol in CS with random matrices was proposed by Bandeira et al.[6] in 2016. The summary of their work is given in Section 3.2. In the sameyear, Zhang et al. [81] proposed the use of Legendre symbol for the construc-tion of deterministic CS matrices. In fact, their construction was based ontheir previous work in 2002 which was done in the context of coding theory,and has the feature of being binary, and having low coherence. Moreover,since any prime number can be used as the number of measurements, thedifference between size of two adjacent matrices in their construction is lowcompared to many other deterministic constructions.As outlined below, another important feature that a CS matrix can haveis the circulant matrix structure (see Section 3.3). The use of circulantmatrices in random CS was first proposed by Bajwa et al. [4] in 2007. Anequivalent approach was proposed by Romberg [68] in 2009. In the latterapproach, given a signal x 2 Rn, first a convolution of x, of the form Hx isconsidered, where H can be written in the form ofH =1pnF \u21e4\u2303F673.1. Introduction and preliminariesHere, n is as usual the ambient dimension, F is the discrete Fourier matrix,and \u2303 is a diagonal matrix whose diagonal entries are complex numberswith unit norm, and random phases. Following this step, we subsample themeasurements. Therefore, the measurement matrix in this approach can bewritten as \u0000 = R\u2326H, where \u2326 \u2713 {1, 2, \u00b7 \u00b7 \u00b7 , n} is a set with m elements,and R\u2326 is a sampling operator that restricts the rows to a random set \u2326,i.e., a random choice of a set \u2326 among all possible\u0000nm\u0000such sets. Basedon this idea, Li et al. [50] considered a measurement matrix of the form\u0000 = R\u2326A, where A is a deterministic matrix. Since R\u2326 here is a randomsampling operator, the measurement matrix in their construction can not beconsidered as deterministic yet. To the best of our knowledge, the only classof circulant deterministic matrices considered in the literature so far is theclass of matrices introduced by Cui [25]. In their paper, they constructed thecirculant matrix A by first writing it of the form A = 1pnF \u21e4diag(\u0000)F , then,considering the sequence \u0000 as a decimated Legendre sequence, and finallyconsidering the first m rows of A as the measurement matrix. They showempirically that this matrix performs very well as a measurement matrix,but no proof was given in this regard. Note that there are of course otherdeterministic binary matrices given in the literature. In Section 1.2, wegave a summary for deterministic binary construction introduced by DeVore.Other binary deterministic constructions have been introduced in [1, 51, 58].To the best of our knowledge, the construction we introduce in this chap-ter is the first deterministic binary circulant construction in CS which isproved to have low coherence, and hence, can be used for recovery of sparsesignals. Compared to the work of Cui [25], in addition to giving a proof forwhy the construction can be used in CS, our construction has the advantageof having a simple, explicit formula for each entry of the measurement ma-trix itself (as opposed to its diagonalization). The circulant structure of ourconstruction allows us to perform a fast matrix-vector multiplication, and afast recovery algorithm. Moreover, our construction has a small differencebetween the sizes of two adjacent matrices. (as we will see below, the num-ber of measurements in our construction is chosen as dp3\/4e, where n = p isa prime number and is assumed to be the ambient dimension). Lastly, wewill show that we can perform the one-stage \u2303\u0000 quantization as describedin Chapter 2 using our construction. Similar to the constructions given in[6, 25, 81], our construction exploits Legendre symbol. Therefore, we firstgive a summary regarding Legendre symbol and its properties.Let p \u0000 3 be a prime number. For a 2 R, we assume that dae representsthe smallest integer greater than or equal to a. For a 2 Z, we assume that683.2. Compressed sensing matrices using Legendre sequence\u21e3ap\u2318denotes the Legendre symbol and is equal to 0 if a is divisible by p,is equal to 1 if a is a quadratic residue mod p, and is equal to -1 if a is aquadratic non-residue mod p. These basic properties of the Legendre symbolcan be found in any elementary number theory textbook, e.g., in [45] :1.\u21e3abp\u2318=\u21e3ap\u2318\u21e3bp\u2318for any a, b 2 Z and a prime number p.2.\u21e3ap\u2318= ap\u000012 mod p for a 2 Z and an odd prime number p.3.\u21e3qp\u2318=\u21e3pq\u2318(\u00001) p\u000012 q\u000012 for all odd primes p and q.Legendre symbol is a special example of Dirichlet characters. A Dirichletcharacter is a function \u0000 : Z! C that has the following properties:1. There exists a positive integer k such that \u0000(n) = \u0000(n + k) for alln 2 Z. The number k is called the modulus of the character.2. If (n, k) > 1 (where (n, k) denotes the greatest common divisor of nand k), then \u0000(n) = 0; If (n, k) = 1, then \u0000(n) 6= 0.3. \u0000(mn) = \u0000(m)\u0000(n) for all m,n 2 Z. Note that this property impliesthat \u0000(1) = 1 as \u0000(1) 6= 0 by 2.A character is called principal if it assumes the value 1 for the argumentscoprime to its modulus and otherwise is 0.It is obvious that when p is a prime,\u21e3\u00b7p\u2318is a Dirichlet character withmodulus k = p.Remark 10. Dirichlet characters with modulus k = p form a group undermultiplication where the principal character is the identity element. If \u0000 isan element of this group, then \u0000(ap\u00001) = \u0000(1) = 1 for every integer a whichis not divisible by p since ap\u00001 = 1 mod p. Therefore \u0000p\u00001 is always equalto the principal character. Hence, if \u0000 is a character modulus p of order d,then d|p\u0000 1.3.2 Compressed sensing matrices using LegendresequenceFix a prime number p, and an element x 2 Zp. Using the Legendre symbolintroduced above, we observe that the entries of any vector of the form v =693.3. Circulant matrices(\u21e31+xp\u2318,\u21e32+xp\u2318, \u00b7 \u00b7 \u00b7 ,\u21e3p+xp\u2318) are evenly distributed with \u00b11 entries. Hence,one can construct CS matrices using these vectors (or similar vectors). In[6], Bandeira et al. constructed a class of m \u21e5 n random matrices whose(i, j)th entry is obtained via\u0000i,j :=1pm\u21e3x+m(j \u0000 1) + ip\u2318(3.1)where p is a large prime number, and x is drawn uniformly from a set of theform {0, 1, 2, \u00b7 \u00b7 \u00b7 , 2h\u00001}. They show that for appropriate choices of k,m, n, \u0000,and h, such matrices are RIP of order k and constant \u0000 with high probability.They also make a conjecture implying that the class of deterministic matricesobtained from fixing x = 0 in (3.1) are RIP in the optimal regime (and hence,would break the square-root barrier). As mentioned above, one can also usethe Legendre symbol to construct binary deterministic matrices. Examplesof such constructions are given in [25, 81].Since our goal in this chapter is to consider a class of deterministic binarycirculant matrices, we next summarize the definition and main features ofcirculant matrices in the context of CS.3.3 Circulant matricesThe use of (random) circulant matrices in CS has been originally suggestedby Bajwa et al. [4] in 2007, followed by the results by Romberg [66, 68] in2009. As stated in [66], there is an advantage for using circulant matricescompared to Gaussian and Bernoulli random matrices in CS, as generatingthese matrices is faster, and more importantly the matrix-vector multiplica-tion process is faster when we use these matrices which in turn makes thereconstruction algorithm faster. Moreover, they arise in applications such asidentifying linear time-invariant systems [3]. We know that CS can be ap-plied in MRI using partial Fourier matrices. However, application of CS inMRI with using a generalization of circulant matrices called Toeplitz matri-ces has been proposed in [52], and has been shown that it has an advantageover the classical method in the sense that it spreads out the image energymore evenly.703.3. Circulant matricesDefinition 9. A circulant matrix is a matrix of the formC =26664c1 c2 .... cncn c1 .... cn\u00001...c2 c3 ... c137775We say that C is generated by the vector v = (c1, c2, ..., cn).It is seen in the definition above that a circulant matrix is a squarematrix and hence, is not appropriate to use in the context of CS. Therefore,in practice, random rows of such matrices have been considered to obtaina class of matrices called \u201cpartial circulant matrices\". It is shown in [67]that if a circulant matrix is generated by a Rademacher sequence, then withhigh probability, the partial circulant matrix obtained from choosing m rowsof the (square) n \u21e5 n circulant matrix satisfies RIP with \u00002k < 1\/p2 ifm & k3\/2 log3\/2 n. This condition was later improved by Krahmer et al.[46] to the (suboptimal) condition m & k log2 k \u00b7 log2 n. Hence, to comparethese random matrices with the sub-Gaussian random matrices, we observethat there is an additional log2 k factor in the expression for the minimumnumber of measurements. However, on the positive side, these matricescan be diagonalized using Fourier transform, and therefore, using FFT, theprocess of matrix-vector multiplication as well as running the reconstructionalgorithm become faster.Here, we justify these facts. First, we show that the Discrete FourierTransform matrix (DFT) diagonalizes the circulant matrix above. Let ! =e2\u21e1i\/n, and let vk = (1,!k,!2k, ...,!(n\u00001)k)T (for 0 \uf8ff k \uf8ff n\u0000 1). Then,Cvk =26664c1 c2 .... cncn c1 .... cn\u00001...c2 c3 ... c137775266641!k...!(n\u00001)k37775 =26664c1 + c2!k + \u00b7 \u00b7 \u00b7 cn!(n\u00001)kcn + c1!k + \u00b7 \u00b7 \u00b7+ cn\u00001!(n\u00001)k...c2 + c3!k + \u00b7 \u00b7 \u00b7+ c1!(n\u00001)k37775=\u21e3c1 + c2!k + \u00b7 \u00b7 \u00b7+ cn!(n\u00001)k\u2318266641!k...!(n\u00001)k37775Hence, vk is an eigenvector of C with eigenvalue \u0000k = c1 + c2!k + \u00b7 \u00b7 \u00b7 +713.3. Circulant matricescn!(n\u00001)k. Thus, the Discrete Fourier Transform (DFT) matrixF =1pn26666641 1 1 \u00b7 \u00b7 \u00b7 11 ! !2 \u00b7 \u00b7 \u00b7 !(n\u00001)1 !2 !4 \u00b7 \u00b7 \u00b7 !2(n\u00001)...1 !n\u00001 !2(n\u00001) \u00b7 \u00b7 \u00b7 !(n\u00001)(n\u00001)3777775diagonalizes the matrix C. Therefore, to compute Cx = FDF \u21e4x (for a vectorx), we need to compute F \u21e4x using fast fourier transform (FFT) (which takesas we know O(n log n) operations instead of the standard n2 operations),then we multiply the ith entry of the resultant vector with \u0000i. Note thatby computing FFT of x = (1, 1, \u00b7 \u00b7 \u00b7 , 1)T , we find the vector consisting allthe eigenvalues. At last, we multiply F with the resultant vector usingFFT which takes another O(n log n) operations. Accordingly, Cx can becomputed using O(n log n) operations. Note that if we use a partial circulantmatrix as the measurement matrix, we should multiply the resultant vectorby the matrix RT from left, where T = {j1, ..., jm} determines the rows ofthe circulant matrix that are going to be selected, and RT is an m\u21e5n matrixwhose (i, ji)th entry is one, and all its other entries are zeros. Since RT is asparse matrix, this last multiplication is also fast.As in this thesis, we focus on deterministic matrices as well as quantiza-tion, we should also investigate about performing quantization when partialcirculant matrices are used as CS matrices. In the recent publications, Dirk-sen et al. [29] showed how to do one-bit quantization using random partialcirculant matrices, and also Feng et al. [32] showed how to do \u2303\u0000 quanti-zation using these matrices. The essence of the main result in [32] is in factProposition 7, which as we saw in Section 2.1 leads to Theorem 16. Usingthe notation used in Chapter 2, the main technical challenge in [32] is toprove that UT\u0000 satisfies (P1) for appropriate choices of k and ` wheneverthe matrix \u0000 is a random partial circulant matrix. In Section 3.5, we willconsider the problem of one-stage recovery using \u2303\u0000 quantization for a spe-cific class of deterministic partial binary circulant matrices which we proposeas follows.Now, we give an explicit formula for the novel class of deterministic partialcirculant matrices. We obtain bounds for the coherence of these matrices,which we then use to identify requirements that relate the sparsity level k tothe number of measurements m (as well as the ambient dimension n) to en-sure that these matrices can be used as CS matrices. We also perform somenumerical experiments to compare the performance of these matrices with723.4. A novel, explicit constructionother deterministic and random CS matrices. Finally, we investigate theproblem of quantization when these matrices are used as measurement ma-trices, and we provide theoretical guarantees for the error in reconstructionusing one-stage recovery method in the case of rth order \u2303\u0000 quantization.3.4 A novel, explicit constructionConsider the p\u21e5 p deterministic matrix A defined byAi,j =(\u21e3j\u0000ip\u2318if i 6= j1 if i = jwhere\u21e3.p\u2318denotes the Legendre symbol, and 1 \uf8ff i, j \uf8ff p.Proposition 9. Let p be any prime number, then the matrix A as definedabove is a circulant matrix.Proof. First, we define the operator S : Rp ! Rp as follows.S\u21e3(x1, x2, ..., xp)\u2318:= (xp, x1, \u00b7 \u00b7 \u00b7 , xp\u00001)Now, if we denote the rows of A by A1, A2, ..., Ap, then to prove A is acirculant matrix, we need to show S(Ai) = Ai+1 for 1 \uf8ff i \uf8ff p\u0000 1. Next,S(Ai) = S(Ai1, Ai2, ..., Aip) = S\u21e3(1\u0000 ip), (2\u0000 ip), ..., (p\u0000 ip)\u2318=\u21e3(p\u0000 i)p), (1\u0000 ip), ..., (p\u0000 1\u0000 ip)\u2318=\u21e3(1\u0000 (i+ 1)p), (2\u0000 (i+ 1)p), ..., (p\u0000 (i+ 1)p)\u2318= (Ai+1,1, Ai+1,2, ..., Ai+1,p) = Ai+1as desired.Given the matrix A above, we define the dp3\/4e \u21e5 p measurement matrix \u0000by considering the first dp3\/4e rows of the matrix A (and then normalizingit).733.4. A novel, explicit constructionDefinition 10. Let p \u0000 3 be a prime number. The (i, j) th entry of themeasurement matrix \u0000, of our construction is defined as follows.\u0000i,j =8<:1pdp3\/4e\u21e3j\u0000ip\u2318if i 6= j1pdp3\/4e if i = jwith 1 \uf8ff i \uf8ff dp3\/4e, and 1 \uf8ff j \uf8ff pBased on this definition, \u0000 is a deterministic partial circulant matrix,and hence, all the features and benefits of using a circulant measurementmatrix, such as less space required to store the matrix, or fast matrix-vectormultiplication, and fast reconstruction scheme can be applied to these ma-trices.Theorem 24. There exists p0 \u0000 23 such that for p \u0000 p0, the coherence \u00b5 ofthe dp3\/4e \u21e5 p matrix \u0000 defined above satisfies\u00b5 \uf8ff 3p1\/2 log pp3\/4=3 log pp1\/4Hence, for these matrices\u0000k \uf8ff k\u00b5 \uf8ff 3k log pp1\/4Thus, in the context of CS, these matrices can be used for measurement and`1 recovery of vectors with the sparsity level of k = O( p1\/4log p).Note that if we use \u0000 as a measurement matrix, then the maximum sparsitylevel k, and the number of measurements m are related via k = O\u21e3m1\/3logn\u2318since as we observed above, the number of measurements is of order p3\/4,and the the maximum sparsity level k is of order p1\/4log p . This clearly comparesunfavourably to the random case; however note that our construction isexplicit and fast.Remark 11. Similar results will hold if we construct a dp\u21b5e\u21e5p CS matrix \u0000with the similar definition (with 1\/2 < \u21b5 < 1) given in section II. Specifically,we define the dp\u21b5e \u21e5 p matrix \u0000 as\u0000i,j =1pdp\u21b5e\u21e3j \u0000 ip\u2318743.4. A novel, explicit constructionwhere 1 \uf8ff i \uf8ff dp\u21b5e and 1 \uf8ff j \uf8ff p. Such matrix \u0000 can be used for the exactrecovery of signals with the sparsity level k satisfyingk \uf8ff p\u21b5\u00001\/218p2 log p= O\u21e3m\u21b5\u00001\/2\u21b5log n\u2318where we used the fact that m = dp\u21b5e, and n = p. Hence, if we choose alarger value for \u21b5 (close to 1 ), then the matrix \u0000 can be used for recovery ofa larger class of signals but we need more number of measurements. While ifwe choose a smaller value for \u21b5 (close to 1\/2), then the matrix \u0000 is closer tobe an ideal CS matrix (where the number of measurements is much less thanthe ambient dimension) but on the other hand \u0000 can be used for recovery ofa smaller class of signals. Note that 0 < \u21b5\u00001\/2\u21b5 < 1\/2 for 1\/2 < \u21b5 < 1,which is within boundaries of square-root barrier for deterministic matrices.To prove Theorem 24, we should consider the inner product of two distinctcolumns of our measurement matrix. As we will see below, the inner prod-ucts are related to a quantity called \u201cincomplete Weyl sums\". Let \u0000p be acharacter modulus p, and let f(x) 2 Zp[x] be a polynomial. A complete Weylsum is a sum of the formp\u00001Xx=0\u0000p\u21e3f(x)\u2318.It can be shown [78] that if f(x) 2 Zp[x] is monic of degree d \u0000 1 and f hasdistinct roots in Zp, then \u0000\u0000\u0000 p\u00001Xx=0\u0000p\u21e3f(x)\u2318\u0000\u0000\u0000 < dppAn incomplete Weyl sum is a sum of the formNXx=M\u0000p\u21e3f(x)\u2318,where M,N 2 Zp. In the case of f(x) = x, an upper bound was found byPolya in 1918 : \u0000\u0000\u0000 NXx=M\u0000p(x)\u0000\u0000\u0000 \uf8ff pp log pIn the case of an arbitrary polynomial f(x), the following bound on theIncomplete Weyl sums can be derived.753.4. A novel, explicit constructionTheorem 25. (Incomplete Weyl-Sum Estimate, Theorem 9.2 in [78]) : Thereexists p0 > 0 such that for any prime number p \u0000 p0, and for any monicpolynomial f(x) 2 Zp of degree d \u0000 1 and with distinct roots in Zp we have\u0000\u0000\u0000 NXx=0\u0000p\u21e3f(x)\u2318\u0000\u0000\u0000 \uf8ff d(1 + log p)ppfor any integer N 2 Zp.Proof of Theorem 24. Suppose that \u0000a and \u0000b are two distinct columns of\u0000, i.e., a 6= b. Thenh\u0000a,\u0000bi = 1dp3\/4edp3\/4eXi=1\u21e3b\u0000 ip\u2318\u21e3a\u0000 ip\u2318(3.2)Next, let f(x) := (x\u0000 a)(x\u0000 b), \u0000(n) :=\u21e3np\u2318(which is a character modulusp), and N = dp3\/4e. Then, f(x) is of degree d = 2, and hence we canuse Theorem 25. Also, note that for any integer i 2 Zp, we have f(i) =(b\u0000 i)(a\u0000 i), and hence,\u0000\u21e3f(i)\u2318=\u21e3(b\u0000 i)(a\u0000 i)p\u2318= (b\u0000 ip)(a\u0000 ip)Therefore,\u0000\u0000\u0000 dp3\/4eXi=1\u21e3b\u0000 ip\u2318\u21e3a\u0000 ip\u2318\u0000\u0000\u0000 = \u0000\u0000\u0000 dp3\/4eXi=1\u0000p\u21e3f(i)\u2318\u0000\u0000\u0000\uf8ff\u0000\u0000\u0000 dp3\/4eXi=0\u0000p\u21e3f(i)\u2318\u0000\u0000\u0000+ 1\uf8ff 2(1 + log p)pp+ 1 \uf8ff (3 + 2 log p)pp \uf8ff (3 log p)ppwhere we used the assumption p \u0000 23 to conclude log p \u0000 3, i.e., p \u0000 e3.Since the inequality above is valid for any two distinct values of a and b,using (3.2) we can conclude\u00b5 = maxa 6=b\u0000\u0000\u0000h\u0000a,\u0000bi\u0000\u0000\u0000 \uf8ff 3p1\/2 log pp3\/4=3 log pp1\/4as desired.763.5. One-stage recovery for \u2303\u0000-quantized compressed sensing with deterministic partial circulant matrices3.5 One-stage recovery for \u2303\u0000-quantizedcompressed sensing with deterministic partialcirculant matricesWe saw in Chapter 2 that we can perform a one-stage reconstruction \u2303\u0000quantization method for any CS measurement matrix as far as the mea-surement matrix \u0000 satisfies the property (P1) of order (k, `), as defined inSection 2.1. In particular, Theorem 16 states that if we write the singularvalue decomposition of Dr in the form Dr = U\u2303V T , and if the (original)measurement matrix satisfies (P1) of order (k, `), then we can use a (mod-ified) measurement matrix \u0000\u02dc = U\u0000, and the error in reconstruction usingthe algorithm (2.1) satisfies the bound given in (2.3).Proposition 10. The partial circulant matrix \u0000, as defined in Definition10, satisfies the property (P1) of order (k, `) for ` = dp5\/8e, and k = d p1\/8log pe.Hence, in terms of number of measurements m = dp3\/4e, and the ambientdimension n = p, we have ` = O(m5\/6), and k = O(m1\/6logn ).Proof. Recall that in one stage reconstruction method, we start with a matrixthat satisfies RIP. Then, we consider the first ` (for ` as small as possible)rows of the m \u21e5 n matrix \u0000, and the resultant matrix still satisfies RIP.The technical issue is that such property does not hold for an arbitrarymeasurement matrix, e.g., see Section 2.3. However, our construction herehas the advantage that the matrix obtained by considering its first ` rows isstill a partial circulant binary matrix. Hence, to obtain the RIP constantsof a matrix of the form\u0000 =1p`\u0000` (3.3)where \u0000` is the restriction of \u0000 as defined above to its first rows, we simplyreplace m = dp3\/4e in the proof of Theorem 24, with ` = dp5\/8e. Then, weconclude that the coherence of this matrix given in (3.3), satisfies\u00b5 \uf8ff 18pp log pp5\/8=18 log pp1\/8Hence, if the sparsity level satisfies k \uf8ff p1\/8log p , we have\u0000k < k\u00b5 =19773.6. Numerical experimentsThe following Corollary is immediate by combining the theorem above, andTheorem 16.Corollary 6. Consider the deterministic partial circulant matrix \u0000 of orderdp3\/4e \u21e5 p, as defined in Definition 10. Then, for k \uf8ff p1\/8log p , any k-sparsesignal x can be approximated with the vector x\u02c6, the solution to (2.1). Here,the measurement matrix that we use is U\u0000, and q is obtained from rth order\u2303\u0000 quantization. Moreover, as we increase the number of measurements p,the error in approximation decreases according tokx\u0000 x\u02c6k2 \uf8ff 2CrC4(3\u21e1r)r(m1\/6)\u0000r+1\/2 + 2C4\u270fpm1\/6 (3.4)for constant C4 depending only on RIP constant of \u0000, and where m = dp3\/4edenotes the number of measurements.3.6 Numerical experimentsFirst, we give an example of the new construction when p = 997. Figure 3.1shows the matrix constructed with this method.To compare the construction we introduced in this chapter with otherexisting deterministic constructions extensively, we run four numerical ex-periments.In the first experiment, we compute the coherence of matrices as definedin Definition 10, in order to find out about the tightness of the theoreticalbound we derived in this chapter. In this experiment, for each value of p, with71 \uf8ff p \uf8ff 1193, we start with the dp3\/4e\u21e5p matrix \u0000, then we construct \u0000\u21e4\u0000.Next, we calculate the maximum off-diagonal entry (in absolute value) of thisGramian matrix. This gives the coherence of the matrix \u0000 and we plot thisin log-log scale in Figure 3.2. As seen in this Figure, the coherence of thesematrices as a function of p, behaves almost like f(p) = 1p1\/4, which apartfrom a logarithm factor, matches with the bound we derived in Theorem 24.In order to make a comparison, we also plot the coherence of chirp sensingmatrices (of the size p \u21e5 p2) as a function of p. Note that the coherence ofchirp sensing matrices matches precisely with f(p) = p\u00001\/2.In the second experiment, we fix a measurements matrix \u0000 (randomor deterministic) and we consider a k-sparse signal x 2 R300. For each2 \uf8ff k \uf8ff 102, we choose a random support T \u2713 {1, 2, ..., 300} with k elements,and we choose non-zero entries of x independently from standard Gaussiandistribution. Then, we compute the measurement vector y = \u0000x, and we783.6. Numerical experimentsuse BP to to find the reconstruction vector x\u02c6. Next, we compute SNR forthe signal x defined bySNR(x) = 10 \u00b7 log10\u21e3 kxk2kx\u0000 x\u02c6k2\u2318dB (3.5)If we obtain SNR(x) > 50, we consider the reconstruction as a successfulrecovery and otherwise an unsuccessful recovery. For each k, we repeat thisexperiment 10 times and we let f(k) := number of successful recoveries10 . InFigure 3.3, we plot f(k) vs. k for different constructions. For our proposedconstruction, we choose a prime number close to 1000, say p = 997, whichleads to the number of measurements m = d9973\/4e = 178. We can choosethe same number of measurements for random Bernoulli matrix. For Reed-Muller construction, since the number of measurements is a power of 2, wechoose the smallest power of 2, greater than 178, namely, m = 256. ForDeVore construction, we choose m = 169 (the closest integer of the formp2 to 178). Accordingly, in Figure 3.3, we use Reed-Muller construction ofthe size 256 \u21e5 236, restricted to its first 300 columns (as x 2 R300), DeVoreconstruction of the size 132 \u21e5 133, restricted to its first 300 columns, thenovel construction of the size d9973\/4e \u21e5 997 = 178 \u21e5 997, restricted to itsfirst 300 columns, and also random Bernoulli of the size 178\u21e5300. As we seein this Figure, the novel construction introduced in this chapter has the bestperformance among the other deterministic constructions mentioned above,and its performance is comparable to the random Bernoulli matrices.In the third experiment, we fix k = 10, and we consider a k-sparsesignal x 2 R40, with random support and non-zero entries chosen indepen-dently from the standard Gaussian distribution, and we consider the newconstruction with m = bp3\/4c, and n = p (with 41 \uf8ff p \uf8ff 293). For eachvalue of p, we evaluate y = \u0000x, we approximate x with x\u02c6 using BP, andwe evaluate SNR using (3.5). Similar to above, if SNR > 50, we con-sider the reconstruction as a successful recovery, and otherwise an unsuc-cessful recovery. We repeat the same experiment 10 times and we definef(p) := number of successful recoveries10 . We plot f(p) vs. p in Figure 3.4.We repeat the same process above with k = 20. Lastly, we perform thesame process with k = 10 and k = 20 using random Bernoulli matrices withexactly same sizes, i.e., bp3\/4c \u21e5 p, with 41 \uf8ff p \uf8ff 293. Figure 3.4 also sug-gests that the performance of the new construction is comparable with therandom Bernoulli matrices.In the last experiment, for a fixed value of p (which fixes the number ofmeasurements m = dp3\/4e), and with 113 \uf8ff p \uf8ff 197, we consider a 1-sparse793.6. Numerical experiments100 200 300 400 500 600 700 800 90050100150Figure 3.1: The binary matrix given by the new construction for p = 997.102 103p10-1100CoherenceFigure 3.2: Coherence of the matrices introduced in this chapter, and alsochirp sensing matrices in log-log scale, accompanied with best fitted lines. Asseen in this Figure, the coherence of our construction behaves as \u21e0 m\u00001\/3,and for chirp sensing matrices behaves as \u21e0 p\u00001\/2 = m\u00001\/2.803.6. Numerical experiments0 20 40 60 80 100 120Sparsity level k00.10.20.30.40.50.60.70.80.91Fraction of successful recoveryFigure 3.3: The fraction of exactly recovered vectors versus sparsity for afixed number of measurements. The number of measurements is chosen asm = 256 for the Reed-Muller matrix, m = 178 for the new construction andalso for the Bernoulli matrix, and m = 169 for the DeVore\u2019s construction.The ambient dimension of all signals is n = 300signal x 2 R100 with random support and the non-zero entry chosen from astandard Gaussian distribution. Then, we reconstruct it using BP, and wemeasure SNR(x). We repeat the experiment 50 times for 50 signals x and ifthe minimum value of SNR(x) becomes greater than 50 dB, we consider thatlevel of sparsity as exactly recoverable and we increase the sparsity level by1 unit. Then, we repeat the same experiment until the minimum value ofSNR(x) (among all 50 experiments) becomes less than 50 dB for a sparsitylevel k1. Then we define g(p) := k1 \u0000 1 as the maximum level of recoverablesparsity corresponding to p. The graph of g(p) versus p is plotted in Figure3.5 in log-log scale. In the suboptimal case, we know that the maximumlevel of sparsity is m\/ log(n\/m) which will be p3\/4\/(4 log p). Therefore, inaddition to the graph of g(p), we plot the graph of f(p) = p3\/4 and wecompare these graphs. As it can be seen in this Figure, our construction,like the random CS matrices has the feature that numerically, the maximumrecoverable sparsity level behaves close to the suboptimal case.813.6. Numerical experiments0 50 100 150 200 250 300Ambient dimension p00.10.20.30.40.50.60.70.80.91Fraction of successful recoveryFigure 3.4: The graph of fraction of exactly recovered vectors (for 10 ex-periments) versus prime number p for a fixed level of sparsity (k = 10 or20) for the new construction and the Bernoulli matrices. Note that onlythree graphs are shown because the graphs corresponding to k = 20 forthe new construction and random Bernoulli exactly overlap with each other.This suggests that our proposed deterministic construction has a very similarperformance to random Bernoulli.120 130 140 150 160 170 180 190Ambient dimension p8101214161820Sparsity level kFigure 3.5: The maximum sparsity level of recoverable signals g(p) versusthe prime number p compared with the graph of f(p) = p3\/4.82Chapter 4RIP constants for deterministiccompressed sensingmatrices-beyond GershgorinWe saw in Section 1.2 that given a deterministic CS matrix, one of the mostcommon ways to bound its RIP constant is by relating its RIP constant withits mutual coherence via\u0000k \uf8ff \u00b5(k \u0000 1) (4.1)Throughout this chapter, we will call this bound, the Gershgorin bound onthe RIP constants, or simply the Gershgorin bound as this is the boundobtained from Gershgorin circle theorem. Moreover, the tightest bound thatrelates the RIP constants of a matrix to its performance as a CS measurementmatrix is due to [18] and it states that to ensure recovery of k-sparse (orcompressible) signals, we need \u00002k < 1p2 . On the other hand, using (4.1), inorder for a matrix to have small enough RIP constant, it is sufficient thatthe maximum sparsity level satisfies k < 12\u00b5p2+ 12 . Considering the Welchbound for the coherence of an m \u21e5 n matrix, this imposes a square-rootbarrier on the sparsity level of signals, namely, k = O(pm). Comparingthis level of sparsity with the maximum level of sparsity for sub-Gaussianmatrices, which is found to be k = O( mlog nm ), we observe that there is a hugedifference between these two.In fact, as mentioned in [37], p. 141, finding a deterministic CS matrixthat satisfies RIP in the optimal regime is a major open problem. Herewe quote a few sentences from [37] that explains the intrinsic difficulty ofreaching RIP in the optimal regime :\u201cThe intrinsic difficulty in bounding the restricted isometry constants ofexplicit matrices A lies in the basic tool for estimating the eigenvalues ofA\u21e4SAS \u0000 Id, namely, Gershgorin\u2019s disk Theorem ... the quadratic bottle-neck is unavoidable when using Gershgorin\u2019s theorem to estimate restrictedisometry constants. It seems that not only the magnitude of the entries of83Chapter 4. RIP constants for deterministic compressed sensing matrices-beyond GershgorinGramian A\u21e4A but also their signs should be taken into account in order toimprove estimates for deterministic matrices, but which tools to be used forthis purpose remain unclear.\" We will verify the fact that one needs to takeaccount the signs of entries of the Gramian matrix (in addition to the mag-nitudes) to obtain a bound that improves the Gershgorin bound. See Section4.1. Moreover, recall from Section 1.2 that as shown in [76], whenever thenumber of measurements satisfies m\u0000 k1+\u21b5s\u00002 log n, with \u21b5 2 [0, 1), thereis no polynomial time algorithm that can certify the RIP constants satisfy\u0000k \uf8ff s. This shows the significance of finding RIP matrices, even in thesuboptimal regime m\u0000 k1+\u21b5s\u00002 log n, with \u21b5 2 [0, 1). The first step to doso, is going beyond the Gershgorin bound, namely, the bound given in (4.1).In this chapter, we will propose two different tools to replace Gershgorincircle theorem for bounding eigenvalues of the Gramian matrix to estimatethe isometry constants for a specific construction. In the first approach, wecompare the Gramian matrices of this construction with the skew-adjacencymatrices of specific graphs to obtain bounds on the extreme eigenvalues ofthe Gramian matrices and hence, will estimate the RIP constants.To explain the idea used in the second approach, first note that Gersh-gorin circle theorem bounds every eigenvalue of a matrix uniformly. That is,it does not distinguish the extreme eigenvalues with other eigenvalues and itstates that every eigenvalue lies in one of Gershgorin circles. However, theisometry constants only depend on the minimum and maximum eigenvaluesof the Gramian matrix. There is in fact, a bound called \u201cDembo bound\"which provides bounds for the maximum and minimum eigenvalues of a pos-itive semidefinite Hermitian matrix. The goal in this chapter is to use oneof these two approaches to achieve an improved bound for the isometry con-stant, i.e., something better than \u0000k \uf8ff (k \u0000 1)\u00b5. We will see that using thefirst approach mentioned above, one can improve the classical Gerhsgorinbound by a multiplicative constant while using the second approach one canhave a small additive improvement. However, the second approach has itsown significance because using this approach we will give a pathway by pro-viding an explicit conjecture regarding the distribution of quadratic residues,to break the square-root barrier via k = O(m5\/7) (if the conjecture holds).All results in the literature on sparse recovery using the standard RIP relyon the Welch bound or its variants using `1-coherence. The only exception tothis, until our work, is the work of Bourgain et al. [16]. For a prime numberp, they constructed an explicit CS matrix of the order p\u21e5 p1+\u270f (where \u270f > 0is a small number and m = p is the number of measurements) such that thismatrix satisfies RIP with \u0000k < 1\/p2 when k = bp 12+\u270f0c (with \u270f0 < \u270f is also a844.1. Paley tight frames for compressed sensingsmall constant) and p is large enough. As mentioned above, while we can notbreak the square-root barrier, we will propose novel approaches to improvethe bounds based on coherence or `1-coherence. Lastly, we will propose aconjecture that if it holds, we would have an improved version of breakingthe square-root barrier compared to the one given in [16]. This improvementwill be on how close to unity the power \u21b5 can be chosen in k = O(m\u21b5), andon the lower bound on the minimum number of measurements.4.1 Paley tight frames for compressed sensingIn this chapter, we will investigate the behaviour of the RIP constants of aspecific class of matrices, and will show that it behaves better than what isexpected using the Gerhsgorin circle theorem, i.e., the bound given by (4.1).In order to choose such a class of matrices, first note that for a (normalized)measurement matrix \u0000, with the coherence \u00b5, the 2 \u21e5 2 Gramian matrixof the form\uf8ff1 cc\u21e4 1\u0000, with |c| = \u00b5, has the extreme eigenvalues 1 \u00b1 \u00b5, andhence, \u00002 = \u00b5 as predicted by (4.1). In the next step, we consider a Gramianmatrix of order 3 of the form241 \u00b5 \u00b5\u00b5 1 \u00b5\u00b5 \u00b5 135, and we observe that the extremeeigenvalues of this matrix are of the form 1\u00b1 2\u00b5. However, if we consider aGramian matrix of the form24 1 i\u00b5 i\u00b5\u0000i\u00b5 1 \u00b5\u0000i\u00b5 \u0000i\u00b5 135 (with i = p\u00001), the extremeeigenvalues are of the form 1 \u00b1 p3\u00b5. Moreover, it can be seen that fora larger value of k, the spectral radius of the Gramian matrix of order kcan reduce further if all non-diagonal entries are imaginary numbers anda mixture of the above diagonal entries have negative imaginary parts (asopposed to all above diagonal entries having positive imaginary parts, or allhaving negative imaginary parts). Therefore, we search among measurementmatrices with the property that the inner product of distinct columns areimaginary numbers, and also for large enough k, a mixture of above-diagonalentries of Gramian matrices of order k, have negative imaginary parts. Sucha construction is based on Paley tight frame as proposed in [7]. Specifically,we will consider the following matrices.Definition 11. Let p \u2318 3 mod 4 be a prime number, and consider thep\u21e5 p DFT matrix whose (m,n)th entry is given by e 2\u21e1ip mn. Next, choose the(p+1)\/2 rows of the DFT matrix whose indices are quadratic residues mod p854.1. Paley tight frames for compressed sensing(starting with the row corresponding to m = 0). We denote this (p+1)\/2\u21e5pmatrix by H, which we normalize to obtain the measurement matrix \u0000 :\u0000 := DH,where D is the diagonal matrix whose first diagonal entry isq1p , and therest of its diagonal entries areq2p .Hence, our measurement matrix is a (p + 1)\/2 \u21e5 p matrix with unitnorm columns. For example, for p = 7, we should consider the 7 \u21e5 7 DFTmatrix, and then consider the 1st, 2nd, 3rd, and 5th rows (corresponding tothe quadratic residues m = 0, 1, 2, 4 in Z7), and subsequently normalize theresultant matrix as mentioned above to obtain the 4 \u21e5 7 Paley CS matrix.Using Proposition 4, we compute the inner product between the columnscorresponding to n, n0 2 Zp ash\u0000n,\u0000n0i = 1p +2pp\u000012Xm=1e2\u21e1i(n\u0000n0)p m2=1pp\u00b7p\u00001Xm=0e2\u21e1i(n\u0000n0)p m2=\u21e3n\u0000 n0p\u2318 ipp.Here,\u21e3n\u0000n0p\u2318denotes the Legendre symbol.One way to bound the RIP constants of this construction is using (4.1),which gives\u0000k \uf8ff k \u0000 1pp (4.2)On contrary, numerically, we observe that at least the lower bound on theRIP constant behaves much better. In fact, what we observe in Figure 4.1would be consistent with\u0000k \uf8ff k\u0000pp(4.3)for \u0000 \u21e1 0.65. Note that if (4.3) is proved, then the square-root barrier wouldbe broken. In this chapter, we will show that for this construction, the bound(4.2) can be improved by an additive or a multiplicative constant using twonovel approaches. We will also propose a conjecture regarding distributionof quadratic residues in Zp that leads to (4.3). Next, we explain how weobtain Figure 4.1.Fix a value of p, say p = 103, and consider the Paley CS matrix \u0000 asdefined above. Also, fix a value of k, say k = 30, and choose a signal withrandom support T \u2713 {1, 2, ..., p} and with |T | = k. Let T = {r1, ..., rk}, and864.1. Paley tight frames for compressed sensing101Sparsity level k100RIP constant estimationFigure 4.1: The graph of lower bound of the RIP constants, compared withthe Gershgorin bound and the new improved bound, as given in Section 4.2,on the RIP constants.for each 1 \uf8ff j \uf8ff k, let d(j) be an estimation for the RIP constant of orderj defined byd(j) = max{\u0000max(Gj)\u0000 1, 1\u0000 \u0000min(Gj)}where Gj := \u0000\u21e4Tj\u0000Tj , and Tj := {r1, ..., rj}. The graph of d(j) as a functionof j is shown in Figure 4.1 in log-log scale. In this figure, we also plot theclassical Gershgorin bound as well as the new improved bound (which will bederived in Section 4.2) on the RIP constants. As we observe in this figure andas suggested by the method of least squares, the lower bound function d(j) forthe RIP constants behaves like j\u0000 for \u0000 \u21e1 0.65308. Note that as mentionedabove, d(j) is a lower bound for the RIP constants since it is obtained byusing only one random support set, while the precise value of RIP constantsare obtained by considering the worst case over exponentially many supportsets. Accordingly, we perform another experiment in which we compare thebehaviour of d(j) as defined above obtained from a single random supportset with d0(j) obtained from the worst case of 1000 random support sets. Aswe observe in Figure 4.2, as we increase the number of random support sets,the behaviour of RIP constant estimation remains almost the same. We willuse the estimated value of \u0000 in d(j) = j\u0000 later in this chapter.Remark 12. In the construction used in [7], it is assumed that p \u2318 1.However, as our goal in this chapter is to improve the Greshgorin bound, wewould not be able to do so with the same assumption. The reason is that the874.1. Paley tight frames for compressed sensing12 14 16 18 20 22 24 26 28 30Sparsity level k0.60.650.70.750.80.850.9RIP constant estimationEstimation using single support setEstimation using the worst case in 1000 support setsFigure 4.2: Comparison of lower bounds of the RIP constants obtained froma single random support set and using the worst case among 1000 randomsupport sets. We observe that the slope of the graph (in log-log scale) ob-tained from a single support set almost remains constant, as we increase thenumber of support sets from 1 to 1000.computation above shows that the inner product of nth and n0th columns of\u0000 is given by\u21e3n\u0000n0p\u2318\u00b7 1pp . Now, consider a set T = {r1, ..., rk} such that\u21e3ri\u0000rjp\u2318= 1 whenever i < j. Then, the Gramian matrix will be a k \u21e5 kmatrix of the following formG =26641 1pp \u00b7 \u00b7 \u00b7 1pp...1pp1pp \u00b7 \u00b7 \u00b7 13775The Gershgorin bound for the maximum eigenvalue of this matrix is \u2318 =1 + k\u00001pp . This is in fact the maximum eigenvalue of this matrix sincedet2664\u0000k\u00001pp 1pp \u00b7 \u00b7 \u00b7 1pp...1pp1pp \u00b7 \u00b7 \u00b7 \u0000k\u00001pp3775 = 0This can be verified by adding rows 2, 3, ..., k to the first row, which makesthe first row the zero vector. For this reason, we change the assumption top \u2318 3 mod 4, which as we will see later will lead to improving the Gershgorinbounds.884.2. Improving the Gershgorin bound using skew-adjacency matrices4.2 Improving the Gershgorin bound usingskew-adjacency matricesIn this section, we propose an approach that will enable us to improve theGershgorin bound by a multiplicative constant for the construction given inDefinition 11.We start by considering the construction given in Definition 11, anddecompose the Gramian matrix of order k for this construction, denoted byGk as follows.Gk = (gij) = Ik +Ak (4.4)where Ik is the identity matrix of order k, Ak = (aij), aii = 0, and aij =p\u00001ppor \u0000p\u00001pp for i 6= j.Recall that to compute the RIP constant of order k of the measurementmatrix \u0000\u02dc, we need to consider the Gramian matrices Gmaxk , and Gmink withlargest maximum and smallest minimum eigenvalues respectively (among allGramian matrices of the same order). Decompose these matrices as Gmaxk =I+Amaxk , and Gmink = I+Amink as in (4.4). For any matrix M , let \u0000max(M),\u0000min(M), and \u21e2(M) denote the maximum and minimum eigenvalues of M ,and the spectral radius of M respectively. Then, we have\u0000max(Gmaxk ) = 1 + \u0000max(Amaxk )and\u0000min(Gmink ) = 1 + \u0000min(Amink )On the other hand, each Ak is a skew-symmetric matrix and hence, if \u0000 isan eigenvalue of Ak, then \u0000\u0000 is also an eigenvalue for Ak. This means that\u0000max(Amaxk ) = \u21e2(Amaxk ) and \u0000min(Amink ) = \u0000\u21e2(Amaxk ). Therefore,\u0000k = max{\u0000max(Gmaxk )\u0000 1, 1\u0000 \u0000min(Gmink )} = \u21e2(Amaxk ) (4.5)It remains to find a bound for \u21e2(Amaxk ). Note that Amaxk can be written inthe form ofAmaxk =ippCk, (4.6)where i =p\u00001, and Cmaxk is a skew-symmetric matrix with zero diagonals,and whose every other entry is 1, or -1, and it has the largest spectral radius(among all matrices of the same form). In order to bound the spectral radiusof Cmaxk , we view Cmaxk as the skew adjacency of an oriented graph, and use894.2. Improving the Gershgorin bound using skew-adjacency matricesthe results in the literature about the spectral radius of these matrices tofind bounds on the extreme eigenvalues of Cmaxk . First, we need the followingdefinition.Definition 12. Let G be a simple undirected graph of order n. By G\u0000 wedenote a directed (or oriented) graph that assigns a direction to every edge ofG. The skew-symmetric adjacency matrix of G\u0000, denoted by S(G\u0000) = (sij)is an n\u21e5 n skew symmetric matrix such that si,j = 1 and sj,i = \u00001 if i! jis an arc of G\u0000. If there is no arc between the vertices i and j, we definesi,j = sj,i = 0. The skew spectral radius of G\u0000, denoted by \u21e2S(G\u0000) is definedas spectral radius of S(G\u0000).Now, to find a bound for \u21e2(Amaxk ), we need to consider the skew adjacencyof a simple graph with largest (among all oriented graphs of order k) spectralradius. It turns out [27] that the oriented graph whose skew adjacencymatrix has zero diagonals, whose upper diagonals are all 1, and whose lowerdiagonals are all -1 has the largest spectral radius. In particular, let Kn bethe complete graph of order n, and let K\u2327n be the oriented complete graphwith the adjacency matrix with zero diagonals, with 1\u2019s located in the upperdiagonal entries, and -1\u2019s located in the lower diagonal entries. In otherwords,S(K\u2327n) =266640 1 1 \u00b7 \u00b7 \u00b7 1\u00001 0 1 \u00b7 \u00b7 \u00b7 1...\u00001 \u00001 \u00001 \u00b7 \u00b7 \u00b7 037775Theorem 26. For any oriented graph G\u0000 of order n,\u21e2S(G\u0000) \uf8ff \u21e2S(K\u2327n) = cot(\u21e12n)Equality holds if and only if S(G\u0000) = QTS(K\u2327n)Q for some signed permuta-tion matrix Q.Based on this theorem and using our notation, we can conclude that\u21e2(Cmaxk ) \uf8ff cot(\u21e12k) \uf8ff 2k\u21e1(4.7)where we used the fact that cot(x) \uf8ff 1\/x for x > 0. This inequality comesfrom the standard inequality x < tanx for x > 0. Lastly, by combining(4.5), (4.6), and (4.7) we obtain the following theorem.904.3. Improving the Gershgorin bound using Dembo boundsTheorem 27. Let p \u0000 3 be a prime number. The RIP constant of the matrix\u0000\u02dc as defined in Definition 11 satisfies\u0000k \uf8ff 2\u21e1 \u00b7kppfor any k \uf8ff p.Therefore, the maximum sparsity level k for which we have a guaranteefor recovery through BPDN using this construction must now satisfy2k 0, therefore,k + 1\u0000 1\/c(k + 1)\u0000 k \u0000 1\/ck2\u0000r(k \u0000 1\/ck)24+ k + 1which implies the Lemma.Next, we give the proof for Theorem 29.Proof of Theorem 29. To prove this theorem, we use induction on k. Letp \u2318 3 mod 4 be a prime satisfying the condition of the theorem, and let \u0000\u02dcbe the (p + 1)\/2 \u21e5 p measurement matrix as defined in Definition 11. Let\u0000 \u2713 {1, 2, ..., p}, and by \u0000\u02dc\u0000 we mean an m\u21e5 |\u0000| matrix defined by restrictionof \u0000\u02dc to the columns indexed by the elements of the set \u0000. Define \u0000mink and\u0000maxk as\u0000maxk = max\u0000:|\u0000|\uf8ffk\u0000max(\u0000\u21e4\u0000\u0000\u0000) = \u0000max(\u0000\u21e4\u00000\u0000\u00000),\u0000mink = min\u0000:|\u0000|\uf8ffk\u0000min(\u0000\u21e4\u0000\u0000\u0000) = \u0000min(\u0000\u21e4\u00001\u0000\u00001),(4.12)where \u00000 and \u00001 are sets with k elements, \u0000max(A) and \u0000min(A) denotethe maximum and minimum eigenvalues of a matrix A respectively, andGmaxk := \u0000\u21e4\u00000\u0000\u00000 and Gmink := \u0000\u21e4\u00001\u0000\u00001 denote the Gramian matrices withlargest maximum eigenvalue and smallest minimum eigenvalue respectively.Note that the RIP constant \u0000k of \u0000\u02dc is given by\u0000k = max{1\u0000 \u0000mink ,\u0000maxk \u0000 1} (4.13)In order to prove the theorem, we find an upper bound for the maximumeigenvalue of Gmaxk and a lower bound for the minimum eigenvalue of Gminkrespectively.Proving \u0000max(Gmaxk ) \uf8ff 1+k\u00001\u0000 1c(k\u00001)pp for k \u0000 3: First, note that the resultholds for k = 3 by Lemma 3. Indeed by this lemma, \u0000max(Gmax3 ) \uf8ff 1+p3pp =964.3. Improving the Gershgorin bound using Dembo bounds1 +2\u0000 12cpp for c satisfying12c = 2\u0000p3, i.e., c = 14\u00002p3 .Now assume that the statement is valid for k, then \u0000maxk+1 \uf8ff 1 + k\u00001\/ckpp . Wewill show that \u0000maxk+2 \uf8ff 1 + k+1\u00001\/c(k+1)pp .To bound \u0000maxk+2 in terms of \u0000maxk+1, we will use the Dembo bounds as stated inTheorem 28. In particular, if R is a positive semidefinite Hermitian matrixsuch that R =\uf8ffc b\u21e4b Q\u0000, Q is a (k + 1) \u21e5 (k + 1), positive semidefiniteHermitian matrix, and \u00001 \uf8ff \u00002 . . . \uf8ff \u0000k+2 are eigenvalues of R, then\u0000k+2 \uf8ff c+ \u2318k+12 +r(c\u0000 \u2318k+1)24+ b\u21e4b (4.14)and\u00001 \u0000 c+ \u231812\u0000r(c\u0000 \u23181)24+ b\u21e4b (4.15)for any \u23181 and \u2318k+1 such that \u0000max(Q) \uf8ff \u2318k+1 and \u0000min(Q) \u0000 \u23181. In ourcase, Q is the matrix \u0000\u02dc\u21e4\u00000\u0000\u02dc\u00000 , and R is the matrix \u0000\u02dc\u21e4\u0000\u0000\u02dc\u0000, where \u0000,\u00000 \u2713{1, 2, ..., p} with |\u00000| = k + 1 and \u0000 \u25c6 \u00000 with |\u0000| = k + 2. Say, \u00000 ={j2, j3, ..., jk+2}, and \u0000 = {j1, j2, ..., jk+2}. Then \u0000\u02dc\u0000 = [\u0000j1 ,\u0000j2 , ...,\u0000jk+2 ],and\u0000\u02dc\u21e4\u0000\u0000\u02dc\u0000 =26664h\u0000j1 ,\u0000j1i h\u0000j1 ,\u0000j2i . . . h\u0000j1 ,\u0000jk+2ih\u0000j2 ,\u0000j1i h\u0000j2 ,\u0000j2i . . . h\u0000j2 ,\u0000jk+2i...h\u0000jk+2 ,\u0000j1i h\u0000jk+2 ,\u0000j2i . . . h\u0000jk+2 ,\u0000jk+2i37775so we have c = 1 (as k\u0000jk = 1 for all j), b = [h\u0000j2 ,\u0000j1i h\u0000j3 ,\u0000j1i . . . h\u0000jk+2 ,\u0000j1i]T ,andQ =26664h\u0000j2 ,\u0000j2i h\u0000j2 ,\u0000j3i . . . h\u0000j2 ,\u0000jk+2ih\u0000j3 ,\u0000j2i h\u0000j3 ,\u0000j3i . . . h\u0000j3 ,\u0000jk+2i...h\u0000jk+2 ,\u0000j2i h\u0000jk+2 ,\u0000j3i . . . h\u0000jk+2 ,\u0000jk+2i37775Also, by induction hypothesis we have \u2318k+1 \uf8ff 1 + k\u00001\/ckpp . Hence (4.14)implies that,\u0000k+2 \uf8ff 1 + k \u0000 1\/ck2pp +s(k \u0000 1\/ck)24p+ b\u21e4b974.3. Improving the Gershgorin bound using Dembo boundsOn the other hand, using the fact that each entry of b is \u00b1 ipp we haveb\u21e4b =Pk+1i=11p =k+1p . Therefore, to prove\u0000k+2 \uf8ff 1 +k + 1\u0000 1c(k+1)pp, (4.16)it is enough to prove :k \u0000 1\/ck2pp+s(k \u0000 1\/ck)24p+k + 1p\uf8ff k + 1\u0000 1\/c(k + 1)pp(4.17)This inequality holds by Lemma 4. Next, we observe that to calculate \u0000maxk+2,we have to consider all such matrices R as mentioned above and take maxi-mum over all such choices. In other words,\u0000maxk+2 = max\u0000\u0000k+2(\u0000\u02dc\u21e4\u0000\u0000\u02dc\u0000)However, as it was seen in (4.16), the value of \u0000k+2(\u0000\u02dc\u21e4\u0000\u0000\u02dc\u0000) only depends on|\u0000| = k + 2, and not the elements of \u0000. Therefore, the same upper boundholds for \u0000maxk+2, i.e.,\u0000maxk+2 \uf8ff 1 +k + 1\u0000 1c(k+1)ppProving \u0000mink \u0000 1 \u0000k\u00001\u0000 1c(k\u00001)pp for k \u0000 3: Similar to the argument givenabove, the induction base holds by Lemma 3. Assuming that the statementis valid for (k + 1) (induction hypothesis), we prove it for (k + 2). Usingthe same notation used above, and using Dembo bound (4.15), we can finda lower bound for the minimum eigenvalue of R, \u00001(R), as follows.\u00001 \u0000 1\u0000k \u0000 1\/ck2pp\u0000s(k \u0000 1\/ck)24p+ b\u21e4b = 1\u0000k \u0000 1\/ck2pm\u0000s(k \u0000 1\/ck)24p+k + 1pwhere we used the fact that b\u21e4b = k+1p . Hence, using (4.17), we obtain\u00001 \u0000 1\u0000 k + 1\u0000 1\/c(k + 1)pp(4.18)Again, note that \u0000mink+2 can be calculated by considering all such matrices Rand taking a minimum over them, i.e.,984.4. A generalized Dembo approach\u0000mink+2 = min\u0000\u00001(\u0000\u02dc\u21e4\u0000\u0000\u02dc\u0000)and as seen in (4.18), the value ofmin\u0000 \u00001(\u0000\u02dc\u21e4\u0000\u0000\u02dc\u0000) depends only on |\u0000| = k+2,and not the elements of \u0000. Therefore, the same lower bound holds for \u0000mink+2,i.e.,\u0000mink+2 \u0000 1\u0000k + 1\u0000 1c(k+1)ppTherefore,\u0000k+2 = max{\u0000maxk+2 \u0000 1, 1\u0000 \u0000mink+2} \uf8ffk + 1\u0000 1\/c(k + 1)ppas desired.4.4 A generalized Dembo approachThroughout this chapter, we have focused so far on the bounds on the max-imum and minimum eigenvalues of a Hermitian positive semidefinite matrixA given by Dembo bounds as stated in Theorem 28. These bounds are ob-tained by considering the maximum and minimum eigenvalues of 2\u21e52 blockmatrices R1 and R2 satisfying R1 \u0000 A, and A \uf8ff R2 respectively. In thissection, our goal is to tighten these bounds by following a similar idea. Inparticular, we would like to consider the maximum and minimum eigenvaluesof 3\u21e5 3 block matrices Q1 and Q2 satisfying Q1 \u0000 A, and A \uf8ff Q2, in orderto obtain bounds on the extreme eigenvalues of A.Lemma 5. Suppose that a (k + 2)\u21e5 (k + 2) positive semidefinte Hermitianmatrix R can be written as R =24 a b cb\u21e4 a dc\u21e4 d\u21e4 Q35 where Q is a k \u21e5 k positivesemidefinite Hermitian matrix, and c and d are k \u21e5 1 vectors, and a, b 2 C.Also, suppose that \u00001 \uf8ff \u00002 \uf8ff ... \uf8ff \u0000k+1 \uf8ff \u0000k+2 are the eigenvalues of R.Then \u0000k+2 \uf8ff \u232bmax and \u00001 \u0000 \u232bmin where \u232bmax and \u232bmin are the maximumand minimum roots of characteristic polynomials of R1 =24 a b cb\u21e4 a dc\u21e4 d\u21e4 \u2318kI35994.4. A generalized Dembo approachand R2 =24 a b cb\u21e4 a dc\u21e4 d\u21e4 \u23181I35 respectively. Here, as before, \u23181 is any lowerbound on the minimum eigenvalue of Q and \u2318k is any upper bound on themaximum eigenvalue of Q.Proof. Since Q\u0000 \u23181I \u0000 0, we have R\u0000R2 \u0000 0, and so \u0000min(R) \u0000 \u0000min(R2).Similarly, \u2318kI \u0000Q \u0000 0, implies that \u0000max(R) \uf8ff \u0000max(R1).Next, to estimate the extreme eigenvalues of matrices of the form R1 orR2 mentioned above, we need to obtain a formula for determinant of thesematrices. To that end, we use the so called Schur determinant formula.Lemma 6. [5, p. 50] (Schur determinant formula) Let M be a 2 \u21e5 2 blockmatrix of the formM =\uf8ffP QR S\u0000where P is a p\u21e5 p matrix, S is an s\u21e5 s matrix, Q is a p\u21e5 s matrix, and Ris an s\u21e5 p matrix. If P is invertible, thendet(M) = det(P ) \u00b7 det(S \u0000RP\u00001Q)Similarly, if S is invertible, thendet(M) = det(S) det(P \u0000QS\u00001R)An immediate corollary of the lemma above is that for a 2\u21e5 2 block matrixof the formR =\uf8ffa bc \u2318Ik\u0000where \u2318 6= 0, and b, and c are 1\u21e5 k, and k\u21e5 1 vectors respectively, we havedet(R) = \u2318k(a\u0000 b\u2318\u00001c)Now, we are ready to derive a formula regarding the determinant of the 3\u21e53block matrices of the form mentioned above.Lemma 7. Let R be a (k + 2) \u21e5 (k + 2) matrix that can be written of theform R =24 a b cb\u21e4 a dc\u21e4 d\u21e4 \u2318Ik35, where c = (c1, c2, ..., ck), d = (d1, d2, ..., dk) are1004.4. A generalized Dembo approach1\u21e5k vectors. For each 1 \uf8ff i \uf8ff k, define the vectors ci, and di as 1\u21e5 (k\u00001)vectors obtained by removing the ith entry ci and di from the vectors c andd respectively. Then we havedet(R) = \u2318k\u00002\u21e3a2\u23182 \u0000 a\u2318(dd\u21e4 + cc\u21e4)\u0000 bb\u21e4\u23182 + 2Re(bdc\u21e4) + \u0000\u2318where \u0000 is defined as \u0000 :=Pki=1 |ci|2did\u21e4i \u0000Pki=1 cid\u21e4idic\u21e4i .Note that the determinant of a Hermitian matrix must be a real number.Now, apart from the termPki=1 cid\u21e4idic\u21e4i , other terms are obviously real.This term is also real because if say c`, d` 6= 0, then the term c`d\u21e4`d`c\u21e4` willbe appeared in the sum above. Now, if cm, dm 6= 0, for some m 6= `, thenc\u21e4mdm will be one of the terms appearing in the expansion of d`c\u21e4` . Hence,c`d\u21e4`c\u21e4mdm will be a generic non-zero term of the expansion of c`d\u21e4`d`c\u21e4` . Next,note that this term will be accompanied by cmd\u21e4mc\u21e4`d` which is a genericterm in the expansion of cmd\u21e4mdmc\u21e4m. Thus, every term in this sum will beaccompanied by its complex conjugate, and hence, this term is also real.Proof of Lemma 7. Expanding the determinant along the first row we obtaindet(R) = a det\uf8ffa dd\u21e4 \u2318Ik\u0000\u0000 b det\uf8ffb\u21e4 dc\u21e4 \u2318Ik\u0000+ c1 det2666664b\u21e4 a d2 d3 ... dkc\u21e41 d\u21e41 0 0 ... 0c\u21e42 d\u21e42 \u2318 0 ... 0...c\u21e4k d\u21e4k 0 0 ... \u23183777775\u0000\u0000 c2 det266666664b\u21e4 a d1 d3 . . . dkc\u21e41 d\u21e41 \u2318 0 . . . 0c\u21e42 d\u21e42 0 0 . . . 0c\u21e43 d\u21e43 0 \u2318 ... 0...c\u21e4k d\u21e4k 0 0 . . . \u2318377777775+ . . .+ (\u00001)k+1ck266666664b\u21e4 a d1 d2 d3 . . . dk\u00001c\u21e41 d\u21e41 \u2318 0 0 . . . 0c\u21e42 d\u21e42 0 \u2318 0 . . . 0c\u21e43 d\u21e43 0 0 \u2318 ... 0...c\u21e4k d\u21e4k 0 0 0 . . . 03777777751014.4. A generalized Dembo approachFor the first two terms above, we use Schur determinant formula, and weexpand the remaining terms along the rows with more number of zeros.det(R) = a\u2318k(a\u0000 \u2318\u00001dd\u21e4)\u0000 b\u2318k(b\u21e4 \u0000 \u2318\u00001dc\u21e4) + c1\u21e3\u0000 c\u21e41 det\uf8ffa d1d\u21e41 \u2318Ik\u00001\u0000+ d\u21e41\uf8ffb\u21e4 d1c\u21e41 \u2318Ik\u00001\u0000\u2318\u0000 c2\u21e3c\u21e42\uf8ffa d2d\u21e42 \u2318Ik\u00001\u0000\u0000 d\u21e42 det\uf8ffb\u21e4 d2c\u21e42 \u2318Ik\u00001\u0000\u2318+ . . .+ (\u00001)k+1ck\u21e3(\u00001)k+2c\u21e4k\uf8ffa dkd\u21e4k \u2318Ik\u00001\u0000+ (\u00001)k+3d\u21e4k det\uf8ffb\u21e4 dkc\u21e4k \u2318Ik\u00001\u0000\u2318Next, we use the Schur determinant formula to expand the determinant of2\u21e5 2 block matrices above.det(R) = a2\u2318k \u0000 a\u2318k\u00001dd\u21e4 \u0000 bb\u21e4\u2318k + b\u2318k\u00001dc\u21e4\u0000\u21e3c1c\u21e41\u2318k\u00001a\u0000 \u2318k\u00002c1c\u21e41d1d\u21e41 + c2c\u21e42\u2318k\u00001a\u0000 \u2318k\u00002c2c\u21e42d2d\u21e42 + ...+ ckc\u21e4k\u2318k\u00001a\u0000 \u2318k\u00002ckc\u21e4kdkd\u21e4k\u2318+ c1d1b\u21e4\u2318k\u00001 \u0000 c1d1d1c\u21e41\u2318k\u00002 + . . .+ ckd\u21e4kb\u21e4\u2318k\u00001 \u0000 ckd\u21e4kdkc\u21e4k\u2318k\u00002= \u2318k\u00002 a2\u23182 \u0000 a\u2318dd\u21e4 \u0000 bb\u21e4\u23182 + b\u2318dc\u21e4 \u0000 acc\u21e4\u2318 +kXi=1cic\u21e4idid\u21e4i+ \u2318b\u21e4cd\u21e4 \u0000kXi=1cid\u21e4idic\u21e4i!= \u2318k\u00001 a2\u2318 \u0000 add\u21e4 \u0000 bb\u21e4\u2318 + 2Re(bdc\u21e4)\u0000 acc\u21e4!+ \u2318k\u00002 kXi=1cic\u21e4idid\u21e4i \u0000kXi=1cid\u21e4idic\u21e4i!= \u2318k\u00002\u21e3a2\u23182 \u0000 a\u2318(dd\u21e4 + cc\u21e4)\u0000 bb\u21e4\u23182 + 2Re(bdc\u21e4) + \u21b5\u2318where \u0000 is the expression defined as \u0000 := Pki=1 cic\u21e4idid\u21e4i \u0000Pki=1 cid\u21e4idic\u21e4i!.Using the lemmas proved above, we show that the RIP constant of thePaley CS matrices\u2013 as defined in Definition 11\u2013 can be improved where theimprovement term is a universal constant, unlike the situation in the previoussection (see Theorem 29), where the improvement term was dependent onthe sparsity level.1024.4. A generalized Dembo approachTheorem 30. Let p \u0000 7 be a prime number such that p \u2318 3 mod 4. TheRIP constants of the measurement matrix \u0000\u02dc as given in Definition 11 satisfies\u0000k \uf8ff k \u0000 1\u000023(2\u0000p3)ppProof. The idea of the proof is similar to the one given for Theorem 29, andwe use similar notation. Hence, we assume that the result holds for k whichgives an upper bound and a lower bound for the eigenvalues of a Gramianmatrix of the size k \u21e5 k, and we prove the statement for (k + 2). Therefore,the proof includes two main steps, one regarding the maximum eigenvalue,and one regarding the minimum eigenvalue.1. We will prove that \u0000max(Gmaxk ) \uf8ff 1 +k\u00001\u0000 23 (2\u0000p3\u0000\u21b5)pp for k \u0000 3 usinginduction. To that end, we first verify the statement for k = 3 andk = 4; then we finish by assuming it holds for k, and proving thatthis implies it holds for (k + 2). The induction base (k = 3) holds byLemma 3, since by this lemma,\u0000max3 \uf8ff 1 +p3pp= 1 +2\u0000 (2\u0000p3)pp\uf8ff 1 + 2\u000023(2\u0000p3)pp.The other induction base (k = 4) also holds by Theorem 29. Settingk = 4 in this theorem, we obtain \u00004 \uf8ff 3\u000013cpp , which implies\u0000max4 \uf8ff 1 +3\u0000 13cpp= 1 +3\u0000 23 12cpp= 1 +3\u0000 23(2\u0000p3)ppNext, consider Gmaxk+2, the (k + 2) \u21e5 (k + 2) matrix obtained from theGramian matrix indexed by the set \u0000 = {r1, r2, ..., rk+2} \u2713 {1, 2, ..., p}(with |\u0000| = k + 2,) and we write it in the following form.Gmaxk+2 =24 1 b cb\u21e4 1 dc\u21e4 d\u21e4 Q35 (4.19)Here, b = h\u0000\u02dcr1 , \u0000\u02dcr2i, c = (c1, ..., ck), d = (d1, ..., dk), ci = h\u0000\u02dcr1 , \u0000\u02dcri+2i,di = h\u0000\u02dcr2 , \u0000\u02dcri+2i (with 1 \uf8ff i \uf8ff k). By the construction of \u0000\u02dc, eachnon-diagonal entry of Gmaxk+2 (including b, ci\u2019s and di\u2019s) is \u00b1 ipp . On the1034.4. A generalized Dembo approachother hand, we know by Lemma 5, that the maximum eigenvalue ofGmaxk+2 is bounded from above by the maximum eigenvalue ofBmaxk+2 =24 1 b cb\u21e4 1 dc\u21e4 d\u21e4 \u2318kI35 (4.20)where \u2318k is the upper bound on the maximum eigenvalue of Q and byinduction hypothesis can be written in the form \u2318k = 1 + Dpp , withD = k \u0000 1 \u0000 23(2 \u0000p3). Next, let \u0000 = 1 + Cpp , with C > D + 2 =k + 1 \u0000 23(2 \u0000p3). If we show that p(\u0000) 6= 0, where p(x) is thecharacteristic polynomial ofBmaxk+2 , then this shows that 1+k+1\u0000 23 (2\u0000p3)ppis an upper bound for the maximum eigenvalue of Bmaxk+2 , and hence, forGmaxk+2 as desired. Note that p(x) = det(Bmaxk+2 \u0000 xI) can be written asp(x) = det241\u0000 x b cb\u21e4 1\u0000 x dc\u21e4 d\u21e4 (\u2318k \u0000 x)I35, where each non-diagonal entryis \u00b1i\/pp. Following the notation of Lemma 4, we havep(x) = (\u2318k \u0000 x)k\u00002\u21e3(1\u0000 x))2(\u2318k \u0000 x)2 \u0000 (1\u0000 x)(\u2318k \u0000 x)(dd\u21e4 + cc\u21e4)\u0000(\u2318k \u0000 x)2bb\u21e4 + 2(\u2318k \u0000 x)Re(bdc\u21e4) + \u0000\u2318(4.21)where \u0000 :=Pki=1 |ci|2did\u21e4i \u0000Pki=1 cid\u21e4idic\u21e4i . Next, we prove that\u0000 \u0000 0. Note that each entry of vectors c and d is \u00b1 ipp . Thus,Pki=1 cic\u21e4idid\u21e4i =k(k\u00001)p2 . Also, for each 1 \uf8ff i \uf8ff k, dic\u21e4i contains(k \u0000 1) terms, each with magnitude 1p . Thus,Pki=1 cid\u21e4idic\u21e4i \uf8ff k(k\u00001)p2 .This implies that \u0000 \u0000 0.Furthermore, since C > D, we have \u2318k \u0000 \u0000 < 0. Accordingly, to showp(\u0000) 6= 0, considering the fact that \u0000 \u0000 0, it is enough to show thatq(\u0000) = (1\u0000 \u0000)2(\u2318k \u0000 \u0000)\u0000 (1\u0000 \u0000)(dd\u21e4 + cc\u21e4)\u0000 (\u2318k \u0000 \u0000)bb\u21e4 < 0where we used the fact that Re(bdc\u21e4) = 0. Also, using \u2318k = 1 + Dpp ,\u0000 = 1 + Cpp , c\u21e4c = kp , and d\u21e4d = kp , we haveq(\u0000) =1ppp\u21e3C2(D \u0000 C) + C(2k + 1)\u0000D\u2318(4.22)1044.4. A generalized Dembo approachTo show that q(\u0000) < 0 for any C > D + 2, first let C = D + 2 =k \u0000 1\/3 + 2p3\/3, and recall that D = k \u0000 7\/3 + 2p3\/3. Then,q(\u0000) =1ppp\u21e3\u0000 2C2 + C(2k + 1)\u0000 (k \u0000 7\/3 + 2p3\/3)\u2318=1ppp\u21e3\u0000 2(k + 2p33\u0000 13)2 + (k +2p33\u0000 13)(2k + 1)\u0000 k + 73\u0000 2p33\u2318=1ppp\u21e3\u0000\u0000 23(2p3\u0000 1)\u0000k + 2\u0000 29(2p3\u0000 1)2\u2318\uf8ff 1ppp\u21e3\u0000\u0000 23(2p3\u0000 1)\u0000k + 0.651\u2318 < 0for k \u0000 1.On the other hand, if we substitute D = k \u0000 7\/3 + 2p3\/3, and subse-quently differentiate the right hand side of (4.22), we obtainddC\u21e3C2(k \u0000 7\/3 + 2p3\/3\u0000 C) + C(2k + 1)\u2318= \u00003C2 + 2(k \u0000 7\/3 + 2p3\/3)C + 2k + 1 := g(C) < 0for C > k. This is because g(k) = \u0000k2 + (\u00008\/3 + 4p3\/3)k + 1 < 0,and g(C) is decreasing for C > 2k\u000014\/3+43 (p3)6 . Therefore, q(\u0000) < 0 forevery \u0000 = 1 + Cpp and C > k \u0000 1\/3 + 2p3\/3. Hence, an upper boundfor the maximum eigenvalue of (k+2)\u21e5 (k+2) matrix R2 (and hencefor Gmaxk+2) is \u0000 = 1 +k\u00001\/3+2p3\/3pp , as desired. Note this bound onlydepends on |\u0000|, and not the elements of \u0000.2. We will prove that \u0000min(Gmink ) \u0000 1 \u0000k\u00001\u0000 23 (2\u0000p3)pp for k \u0000 3 usinginduction. The proof is similar to above. For the sake of completeness,we state it briefly. The induction base (k = 3) holds by Lemma 3,since by this lemma,\u0000min3 \u0000 1\u0000p3pp= 1\u0000 2\u0000 (2\u0000p3)pp\u0000 1\u0000 2\u000023(2\u0000p3)ppThe other induction base (k = 4) also holds by Theorem 29: Set k = 41054.4. A generalized Dembo approachin this theorem. Then we obtain \u00004 \uf8ff 3\u000013cpp , which implies\u0000min4 \u0000 1\u00003\u0000 13cpp= 1\u0000 3\u00002312cpp= 1\u0000 3\u000023(2\u0000p3)ppNext, consider the (k + 2)\u21e5 (k + 2) Gramian matrix Gmink+2. We writethis matrix of the form given in (4.19), and we consider the matrixBmink+2 of the form given in (4.20), and with \u2318k replaced by \u23181, namely,the lower bound for the minimum eigenvalue of Q. We write \u23181 of theform \u23181 = 1\u0000D\/pp, and we consider \u0000 := 1\u0000C\/pp with C > D+2.We will consider p(x), the characteristic polynomial of Bmink+2, and willshow that p(\u0000) 6= 0. To do so, it is enough to show thatq(\u0000) = (1\u0000 \u0000)2(\u2318k \u0000 \u0000)\u0000 (1\u0000 \u0000)(dd\u21e4 + cc\u21e4)\u0000 (\u2318k \u0000 \u0000)bb\u21e4 > 0The expression for q(\u0000) can be simplified asq(\u0000) =1ppp\u21e3C2(C \u0000D)\u0000 C(2k + 1) +D\u2318(4.23)Now, let C = k \u0000 1\/3 + 2p3\/3, and recall that D = k \u0000 7\/3 + 2p3\/3.Next,q(\u0000) =1ppp\u21e32C2 \u0000 C(2k + 1) + (k \u0000 7\/3 + 2p3\/3)\u2318\u0000 1ppp\u21e3\u000023(2p3\u0000 1)\u0000k \u0000 0.651\u2318 > 0for k \u0000 1.On the other hand, if we substitute D = k \u0000 7\/3 + 2p3\/3, and subse-quently differentiate the right hand side of (4.23), we will getddC\u21e3C2(\u0000k + 7\/3\u0000 2p3\/3 + C)\u0000 C(2k + 1)\u2318= 3C2 \u0000 2(k \u0000 7\/3 + 2p3\/3)C \u0000 2k \u0000 1 := g(C) > 0for C > k. This is because g(k) = k2 + (8\/3 \u0000 4p3\/3)k \u0000 1 > 0,and g(C) is increasing for C > 2k\u000014\/3+43 (p3)6 . Therefore, q(\u0000) > 0 for1064.5. A path to break the square-root barrier using Dembo boundsevery \u0000 = 1\u0000 Cpp and C > k\u0000 1\/3+ 2p3\/3. Hence, a lower bound forthe minimum eigenvalue of (k+2)\u21e5 (k+2) matrix R2 (and hence forGmink+2) is \u0000 = 1 \u0000 k\u00001\/3+2p3\/3pp , as desired. Note that this bound alsoonly depends on |\u0000|, and not the elements of \u0000.Considering the calculations we did for the maximum and minimumeigenvalues of the Gramian matrices R1 and R2, we conclude that theRIP constant of order (k + 2) satisfies\u0000k+2 \uf8ff k \u0000 1\/3 + 2p3\/3ppproving the theorem using induction.4.5 A path to break the square-root barrier usingDembo boundsIn this section, we propose an approach that can lead to breaking the square-root barrier for the construction given in Definition 11, if a specific conjec-ture regarding the distribution of quadratic residues holds. Our approach isbased on the generalized Dembo bounds as derived and explained in section4.4. Let \u0000 denote the measurement matrix as defined in Definition 11. Wesaw in Section 4.2 that if all upper diagonal entries of the Gramian matrixG = \u0000\u21e4T\u0000T , corresponding to the index set T = {r1, r2, ..., rk}, are i\/pp(and all the lower elements are \u0000i\/pp) then we would have a multiplicativeimprovement for Gershgorin bound but the square root barrier can not bebroken. Therefore, in such case,\u21e3r1 \u0000 r3p\u2318=\u21e3r1 \u0000 r4p\u2318= ... =\u21e3r1 \u0000 rkp\u2318= 1,\u21e3r2 \u0000 r3p\u2318=\u21e3r2 \u0000 r4p\u2318= ... =\u21e3r2 \u0000 rkp\u2318= 1If we re-tag the columns as r01 = rk, r02 = rk\u00001, ..., r0k := r1, then\u21e3r01 \u0000 r03p\u2318=\u21e3r01 \u0000 r04p\u2318= ... =\u21e3r01 \u0000 r0kp\u2318= \u00001,\u21e3r02 \u0000 r03p\u2318=\u21e3r02 \u0000 r04p\u2318= ... =\u21e3r02 \u0000 r0kp\u2318= \u000011074.5. A path to break the square-root barrier using Dembo boundsIn either case, we have Xi2I\u0000(i)\u0000(i+ a) = |I|where \u0000(x) =\u21e3xp\u2318denotes the Legendre symbol (and hence, is a Dirichletcharacter), I := {r1\u0000 r3, r1\u0000 r4, ..., r1\u0000 rk}, and a = r2\u0000 r1. Therefore, onecan hope that if the opposite to this situation occurs in the following sense,then the square-root barrier may be broken.Conjecture 1. There exists constants 0 < \u21b5 < 1, and \u232b > 1\/2, and apositive integer m\u21b5 such that for any set {r1, ..., rk} in Zp, with m\u21b5 \uf8ff k \uf8ff p\u232bthere exist indices 1 \uf8ff i < j \uf8ff k satisfying the following inequality|P`2Iri,rj \u0000(`)\u0000(`+ a)||Iri,rj |< \u21b5where \u0000(x) =\u21e3xp\u2318, a = rj\u0000ri, and Iri,rj := {ri\u0000r` : 1 \uf8ff ` \uf8ff k, ` 6= i, ` 6= j}.We call Iri,rj a one-sided difference set.Note that this conjecture is not just based on what is needed to breakthe square-root barrier, but also based on a similar result already known inNumber Theory. We briefly mention this result as stated in [75].Let G be a finite (additive) group, and let D be a subset of G (called adifference set) with k elements such that every non-zero element of G canbe uniquely written as d1 \u0000 d2. Then,|Xd2D\u0000(d)| = pk \u0000 1Hence, for any 0 < \u21b5 < 1, we have|Pd2D \u0000(d)||D| =pk \u0000 1k< \u21b5provided that k is sufficiently large. We also verify this conjecture numeri-cally with few examples.Set \u21b5 := 0.8, and m\u21b5 := 5. The first prime satisfying p \u0000 m\u21b5 is p = 5in which case we should start with a set with 5 elements. We only have onechoice, and that is A = Z5 (or any permutation of it). Setr1 = 0, r2 = 1, r3 = 2, r4 = 3, r5 = 41084.5. A path to break the square-root barrier using Dembo boundsThen,\u21e3r1 \u0000 r3p\u2318\u21e3r2 \u0000 r3p\u2318= \u00001,\u21e3r1 \u0000 r4p\u2318\u21e3r2 \u0000 r4p\u2318= 1,\u21e3r1 \u0000 r5p\u2318\u21e3r2 \u0000 r5p\u2318= \u00001Thus,|P5i=3 \u21e3 r1\u0000rip \u2318\u21e3 r2\u0000rip \u2318|3= 1\/3 < \u21b5We also verify for p = 19. As we know, we should start with a set with at least5 elements, say, we start with a set with 12 elements. We choose a randomsupport set T with 12 elements, say T = {8, 15, 5, 13, 10, 2, 17, 4, 1118, 16, 19}(i.e., r1 = 8, r2 = 15, ..., r12 = 19). Then, among 10 elements of the sequence{\u21e3r1\u0000r3p\u2318\u00b7\u21e3r2\u0000r3p\u2318, ....,\u21e3r1\u0000r012p\u2318\u00b7\u21e3r2\u0000r12p\u2318}, we have three -1\u2019s and seven 1\u2019s.Hence,|P12i=3 \u21e3 r1\u0000rip \u2318\u21e3 r2\u0000rip \u2318|10= 0.4 < \u21b5In the next experiment, we again work with p = 19, but we try to considerthe worst case (that could potentially fail the conjecture). Such case wouldoccur if we choose a set {r1, ..., rk} such that\u21e3ri\u0000rjp\u2318= 1 whenever i < j.In particular, we can choose T := {1, 2, 18, 16, 15, 14, 8, 7, 6, 4}. Then, all10 elements of the sequence {\u21e3r1\u0000r3p\u2318\u00b7\u21e3r2\u0000r3p\u2318, ....,\u21e3r1\u0000r12p\u2318\u00b7\u21e3r2\u0000r12p\u2318} are1\u2019s which is opposite to what we need. However, we can simply chooser01 := r7 = 8, and r02 = r8 = 7, {r03, ..., r010} = T \\ {7, 8} (the order isirrelevant). Then, we would have|P12i=3 \u21e3 r1\u0000rip \u2318\u21e3 r2\u0000rip \u2318|10=210< \u21b5Therefore, in all these experiments the conjecture was verified for \u21b5 = 0.8(in these cases, we could even choose a smaller \u21b5, e.g., \u21b5 = 0.5).Now, based on this conjecture, we prove that the square-root barrier canbe broken for the construction given in Definition 11. In the following, weprovide a proof using induction on k. For the induction base, we need to usea value for the power \u0000 in \u0000k \uf8ff k\u0000pp . Since numerical experiments (see Figure4.1) suggests \u0000 < 0.7, we use \u0000 = 0.7 as it seems that this is the value thatworks for any k-value.1094.5. A path to break the square-root barrier using Dembo boundsProposition 11. (breaking the square-root barrier). Suppose Conjecture 1holds with \u21b5, \u232b,m\u21b5 as defined in the statement of this conjecture. Let \u0000 = 0.7,and let c\u21b5 be a fixed integer such that the following inequality holds:12c1+\u0000\u21b5 < (1\u0000 \u21b5)c2\u21b5 \u0000 2c\u21b5,Also, let b\u21b5 = max{m\u21b5 + 2, c\u21b5 + 2}, and suppose that p \u0000 b2\u21b5. Then, for theconstruction given in Definition 11, and for k \uf8ff min{p\u232b2 , p12\u00002 }, we have\u0000k < 1\/p2Hence, the square-root barrier would be broken for this construction.Remark 15. Here, we make the above statement more concrete. Set \u21b5 =0.8,m\u21b5 = 5, and \u232b = 0.8. Then, c\u21b5 is the smallest integer satisfying12x1.7 \uf8ff 0.2x2 \u0000 2xNumerically we check that we can set c\u21b5 = 899, 998, and this gives b\u21b5 =900, 000. So the proposition above then reduces to:\u201cSuppose Conjecture 1 holds with the values mentioned above, and letp \u2318 3 mod 4 be a prime number satisfying p \u0000 81 \u21e5 1010, and considerthe construction given in Definition 11. Then, the RIP constants of suchconstruction satisfy \u00002k < 1\/p2 for any k < p5\/72 \".Proof of Proposition 11. First, let k \uf8ff b\u21b5. Then, k \uf8ff pp, and hence, byTheorem 27, the RIP constant of the measurement matrix \u0000\u02dc satisfies\u0000k \uf8ff 2\u21e1kpp\uf8ff 2\u21e1<1p2as desired. Next, let k = k0 \u0000 b\u21b5 be an arbitrary integer (with k0 \uf8ff p\u232b),and we prove that\u0000k \uf8ff k\u0000pp(4.24)holds for k = k0 using induction. To do that, first verify (4.24) for k = b\u21b5 \u00001, b\u21b5 \u0000 2 numerically (induction base). Now, since k0 \u0000 m\u21b5, by Conjecture1, there exist indices 1 < i < j \uf8ff k0 such that if we set r01 = ri, r02 = rj , then\u0000\u0000\u0000Pk0`=3 \u21e3 r01\u0000r0`p \u2318\u21e3 r02\u0000r0`p \u2318\u0000\u0000\u0000k0 \u0000 2 < \u21b5 (4.25)1104.5. A path to break the square-root barrier using Dembo boundswhere r03, ..., r0k0 are the elements of the set {r1, ..., rk0} \\ {r01, r02} (the orderis irrelevant). In the next step, we consider I3 := {r03, r04, ..., r0k0}, and ifk0 \u0000 2 = |I3| \u0000 m\u21b5, then we apply Conjecture 1 again, and after possibly apermutation, we can assume that\u0000\u0000\u0000Pk0`=5 \u21e3 r03\u0000r0`p \u2318\u21e3 r04\u0000r0`p \u2318\u0000\u0000\u0000k0 \u0000 4 < \u21b5. (4.26)Continuing this process, in the last step we reach\u0000\u0000\u0000Pk0`=k0\u0000b\u21b5+2 \u21e3 r0k0\u0000b\u21b5\u0000r0`p \u2318\u21e3 r0k0\u0000b\u21b5+1\u0000r0`p \u2318\u0000\u0000\u0000b\u21b5 \u0000 1 < \u21b5 (4.27)if k0 \u2318 b\u21b5 + 1 mod 2, and\u0000\u0000\u0000Pk0`=k0\u0000b\u21b5+3 \u21e3 r0k0\u0000b\u21b5+1\u0000r0`p \u2318\u21e3 r0k0\u0000b\u21b5+2\u0000r0`p \u2318\u0000\u0000\u0000b\u21b5 \u0000 2 < \u21b5 (4.28)if k0 \u2318 b\u21b5 mod 2. Now, let \u0000maxk0 and \u0000mink0 denote the maximum and minimumeigenvalues of the Gramian matrices of order k0 with the largest maximumeigenvalue and the smallest minimum eigenvalues respectively. We prove thetheorem using two main steps.Step 1: Proving \u0000maxk0 \uf8ff 1 +k\u00000pp . As mentioned above, our inductionbase consists of verifying (4.24) for k = b\u21b5 \u0000 1, b\u21b5 \u0000 2 numerically. Next, weconsider two cases.Case (I): k0 \u2318 b\u21b5 mod 2. Our goal is to estimate the eigenvalues of amatrix of the form G = Gmaxk0 = \u0000\u21e4T\u0000T , with T = {r1, r2, ..., rk0}. Afterpossibly a proper permutation, we can assume that T = {r01, ..., r0k0} is suchthat all of (4.25), (4.26),...,(4.28) hold. Now, let T1 = {rk0\u0000b\u21b5+3, ..., rk0},k1 := b\u21b5\u00002, G1 = \u0000\u21e4T1\u0000T1 (G1 is obtained by considering the last k1 = b\u21b5\u00002rows and columns of G). We start by estimating eigenvalues of this matrix.By induction hypothesis, an upper bound for the largest eigenvalue of G1 isgiven by\u2318k1 = 1 +k\u00001ppNow, we show that\u2318k2 = 1 +(k1 + 2)\u0000pp1114.5. A path to break the square-root barrier using Dembo bounds(with k2 = k1 + 2) is an upper bound for the maximum eigenvalue of theGramian matrix G2 obtained by considering the last k1+2 rows and columnsof G. In order to do that, we write G2 in the formG2 =24 1 b cb\u21e4 1 dc\u21e4 d\u21e4 G135with c = (c1, ..., ck), d = (d1, ..., dk), ci = h\u0000\u02dcrk0\u0000k1\u00001 , \u0000\u02dcrk0\u0000k1+ii, and di =h\u0000\u02dcrk0\u0000k1 , \u0000\u02dcrk0\u0000k1+ii (with 1 \uf8ff i \uf8ff k1). Note that each of the entries b, ci\u2019sand di\u2019s are \u00b1 ipp . (similar to what we did in (4.19)). We also define B2 viaB2 =24 1 b cb\u21e4 1 dc\u21e4 d\u21e4 \u2318k135 (4.29)where \u2318k1 = 1 + Dpp , with D = k\u00001 , and we let \u0000 = 1 + C\/pp with C >(k1 + 2)\u0000 . We will show that p(\u0000) 6= 0, where p(x) is the characteristicpolynomial of B2. First, we write the expression for p(x) as given in (4.21).p(x) = (\u2318k \u0000 x)k\u00002\u21e3(1\u0000 x))2(\u2318k \u0000 x)2 \u0000 (1\u0000 x)(\u2318k \u0000 x)(dd\u21e4 + cc\u21e4)\u0000(\u2318k \u0000 x)2bb\u21e4 + 2(\u2318k \u0000 x)Re(bdc\u21e4) + \u0000\u2318Using Re(bdc\u21e4) = 0, c\u21e4c = d\u21e4d = k1p , and \u2318k \u0000 \u0000 6= 0, we observe that toshow p(\u0000) 6= 0, it is enough to showq(C) := C2(D \u0000 C)2 \u0000 (\u0000C)(D \u0000 C)(2k1)\u0000 (D \u0000 C)2 + \u0000 > 0 (4.30)i.e.,q(C) = C2(C \u0000D)2 \u0000 C(C \u0000D)(2k1)\u0000 (C \u0000D)2 + \u0000 > 0In order to show this inequality, we show thatC(C \u0000D)(2k1) + (C \u0000D)2 < \u0000 (4.31)where we used the fact that \u0000 \u0000 0. We call the left hand side and right handside of the inequality above as LHS and RHS respectively. Next,LHS = (C \u0000D)\u21e32k1C + (C \u0000D)\u2318= (C \u0000D)\u21e3(2k1 + 1)C \u0000D\u23181124.5. A path to break the square-root barrier using Dembo boundsOur goal is to prove (4.31) for any C > (k1+2)\u0000 , but we start by consideringC = (k1 + 2)\u0000 . Then,C \u0000D = (k1 + 2)\u0000 \u0000 k\u00001 < 2where we used the fact that if we set f(x) = ax \u0000 (a \u0000 2)x, then f(1) = 2,and f(x) is increasing for 0 \uf8ff x \uf8ff 1. Hence,LHS \uf8ff 2\u21e3(2k1 + 1)(k1 + 2)\u0000 \u0000 k\u00001\u2318< 2\u21e3(3k1)(2k1)\u0000\u2318< 12k1+\u00001where we used the fact that 2k1 + 1 < 3k1, and k1 + 2 < 2k1.On the other hand,RHS = \u0000 =k1Xi=1cic\u21e4idid\u21e4i \u0000k1Xi=1cid\u21e4idic\u21e4i = k1(k1 \u0000 1)\u0000k1Xi=1cid\u21e4idic\u21e4iIn above, ci, di are vectors c and d excluding their ith entry (so they are (k1\u00001)-dimensional vectors). Now, we find an upper bound forPk1i=1 cid\u21e4idic\u21e4i :k1Xi=1cid\u21e4idic\u21e4i \uf8ffk1Xi=1|d\u21e4i ci| \uf8ff k1maxi|d\u21e4i ci|On the other hand, for each 1 \uf8ff j \uf8ff k1, by (4.28), we have|d\u21e4jcj | \uf8ff\u0000\u0000\u0000 k0Xi=k0\u0000k1+1\u21e3r0k0\u0000k1\u00001 \u0000 r0ip\u2318\u21e3r0k0\u0000k1 \u0000 r0ip\u2318\u0000\u0000\u0000+ 1 \uf8ff \u21b5k1 + 1Thus,\u0000 \u0000 k1(k1 \u0000 1)\u0000 \u21b5k21 \u0000 k1 = (1\u0000 \u21b5) \u00b7 k21 \u0000 2k1and hence, \u0000 \u0000 12k1+\u00001 . In above, we used the fact that for k \u0000 c\u21b5, we have12k1+\u0000 \uf8ff (1\u0000\u21b5)k2\u0000 2k. Therefore, q(C) > 0 for C = (k1+2)\u0000 . Next, notethat the value of \u0000 does not depend on C, and note that C\u0000D is increasing asa function of C. Hence, if we show that g(C) := C2(C\u0000D)\u00002k1C\u0000(C\u0000D)is an increasing function of C, then we can conclude that q(C) > 0 forC > (k1 + 2)\u0000 . Now, we haveddCg(C) = 3C2 \u0000 2k1 \u0000 2CD \u0000 1 = 3C2 \u0000 2C \u00b7 k\u00001 \u0000 2k1 \u0000 1 > 01134.5. A path to break the square-root barrier using Dembo boundsfor C \u0000 (k1 + 2)\u0000 . To justify the last inequality, we define h(C) := 3C2 \u00002C \u00b7 k\u00001 \u0000 2k1 \u0000 1, and we verify that h\u21e3(k1 + 2)\u0000\u2318> 0, and ddCh(C) > 0 forC > (k1 + 2)\u0000 :h\u21e3(k1 + 2)\u0000\u2318= 3(k1 + 2)2\u0000 \u0000 2(k21 + 2k1)\u0000 \u0000 2k1 \u0000 1= 3(k21 + 4k1 + 4)\u0000 \u0000 2(k21 + 2k1)\u0000 \u0000 2k1 \u0000 1\u0000 3(k21 + 2k1)\u0000 \u0000 2(k21 + 2k1)\u0000 \u0000 2k1 \u0000 1 = (k21 + 2k1)\u0000 \u0000 2k1 \u0000 1 > 0,for k1 \u0000 4. We also haveddC(3C2 \u0000 2Ck\u00001 \u0000 2k1 \u0000 1) = 6C \u0000 2k\u00001 > 0for C > k\u00001 \/3. Therefore, if we set k2 := k1+2,we have shown that 1+(k2)\u0000pp isan upper bound for the maximum eigenvalue of G2 = \u0000T2\u0000T2 , with T2 beingthe set containing the last k2 entries of T , i.e., T2 = T1[{rk\u0000b\u21b5+2, rk\u0000b\u21b5+1}.After repeating a similar reasoning mentioned above, we can conclude that1+k\u00003pp is an upper bound for G3 = \u0000\u21e4T3\u0000T3 , with k3 := k2 +2, and T3 beingthe set containing the last k3 elements of T . By continuing this process, weconclude that in the last step, 1 + k\u0000p`p is an upper bound for the maximumeigenvalue of G` = G = \u0000T`\u0000T` , where k` = k0, and T` = T .Case (II). k0 \u2318 b\u21b5+1 mod 2. In this case, to find an upper bound for themaximum eigenvalue ofG = Gmaxk0 = \u0000\u21e4T\u0000T (again with T = {r1, r2, ..., rk0}),we begin with estimating the eigenvalues of G1 := \u0000\u21e4T1\u0000T1 , with T1 being theset containing the last k1 := b\u21b5 \u0000 1 elements of T . By induction hypothesis,the upper bound for the largest eigenvalue of G1 is given by1 +k\u00001ppNext, it can be shown (similar to above) that 1 + k\u00002pp is an upper bound forthe largest eigenvalue of G2 = \u0000\u21e4T2\u0000T2 , with k2 := k1+2, and T2 being the setcontaining the last k2 entries of T . Continuing this process, we conclude thatthe 1+ k\u0000p`p is an upper bound for the largest eigenvalue of G = G` = \u0000\u21e4T`\u0000T` ,with k` = k0 and T` = T .Step 2: Proving \u0000mink0 \u0000 1 \u0000k\u00000pp . The proof of this step is similar tothe one given for Step 1 (and consists of two cases). In each case, we startby concluding from induction hypothesis that \u23181,k1 := 1\u0000 Dpp , with D = k\u000011144.5. A path to break the square-root barrier using Dembo boundsis a lower bound for the minimum eigenvalue of G1 = \u0000\u21e4T1\u0000T1 . Then, welet \u0000 = 1 \u0000 Cpp with C > D + 2, and we show that p(\u0000) 6= 0, where p(x)is the characteristic polynomial of the matrix B02 (obtained from the matrixB2 as given in (4.29), but \u2318k1 replaced by \u23181,k1). To do so, it is enough toshow (4.30) holds, which we already know its validity as proved in Step 1.Therefore, \u23181,k2 = 1\u0000 k\u00002pp (with k2 = k1+2) is a lower bound for the minimumeigenvalue of B02 (and hence the minimum eigenvalue of G2). Continuing thisprocess, after ` steps, we conclude that \u23181,k` = 1 \u0000 k\u0000p`p is a lower bound forthe minimum eigenvalue of G` = G = \u0000\u21e4T`\u0000T` , with k` = k0, and T` = T .Gathering the results proved in Steps 1 and 2, we conclude that the RIPconstants of \u0000\u02dc satisfy\u0000k = max{\u0000maxk \u0000 1, 1\u0000 \u0000mink } \uf8ffk\u0000ppfor b\u21b5 \uf8ff k \uf8ff p\u232b . Therefore, \u0000k < 1\/p2 for k \uf8ff min{p\u232b2 , p12\u00002 }, as desired.115Chapter 5Concluding remarksCompressed sensing (CS) is a sampling paradigm in which signals are consid-ered as vectors in Rn, for a large n, and are generally assumed to be sparseor compressible with respect to a basis or frame. Under such assumption,the goal of CS is to recover the original signal with seemingly insufficientnumber of measurements using a proper measurement matrix and an effi-cient algorithm. The standard results of CS indicate that certain randommatrices can be used as measurement matrices, and they satisfy \u201cRestrictedIsometry Property\" (RIP) with high probability, for optimal number of mea-surements. Using these matrices comes with one caveat: storing these ma-trices are costly and for this reason certain deterministic matrices have beenconsidered. Using these matrices also comes with one significant caveat: inorder for these matrices to satisfy RIP, the number of measurements mustsatisfy m = O(k2), i.e., k = O(pm). This is called the \u201csquare-root barrier\"for deterministic matrices in CS, which in fact is derived by combining theso-called Welch bound with Gershgorin circle theorem.In Chapter 3, we proposed a novel deterministic construction which hasthe advantage of being partial circulant, binary, and each entry of this con-struction has a simple and explicit formula. This means that these matricesare easy to store, and also using FFT, these matrices come with fast matrix-vector multiplication, and a fast reconstruction algorithm. We proved thatthese matrices have small coherence (and hence, can be used as CS matri-ces). In addition, we provided numerical experiments comparing our pro-posed construction with some existing deterministic constructions. In fact,we observed that our construction has a better performance compared to thedeterministic constructions considered here, and has a performance compa-rable with random Bernoulli matrices.In today\u2019s digital world, quantizing the measurement vector is a crucialstep in the sampling process, which was mostly ignored in early literatureof CS. One known efficient method of quantization in CS is a method calledrth-order \u2303\u0000 quantization, which was accompanied with a one-stage recon-struction method. This method was shown to be robust respect to noiseand stable respect to compressible signals, but came with one caveat: it116Chapter 5. Concluding remarkswas applied only for the class of sub-Gaussian matrices. In Chapter 2, weproposed two novel approaches to generalize this method to random restric-tions of bounded orthonormal systems, such as random restrictions of DFTmatrices (which are of high importance due to the applications in MRI). Wealso generalized this method to certain class of deterministic measurementmatrices, namely, certain submatrices of chirp sensing matrices. For eachof these cases, we provided numerical experiments confirming the boundsderived for the errors in approximation.In CS, the performance of a measurement matrix is normally judged bythe estimates for its RIP constants, since calculating the exact values of RIPconstants is an NP-hard problem\u2013 at least in a vast regime for number ofmeasurements m (vs. the sparsity level k). A common bound for RIP con-stants of a matrix, namely, \u0000k \uf8ff (k \u0000 1)\u00b5 is derived by applying Gershgorincircle theorem on Gramian matrices. However, one should note that Gersh-gorin circle theorem estimates the eigenvalues of a matrix uniformly, whilethe RIP constants depend only on the maximum and minimum eigenval-ues of the Gramian matrices. Furthermore, Gershgorin circle theorem canbe applied to any square matrix, and hence it does not use the fact thatthe Gramian matrices are positive semidefinite. In Chapter 4, we deployedthe so-called Dembo bounds, which estimate the maximum and minimumeigenvalues of a positive semidefinite matrix, to improve the classical bound\u0000k \uf8ff (k \u0000 1)\u00b5 by an additive constant for the so-called Paley tight frames.However, we showed that this method has a great potential in general. Infact, we showed that if a particular conjecture regarding the distribution ofquadratic residues holds, then we can generalize Dembo bounds to break thesquare-root barrier via k = O(m5\/7). We substantiated this conjecture bynumerical experiments, and we also theoretically discussed it. Furthermore,we used the notion of skew-symmetric adjacency matrices and a recent (2018)result regarding a bound on the spectral radius of an oriented graph to derivea multiplicative constant improvement on the classical bound \u0000k \uf8ff (k \u0000 1)\u00b5for the Paley tight frames. In particular, we showed that the maximumsparsity level satisfies 2k < \u21e12 \u00b7 1\u00b5p2 (opposed to 2k < 1\u00b5p2 + 1).117Bibliography[1] A. Amini and F. Marvasti. Deterministic construction of binary, bipolarand ternary compressed sensing matrices. Information Theory, IEEEtransactions on, 57(4):2360\u20132370, 2011.[2] L. Applebaum, S. D. Howard, S. Searle, and R. Calderbank. Chirp sens-ing codes: Deterministic compressed sensing for fast recovery. Appliedand Computational Harmonic Analysis, 26(2):283\u2013290, 2009.[3] W. Bajwa, J. Haupt, G. Raz, and R. Nowak. Compressed channelsensing. Proc. of Conf. on Information Sciences and Systems, 2008.[4] W. Bajwa, J. Haupt, G. Raz, S. Wright, and R. Nowak. Toeplitz-structured compressed sensing matrices. IEEE Workshop SSP., 2007.[5] T. Banachiewicz. Zur berechnung der determinanten, wie auch der in-versen, und zur darauf basierten auflosung der systeme linearer gle-ichungen. Acta Astronomica, Serie C(3):41\u201367, 1937.[6] A. Bandeira, M. Fickus, D. G. Mixon, and J. Moreira. Derandomizngrestricted isometries via the legendre symbol. Constructive Approxima-tion, June 2014.[7] A.S. Bandeira, M. Fickus, D. Mixon, and P. Wong. The road to de-terministic matrices with the restricted isometry property. Journal ofFourier Analysis and Applications, 19:1123\u20131149, 2013.[8] H. E. Bell. Gerschgorin\u2019s theorem and the zeros of polynomials. Amer.Math. Monthly, 72:292\u2013295, 1965.[9] J. Benedetto, A. Powell, and O. Yilmaz. Sigma-delta quantization andfinite frames. IEEE International Conference on Acoustics, 52(5):1990\u20132005, 2004.[10] J. Benedetto, A. Powell, and O. Yilmaz. Second-order sigma-deltaquantization of finite frame expansions. Applied and ComputationalHarmonic Analysis, 20(1), 2006.118Bibliography[11] J. Blum, M. Lammers, A. Powell, and O. Yilmaz. Sobolev duals in frametheory and sigma-delta quantization. Journal of Fourier Analysis andApplications, 16(3):365\u2013381, 2010.[12] B. Bodmann and V. Paulsen. Frames, graphs, and erasures. LinearAlgebra and its Applications, 404, 2005.[13] P. Boufounos, L. Jacques, F. Krahmer, and R. Saab. Quantization andcompressed sensing, chapter in \u201cCompressed sensing and its applica-tions\" (edited by H. Boche, R. Calderbank, G. Kutyniuk, J. Vybiral).Springer, 2015.[14] P. T. Boufounos and R. G. Baraniuk. One-bit compressive sensing.Proc. of Conf. on Information Sciences and Systems, March 2008.[15] J. Bourgain, S. Dilworth, K. Ford, S. Konyagin, and D. Kutzarova.Breaking the k2 barrier for explicit rip matrices. STOC, 2011.[16] J. Bourgain, S. Dilworth, K. Ford, S. Konyagin, and D. Kutzarova.Explicit constructions of rip matrices and related problems. Duke Math,159(1):145\u2013185, 2011.[17] J. Bourgain and A. Glibichuk. Exponential sum estimate over subgroupin an arbitrary finite field. Journal d\u2019Analyse Mathematique, 115(1):51\u201370, 2011.[18] T. T. Cai and A. Zhang. Sparse representation of a polytope and re-covery of sparse signals and low-rank matrices. IEEE Transactions onInformation Theory, 60(1), January 2014.[19] E. Candes, J. Romberg, and T. Tao. Robust uncertainty principle: Ex-act signal reconstruction from highly incomplete frequency information.IEEE Transactions on Information Theory, 52(2):489\u2013509, 2006.[20] E. Candes, J. Romberg, and T.Tao. Stable signal recovery from in-complete and inaccurate measurements. Communications on pure andApplied Mathematics, 59(8):1207\u20131223, August 2006.[21] E. Candes and T. Tao. Near-optimal signal recovery from random pro-jections: universal encoding strategies ? IEEE Transactions on Infor-mation Theory, 52:5406\u20135425, 2006.[22] E. Candes and T. Tao. Decoding by linear programming. InformationTheory, IEEE transactions on, 51(12), December 2005.119Bibliography[23] Z. Chen and J. Dongarra. Condition numbers of gaussian random ma-trices. SIAM J. Matrix Anal. and Appl., 27(3):603\u2013620, 2005.[24] E. Chou, S. Gunturk, F. Krahmer, R. Saab, and O. Yilmaz. Noise-shaping quantization methods for frame-based and compressive samplingsystems, chapter 4 in \"Sampling Theory, A Renaissance\" (edited by G.Pfander). Birkhauser, Boston, 2015.[25] X. Cui. Construction of deterministic measurement matrices using dec-imated legendre sequence. MATEC Web of conferences, page 22, 2015.[26] A. Dembo. Bounds on the extreme eigenvalues of positive-definitetoeplitz matrices. Information Theory, IEEE transactions on,34(2):352\u2013355, 1988.[27] B. Deng, X. Li, B. Shader, and W. So. On the maximum skew spectralradius and minimum skew energy of tournaments. Linear and Multilin-ear Algebra, 66(7), 2018.[28] R. DeVore. Deterministic cinstructions of compressed sensing matrices.Journal of Complexity, 23(4-6):918\u2013925, 2007.[29] S. Dirksen, H. Jung, and H. Rauhut. One-bit compressed sensing withpartial gaussian circulant matrices. ArXiv preprint ArXiv: 1710.03287,2017.[30] D. Donoho. Compressed sensing. IEEE Transactions on Signal Pro-cessing, 52(4):1289\u20131306, 2006.[31] D. Donoho, M. Elad, and V. Telmyakov. Stable recovery of sparse over-complete representations in the presence of noise. Information Theory,IEEE transactions on, 52(1):6\u201318, January 2006.[32] J. Feng, F. Krahmer, and R. Saab. Quantized compressed sensing forpartial random circulant matrices. 2017 International Conference onSampling Theory and Applications, 2017.[33] M. Fornasier. Numerical methods for sparse recovery, chapter 2 in:\u201cTheoretical foundations and numerical methods for sparse recovery\"(edited by M. Fornasier). Walter de Gruyter & Co., 2010.[34] S. Foucart. Sparse recovery algorithms: sufficient conditions in terms ofrestricted isometry constants. Chapter 5 in: Approximation theory XIII:San Antonio 2010, volume 13. Springer Proceedings in Mathematics,2012.120Bibliography[35] S. Foucart. Stability and robustness of `1 minimization with weibull ma-trices and redundtant dictionaries. Linear Algebra and its Applications,441(15):4\u201321, 2014.[36] S. Foucart, A. Pajor, H. Rauhut, and T. Ullrich. The gellfand widths oflp-balls for 0 < p \uf8ff 1. Journal of Complexity, 26(6):629\u2013640, December2010.[37] S. Foucart and H. Rauhut. A mathematical introduction to compressivesensing. Birkhauser Verlag, 2013.[38] A. Y. Garnaev and E. D. Gluskin. On widths of the euclidean ball.Soviet Mathematics. Doklady, 30(200-204), 1984.[39] S. Gerschgorin. Uber die abgrenzung der eigenwerte einer matrix. Bul-letin de l\u2019Academie des Sciences de l\u2019URSS. Classe des sciences mathe-matiques et na, (6):749\u2013754, 1931.[40] C.S. Gunturk, M. Lammers, A. Powell, R. Saab, and O. Yilmaz. Sobolevduals for random frames and sigma-delta quantization of compressedsensing measurements. Foundation of Computational Mathematics,13(1):1\u201336, 2013.[41] S.D. Howard, A. R. Calderbank, and S. J. Searle. A fast reconstructionalgorithm for deterministic compressive sensing using second order reed-muller codes. Proc. of Conf. on Information Sciences and Systems,pages 11\u201315, 2008.[42] H. Inose and Y. Yasuda. A unity bit coding method by negative feed-back. Proc. IEEE, 51(11):1524\u20131535, 1963.[43] K. Ireland and M. Rosen. A classical introduction to modern numbertheory. Springer Science+ Business Media New York, 2000.[44] L. Jacques, J. N. Laska, P. T. Boufounos, and R. G. Baraniuk. Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors.IEEE Transactions on Information Theory, 59(4), April 2013.[45] G. A. Jones and J. M. Jones. Elementary Number Theory. Berlin:Springer-Verlag, 1998.[46] F. Krahmer, S. Mendelson, and H. Rauhut. Suprema of chaos processesand the restricted isometry property. Communications on pure andApplied Mathematics, 67(11):1877\u20131904, 2014.121Bibliography[47] F. Krahmer, R. Saab, and R. Ward. Root-exponential accuracy forcoarse quantization of finite frame expansions. Information Theory,IEEE transactions on, 58(2):1069\u20131079, 2012.[48] F. Krahmer, R. Saab, and O. Yilmaz. Sigma-delta quantization of sub-gaussian frame expamsions and its application to compressed sensing.Information and Inference, 3(1):40\u201358, 2014.[49] F. Krahmer and R. Ward. Lower bounds for the error decay incurredby coarse quantization schemes. Applied and Computational HarmonicAnalysis, 32(1):131\u2013138, 2012.[50] K. Li, L. Gan, and C. Ling. Convolutional compressed sensing us-ing deterministic sequences. IEEE Transactions on Signal Processing,61(3):740\u2013752, 2013.[51] S. Li, F. Gao, G. Ge, and S. Zhang. Deterministic construction ofcompressed sensing matrices via algebraic curves. Information Theory,IEEE transactions on, 58(8):5035\u20135041, 2012.[52] M. Lustig, D. L. Donoho, J. M. Santos, and J. M. Pauly. Compressedsensing mri. IEEE Signal Processing Magazine, 25(2):72\u201382, 2008.[53] F. J. MacWilliams and N. J. A. Sloane. The theory of error correctingcodes. Elsevier, 1976.[54] K. Melnykova and O. Yilmaz. Memoryless scalar quantization for ran-dom frames. preprint, 2018.[55] S. Mendelson and A. Pajor. On singular values of matrices with inde-pendent rows. Bernoulli, 12(5):761\u2013773, 2006.[56] D. G. Mixon. Explicit matrices with the rip: Breaking the square-rootbottleneck. Book chapter, Compressed sensing and applications, 2014.[57] Hugh L. Montgomery. Ten lectures on the interface between analyticnumber theory and harmonic analysis. American Mathematical Society,1994.[58] R. R. Naidu, P. V. Jampana, and C. S. Sastry. Deterministic compressedsensing matrices: Construction via euler squares and applications. IEEETransactions on Signal Processing, 64(14):3566\u20133575, April 2016.[59] B. K. Natarajan. Sparse approximate solutions to linear systems. SIAMJournal on Computing, 24:227\u2013234, 1995.122Bibliography[60] J. L. Nelson and V. N. Temlyakov. On the size of incoherent systems.Journal of Approximation Theory, 163(9):1238\u20131245, September 2011.[61] K. Ni, S. Datta, P. Mahanti, S. Roudenko, and D. Cochran. Using reed-muller sequences as deterministic compressed sensing matrices for imagereconstruction. Acoustic Speech and Signal Processing, pages 465\u2013468,March 2010.[62] J. A. Nikara, J. H. Takala, and J. T. Astola. Discrete cosine and sinetransforms: regular algorithms and pipeline architectures. Signal Pro-cessing, 86(2):230\u2013249, 2006.[63] Y. Plan and R. Vershynin. Robust 1-bit compressed sensing and sparselogistic regression: a convex programming approach. Information The-ory, IEEE transactions on, 59(1):482\u2013494, December 2012.[64] Y. Plan and R. Vershynin. One-bit compressed sensing by linearprogramming. Communications on pure and Applied Mathematics,66(8):1275\u20131297, 2013.[65] A. Powell, R. Saab, and O. Yilmaz. Quantization and finite frames,chapter 8 in \u201cFinite frames: Theory and Applications\" (edited by P.Casazza and G. Kutyniok). Birkhauser, Boston, 2012.[66] H. Rauhut. Circulant and toeplitz matrices in compressed sensing. Proc.SPARS\u201909, 2009.[67] H. Rauhut, J. Romberg, and J. Tropp. Restricted isometries for par-tial random circulant matrices. Applied and Computational HarmonicAnalysis, 32:242\u2013254, 2012.[68] J. Romberg. Compressive sensing by random convolution. SIAM J.Imaging Sci., 2(4):1098\u20131128, 2009.[69] M. Rudelson and R. Vershynin. Sparse reonstruction by convex relax-ation: Fourier and gaussian measurements. 40th Annual Conference onInformation Science and Systems, pages 207\u2013212, 2006.[70] M. Rudelson and R. Vershynin. Non-asymptotic theory of randommatrices: extreme singular values. Proceedings of the InternationalCongress of Mathematicians, Hyderabad, India, 2010.[71] R. Saab, R. Wang, and O. Yilmaz. From compressed sensing to com-pressed bit-streams: practical enncoders, tractable decoders. IEEETransactions on Information Theory, 64(9):6098\u20136114, 2017.123Bibliography[72] R. Saab, R. Wang, and O. Yilmaz. Quantization of compressive sampleswith stable and robust recovery. Applied and Computational HarmonicAnalysis, 44(1):123\u2013143, 2018.[73] R. Schreier and G. Temes. Understanding delta-sigma data converters.Wiley, Piscataway, New Jersey, 2004.[74] A. Soshnikov and Y. Fyadorov. On the largest singular values of randommatrices with independent cauchy entries. Journal of MathematicalPhysics, 46, 2005.[75] R. J. Turyn. Character sums and difference sets. Pacific Journal ofMathematics, 15(1), 1965.[76] T. Wang, Q. Berthet, and Y. Plan. Average-case hardness of rip cer-tification. Proceedinngs of the 30th International Conference on Neu-ral Information Processing Systems. NIPS\u201916, Curran Associates Inc.,USA, 2016.[77] L. R. Welch. Lower bounds on the maximum cross-correlation of signals.IEEE Transactions on Information Theory, 20(3):397\u2013399, 1974.[78] S. Wright. Quadratic residues and non-residues. Springer InternationalPublishing, 2016.[79] O. Yilmaz. Coarse quantization of highly redundant time-frequency rep-resentations of square-integrable functions. Applied and ComputationalHarmonic Analysis, 14(2):107\u2013132, 2003.[80] O. Yilmaz. On coarse quantization of tight gabor frame expansions.International Journal of Wavelets, 3(2):283\u2013299, 2005.[81] G. Zhang, R. Mathar, and Q. Zhou. Deterministic bipolar measurementmatrices with flexible sizes from legendre sequence. Electronic Letters,52(11), May 2016.[82] G. Zhang and Q. Zhou. Pseudonoise codes constructed constructed bylegendre sequence. Electronic Letters, 38(8):376\u2013377, 2002.124","type":"literal","lang":"en"}],"http:\/\/www.europeana.eu\/schemas\/edm\/hasType":[{"value":"Thesis\/Dissertation","type":"literal","lang":"en"}],"http:\/\/vivoweb.org\/ontology\/core#dateIssued":[{"value":"2019-11","type":"literal","lang":"en"}],"http:\/\/www.europeana.eu\/schemas\/edm\/isShownAt":[{"value":"10.14288\/1.0381024","type":"literal","lang":"en"}],"http:\/\/purl.org\/dc\/terms\/language":[{"value":"eng","type":"literal","lang":"en"}],"https:\/\/open.library.ubc.ca\/terms#degreeDiscipline":[{"value":"Mathematics","type":"literal","lang":"en"}],"http:\/\/www.europeana.eu\/schemas\/edm\/provider":[{"value":"Vancouver : University of British Columbia Library","type":"literal","lang":"en"}],"http:\/\/purl.org\/dc\/terms\/publisher":[{"value":"University of British Columbia","type":"literal","lang":"en"}],"http:\/\/purl.org\/dc\/terms\/rights":[{"value":"Attribution-NonCommercial-NoDerivatives 4.0 International","type":"literal","lang":"*"}],"https:\/\/open.library.ubc.ca\/terms#rightsURI":[{"value":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/","type":"literal","lang":"*"}],"https:\/\/open.library.ubc.ca\/terms#scholarLevel":[{"value":"Graduate","type":"literal","lang":"en"}],"http:\/\/purl.org\/dc\/terms\/title":[{"value":"Compressed sensing with deterministic measurement matrices","type":"literal","lang":"en"}],"http:\/\/purl.org\/dc\/terms\/type":[{"value":"Text","type":"literal","lang":"en"}],"https:\/\/open.library.ubc.ca\/terms#identifierURI":[{"value":"http:\/\/hdl.handle.net\/2429\/71789","type":"literal","lang":"en"}]}}